You can’t tell whether your delivery is normal

Your last config change took nine days to get from commit to production. There were two deployments that month, and one of them went back out the same evening to fix what the first one broke. Is that fine, bad, or quietly better than most? Your internal deployment history alone doesn’t tell you how you compare with other teams.

That’s the gap the 2024 Accelerate State of DevOps Report fills, and it’s the reason to read it even if you don’t care about survey research. It’s the tenth report in the series, published by DORA, the research team inside Google Cloud led by Nathen Harvey and founded by Nicole Forsgren, Jez Humble and Gene Kim before Google acquired it. Two populations get quoted interchangeably and shouldn’t be. DORA has heard from “roughly 39,000 professionals” across the decade of the research programme; this year’s survey is the smaller slice, “nearly 3,000 working professionals”, from 104 countries. What they produce is four numbers and a table, and the table is a useful free cross-industry yardstick. This year it also says something uncomfortable about the tools everyone in your organisation is busy adopting.

One disambiguation first, because Europe has its own DORA. The Digital Operational Resilience Act, Regulation (EU) 2022/2554, applies from 17 January 2025 and covers ICT risk management, incident reporting and third-party oversight for financial entities. Different document, different problem, nothing to do with deployment frequency. Everything below is the other DORA.

Four metrics became five

Start with what the report measures, because one metric is new this year.

What is the fifth DORA metric?

Rework rate: the percentage of your deployments over the last six months that weren’t planned, but fixed a user-facing bug.

DORA has long suspected that change failure rate is really a proxy for rework, so this year they asked directly, and the survey question asks for a share, not a count: “Approximately what percentage of deployments in the last 6 months were not planned but were performed to address a user-facing bug in the application?” The two turned out to be related, and together they now form the factor the report calls software delivery stability.

That regrouping matters more than the new question. Change lead time, deployment frequency and failed deployment recovery time are now the throughput factor. Change failure rate and rework rate are the stability factor. If you learned the four keys as two-and-two, recovery time has moved sides.

Two definitions are worth nailing down. Change lead time runs from the commit to the change going live, not from a merge to main. Deployment frequency counts changes reaching production, not merges.

Recovery time is where we’ve seen some clients start, because it’s easier to measure than the others. Disaster recovery and business continuity sit in many compliance frameworks, so an RTO is usually written down. But an RTO is a target: how fast you intend to recover, not how fast you did. DORA asks a narrower question anyway, recovery from a failed deployment rather than from a site-wide disaster, and those durations live in your incident records, not in the DR policy. The clients who do measure lead time get an answer in months, sometimes years.

If your platform counts for you, GitLab reports all four on Ultimate, with three mismatches to watch. Its lead time starts at merge request merge time, not at the commit, so the time before that merge needs instrumenting separately. Its time to restore service is the median time incidents stayed open on a production environment, not only the ones a deployment caused. And its change failure rate divides incidents by deployments to a production environment with nothing tying one to the other, so an unrelated outage counts against you and two incidents from one deployment count twice. Issue 444295, open since March 2024, proposes a fix. Before you compare with DORA’s table, filter to deployment-related incidents and base the percentage on distinct deployments that caused a failure somebody had to remediate.

What elite looks like this year

Definitions only matter because of the table they feed, and the table is what everyone screenshots.

What counts as elite in 2024?

Under a day from commit to production, deployments on demand, a 5% change failure rate, and recovery from a failed deployment in under an hour.

The four clusters, with the share of respondents in each:

• Elite, 19%: change lead time under one day, deployment on demand, 5% change failure rate, recovery in under an hour.
• High, 22%: lead time between one day and one week, deployment between daily and weekly, 20% change failure rate, recovery in under a day.
• Medium, 35%: lead time between one week and one month, deployment between weekly and monthly, 10% change failure rate, recovery in under a day.
• Low, 25%: lead time between one month and six months, deployment between monthly and twice a year, 40% change failure rate, recovery in one week to one month.

These levels aren’t set in advance. DORA runs a cluster analysis on the responses and sees what separates, so the boundaries move with the sample each year, and a tier describes this year’s respondents rather than a standard. Compared with the low cluster, the report puts elite performers at 127 times faster lead time, 182 times more deployments per year, 8 times lower change failure rate and 2,293 times faster recovery.

Read the low row again, because a quarter of this year’s respondents sit there. A change takes between one month and six months to reach production, and two deployments in five cause a failure. That’s an unpleasant place to work and a worse place to be on call.

The useful move is to find your row, then ignore the label and look at which of the five numbers you could change this quarter.

Medium performers fail less than high performers

One row of that table contradicts the advice we all repeat, and it’s worth sitting with rather than skipping.

Why does the medium cluster fail less often than the high cluster?

Because speed and safety aren’t a single axis, and the cluster names flatten them into one.

High performers report a 20% change failure rate. Medium performers report 10%, while deploying less often and taking longer to get changes out. Smaller and more frequent is supposed to be safer, and here the slower group fails half as often. The report doesn’t hide it. A section called “Throughput or stability?” takes the contradiction on directly. The correlation between the two factors “persists even in the medium performance cluster, where throughput is lower and stability is higher than in the high performance cluster”. DORA reads that as a sign that “factors besides throughput and stability influence cluster performance”. Its advice to the medium cluster is that it “may benefit from shipping changes more frequently”. DORA also concedes that calling the faster teams high performers shows one of the potential pitfalls of the levels.

The low row is easier to explain, with one condition attached. A 40% change failure rate goes with deployments that are months apart, and if people kept committing through those months, each deployment carries all of it. How much that is depends on how many people commit and how often: two developers producing a couple of commits a week don’t build the same batch over a month that a thousand-commit organisation does, and the rest of the metrics follow from that volume.

So the practical read of the table isn’t your tier. It’s your delivery batch: how much change rides on one deployment, and how fast you can take it back out.

AI adoption correlated with better code and worse delivery

Batch size is also where the report’s headline finding lands, which makes this the section most teams will argue about.

Does AI adoption improve software delivery?

Not in this data. It tracks with improvement in almost every step of the process and still comes out negative on both delivery factors.

For every 25% increase in an individual’s AI adoption, the report estimates a 7.5% increase in documentation quality, 3.4% in code quality, 3.1% in code review speed, 1.3% in approval speed, and a 1.8% decrease in code complexity. Those are the inputs DORA’s own prior research says should improve delivery. The same 25% increase is associated with a 1.5% decrease in delivery throughput and a 7.2% decrease in delivery stability.

These are associations in survey data, so read the explanation the same way DORA writes it. They hypothesise that “since AI allows respondents to produce a much greater amount of code in the same amount of time, it is possible, even likely, that changelists are growing in size”. Their conclusion is more firmly stated than the mechanism: improving the development process does not automatically improve software delivery, “at least not without proper adherence to the basics of successful software delivery, like small batch sizes and robust testing mechanisms”.

Adoption itself is no longer the interesting question. 75.9% of respondents rely on AI in at least one of their daily responsibilities, with code writing at 74.9% and summarising information at 71.2%, and 81% say their organisation has shifted priorities toward building AI into its products. Stack Overflow’s 2024 survey puts current use at 62%, up from 44% a year earlier. Whichever number you prefer, using an assistant is the baseline rather than the experiment, which is a change from the advice we were giving a year ago.

Two caveats are worth carrying into your own team. The individual numbers are positive but small: flow up 2.6%, job satisfaction up 2.2%, productivity up 2.1%, and time spent on work people consider valuable down 2.6%. It feels better than it measures. And the output isn’t stable between runs, so whatever sits between the model and production has to be an actual check. On Alphabet’s Q3 2024 earnings call, Sundar Pichai said that “more than a quarter of all new code at Google is generated by AI, then reviewed and accepted by engineers”. The review half of that sentence is the part to copy.

The platform you built to help

AI isn’t the only thing in this report that helps the people and hurts the delivery.

Is your internal developer platform making delivery faster?

Not on its own. Respondents using an internal developer platform reported roughly 8% lower throughput and 14% lower change stability than those who don’t.

The report is positive about platforms on productivity, team performance and organisational performance, and reports that trade-off anyway. Mandated platforms get a separate association: respondents required to “exclusively use the platform to perform tasks for the entire app lifecycle” showed a 6% decrease in throughput. That is its own comparison, not an extra penalty stacked on the 8%, so don’t add the two. DORA offers three hypotheses for the stability drop: the platform gives teams confidence to push changes they can quickly remediate, the platform isn’t catching bad changes, or teams that already had instability and burnout are the ones who build platforms. They also found that instability combined with a platform is linked to higher burnout, which is the opposite of what platform teams are funded to deliver.

What developers want from a platform is unglamorous and easy to describe: deploy from a terminal without logging into Jenkins to trigger the job, SonarQube to read the coverage, and the artifact repository to confirm what shipped. Cutting that down is real work with a real payoff. The harder half is that infrastructure keeps getting more complicated for good reasons. You go multi-region for resilience, you split accounts for compliance, and now a developer chasing a failed request has to know which region served it and where those logs went. Hide that completely and nobody can debug it; expose all of it and you’ve put the whole estate on top of the business logic they were hired to write.

This is where we don’t agree. Mattias is split on the fully hidden version: when you deploy an application, you should know roughly where it runs and what it talks to. Andrey’s answer is that it depends on headcount. With a team of ten you can hire people who clear that bar, and at ten thousand you work with the people you can get, which is the whole point of a platform that makes less specialised engineers productive.

Priorities that keep moving

The most expensive finding in the report has nothing to do with tooling.

What do constantly shifting priorities cost?

Small but meaningful drops in productivity, substantial increases in burnout, and the report could not find anything that counteracts it.

DORA tested the obvious mitigations. Strong leaders, good internal documents and a user-centred approach to software development were all examined for whether they soften the effect of unstable priorities on burnout: “The answer is: They can’t. An organization can have all these positive traits and, if priorities are unstable, employees will still be at risk of experiencing burnout.” Their explanation is chronic uncertainty rather than change itself. Business direction should shift; it’s the frequency that does the damage.

Leadership moves the same outcomes in the other direction and moves them hard. A 25% increase in transformational leadership is associated with a 9.1% increase in job satisfaction, an 8.7% increase in productivity, an 8.7% increase in team performance and a 9.9% decrease in burnout. Those are larger numbers than AI adoption produced anywhere in the same report.

Then the finding that should stop you: when priorities stabilise, software delivery performance declines. DORA calls it puzzling and hypothesises that teams with stable priorities may be shipping less often, “and in larger batches than recommended”.

If you’re the person between leadership and the team, the legitimate job is the one DORA recommends, to shield employees from the day-to-day churn: push back, ask for the clarification nobody wrote down, and get the priority confirmed before it reaches the board. There’s a dirtier version, and the warning travels with it: don’t replicate this at your workplace. Leave the request sitting on the board untouched until somebody asks about it a second or third time, on the theory that most of them never come back. We can’t tell you it works, we wouldn’t write it into a process document, and it’s unlikely to win you a career. It might win you some points with your team.

What this means for teams

The report isn’t a scorecard, and treating it as one is how it gets misused. It’s a set of comparisons that make your own numbers interpretable, plus one argument worth taking seriously: the practices that make each step of development better don’t automatically make delivery better, and DORA’s best guess at what sits in between is how much change you ship at once.

Four things worth doing this week:

• Get one real lead time. Pick your most active service, take the last twenty production deployments, and measure from each commit to the moment it served traffic. That gives you a lead-time band, which is one column of the table above. It isn’t a performance level, so measure the other three before you tell anyone which cluster you’re in.
• Work out your rework rate. Count the deployments in the last six months that were unplanned and went out to fix a user-facing bug, divide by all your deployments over the same six months, and multiply by 100. That percentage is what DORA’s questionnaire asks for. It takes an afternoon in your deployment history and nobody needs to buy anything.
• Start recording change size. If AI assistance is in your editors, track the median diff per deployment from now on. DORA’s explanation for the stability drop is a hypothesis about your repositories, and your repositories can test it.
• Check whether your platform is mandatory. If teams must use it for the whole lifecycle, find out what that is associated with in your own throughput before you add the next gate to it.

Common questions, answered

How do I measure DORA metrics without buying a tool?

Start from two timestamps you already have: the commit, and the moment the deployment finished. Lead time is the difference, deployment frequency is a count per service per week, and both come out of your CI history with a script. Recovery time is usually in your incident records. GitLab’s DORA reporting does the arithmetic on Ultimate, with three adjustments first: its lead-time clock starts at merge, its recovery clock counts every production incident, and its change failure rate is incidents divided by deployments with nothing tying one to the other, so an unrelated outage or a second incident from one deployment inflates it. Count the share of distinct deployments that caused a failure somebody had to remediate.

Is deploying once a month actually bad?

Not by itself, and the 2024 data makes the case. Teams in the medium cluster deploy between weekly and monthly with a 10% change failure rate, half the rate of the cluster above them, and at 35% of respondents they’re the largest group. What makes monthly deployment dangerous is what accumulates in between: how much rides on each release, and how fast you could pull one back out. DORA’s own note is that the cluster may benefit from shipping more frequently.

Our AI-assisted changes keep breaking production. What do we fix first?

Check change size, review quality and test coverage; if changes have grown, try smaller batches and measure. The report estimates a 7.2% decrease in delivery stability for every 25% increase in AI adoption, and hypothesises bigger changelists; its recommendation pairs small batches with robust testing. Keep one logical change per deployment, and run your tests against generated code rather than around it. When stability recovers as the diffs shrink, you have tested DORA’s hypothesis on your own code. If it doesn’t, you may not have enough deployments to tell yet.

Related episodes

• Episode 69, Who is Paulina? is change lead time seen from inside client work: three-hour test runs before a merge, and approvals that sat for two months.
• Episode 61, GitHub Actions and the evolution of CI/CD covers the machinery deployment frequency actually rests on, and why pipelines resist being tested.
• Episode 60, ChatGPT anniversary is where we told practitioners to experiment with these tools. DORA’s 2023 report already examined AI adoption and software delivery performance, and the 2024 edition adds further evidence on that relationship.
• Episode 56, Backstage and internal development platforms is the platform conversation a year before DORA put a throughput and stability number on it.

Resources

• 2024 Accelerate State of DevOps Report, PDF is the primary source for everything quoted above, with the page numbers linked inline.
• 2023 Accelerate State of DevOps Report, PDF is the previous year’s look at AI adoption and its effect on team-level outcomes, which is what the 2024 numbers build on.
• DORA’s 2024 survey questions settle exactly how rework rate and change failure rate were asked, both as a percentage between 0 and 100.
• DORA Quick Check answers where your team sits against the industry without a measurement project first.
• Stack Overflow 2024 Developer Survey, AI section is an independent adoption number from a different population: 62% currently using AI tools, up from 44%.
• Alphabet Q3 2024 earnings call transcript, 29 October 2024 is the primary record of what Sundar Pichai said about the share of new code at Google that AI generates.
• GitLab’s DORA metrics documentation settles which DORA metrics a platform can report for you, on which tier, and which events it starts and stops counting from.
• GitLab issue 444295 is GitLab’s own record that its change failure rate does not associate an incident with the deployment that caused it.

Since we recorded

The 2025 edition is the State of AI-assisted Software Development report, published in September 2025, and it reports AI adoption at 90% of respondents, a throughput relationship that has flipped positive, and instability that has not: more change failures and more rework. DORA’s framing there is that AI amplifies whatever your organisation already does, which is a sharper version of the batch size argument above. We came back to what AI does and doesn’t do for delivery in Episode 84, and to what platforms are really buying teams in Episode 95.