# MTTR is the Wrong Metric for AI-Era Engineering Teams --- AI tools produce 41% more bugs and 98% more pull requests. MTTR can't keep up. Here's how MTTF shifts your team from incident response to prevention. --- By Evan Marshall · 2026-07-23 --- AI makes code generation and deployment faster, but our safeguards haven’t caught up. For over a decade, mean time to recovery, or MTTR, was the industry’s primary protective measure against bad code. Organizations assumed failure was inevitable and used MTTR to measure how fast they could recover. The smaller your MTTR, the better. MTTR, however, doesn’t map to AI-era failure modes. It doesn’t meaningfully capture the speed of AI-generated code production, nor the potential blast radius of autonomous agents. Every engineering team is already rethinking their deployment process, so it’s time to rethink recovery, too. Enter mean time to failure, or MTTF. MTTF first emerged in hardware contexts, where it referred to the average operating time before a non-repairable component broke and required replacement. In software, we can use the same framework to focus on the component *we* can’t have broken: Your customer’s experience. ## DORA, SRE, and the assumption of failure MTTR might feel like it’s been around forever, but it’s only been around for about a decade. It has its roots in DORA and site reliability engineering (SRE), both of which were responses to the rise of distributed computing. MTTR was a valid response to a real shift in how we produce software, but with the rise of AI, old ideas working from previously valid principles can cause damage. That’s why it’s worth rewinding to see why MTTR emerged, and why it no longer suits what we need today. ### DORA: DevOps Research & Assistance The first study in [what would become DORA](https://dora.dev/insights/dora-metrics-history/) attempted to tie IT performance to organizational performance. The goal was ambitious: How could organizations link sometimes abstract-feeling technical goals to real business value? The research turned up four values, split into two categories: - 1 Throughput measures: Deployment frequency and deployment lead time. - 2 Stability measures: MTTR and change fail rate. Note that, in the years since, there have been updates. In 2023, DORA renamed MTTR to failed deployment recovery time to focus on failures caused by software changes, rather than events like data outages. And in 2024, DORA added a fifth metric: deployment rework rate. By then, however, MTTR had already become the de facto term. In the years since, DORA has become the preeminent methodology for measuring performance, replacing (and in many cases upgrading) other methods. These five metrics subsequently became canonized, and even organizations that don’t use DORA know to prize these metrics. ([Source](https://www.aviator.co/blog/everything-wrong-with-dora-metrics/)) As DORA metrics were taking off, Google was formalizing a new role and philosophy: Site Reliability Engineering, or SRE. The SRE approach emphasizes automation, reliability, and hands-on incident management. This places SRE engineers in charge of building fault-tolerant systems and minimizing failure. MTTR became a key metric for Google’s SREs, resulting in MTTR’s spread as other organizations adopted SRE roles. As the metric spread, business leaders keyed in, too. Many organizations defaulted to these metrics, along with the frameworks they implied, to build consensus across teams. Over time, organizations could capture failure in one simple number, and trust everyone understood what they meant. Shared understandings can be dangerous, however, when new concepts unseat previously foundational assumptions. ## The limitations of MTTR MTTR is a widely accepted standard. The problem with standards is that the map can be mistaken for the territory. What happens then when the territory changes underneath this map? ### Goodhart’s Law and its consequences When Anthropic published a case study with Spotify, the online response wasn’t entirely positive, with developers pointing out that velocity doesn’t equal value, and fewer bugs don’t mean a better experience. As Mitchell Hashimoto, cofounder of HashiCorp, writes in a [representative post](https://x.com/mitchellh/status/2071971627748020409), *“This is the problem with a lot of AI reporting. It reports completely meaningless metrics.”* Production deploys per day, proportion of AI-assisted PRs, MTTR: They’re all classic cases of Goodhart’s Law, which states, *“When a measure becomes a target, it ceases to be a good measure.”* Goodhart’s Law is a common citation in these conversations, but it can be easy to dismiss. As Hillel Wayne, a software development consultant, [writes](https://buttondown.com/hillelwayne/archive/goodharts-law-in-software-engineering/), *“The weak version of this is that people have perverse incentives to game the metrics.” Measure development productivity by lines of code, for example, and you’ll find yourself with verbose code, not greater productivity.* Wayne, instead, emphasizes a strong version: *“Even 100% honest pursuit of a metric, taken far enough, is harmful to your goals, and this is an inescapable consequence of the difference between metrics and values.” This version requires no bad faith or gaming. An honest pursuit of one metric among others has a way of twisting things in unintended ways.”* The philosophy underpinning MTTR (that you can't prevent every failure) isn’t wrong, but it licensed the treatment of customer pain as a cost of doing business. As development accelerates, MTTR, followed faithfully and innocently, can cause teams to rely on customers having negative experiences, especially as they scale up velocity. ### The pain of MTTR in the age of AI MTTR sanitizes what failure means. Ultimately, customers had to experience a failure, report it, file a ticket, and wait for it to be fixed. That’s true no matter how low your MTTR is. Depending on your scale, that could mean tens, hundreds, thousands, or millions of people having bad experiences. And because there’s typically a delay between a bug happening and a user reporting it, users can live with bad experiences for long periods. Worse, only a fraction of users ever report bugs, meaning many will suffer through the experience, feeling discouraged or becoming churn risks. With AI, the risk and scale of this failure pattern is only increasing. According to research from [DORA and Google](https://cloud.google.com/blog/products/devops-sre/announcing-the-2024-dora-report), an increase in AI adoption has been accompanied by a 1.5% decrease in delivery throughput and a 7.2% decrease in delivery stability. And they’re not alone. Research from Uplevel complements these results, showing a [41% increase in bug rate](https://uplevelteam.com/blog/genai-developers) resulting from the use of generative AI tools. If you measure success based on MTTR, you're allowing your customers to experience paper cuts frequently and big pains periodically. If you accept that pattern in the name of fixing it fast, you accept a bad experience in disguise. This was just as true before AI as it is now, but with AI, velocity *and* bugs are rising. You’re putting more work in front of your customer than ever, and it's buggier than ever. Bad experiences will occur more frequently. Even if AI helps you resolve them, you're still making your customers play quality assurance for you. Sadly, the pain is also disproportionate in a way no company wants: The customers suffering the most will be your power users and early adopters. Organizations end up in a situation where AI transforms everything except their ability to reliably provide quality user experiences. Fixing it fast still means you're on day zero, again and again and again. That's no way to build a product, and it's certainly not a good way to measure quality. That's why MTTR has to go. ## MTTF: An old metric with a new life You’ve likely heard of MTTF, or mean time to failure. MTTF has its origins in hardware, where organizations used it to measure the average time before components broke down and required replacement. Reframed for software, MTTR has a new lease on life and the capacity to change how we think of failure in the wake of AI. Here, failure isn’t an outage or a dip in reliability but a customer having some level of negative experience. This shift in metrics also requires a shift in emphasis away from post-deployment response to pre-deployment review and [runtime analysis](https://ito.ai/blog/static-analysis-misses-bugs-that-reach-production). ### Code review is the current bottleneck With AI, we can accelerate sheer code output. However, a 10x increase in code also requires a 10x increase in code review capacity. Unfortunately, as an industry, we’ve spent much more time increasing code production than improving our review processes. Anyone familiar with systems building knows what happens next. In *The Phoenix Project*, written almost 15 years ago, the authors write, *“Any improvement made after the bottleneck is useless because it will always remain starved, waiting for work from the bottleneck. And any improvements made before the bottleneck merely results in more inventory piling up at the bottleneck.”* Right now, we’re just putting more pressure on the code review bottleneck and hoping for increased throughput anyway. The results speak for themselves. [Faros AI’s 2025 telemetry study of 10,000+ developers](https://arxiv.org/pdf/2605.01160) found that “A 98% increase in merged pull requests coincides with a 91% increase in review time and flat organizational delivery metrics.” The *New York Times*, providing an even more vivid example, spoke with a [financial services company](https://www.nytimes.com/2026/04/06/technology/ai-code-overload.html) that went from producing 25,000 lines of code per month to 250,000, resulting in a backlog of 1 million lines of unreviewed code. Some metrics moved (PRs and lines of code), but did the most important ones change for the better? ### The blast radius of bad code is growing Despite the bottleneck, the risk of rushing code review is enormous. Research from [METR](https://metr.org/blog/2025-03-19-measuring-ai-ability-to-complete-long-tasks/) shows that the length of tasks that AI agents can complete on their own is doubling about every 7 months. As true AI-generated code scales, the volume of potential bugs increases (recall the 41% increase in bug rate cited earlier). With [agents creating code autonomously](https://ito.ai/blog/what-is-agentic-qa), including bugs, the blast radius of failures can increase in tandem. More bugs across more code can result in domino effects that spread across systems, and because agents author most of the code, the failures can be inscrutable to human incident response teams. Code review processes, as they stand today, are not up to the task. You try to catch things before they ship, knowing that you likely won’t catch everything. That leaves you acknowledging that you’ll only catch the issue later, in production. From there, you can roll back, fix, and keep compressing, but *never eliminate*, MTTR. This is especially risky when MTTR fails to capture a complete view of poor customer experiences. ### Do you know what you’re recovering from? According to research from [VOID](https://www.infoq.com/articles/incident-metrics-void/), there is no correlation between incident duration and incident severity, meaning MTTR can be misleading. According to the company’s research, organizations *“can have long or short incidents that are very minor, existentially critical, and nearly every combination in between. Not only can duration not tell a team how reliable or effective they are, but it also doesn’t convey anything useful about the event's impact or the effort required to deal with the incident.”* As a result, MTTR captures skewed data when we need precise data more than ever. The more code you generate, the more bad experiences MTTR will fail to capture and prevent. You’ll never know which issues were fine and only seen by one small customer, and which ones hit your core user base on a core workflow, or a key buyer on the day they’re making a purchasing decision. ### New metrics, new workflows MTTF implies a new way of thinking. Suppose there’s a bug in the code. Do we care? The answer requires a follow-up question: Does the bug cause a bad customer experience? If no, we don’t care; if yes, we do care, and we care enormously. The throughline is not quality for quality’s sake, but confidence that customers will not have bad experiences. Short-term, that means accelerating developers with AI, relieving the code review bottleneck without sacrificing or driving your engineers crazy with endless code review, and trusting that user experiences will remain good throughout. Long-term, this enables teams to ask difficult questions without worrying about losing quality. For example, a common debate among many engineering teams is whether you should let non-technical people vibe code to production? The cost of doing this can be high, but with the right safeguards, so can the rewards. With MTTF, you safeguard the shift because engineers won't have to measure how fast they can fix what the vibe coders mess up. They can instead ensure the right guardrails are in place, so that only the good results of vibe coding make it to production. MTTR, however, isn’t a hot-swappable metric. Industry workflows will have to change as organizations shift to MTTF. For example, when developers have [agents working in parallel](https://ito.ai/blog/ai-scaled-engineering-org-needs-big-org-processes) with them and each other, teams will have to think about the pace of work differently. Rushing to production – shipping fast and rolling back later – won’t be the default right move anymore. Not when code review is as scalable as code generation. To build the bridge to this future state, organizations can use a combination of tools to provide multiple code review passes: 1. Run [static analysis tools](https://ito.ai/blog/static-analysis-misses-bugs-that-reach-production) to find issues in the code. Fix the code until the static analysis greenlights it. 2. Run Ito to detect behavioral issues through runtime analysis. Fix the code until Ito greenlights it. 3. Ask a human to check the code. Fix the code until the reviewer is happy. 4. Run smoke tests and/or E2E tests as a final pass. 5. Merge. Each pass will catch different issues, and in combination safeguard your customers from having bad experiences when you ship. Now, you do have to remember that MTTF is not a silver bullet. Don’t treat MTTF as another singular metric, or it will become another victim of Goodhart’s Law. Start by defining the unit: a customer-impacting failure is any discrete moment a real user hits a degraded experience. MTTF is the interval between those moments. Instead of reducing it to a single, simplistic number, track it as an impact-weighted distribution, separating the cohorts that suffer first and churn fastest (power users, new users, high-value accounts) and the workflows you can't afford to break (checkout, login). Then, pair that lagging indicator with a leading one: how often a customer-facing issue slips past static analysis, runtime analysis, human review, and smoke and E2E tests. Together, you can build a scorecard that lets you track and improve MTTF. By putting this into practice, you’ll stop asking how fast you cleaned up after a customer got hurt, and start asking how long it's been since a customer got hurt at all. ## Skills, not cycles Before AI, the industry mapped software development linearly, with the SDLC aiming to reduce MTTR (treating failure as built-in). With AI, we can rethink these foundational assumptions and view our organization as building skills rather than following cycles. For example, in a video game like Runescape, you build skills over time, and the decisions you make at the early levels can shape the decisions you’re able to make at the later levels. As you progress in your skill tree, your early decisions compound, and by the end game, you can find yourself grateful or regretful, victorious or struggling. If you want to reach a particular end state, you need to know where you’re going when you start. Thinking ahead to the end state and building experiences that compound instead of subtract is the framework we need to embrace. Prevention compounds, while recovery resets you to day zero again. And that’s the core benefit of our work: catching issues *before* they hit your customers, not helping you respond faster once they do. Ito is the MTTF layer that static code review misses, but your customers experience if you don’t catch them. What will your engineering team prioritize? **CTA**: The world of AI code review tools is complex, especially when comparing static and runtime analysis tools. To learn more about the differences and nuances, check out [our explainer](https://ito.ai/blog/static-analysis-misses-bugs-that-reach-production).