Many developers feel faster the moment they turn on an AI coding tool. In fact, most already use these tools every day. Yet there is a point where the "feeling" and the measurement diverge. In one randomized controlled trial (RCT, an experiment that randomly assigns participants to conditions to compare effects), experienced developers were actually slower when allowed to use AI [Source: METR, 2025]. This piece examines that gap with data. And it gives you a way to decide when to trust these tools and when to verify them.
In this article
- Adoption raced ahead; conviction lagged behind
- The productivity paradox: perceived vs. measured
- Adoption is up, but trust is down
- When it helps and when it gets in the way
- In organizations: AI is an amplifier
- So how do you use these tools well?
- Conclusion: still an open question
Adoption raced ahead; conviction lagged behind
Start with the pace of adoption. In Stack Overflow's 2025 Developer Survey (49,009 respondents across 166 countries), 84% said they use or plan to use AI tools [Source: Stack Overflow, 2025]. That is up from 76% in 2024. Among professional developers, 51% reported using AI tools daily.
Organization-level surveys tell a similar story. In Google's DORA 2025 report, about 90% of respondents said they use AI in their work [Source: DORA, 2025]. The report draws on roughly 5,000 technology professionals and more than 100 hours of qualitative data. In short, adopting AI coding tools is no longer the exception but close to the default.
The catch is that an adoption rate does not equal impact or trust. Using a tool and having that tool actually raise your productivity are two different things. This gap is often called the productivity paradox. From here, we take that paradox apart with data.
The productivity paradox: perceived vs. measured
The most counterintuitive evidence comes from METR, an AI evaluation research organization, and its 2025 randomized controlled trial. Sixteen experienced open-source developers worked through 246 real issues [Source: METR, 2025]. Each task ran about two hours on average, in mature repositories averaging more than 22,000 stars and over a million lines of code. The result ran against expectations. When AI use was allowed, developers took 19% longer to complete issues.
The perception-reality gap was even wider. Before the study, participants expected AI to make them 24% faster [Source: METR, 2025]. Even after the study, they still believed AI had made them 20% faster. They felt this way despite actually being slower. In other words, perceived speed and measured speed pointed in opposite directions.
The conditions, however, must be read alongside the result. The sample was small at 16 people, the participants were experienced, and the repositories were large codebases they already knew well [Source: METR, 2025]. The tools were Cursor Pro and the then-frontier models Claude 3.5/3.7 Sonnet. The authors were explicit that the finding does not mean "AI slows down most developers." They also noted it does not generalize to less-experienced developers or unfamiliar codebases.
What's being measured matters
To read the numbers in this piece without confusion, remember one thing. Each study measures a different target. What METR measured is individual task-completion time [Source: METR, 2025]. That is not the same layer as an organization's overall productivity or delivery performance.
Mixing the layers produces false conclusions. The Stack Overflow and Fastly figures are developers' self-reported perceptions. The DORA figures are correlations with an organization's software delivery performance. The Anthropic figures are scores on a comprehension test about learning. "Feeling 20% faster," "measured 19% slower," and "higher organizational throughput" are claims from different layers. Not chaining them into cause and effect within one sentence is the first step to reading this topic accurately.
Adoption is up, but trust is down
You might expect trust to climb along with adoption, but the data says otherwise. In the Stack Overflow 2025 survey, about 33% said they trust the accuracy of AI answers [Source: Stack Overflow, 2025]. By contrast, about 46% said they do not trust it, a larger share. Set against the 84% adoption rate, the contrast is stark.
The root of the distrust is subtle inaccuracy. In the same survey, the top frustration was AI answers that are "almost right, but not quite," cited by 66% [Source: Stack Overflow, 2025]. And 45.2% of respondents said debugging AI-generated code takes them longer. A fast first draft can hand back the time it saved during verification and fixes.
The overall mood cooled as well. Positive sentiment toward AI, above 70% in 2023-2024, fell to 60% in 2025 [Source: Stack Overflow, 2025]. Trust in accuracy also declined from the prior year. The exact size of that drop is not confirmed by a primary source, so here we note only the direction: lower than the year before.
When it helps and when it gets in the way
The key question is not "does AI help?" but "when does it help?" Gather the evidence and two axes appear. The type of task, and the developer's experience.
By task type
The nature of the work strongly shapes the outcome. Addy Osmani, of Google Chrome's developer experience team, frames this as "the 70% problem" [Source: Addy Osmani, 2024]. AI quickly produces about 70% of a solution, but the last 30% is hard. Edge cases, security, and production integration fall into that 30%. Note that this is a practitioner's observation, not an empirical study.
METR's result points the same way. When experienced developers handled subtle issues in large, familiar codebases, AI added time instead of saving it [Source: METR, 2025]. The summary is this. For repetitive, well-structured work, AI drafts quickly. But in the final stretch that demands deep contextual understanding, people still carry the weight.
By experience level
Experience splits the outcome too. In Fastly's 2025 survey (791 US professional developers), 32% of seniors (10+ years) said they ship more than half of their code as AI-generated [Source: Fastly, 2025]. For juniors (0-2 years) the figure was 13%, so seniors reported it about 2.5 times as often. This is a self-reported, US-only survey, which is a limit to keep in mind. One reading is that seniors have more capacity to filter AI output through their own judgment.
On the other side is the question of learning and skill. In Anthropic's 2026 controlled experiment, 52 mostly-junior engineers learned an unfamiliar library [Source: Anthropic, 2026]. On a comprehension test, the AI-using group averaged 50 versus 67 for the group that wrote code by hand, leaving the AI group 17 points lower. Note that the 17 here is a difference in comprehension scores, not a difference in speed.
The same experiment signals that how you use AI splits the result. Those who asked the AI conceptual questions scored above 65 [Source: Anthropic, 2026]. Those who delegated the code generation itself scored below 40. In other words, how you use the tool decides whether skill atrophy sets in.
In organizations: AI is an amplifier
Move beyond the individual to the organization and the picture shifts again. The DORA 2025 report casts AI as an amplifier. The report puts it this way.
"AI doesn't fix a team; it amplifies what's already there." — DORA, 2025
That is, strong teams get better with AI, while weaker teams see their existing problems surface more [Source: DORA, 2025]. Delivery outcomes split in two directions as well. DORA reported that, unlike the prior year, AI adoption showed a positive relationship with software delivery throughput and product performance. Yet it continued to show a negative relationship with software delivery stability. Throughput can rise while stability wobbles. This is a statement of correlation, and DORA attaches a condition: without controls such as strong testing, version control, and fast feedback, more change tends to lead to instability.
Caution about trust lingers here too. In the same report, 30% said they trust AI-generated code little or not at all [Source: DORA, 2025]. Even inside organizations where adoption is the default, teams do not take the output at face value.
Code-quality signals overlap with this picture. The code-review vendor CodeRabbit analyzed 470 open-source PRs and found AI-coauthored PRs averaged 10.83 issues each, versus 6.45 for human-only PRs, a difference of about 1.7 times [Source: CodeRabbit, 2025]. That said, this is the vendor's own analysis and the sample is limited to 470 PRs. The code-analysis firm GitClear analyzed 211 million lines from 2020 to 2024 and reported that code churn rose from 3.1% to 5.7% [Source: GitClear, 2025]. Here, code churn means the share of new code revised or reverted within two weeks of its first commit. Over the same period, the share of copy-pasted lines grew from 8.3% to 12.3%. Still, these signals only overlap in timing with AI adoption; they are correlation, not causation.
So how do you use these tools well?
The data above leads to a simple conclusion. Use the tools to fit the situation, and verify as you go. What follows turns that principle into practice.
First, verify the output by default. It helps to remember that "almost right, but not quite" answers were the single biggest frustration [Source: Stack Overflow, 2025]. The faster the draft, the more it pays to check it once more before it reaches production.
Second, split what to delegate from what to keep. Hand repetitive drafting to AI, but you are better off keeping the last 30%, such as edge cases, security, and production integration [Source: Addy Osmani, 2024]. For subtle problems in a large, familiar codebase, there is particular reason to be wary of leaning on AI [Source: METR, 2025].
Third, do not loosen code review. Since more issue signals were observed in AI-coauthored code, review is something to strengthen rather than relax [Source: CodeRabbit, 2025]. Fourth, protect junior learning. Asking the AI conceptual questions preserved comprehension better than handing it the code generation wholesale [Source: Anthropic, 2026].
Conclusion: still an open question
If one thing is certain on this topic, it is that we are not yet certain. METR reclassified its 2025 result as historical [Source: METR, 2026]. Based on conversations with participants, it judges that developers are likely faster with AI now, in early 2026. But it stated that, because of selection bias, the evidence for the size of that increase is very weak. In fact, 30-50% of participants did not submit tasks they would rather not do without AI in the first place. So METR is changing its experiment design. The reversal, then, is a qualitative observation, not a measured figure.
Because tools and models keep improving quickly, today's numbers may look different tomorrow. Even the polarizing question of "will AI replace developers?" gets no clean answer from the current data. What the data actually says is conditional. Outcomes depend on when, what, and who uses the tools. So the conclusion settles into a single habit. Choose to fit the situation, and verify as you use. This question is still open, and knowing that much is, for now, the most accurate stance to take.