The Illusion of Productivity: How AI Tools are Slowing Down Experienced Developers — Findings from METR Research

METR conducted a controlled study involving a limited group of 16 experienced developers who tackled genuine programming challenges. The participants were divided into two distinct groups:

One group utilized AI tools, including Cursor and Claude, while the other group operated without any AI assistance.

The tasks included a variety of scenarios sourced from open-source software to mirror actual working conditions. The time taken for each task was precisely measured, and developers were asked to subjectively assess their own performance.

At the conclusion of the study, three main insights emerged:

**Slowing down instead of speeding up**: The average time taken to complete tasks increased by 19% when AI was used.

**Self-deception**: Participants using AI overestimated their efficiency, believing they were working 20% faster.

**Code quality**: Although there was a slowdown in task completion, the quality of the code did not significantly decline nor improve.

The use of AI can be deceptive, creating a false sense of productivity while actual time is spent refining prompts and verifying results.

**Prompt iterations**: Developers invested considerable time fine-tuning their prompts to elicit useful responses from the AI.

**Output verification**: AI frequently generated erroneous code, necessitating further debugging efforts.

**Overestimating the tools**: Experienced developers, accustomed to working independently, often struggled to effectively integrate AI into their workflow.

The METR report emphasizes that these findings pertain to the current tools available (as of the study in 2025) and may evolve with advancements in technology.

This is not the first instance where AI has yielded mixed results. For example, in code analysis or generation tasks, AI can sometimes expedite the work of beginners while hindering experts. METR stresses the importance of empirical testing: «We recommend reading the complete report or the announcement thread rather than just summaries.»

For those using AI in their work:

**Test in practice**: Avoid relying solely on subjective feelings—measure the time taken.

**Optimize prompts**: Develop templates to facilitate quicker interactions with AI.

**Combine with traditional methods**: AI is useful for brainstorming and prototyping, but not for final implementation.

The team at METR concludes: AI is a tool, not a cure-all. In the future, as models improve, the situation may change, but for now, these technologies should be used with caution, without illusions of hyper-productivity.

A detailed report is available on the [METR blog](https://metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study/), and the original thread can be found on the X platform (formerly Twitter — *an organization designated as extremist in Russia*).