Harvard researchers found that AI coding agents increase code generation by 30%, but firms do not see a corresponding rise in software output or employment. The study, conducted across hundreds of software firms, highlights the inefficiencies introduced by AI-generated code.
The study analyzed 300 million work events across over 700,000 employees at 700+ firms from 2021 to 2026. Researchers used aggregated analytics data from Jellyfish to measure the impact of AI coding tools on software development processes.
The introduction of AI coding agents led to a 30% rise in code lines, 20% more commits, and 23% more pull requests. However, the resolution rate for Jira-tracked issues remained unchanged, showing no improvement in software output.
"The code review process significantly increases in length, pull requests are more likely to require revisions, and reviewers leave more comments," the researchers wrote. These findings suggest that the efficiency gains from AI coding are offset by increased review time and effort.
The study also found that AI agents were responsible for only 23.3% of all review comments and 10.8% of pull requests, indicating that humans still handle most code reviews. The researchers noted that AI's impact on the review process has been marginal so far.
While many firms have adopted AI coding agents, the study suggests that the trade-offs between coding time and review time remain a challenge. The researchers concluded that the time and expense of implementing AI coding agents may not be worth it for most companies yet.
Source: arstechnica
