Written by: Mark Hull, Co-Founder and CEO, Exceeds AI
Key Takeaways
- AI coding tools speed up individual tasks by 15–30% but increase bug density 1.7× and raise incidents per PR by 23.5% across the industry.
- Most engineering analytics platforms track throughput yet cannot flag which specific lines in a commit came from AI versus a human.
- AI-generated code shows higher complexity, thinner documentation, and 2.74× more security issues than comparable human-written code.
- Tracking AI-touched code for 30–90 days exposes hidden technical debt that appears weeks after merge and drives extra rework.
- Exceeds AI gives leaders code-level visibility to measure real ROI, not just adoption rates, so see how it works for your team.
Why Leaders Need More Than Adoption Stats
Ninety percent of surveyed developers now use an AI coding assistant regularly, and that adoption has pushed AI-authored code to 26.9% of all production code as of early 2026. This scale has drawn the attention of boards and CFOs, who now focus on whether AI investment produces measurable value. Their core concern has shifted from adoption to outcomes.
Platforms like Jellyfish, LinearB, and Swarmia were designed before AI-generated code existed at scale. They report PR cycle times, reviewer load, and commit volumes, yet this metadata cannot distinguish a Cursor-generated function from a hand-crafted one. This blind spot leaves leaders without answers to basic quality questions, such as whether AI-generated code is more defect-prone or more expensive to maintain. Ninety-four percent of engineering leaders say the metrics that matter most are missing from their current measurement frameworks. Adoption dashboards confirm that engineers opened a tool, but they do not show whether that tool improved or degraded the codebase.
Repo-access measurement fills this gap. Commit and PR-level analysis can map which lines are AI-generated, track those lines over 30, 60, and 90 days, and compare outcomes against human-written equivalents across every tool in the stack.
How This Report Weighs the Evidence
The studies cited in this report use different methods that each reveal part of the story. Controlled experiments, such as Anthropic's 2026 randomized trial with 52 Python developers, isolate causal effects but rely on small samples and narrow tasks. Large-scale field studies, such as the MIT IDE analysis of 187,000 GitHub developers, capture real-world behavior but cannot fully control for factors like developer seniority or codebase complexity.
Longitudinal surveys, including Vella and Blincoe's 2026 study of 158 professional engineers, track self-reported experience over time yet rely on perception instead of code-level measurement. Readers should treat any single-study figure as directional and weigh findings based on method, sample size, and context.
Key Findings on Speed, Quality, and Debt
The research points to a consistent pattern. AI coding tools deliver measurable speed gains, while quality and maintainability often suffer in ways that only appear with code-level tracking over time. Specifically:
- Median PR throughput rose modestly as AI coding tool usage increased across engineering organizations, far below the 3× or 10× gains executives sometimes expect.
- Productivity gains from AI coding assistants vary across studies, typically ranging from 21% to 55% in task completion speed or time-to-PR, based on developer data collected through early 2026.
- AI-generated code produces approximately 1.7× more issues per pull request than human-written code. The 1.7× issue rate breaks down into specific error categories, with logic and correctness errors most common, followed by edge-case handling failures and integration issues.
- Rework rates for AI-generated code often run higher within the first month after merge compared to human-written code, especially in complex services.
- AI assistance can reduce post-task mastery scores in controlled experiments, with the largest gaps on debugging questions where deep understanding matters.
- Research finds that AI yields strong productivity gains on simple greenfield tasks but smaller gains on complex legacy code, and in some cases slows work on highly entangled systems.
These findings raise an immediate question about measurement. If AI-generated code introduces more issues and rework, leaders need specific quality signals that go beyond surface-level throughput metrics.
Code-Level Signals That Reveal AI Quality Gaps
Code quality analytics for AI-generated code must rely on signals that extend beyond linting scores or test pass rates. Maintainability analysis for AI-generated code uses cyclomatic complexity, code duplication percentage, comment density, and naming convention adherence. These metrics show that AI-generated code often carries higher complexity and weaker documentation than human-written equivalents.
These quality gaps appear in concrete ways. Higher cyclomatic complexity and lower comment density make AI-touched modules harder to modify or extend, which compounds maintenance costs over time. Security risk also increases. Security issues occur at approximately 2.74× higher rates in AI-generated code than in human-authored code, with timing-attack flaws and input validation problems appearing frequently. Platforms that analyze only metadata never see these patterns because they do not inspect the diff.
Connecting AI Spend to Measurable ROI
Meaningful ROI analysis depends on a unified view of cost and outcomes. Teams need repository data on PR volume, cycle time, and AI commit detection by developer and team. They also need provider cost and usage data from multiple vendor APIs, plus adoption cohort data that segments power users, casual users, and non-adopters.
Without this combined view, leaders cannot link spending to quality and throughput. Organizations with structured measurement programs consistently capture more value from AI tools than those that rely on vendor dashboards alone. The stakes are high. A mid-sized engineering organization with 100 developers can spend a significant amount on AI coding tools before API costs, which requires more justification than acceptance-rate charts.
How AI Adoption Changes Technical Debt
CircleCI data showed feature branch velocity surging while main branch build success rates hit a five-year low during increased AI-assisted development. This pattern aligns with AI-generated code that passes feature-branch review yet erodes trunk stability over time. Code churn doubled after AI adoption, with AI-generated code showing 1.5–2× the human baseline churn rate, which signals architectural inconsistency and rising maintenance burden.
Longitudinal tracking shows that AI-generated code produces hidden technical debt that surfaces weeks after deployment, with production incidents tied to AI-touched code over 30-, 60-, and 90-day intervals. The DORA ROI report models an instability tax where change failure rates rise after AI adoption, creating negative downtime impact for large organizations. This debt pattern is not inevitable. Teams that can identify AI-touched code and monitor it over time can intervene before issues spread.
Comparing AI and Human Code at the PR Level
Direct comparisons between AI-generated and human-written code require commit-level attribution. Aggregate metrics that blend both sources hide the real differences. Where attribution exists, the gap becomes visible. Overall incidents per PR rose 23.5% industry-wide in 2026 as AI coding tools increased throughput. At the same time, trust in AI-generated code accuracy dropped to 29% in 2025, down from 40% in prior years.
These figures only appear on platforms that perform AI Usage Diff Mapping at the PR and commit level. Without that mapping, leaders see faster shipping but cannot see where the extra incidents originate.
Thirty-Day Windows That Expose Hidden Costs
The 30-day window after merge is where many hidden costs of AI-generated code emerge. Code that clears review on day zero can still contain timing-attack vulnerabilities, architectural misalignments, or maintainability issues that only appear under production load. Teams should track AI-touched code for 30 days or more to spot technical debt patterns before they become production incidents.
The proportion of engineers reporting worsened developer experience in at least one dimension nearly doubled from 14% to 27% over a six-month longitudinal study. This shift acts as a lagging signal that only appears with time-series tracking.
Seeing Across the Full AI Toolchain
Engineering teams in 2026 typically use several AI coding tools. Developers may choose Cursor for feature work, Claude Code for large refactors, and GitHub Copilot for inline autocomplete. Most organizations lack a consolidated cross-vendor view of cost per PR, tool-level ROI, idle license detection, and shadow AI usage from personal API keys or unsanctioned tools.
Single-vendor telemetry, which powers most built-in analytics dashboards, stops providing visibility as soon as an engineer switches tools. Multi-signal AI detection that uses code patterns, commit message analysis, and optional telemetry integration is the only architecture that covers the full toolchain.
Measuring GitHub Copilot's Impact on Quality
GitHub Copilot's built-in analytics report acceptance rates and lines suggested, yet they do not show whether accepted lines caused incidents, required rework, or introduced security issues. Developers with access to GitHub Copilot have shown changes in peer collaboration activities, which likely affects quality in ways acceptance-rate dashboards cannot capture.
Proving Copilot's impact on quality requires comparing the long-term outcomes of Copilot-touched PRs against human-only PRs at the commit level, within the same codebase and time window.
Industry-Scale AI Investment and Governance
Worldwide AI spending is expected to reach $2.59 trillion in 2026, a 47% increase over the prior year. At this scale, the absence of code-level measurement becomes a governance risk rather than a reporting inconvenience. MIT's NANDA initiative research found that 95% of enterprise generative AI pilots fail to deliver measurable P&L impact, with failures attributed to learning gaps for both tools and organizations instead of model quality.
The DORA team at Google Cloud concluded that the greatest returns on AI investment come from a strategic focus on the underlying organizational system. That system cannot improve without visibility into which code is AI-generated, how it performs over time, and which adoption patterns produce durable quality gains.
Why Averages Hide AI's Real Impact
Aggregate productivity and quality figures represent averages across very different scenarios, yet AI's impact is not uniform. Research shows that productivity gains from AI coding assistants vary by task complexity and codebase, with higher gains on simpler tasks than on complex legacy code. Niche programming languages and brownfield codebases can even see productivity declines on highly complex tasks.
Developer experience level adds another layer. Some studies report overall productivity gains around 26%, with less-experienced developers often benefiting more than seniors. Experienced developers may face added maintenance burdens that reduce net gains, partly because seniors spend more time reviewing each AI suggestion and apply extra scrutiny around security, scalability, and edge cases. Yet the GitClear 2026 analysis of over 2,000 developer-weeks found that the largest AI productivity gains were concentrated among senior and staff engineers.
This apparent contradiction likely reflects differences in how productivity is measured. Time-to-completion metrics favor juniors using AI on straightforward tasks, while quality-adjusted output favors seniors who catch AI errors before merge. Team-level averages smooth over these differences and hide the spread of outcomes.
Practical Evaluation Questions for Engineering Leaders
- Can your current tooling distinguish AI-generated lines from human-written lines at the commit level across Cursor, Claude Code, GitHub Copilot, and other tools at the same time?
- Do you track 30-day, 60-day, and 90-day incident rates segmented by AI-touched versus human-only PRs?
- Can you see which teams or individuals show high AI adoption with stable quality versus high adoption with elevated rework?
- Does your measurement approach cover the full multi-tool AI stack, or does it rely on telemetry from a single vendor?
- Can you present the board with a before-and-after comparison of code quality outcomes, not just adoption percentages, tied to specific AI tool investments?
If any answer is no, the gap sits in the measurement layer rather than in the AI tools themselves.
Neutral Summary
The research base on AI's impact on software development productivity and code quality continues to grow, yet methods and contexts differ widely. Productivity gains in controlled settings range from negligible to 40%, depending on task complexity, codebase maturity, developer experience, and measurement approach. Quality outcomes, including bug density, rework rates, and incident rates, consistently trend worse for AI-generated code when measured at the commit level over 30 or more days.
Aggregate organizational throughput gains appear real but modest, typically in the 8–15% range rather than the order-of-magnitude improvements sometimes cited. Organizations that capture the most value use structured measurement programs that connect AI usage to business outcomes at the code level, instead of focusing solely on adoption rates.
Frequently Asked Questions
What is AI Usage Diff Mapping and why does it matter for measuring code quality?
AI Usage Diff Mapping analyzes code diffs at the commit and PR level to identify which specific lines were generated by AI tools versus written by human developers. This mapping matters because aggregate metrics such as PR cycle time, commit volume, and deployment frequency blend AI and human contributions into a single number. Without diff-level attribution, a team cannot determine whether a rising incident rate comes from AI-generated code, human code, or a specific tool.
Diff mapping forms the foundation for any credible AI ROI calculation and for longitudinal tracking of AI technical debt.
Does AI-generated code reliably increase technical debt over time?
Under common adoption patterns, evidence points toward higher technical debt from AI-generated code. AI-generated code shows higher bug density, elevated rework rates within 30 days of merge, and higher production incident rates than human-written code when measured separately. This debt pattern is not deterministic. Teams with strong review practices, senior oversight, and longitudinal monitoring can reduce the risk.
The risk peaks when AI-generated code is accepted without distinguishing it from human code, because quality issues that pass initial review can compound over 60 and 90 days before surfacing as incidents. Tracking AI-touched code over time, rather than only at merge, allows teams to catch these patterns early.
How should engineering leaders measure AI ROI across multiple tools like Cursor, Claude Code, and GitHub Copilot?
Multi-tool ROI measurement requires a tool-agnostic detection layer that identifies AI-generated code regardless of which tool produced it. Relying on a single vendor's telemetry, such as GitHub Copilot Analytics, creates blind spots for every other tool in the stack. A complete measurement approach combines code pattern analysis, commit message signals, and optional telemetry integration to attribute AI contributions across the full toolchain.
From that attribution baseline, leaders can compare productivity outcomes such as cycle time and PR throughput, along with quality outcomes such as rework rate, incident rate, and test coverage, between AI-touched and human-only code. Segmenting these comparisons by tool, team, and repository produces the cross-vendor cost-per-PR and tool-level ROI data that a CFO or board can evaluate.
Why do productivity gains from AI coding tools appear larger in controlled experiments than in organizational data?
Controlled experiments usually measure a single developer completing a well-defined task with a new tool, which isolates the AI's contribution to that activity. Organizational data captures the full engineering system, including code review overhead, verification time spent on AI-generated suggestions, downstream testing adjustments, and the learning curve for new workflows.
Only about 14% of developer time goes to hands-on coding, which limits how much an AI tool that focuses on that slice can move aggregate throughput. Controlled experiments also tend to use greenfield tasks where AI performs best, while production codebases with legacy dependencies and complex business rules show smaller gains. The gap between experimental and organizational results reflects the difference between optimizing one activity and changing an entire engineering system.
What should engineering managers look for in a platform that tracks AI code quality over time?
Managers should require commit and PR-level AI attribution that works across all tools in the team's stack. They also need longitudinal outcome tracking that follows AI-touched code for at least 30 days after merge, and direct comparison of AI versus human code on quality metrics such as rework rate, incident rate, and test coverage.
Beyond raw measurement, managers benefit from platforms that translate data into prescriptive guidance. Useful platforms highlight which teams or individuals show high AI adoption with stable quality versus those where adoption correlates with elevated rework, and they surface specific coaching actions instead of leaving managers to interpret dashboards alone. Setup speed matters as well, because platforms that require months of integration create a measurement gap during the period when AI adoption accelerates fastest.