Written by: Mark Hull, Co-Founder and CEO, Exceeds AI | Last updated: July 16, 2026
Key Takeaways for AI vs. Human Defect Tracking
- Defect density at the commit level directly connects AI tool usage to measurable code quality outcomes, unlike aggregate metadata dashboards.
- Nearly a quarter of AI-introduced issues can survive to the latest repository version, so survival tracking matters as much as initial defect counts.
- Successful tracking depends on four prerequisites: repo access, pre-AI baseline data, sprint alignment, and agreed quality targets under 2 bugs per 1,000 lines.
- The six-step process of baseline establishment, commit-level attribution, sprint segmentation, survival tracking, mode and tool correlation, and coaching turns raw data into concrete quality improvements.
- Teams ready to run commit-level AI vs. human defect density tracking can start a free pilot with Exceeds AI and see initial results within hours.
Before You Begin: Four Prerequisites for Reliable Metrics
A reliable defect-density process starts with four prerequisites in place before you collect the first metric.
- Repo access. Read-only access to the repositories where AI tools are active is required because attribution of defects to AI vs. human origin is impossible without code-diff visibility. Metadata alone cannot distinguish which lines a coding assistant produced.
- Baseline defect data. Capture at least four to six weeks of pre-AI or pre-expansion defect history, including escaped defects per release, static analysis findings, and incident counts tied to specific commits. A 3–6 month historical baseline before AI rollout is the recommended standard for before-and-after comparison.
- Sprint or release cadence alignment. Defect density trends require a consistent measurement window. Align tracking to your existing sprint or release boundaries so cohorts remain comparable across time.
- Stakeholder alignment on quality goals. Agree in advance on what “acceptable” looks like. A recommended post-merge defect density target for enterprise teams using AI coding assistants is fewer than 2 bugs per 1,000 lines of code, with explicit comparison of AI-assisted versus human-only pull requests.
Expect first directional insights within hours of connecting your repo. Statistically meaningful longitudinal trends that stand up in an executive review usually require four to eight weeks of post-baseline data collection.
Step-by-Step Tutorial: From Baseline to Coaching
- Establish your defect density baseline by code origin.
Pull historical commit and PR data for the period before your current AI tool rollout or expansion. Calculate defect density separately for AI-assisted and human-authored changes by dividing confirmed defects from static analysis, QA, or incident reports by the volume of code in each cohort, normalized to per-1,000 lines. Tag commits by origin using available signals such as commit message conventions, PR labels, or IDE telemetry. This baseline becomes the control group every future sprint will be measured against, and its accuracy depends on how reliably you can identify AI-generated code.
Pro Tip: Heuristic-based AI detection, such as scanning for patterns or watermarks, tops out around 20–25% accuracy. For a trustworthy baseline, use a provenance layer that captures AI authorship at the machine level rather than inferring it after the fact.
Inputs needed: Issue tracker exports, static analysis reports, commit history, PR metadata.
Completion signal: A documented defect-per-KLOC figure for AI-assisted and human-authored code cohorts covering at least one full release cycle.Reliable defect density tracking depends on knowing, with high confidence, which lines in each commit were produced by which AI tool and in which interaction mode. Deploy a provenance layer such as Exceeds Ink that captures AI authorship on the developer’s machine at commit finalization and writes a line-level attestation alongside every commit. This approach closes the attribution gap that makes aggregate defect rates misleading.
Exceeds AI Impact Report with PR and commit-level insights Connect my repo and start my free pilot to instrument commit-level AI attribution across Cursor, Claude Code, Codex, GitHub Copilot, and Windsurf in a single setup.
Common Mistake: Relying on a single tool’s telemetry creates a coverage gap because most teams use multiple tools. When attribution covers only one vendor, the majority of AI-generated code remains unclassified.
Inputs needed: Per-repo opt-in configuration, AI tool adapters for each tool in use.
Completion signal: Every new commit carries a structured attribution record identifying AI-generated lines by tool, model, and interaction mode.At the close of each sprint or release, calculate defect density for AI-generated and human-authored code separately. Use static analysis run before and after each attributed commit to isolate introduced issues. Cohort-based separation of defect density by AI versus human code origin is the critical method for attributing longitudinal quality changes specifically to AI tool adoption instead of conflating them in aggregate metrics.
Track the Defect Density Delta by dividing AI defects per KLOC by human defects per KLOC. A target ratio below 1.2× the human baseline is healthy, while a ratio above 2× signals significant quality degradation that requires intervention.
Watchout: This ratio can be misleading when your codebase grows quickly. A 2026 causal study of 151 Java repositories found that after agentic AI adoption, lines of code grew 12.8% while architectural smell counts remained flat, which produced an apparent density improvement with no actual defect reduction. Always publish the raw numerator, the absolute defect count, alongside the density ratio to confirm whether a density drop reflects genuine quality improvement or simply codebase growth.
Inputs needed: Static analysis output per commit, attributed line counts by origin.
Completion signal: A per-sprint dashboard showing defect density for AI and human cohorts with trend direction.
Exceeds AI Impact Report shows AI code contributions, productivity lift, and AI code quality Post-merge defect density captures issues at introduction, while longitudinal survival tracking reveals whether those issues are resolved or persist into production. For each attributed commit, monitor how many introduced issues remain unresolved at 30, 60, and 90 days. The empirical baseline from the Debt Behind the AI Boom study shows that nearly a quarter of AI-introduced issues survive to the latest repository version, which provides a starting threshold for identifying a persistent defect problem.
Survival rates above this baseline in AI-generated code cohorts indicate accumulating technical debt that will surface as incidents or rework costs. Teams without automated quality gates on AI code often see growth in hotfix PRs after adoption.
Inputs needed: Issue tracker linkage to originating commits, incident reports tied to code origin.
Completion signal: A survival curve per AI tool showing the percentage of introduced defects still open at each time window.Not all AI usage carries equal defect risk, so you need to see which patterns cause trouble. Agent-mode sessions without a plan phase, large uniform changesets from headless workflows, and copy-paste-heavy outputs each carry distinct defect profiles. Segment defect density by interaction mode, including plan, ask, agent, edit, and headless, and by tool to identify which usage patterns drive quality degradation and which are safe to scale.
Research on coding agents shows that agent-assisted repositories can experience increases in static analysis warnings and cognitive complexity. These signals surface clearly when defect density is segmented by mode instead of aggregated across all AI usage.
Inputs needed: Interaction-mode classification per session from the provenance layer.
Completion signal: A per-tool, per-mode defect density breakdown updated each sprint.Defect density data becomes operationally useful only when it drives behavior change. Map high-defect patterns to specific coaching interventions. Teams with elevated agent-mode defect density need structured plan phases before execution. Teams with high security finding rates need targeted review checklists for OWASP-mapped categories.
Deliver coaching directly into the tools engineers already use instead of relying on a separate dashboard review cycle. When one team’s AI usage pattern produces consistently lower defect density, treat that pattern as a candidate for org-wide distribution as a versioned skill because scaling what works is faster than remediating what does not.
Inputs needed: Defect density segmentation from steps 3–5, coaching surface tooling.
Completion signal: Each high-defect pattern has a named coaching action assigned to a team or individual, with a follow-on sprint to measure impact.Validation and Success Criteria for Your Tracking Process
A defect-density tracking process is working, meaning it can reliably inform decisions about AI tool governance, when three observable conditions hold across consecutive sprints.
Consistent classification accuracy. AI-generated and human-authored code cohorts are correctly separated with high confidence. Lines that cannot be confidently attributed are recorded as unknown instead of silently assigned to either cohort. Classification accuracy is verifiable by spot-checking attributed commits against known AI tool sessions.
Reproducible trends across sprints. Defect density figures for each cohort are stable enough to distinguish signal from noise. A single sprint’s data is directional, while three or more consecutive sprints with consistent trend direction constitute a reliable signal. Key dates such as “Copilot enabled for Team X” must be annotated on dashboards to correlate metric changes with AI adoption events and enable clean before-and-after analysis.
Exceeds AI Repo Leaderboard shows top contributing engineers with trends for AI lift and quality Stakeholder agreement on quality signals. Engineering managers, QA leads, and executive sponsors interpret the same defect density data consistently. The Defect Density Delta is the primary executive-facing signal. A ratio below 1.2× confirms AI tools are not degrading quality relative to human-authored code, and a ratio trending toward or above 2× triggers a governance review using the healthy threshold established in Step 3.
When all three conditions hold, the process is ready to support board-level AI ROI reporting.
Connect my repo and start my free pilot to validate AI vs. human defect density with commit-level attribution across your entire AI toolchain.
Advanced Considerations and Next Steps for Mature Teams
Teams with a functioning defect-density baseline can extend the process in several practical directions.
Scaling across repositories. Apply the same attribution and measurement process to additional repositories, prioritizing those with the highest AI adoption rates or the greatest production incident history. Cross-repo defect density comparison reveals whether quality patterns are team-specific or tool-specific.
Refining interaction-mode attribution. As interaction-mode data accumulates, build mode-specific defect density benchmarks. Teams can then set mode-specific review thresholds, such as requiring additional senior review on commits where agent mode produced more than a defined percentage of the diff in security-sensitive paths.
Linking defect trends to token-spend governance. Defect density per tool, paired with token cost per tool, produces an Agentic ROI signal that shows which tools deliver the lowest defect density per dollar of token spend. This framing connects quality outcomes directly to the budget questions executives are asking in 2026.
Integrating with policy engines. Structured commit-level attestation is a natural input to policy engines and internal developer platform scorecards. Teams can express policies such as blocking deploys when AI authorship exceeds a threshold in sensitive paths, enforced automatically instead of through manual review.
FAQ
What is defect density and how is it calculated for AI-generated code?
Defect density is the number of confirmed defects, including bugs, security findings, or style violations, divided by the volume of code in which they appear, normalized to per-1,000 lines, or KLOC. For AI-generated code, the calculation requires two additional steps: attributing each defect to its originating commit and classifying that commit as AI-generated or human-authored. The resulting figure, AI defects per KLOC versus human defects per KLOC, is the Defect Density Delta, the primary signal for determining whether AI tools are improving or degrading code quality. A ratio below 1.2× the human baseline is healthy, while a ratio above 2× warrants intervention.
How long does it take to see meaningful defect density trends after AI tool adoption?
First directional data appears within hours of connecting a repository and instrumenting commit-level attribution. Statistically meaningful longitudinal trends that are stable enough to present to executives require three to four consecutive sprints of post-baseline data, typically four to eight weeks. Defect survival tracking at 30, 60, and 90 days requires a longer window by definition. Teams that establish a pre-AI baseline before expanding tool access can compress this timeline by using historical data as the control cohort instead of waiting for a parallel human-only period.
How does defect density tracking differ from metadata-only approaches?
Metadata-only tools track PR cycle time, commit volume, and review latency. They cannot identify which specific lines in a pull request were AI-generated, cannot attribute defects to AI versus human origin, and cannot track whether AI-touched code causes incidents 30, 60, or 90 days after merge. Defect density tracking at the commit level requires read-only repo access and a provenance layer that classifies code origin with high confidence. The operational difference is the difference between knowing a PR merged in four hours and knowing that 623 of its 847 lines were AI-generated, those lines had a 2× higher defect rate than the human-authored lines in the same PR, and three of those defects are still open 60 days later.
What should engineering teams do when AI defect density is higher than the human baseline?
A Defect Density Delta above 1.2× is a signal to investigate, not a signal to immediately restrict AI tool access. The first step is segmenting the elevated defect density by tool and interaction mode to identify whether the problem is concentrated in a specific usage pattern, such as agent-mode sessions without plan phases, large uniform changesets, or specific file types. Once the pattern is identified, the response is targeted through structured coaching for the teams or individuals driving the elevated rate, mode-specific review thresholds for high-risk paths, and a follow-on sprint to measure whether the intervention moved the metric. Banning tools outright drives usage underground without quality controls and removes the measurement visibility needed to manage risk.
How does Exceeds AI handle multi-tool environments where engineers switch between Cursor, Claude Code, and GitHub Copilot?
Exceeds AI is built specifically for multi-tool environments. Exceeds Ink uses per-tool checkpoint materializers for Claude Code, Cursor, and Codex, with adapters for GitHub Copilot and Windsurf, covering lighter-weight detection across up to approximately 50 AI tools. Each commit receives a line-level attestation identifying the specific tool, model, session, and interaction mode that produced each attributed line. This design means defect density can be segmented by tool, revealing, for example, whether Cursor agent-mode sessions in a specific repository carry a higher defect rate than Claude Code ask-mode sessions, without requiring engineers to change their workflows or manually tag their commits.
Conclusion: Turning AI Adoption into a Defensible Quality Signal
Defect density code quality tracking provides the operational process that converts AI adoption data into a provable quality signal. The formula is straightforward. Establish a baseline by code origin, instrument commit-level AI attribution, compute defect density per sprint for AI and human cohorts, track defect survival at 30, 60, and 90 days, segment by tool and interaction mode, and surface coaching actions from the patterns that emerge.
Recent research shows why this process matters. Faros.ai analysis of the DORA 2025 dataset found incidents per pull request rose 242.7% over a two-year window while bugs per developer increased 54%. A 2026 analysis of 8.1 million pull requests across 4,800 engineering teams found technical debt increases 30–41% in the year following AI tool adoption. These outcomes do not appear in metadata dashboards. They appear in commit-level defect density trends tracked over time.
Engineering leaders who implement this process can answer the board question about whether AI investment is improving or degrading code quality with specific numbers, reproducible methodology, and longitudinal evidence. Leaders who rely on adoption stats and acceptance rates cannot provide the same level of confidence.
Connect my repo and start my free pilot and get AI vs. human defect density at the commit level, across every AI tool your team uses, with first insights in hours.