Written by: Mark Hull, Co-Founder and CEO, Exceeds AI | Last updated: August 6, 2026
Key Takeaways
- Engineering leaders lack defensible proof that AI tools improve productivity because current metrics ignore technical debt, validation time, and burnout.
- A seven-step measurement system grounded in code-level attribution replaces survey estimates with repeatable, auditable ROI data.
- Four-layer framework across Adoption, Delivery, Quality, and Business Outcomes requires commit-level provenance to connect AI-generated lines to long-term results.
- Immediate throughput gains must be validated against 30-day rework and incident rates to separate genuine productivity from deferred costs.
- Exceeds AI delivers this attribution layer through Exceeds Ink, turning every commit into executive-ready proof; start your free pilot today.
The Operational Pain: Proving AI Is Worth the Spend
Manager-to-IC ratios have stretched from a typical span of roughly 1:5 toward 1:8 or higher, and Microsoft's ICSE 2008 research established that wider management spans are among the strongest predictors of defect-proneness. Stretched managers cannot inspect every PR. They rely on dashboards that show PR cycle times, commit volumes, and acceptance rates, yet these views do not reveal whether AI caused any observed change or whether the code will hold up 30, 60, or 90 days later.
Faros's 2026 AI Engineering Report, analyzing telemetry from 22,000 developers across 4,000 teams, found that bugs per developer are up 54% under high AI adoption, the incident-to-PR ratio has more than tripled, and median PR review time is up 441%. Vendor dashboards that track acceptance rates and active users do not surface these downstream costs. Only 25% of AI initiatives have delivered the ROI executives expected, per IBM Institute for Business Value. A better measurement system closes this gap.
The Four-Layer Measurement Framework for AI Engineering
A complete AI productivity measurement system operates across four layers: Adoption, Delivery, Quality, and Business Outcomes. Adoption covers which tools engineers use and how often. Delivery tracks whether throughput is genuinely increasing. Quality measures whether AI-touched code holds up over time. Business Outcomes connect the investment to dollars and delivery milestones.
Metadata-only platforms that read PR cycle times and commit counts without analyzing code diffs can approximate Adoption and Delivery. They cannot address Quality or Business Outcomes because they lack the code-level attribution layer that connects a specific line of code to the tool, model, session, and interaction mode that produced it. That attribution layer makes the remaining steps in this system possible.
Connect my repo and start my free pilot to see all four layers in a single view within hours.
Step 1: Define Success Outcomes with Stakeholders
Align with engineering leadership, finance, and product on what “AI is working” means in measurable terms before connecting any repository. Throughput gains matter, and code durability matters just as much. McKinsey’s February 2026 study of 4,500 developers found 46% time savings on routine tasks but under 10% on complex work, based on outcome metrics rather than adoption alone.
Common Mistake: Defining success as lines of code written or commit count. AI tools inflate both without necessarily increasing meaningful output. Stanford research found that AI adoption led to a 14% rise in pull-request volume along with a roughly 2.6x increase in code rework.
To avoid these vanity metrics, agreed-upon success criteria should include at minimum PR throughput delta, 30-day rework rate on AI-touched code, and incident rate for AI-attributed commits versus human-authored commits.
Step 2: Grant Scoped Repo Access and Establish a Baseline
Code-level measurement requires read-only repository access. This prerequisite separates attribution from estimation. With repo access, the system can identify which 847 lines in a given PR were AI-generated, track whether those lines required follow-on edits, and compare incident rates across AI-touched and human-authored modules. Without this access, the system remains limited to the same metadata any dashboard can provide.
Watch-Out: False-positive attribution is a real risk with heuristic-based detection. Heuristic and watermark-based AI detection tops out around 20–25% accuracy. The only authoritative method is client-level capture, which observes what actually happens on the engineer's machine at the moment the work is done, then writes a signed attestation alongside the commit.
Baseline the prior 90 days of commit and PR history before any AI measurement begins. This baseline establishes the pre-AI throughput, rework rate, and incident density that all subsequent comparisons require.
Step 3: Install Commit-Time Capture with Portable Git Notes
The capture layer is where most measurement systems fail. Metadata platforms read what Git already records. A provenance layer records what Git does not: which tool wrote which lines, in which interaction mode, at what token cost, and in which session.
Exceeds Ink is the on-machine provenance layer that writes a structured attestation as a Git Note at refs/notes/exceeds-ink at commit finalization. Every line carries its tool, model, session, turn, interaction mode, and timestamp. The attestation is portable, so it travels with the repository across forks and mirrors, and it is auditable by anyone with repo access. Lines that cannot be confidently attributed are recorded as unknown_lines rather than silently assigned to either category.
Pro Tip: Interaction-mode classification, such as plan, ask, agent, edit, or headless mode, is a signal no metadata platform publishes. It forms the foundation for actionable coaching. Agent-mode commits without a plan phase create a coachable pattern that correlates with higher rework rates.
Setup requires only a lightweight per-machine install and standard Git hooks. First insights are available within 60 minutes. Complete historical analysis typically finishes within four hours.
Connect my repo and start my free pilot and have Ink writing attestations across your active repositories today.
Step 4: Map AI vs. Human Lines Across Every Tool in Use
84% of developers are using or planning to use AI tools, and 51% of professional developers use them daily. In practice, engineers on the same team may use Cursor for feature work, Claude Code for large refactors, Codex for batch transforms, and GitHub Copilot for autocomplete within the same sprint. A measurement system that tracks only one vendor's telemetry produces an incomplete picture.
Exceeds AI maps AI versus human lines across all tools in use through five first-class adapters for Claude Code, Cursor, Codex, GitHub Copilot, and Windsurf, with lighter-weight detection across up to approximately 50 AI tools. The AI Adoption Map surfaces adoption rates by team, individual, repository, and tool, enabling cross-tool outcome comparison. The system can answer which tool drives better results on a given codebase with commit-level fidelity. This granular tracking matters because AI code share per repository acts as a leading indicator of quality risk.
Watch-Out: One 2026 survey found AI accounts for 42% of committed code, while analyses indicate that exceeding roughly 40% AI-generated code correlates with higher bug rates, review effort, and code churn. Tracking AI code share per repository is a leading indicator worth monitoring continuously.
Step 5: Calculate Immediate and 30-Day Outcome Deltas
Immediate outcome deltas such as cycle time, review iterations, and first-pass merge rate become visible within the first sprint. These metrics are necessary and still incomplete on their own. A large-scale empirical study analyzing 302,579 verified AI-authored commits across 6,299 GitHub repositories found that more than 15% of commits from every major AI tool introduced at least one detectable issue, and 22.7% of those issues survived at the repository HEAD, including issues introduced more than nine months earlier.
A 2026 analysis of 8.1 million pull requests across 4,800 engineering teams found that technical debt increases 30–41% in the year following AI tool adoption. Measuring only immediate outcomes misses this accumulation entirely. The 30-day outcome delta, which includes incident rate, rework rate, and follow-on edit frequency for AI-attributed commits, separates genuine productivity gain from deferred cost.
Exceeds AI tracks these longitudinal outcomes anchored to Ink's per-commit attestation, providing an early warning system for AI technical debt before it reaches production.
Step 6: Surface Replicable Best-Practice Patterns
Zapier's chief AI transformation officer describes the goal as identifying “golden patterns we want to multiply across peers” versus “anti-patterns we want to coach our way out of.” The same logic applies at the engineering team level.
When one team's AI-touched PRs show three times lower rework rates than another team's, the difference rarely comes from tool selection. Interaction mode, prompt discipline, and review practice usually explain the gap. Exceeds AI's Best Practices Insights feature uses a LangGraph-backed analysis pipeline to distill the team's actual AI-coding patterns into the top skills worth scaling, sorted by confidence. The Coaching Surfaces feature and the ink-prompting-coach skill, which installs directly into the developer's own Claude Code or Cursor agent, distribute those patterns where the work happens rather than in a separate dashboard.
Pro Tip: Developers using both generative AI tools and separate review agents within a task realized an additional 1.5x to 2.5x time improvement over single-tool usage. Identifying which engineers have already developed multi-tool discipline and scaling that pattern creates a high-leverage coaching action.
Step 7: Build and Share the Executive Dashboard
Executive-ready ROI proof requires three elements: a pre-AI baseline, a post-AI outcome delta with AI attribution, and a quality signal that confirms throughput gains are not offset by downstream costs. Audit-ready metrics include spend per engineer per month, throughput change against a pre-AI baseline, rework rate on AI-influenced PRs, revert rate within 30 days, and the percentage of AI-suggested output that survives review.
Exceeds AI generates this dashboard from the commit-level data Ink has been collecting since Step 3. The provenance is machine-readable, auditable, and not dependent on vendor-supplied acceptance rate figures. When the CFO asks whether the AI investment is paying off, the answer is a specific number tied to specific commits, not a survey result or an adoption percentage.
Validation: Observable Success Indicators
The system is working when the following conditions are observable:
- Consistent AI versus human outcome deltas across multiple sprint cycles, showing that AI-attributed commits have measurably different and improving rework and incident rates compared to the pre-AI baseline
- Stakeholder sign-off on the executive dashboard, meaning finance and engineering leadership agree on the methodology and accept the numbers as defensible
- Declining unknown attribution, as Ink's per-tool adapters mature across the team's toolchain and the share of lines recorded as
unknown_linesdecreases, increasing confidence in all reported figures - Coaching uptake visible in interaction-mode data, with agent-mode-without-plan sessions declining after ink-prompting-coach distribution
Advanced Considerations: Scaling the System and Governing Token Spend
98% of organizations track AI infrastructure costs in 2026 per Mavvrik, while a separate 2026 Flexera report found only 31% have accurate visibility into AI software spend. As reasoning models and agentic workflows draw 5 to 20 times the tokens of simple completions, the question shifts from “are we using AI?” to “what are we getting per token?”
Exceeds Ink captures the underlying model behind each session, including Claude/Opus, GPT, Gemini, and others, and reports cost and token usage per agent and model. Cursor billing is read from Cursor's own state database for exact accuracy. Paired with the platform, that spend is correlated with shipped output such as lines attributed, commits produced, and session-to-merge velocity. The result is an Agentic ROI signal that finance and engineering can act on together.
At scale, Ink's structured JSON attestation in the repository becomes the input for policy engines. Organizations can express governance rules, such as requiring additional review on commits where a specific agent produced more than a defined percentage of the diff in sensitive paths, because the attestation is structured, portable, and lives in the repository rather than in a vendor's cloud.
The strongest teams frequently achieve significantly higher net ROI on the same tooling stack compared to the weakest teams. Scaling the system means identifying what the strongest teams are doing differently and distributing those patterns through Skill Transfer, using versioned skills that can be deployed org-wide and rolled back cleanly if they do not land.
Frequently Asked Questions
How is Exceeds AI different from metadata-only platforms like Jellyfish, LinearB, or Swarmia?
Metadata platforms read what Git already records, such as PR cycle times, commit volumes, review latency, and deployment frequency. They cannot tell you which lines in a given PR were AI-generated, whether those lines required more follow-on edits than human-authored lines, or whether AI-touched code caused incidents 30 days after merge. Exceeds AI analyzes code diffs at the PR and commit level to distinguish AI versus human contributions, anchored by Exceeds Ink's line-level attestation written at commit time. That code-level truth makes AI ROI proof defensible rather than estimated. Metadata platforms remain useful for traditional delivery metrics, and Exceeds AI serves as the AI intelligence layer that sits alongside them.
Does the system work if our team uses multiple AI coding tools?
The system is designed specifically for multi-tool environments. Exceeds Ink provides five first-class adapters with deep per-tool fidelity for Claude Code, Cursor, Codex, GitHub Copilot, and Windsurf, plus lighter-weight detection across up to approximately 50 AI tools. Each adapter uses per-tool checkpoint materializers that resolve edit evidence against the actual working tree at commit finalization, so a Cursor multi-edit session correctly retains human-typed lines and a Claude Code rewrite is attributed to Claude. The AI Adoption Map surfaces adoption rates and outcome comparisons across all tools in use, giving leaders an aggregate view rather than a single-vendor slice.
What are the security and privacy implications of granting repo access?
Exceeds AI is built around a read-only access model. For cloud customers, repositories exist on servers for seconds and are then permanently deleted, while only commit metadata and snippet information persists. Code is fetched via API only when needed and is never cloned after onboarding. Exceeds Ink itself never modifies commit messages, never installs a PATH-shimmed git binary, and never mutates global git configuration. The capture layer uses HMAC-SHA256-signed remote ingest with revocable per-machine tokens, LLM-based prompt redaction before any prompt content is persisted, and an aggregate-only mode that keeps transcripts off the wire entirely. Privacy is configurable along four rungs, from local-only where nothing leaves the machine to full identified replay, and different teams in the same organization can operate at different rungs. Exceeds AI has passed enterprise security reviews including a formal two-month evaluation process at a Fortune 500 retailer.
How long does it take to get from repo connection to actionable executive data?
GitHub or GitLab OAuth authorization takes approximately five minutes. Repository scoping takes roughly fifteen minutes. First insights are available within 60 minutes of setup. Complete historical analysis, covering up to 12 months of prior commit history, completes within four hours. Real-time updates appear within five minutes of new commits. Executive-ready ROI reports, which require enough post-AI commit history to establish a statistically meaningful outcome delta, are typically available within weeks. This compares to setup timelines of two to four weeks for LinearB and commonly nine months to first ROI for Jellyfish.
Can Exceeds AI track technical debt introduced by AI tools over time, not just immediate quality signals?
Longitudinal outcome tracking is a core capability of the platform, not an add-on. Because Exceeds Ink writes a per-commit attestation at the time each commit is finalized, every AI-attributed line can be tracked forward in time. The system records whether the line required follow-on edits, appeared in incident post-mortems, accumulated static-analysis warnings, or was rewritten or reverted within 30, 60, or 90 days. This approach provides the only reliable way to distinguish AI code that looks clean at review from AI code that holds up in production. Metadata-only tools cannot make this distinction because they have no record of which lines were AI-generated in the first place.
Conclusion
The seven-step system, which defines outcomes, baselines with scoped repo access, installs commit-time provenance capture, maps AI versus human lines across every tool, calculates immediate and 30-day outcome deltas, surfaces replicable patterns, and builds the executive dashboard, replaces survey-driven estimates and metadata dashboards with code-level proof. Each step depends on the one before it, and all of them depend on a provenance layer that records what actually happened at commit time rather than inferring it afterward.
Exceeds AI supplies that provenance layer through Exceeds Ink, using portable, auditable, line-level attestations written alongside every commit across Cursor, Claude Code, Codex, GitHub Copilot, Windsurf, and the rest of the tools your team uses. The platform turns those attestations into the AI versus human outcome analytics, coaching surfaces, and the ROI reports leadership needs to answer the question executives are already asking.
Connect my repo and start my free pilot and have first insights from your own commit history within the hour.