Written by: Mark Hull, Co-Founder and CEO, Exceeds AI | Last updated: July 2, 2026
Key Takeaways
- Metadata-only dashboards from tools like Jellyfish, LinearB, and Swarmia cannot distinguish AI-generated lines from human-authored code, so leaders lack causal proof of AI ROI.
- Vanity metrics such as lines of code and acceptance rates reward volume over quality and fail to surface long-term defects or technical debt introduced by AI coding assistants.
- Meaningful AI impact measurement requires commit-level provenance that links AI-generated lines to downstream outcomes like defect density, rework rates, and incident frequency at 30-, 60-, and 90-day marks.
- Among the seven platforms reviewed, only Exceeds AI combines multi-tool capture, portable Git Notes attribution, and longitudinal outcome tracking to deliver board-ready proof of AI value.
- Stop guessing whether your AI investment is paying off. Book a demo with Exceeds AI to see code-level attribution and outcome analytics in action.
Why Metadata-Only Tools Fall Short
Eighty-four percent of developers are now using or planning to use AI tools, and 51% of professional developers use them daily. Boards expect engineering leaders to prove that this investment delivers real returns. Most available tools, however, were built before AI coding assistants existed and cannot answer that question.
Platforms like Jellyfish, LinearB, and Swarmia track metadata such as PR cycle times, commit volumes, and review latency. These tools cannot see which lines in a pull request came from a human and which came from Cursor or Claude Code. Without that distinction, you lose the causal chain between AI usage and business outcomes. A 20% drop in cycle time might reflect AI productivity gains, or it might reflect a quieter sprint. Metadata alone cannot tell you which scenario occurred.
DX adds developer experience surveys and an AI Code Insights module, but survey data is subjective and its capture mechanism is a closed-source daemon that stores all attribution in DX Data Cloud. Nothing lives in your own repository. GitHub Copilot’s built-in analytics report acceptance rates and lines suggested, yet remain blind to every other AI tool your engineers use and cannot connect those suggestions to defect rates or incident outcomes.
Metrics to Avoid
Lines of code is the most dangerous vanity metric in the AI era. AI coding assistants can generate hundreds of lines in seconds. A team that measures AI impact by volume of code produced will systematically reward AI-inflated output while missing the quality signal entirely. Tools like Waydev that treat all code the same are especially vulnerable. In those models, more lines equal higher “impact,” regardless of whether those lines introduce bugs, require rework, or pass review only to fail in production.
Acceptance rate, the percentage of AI suggestions a developer accepts, is similarly misleading. A developer who accepts every Copilot suggestion without review scores perfectly on acceptance rate while potentially accumulating architectural debt. Commit volume, story points closed, and deployment frequency share the same flaw. They measure activity, not outcome, so they cannot show whether AI improves quality or stability.
Core Metrics That Actually Show AI Impact
Effective AI measurement connects AI-generated lines to downstream outcomes. That connection requires repository access and a provenance layer that identifies AI-generated lines at commit time. With that foundation in place, you can track metrics that reveal real impact instead of surface activity.
AI-touched code defect density compared to human-authored code provides the baseline. This comparison shows whether AI introduces more bugs per line or holds steady against human output. Beyond initial defects, the rework rate on AI-generated lines, measured as follow-on edits within 30 days, exposes whether AI code requires more maintenance.
Incident rates on AI-touched modules at 30, 60, and 90 days post-merge show whether AI-generated code creates production stability risks over time. Test coverage on AI-generated versus human-generated diffs highlights whether teams test AI code as rigorously as human-written code. Cycle time broken down by AI contribution percentage per pull request reveals whether AI accelerates or complicates the development process.
A Microsoft study presented at ICSE 2008 found organizational-complexity metrics, including team size and management span, to be among the strongest predictors of defect-proneness. As manager-to-IC ratios stretch from the typical 1:5 toward 1:8 or higher, bandwidth for code review and mentorship shrinks. AI coding assistants accelerate output into that gap. Without longitudinal outcome tracking, technical debt accumulates invisibly.
Tool-by-Tool Comparison of AI Measurement Approaches
This comparison focuses on how each platform measures AI impact, where it falls short, and which use cases it supports. Use these criteria to decide which tools answer your specific questions about AI performance, risk, and ROI.
1. GitHub Copilot Analytics
Overview: Built-in dashboard included with GitHub Copilot enterprise licenses.
Strengths: Zero additional setup, with organization-level reporting on acceptance rates, active users, and lines suggested.
Limitations: Completely blind to Cursor, Claude Code, Codex, and Windsurf. Cannot connect suggestions to quality outcomes, defect rates, or incident data. Lacks longitudinal tracking.
Best-Fit Scenarios: Teams that use only GitHub Copilot and need a basic adoption headcount.
AI-Specific Capabilities: Acceptance rate and suggestion volume only, with no code-level outcome attribution.
2. Jellyfish
Overview: Engineering resource allocation and financial reporting platform targeting CFOs and CTOs.
Strengths: Strong Jira and Git metadata aggregation, useful for budget allocation and capacity planning.
Limitations: Metadata-only and treats all code contributions equally regardless of authorship. Setup commonly takes nine months before showing ROI. No AI-specific outcome tracking.
Best-Fit Scenarios: Organizations that need engineering spend reporting for finance, not AI ROI proof for engineering leadership.
AI-Specific Capabilities: None at the code level.
3. LinearB
Overview: Engineering workflow automation platform focused on SDLC process metrics.
Strengths: Solid cycle time and deployment frequency tracking, plus workflow automation rules.
Limitations: No visibility into which lines came from AI versus human developers. Users have reported surveillance concerns and significant onboarding friction. No multi-tool AI attribution.
Best-Fit Scenarios: Teams improving traditional SDLC workflows without AI-specific measurement needs.
AI-Specific Capabilities: None at the code level.
4. Swarmia
Overview: Developer productivity platform built around DORA metrics and Slack-based team nudges.
Strengths: Fast setup, clean DORA reporting, and low friction for smaller teams.
Limitations: Built for the pre-AI era with limited AI-specific context. No code-level attribution, no multi-tool support, and no longitudinal outcome tracking.
Best-Fit Scenarios: Teams that need DORA baselines and have not yet adopted AI coding tools at scale.
AI-Specific Capabilities: Minimal.
5. DX (AI Code Insights)
Overview: Developer experience platform, now an Atlassian company, with an AI Code Insights module and Agent Experience self-grading.
Strengths: Broad engineering intelligence coverage, strong compliance posture (SOC 2, ISO 27001), and Atlassian distribution through Jira and Bitbucket.
Limitations: AI attribution lives in DX Data Cloud only, with nothing portable in your own repository. Capture relies on a closed-source, always-on CLI daemon. The weakest capture tier falls back to filesystem-change heuristics. Survey data remains subjective. DX is SaaS-only with no self-host option and an enterprise sales gate with a median ARR commitment that makes piloting expensive.
Best-Fit Scenarios: Organizations designing strategic AI transformation programs that prioritize developer sentiment data alongside Atlassian tooling.
AI-Specific Capabilities: AI Code Insights and Agent Experience modules with heuristic-based capture and no portable provenance.
6. Git AI
Overview: Open-source AI provenance tool that writes Git Notes attribution alongside commits, and the closest architectural peer to Exceeds Ink.
Strengths: Apache 2.0 open-source core, Git Notes at refs/notes/ai, and fast local install for individual developers.
Limitations: The deployment model creates real operational and security friction. It uses a long-lived per-user daemon, a PATH-shimmed git binary, and on Windows a git.exe copy of git-ai.exe that breaks EDR and AppLocker checks. It also applies a destructive global git config mutation that clobbers existing Trace2 tooling. Attribution is asynchronous, so a fast git push can race ahead of the daemon’s reconciliation. There is no in-agent behavior-change layer. Teams and Enterprise tiers are sales-led.
Best-Fit Scenarios: Individual developers or small teams comfortable with OSS tooling who want basic Git Notes provenance without an analytics platform.
AI-Specific Capabilities: Line-level Git Notes attribution and recently added generic prompt tips with no skill-transfer or rollback capability.
7. Exceeds AI
Overview: AI-impact analytics platform built for the multi-tool AI coding era, powered by Exceeds Ink. Exceeds Ink is a lightweight, hook-direct provenance layer that writes a portable, line-level attestation alongside every commit as a Git Note at refs/notes/exceeds-ink.
Strengths: Exceeds AI is the only platform that combines client-level capture, multi-tool support across Cursor, Claude Code, Codex, GitHub Copilot, and Windsurf, and longitudinal 30+ day outcome tracking. It writes portable Git Notes provenance that lives in your own repository and delivers prescriptive coaching into the developer’s own AI agent through ink-prompting-coach. There is no long-lived daemon, no PATH-shimmed git binary, and no global git config mutation. First insights arrive within 60 minutes of setup. Outcome-based pricing avoids any per-contributor data tax.
Limitations: Exceeds AI requires scoped read-only repository access, which is the necessary condition for any code-level claim. Best fit starts at 50 engineers, and very large enterprises with more than 5,000 engineers remain a future roadmap priority.
Best-Fit Scenarios: VPs of Engineering, Directors, and engineering managers at 50–1,000-engineer software companies who need to prove AI ROI to the board, manage multi-tool AI adoption, and track long-term code quality outcomes.
AI-Specific Capabilities: Line-level AI versus human attribution per tool and interaction mode, AI versus Non-AI Outcome Analytics, longitudinal technical debt tracking, Best Practices Insights, Coaching Surfaces, skill transfer and rollback, interaction-mode classification (plan, ask, agent, edit, headless), and token cost and model attribution per session.
Multi-Tool Reality and AI Technical Debt Tracking
Modern engineering teams rarely standardize on a single AI coding tool. Engineers reach for Cursor for feature development, Claude Code for large-scale refactoring, Codex for batch transforms and headless workflows, and GitHub Copilot for inline autocomplete, often within the same sprint. This tool diversity creates a measurement blind spot because any platform built around a single vendor’s telemetry can only see a fraction of your team’s AI usage.
Most AI-generated code then flows into production before its risks become visible. AI tools can generate code that looks clean, passes review, and ships successfully, yet surfaces subtle bugs, architectural misalignments, or maintainability issues weeks or months later. Metadata-only tools cannot detect this pattern because they only see merge status and cycle time, not what happens to that code afterward. Longitudinal outcome tracking anchored to per-commit AI attribution is the only reliable way to manage AI technical debt before it becomes a production crisis.
Exceeds AI tracks AI-touched code over time, monitoring incident rates, rework patterns, and test coverage degradation on Ink-attested commits. The platform surfaces these signals early so leaders can intervene before problems compound.
Setup Speed and Pricing Reality Check
Setup time and pricing model now matter as much as feature lists. Jellyfish commonly takes nine months before showing ROI. LinearB users report weeks to months of onboarding friction. DX is enterprise sales-led with a consulting-heavy integration process.
Exceeds AI delivers first insights within 60 minutes of GitHub OAuth authorization, complete historical analysis within four hours, and real-time updates within five minutes of new commits. The Pro plan costs $49 per manager per month at Early Partner Pricing, with no per-contributor data tax. You pay for manager seats and the insights you use, not for every engineer analyzed. Exceeds Ink is available as an add-on, and Ink can also run standalone, piping provenance data directly into your own data warehouse and BI tools.
How to Choose the Right Solution
Your choice should match the question you need to answer. If you need to know how to allocate engineering budget across teams, Jellyfish is purpose-built for that. If you want to see whether your DORA metrics are improving, LinearB or Swarmia will surface that data. If you need to know whether your AI investment is paying off, which tools drive the best outcomes, and whether AI-generated code creates hidden risk, none of those platforms can answer that question.
Several criteria separate genuine AI impact measurement from basic metadata reporting. First, confirm whether the tool captures AI authorship at the line level or infers it from patterns and heuristics. Heuristic and watermark-based detection typically reach only 20–25% accuracy. Next, check whether the platform supports every AI tool your team uses or only one vendor’s telemetry. You should also verify where provenance lives, either in your own repository or in a proprietary cloud, and whether the system tracks outcomes beyond the merge date. Finally, ask whether the tool gives managers concrete actions or just another dashboard.
DORA and SPACE frameworks provide useful baselines but fall short without code-level AI signals. A team can hit elite DORA benchmarks while accumulating AI technical debt that surfaces in the next quarter. The measurement layer needs to go deeper than deployment frequency and change failure rate.
Getting Started with Exceeds AI
The gap between knowing AI adoption is happening and proving that it delivers value has become a board-level problem for most engineering organizations. Tools that existed before AI coding assistants became ubiquitous were not designed to close that gap. Metadata dashboards, vendor analytics, and developer surveys all leave the core question unanswered: which lines are AI-generated, by which tool, in which mode, and what those lines did to your codebase over time.
Exceeds AI connects repo-level observability down to specific commits and pull requests touched by AI across every tool your team uses. It pairs that proof with prescriptive guidance so managers can scale what works. Setup is measured in hours, not quarters, and as mentioned earlier, first insights appear within an hour of authorization. Board-ready ROI reports follow in weeks, not months.
Frequently Asked Questions
What is the difference between heuristic AI detection and client-level capture?
Heuristic detection infers AI authorship after the fact by looking for patterns such as a large block of code written in a short window, formatting conventions associated with a particular model, or watermarks that some tools embed in their output. This approach is fallible by design. It cannot tell you whether an engineer had a thoughtful back-and-forth with an AI agent or simply accepted a bulk suggestion, and it cannot reliably distinguish AI-generated lines from fast human typing.
Client-level capture observes what actually happens on the engineer’s machine at the moment the work is done. It records which tool was active, what mode the session used, how many iterations occurred, and which lines the engineer modified after the AI generated them. Exceeds Ink uses client-level capture through per-tool checkpoint materializers that resolve edit evidence against the actual working tree at commit finalization. Lines that cannot be confidently attributed are recorded as unknown rather than silently assigned to either category.
The practical difference resembles the gap between a trustworthy audit record and an educated guess. Client-level capture produces evidence you can defend in front of a board, regulator, or customer.
Why does AI technical debt tracking require more than 30 days of outcome data?
AI coding assistants are tuned to produce code that passes review and tests at the time of submission. The failure modes that matter most, such as subtle architectural misalignments, edge-case bugs, and maintainability issues that compound over time, typically do not surface in the initial review cycle. A pull request that merges cleanly on day one may generate three follow-on bug fixes by day 45 and contribute to an incident by day 90.
Metadata-only tools that measure cycle time and merge status capture none of this behavior. Longitudinal outcome tracking anchored to per-commit AI attribution is the only way to answer whether AI-generated code performs differently from human-authored code over time. It also reveals whether specific tools, interaction modes, or team patterns correlate with higher or lower downstream risk.
Exceeds AI tracks AI-touched code at 30, 60, and 90 days post-merge, monitoring incident rates, rework patterns, and test coverage changes on Ink-attested commits. These views turn AI technical debt from a vague concern into a measurable, manageable risk.
Can Exceeds AI work alongside existing developer analytics platforms like LinearB or Jellyfish?
Exceeds AI is designed to complement, not replace, traditional developer analytics platforms. Most customers deploy Exceeds as the AI intelligence layer that sits on top of their existing stack. LinearB and Jellyfish track metadata well, including cycle time, deployment frequency, review latency, and engineering resource allocation.
What those tools cannot do is distinguish AI-generated from human-authored lines, connect AI usage to quality outcomes, or track technical debt introduced by AI tools over time. Exceeds integrates with GitHub, GitLab, Azure DevOps, Jira, and Linear, and surfaces AI-specific signals that extend what your existing tools already provide. Most customers run Exceeds alongside their current platform and use it specifically to answer the AI questions those platforms cannot.
How does Exceeds AI handle teams using five or more different AI coding tools simultaneously?
Exceeds Ink is tool-agnostic by design. It ships five first-class adapters with deep per-tool fidelity for Claude Code, Cursor, Codex, GitHub Copilot, and Windsurf, plus lighter-weight detection across up to approximately 50 AI tools. Each adapter uses native per-tool hooks and per-tool checkpoint materializers that resolve edit evidence against the actual working tree at commit finalization.
This approach means a multi-edit Cursor session correctly retains human-typed lines, and a Claude Code rewrite is attributed to Claude rather than to the engineer. The platform then surfaces aggregate AI impact across all tools and enables tool-by-tool outcome comparison. You can see whether Cursor or Copilot drives better results for a given team, which tools correlate with higher rework rates, and where adoption is concentrated versus sparse.
This cross-tool visibility answers the question your CFO actually asks. The real concern is not which tool engineers prefer, but whether the aggregate AI investment is paying off.
What does Exceeds AI’s provenance format mean for governance and audit requirements?
Exceeds Ink writes a structured attestation as a Git Note at refs/notes/exceeds-ink. The attestation is line-level, so every line carries its tool, model, session, interaction mode, and timestamp. It is portable across forks and mirrors and stored as machine-readable JSON that anyone with repository access can resolve.
Because the attestation lives in your own repository rather than in a proprietary cloud, it survives outside the Exceeds platform entirely. This design matters for governance in several concrete ways. Boards and legal counsel can ask what percentage of the codebase was produced with AI and receive an auditable answer rather than an estimate. When AI-assisted code causes an incident, the exact session, prompt, developer, and tool are traceable. Patent examiners can see precisely what role AI played in a given file.
The Git Notes format also enables policy enforcement. You can block deploys when AI authorship exceeds a threshold in sensitive paths or require additional review on commits where agent mode produced more than a defined percentage of the diff. These controls work because the attestation is structured data in the repository, not just a number on a dashboard.