Written by: Mark Hull, Co-Founder and CEO, Exceeds AI
Key Takeaways for Engineering Leaders
- Span AI code detector relies on ML-based chunk classification that degrades sharply under distribution shift and cannot verify accuracy in real-world engineering environments.
- ML detectors cannot distinguish which AI tool produced code, capture interaction modes, or provide longitudinal outcome tracking for governance.
- Exceeds Ink delivers deterministic, client-level provenance by writing structured Git Notes with line-level attribution for tool, model, session, and interaction mode at commit time.
- Only Exceeds Ink enables board-ready ROI proof, AI technical debt tracking, and in-agent coaching that ML detectors fundamentally cannot support.
- Engineering leaders seeking reliable AI governance should connect their repo and start a free pilot with Exceeds AI to replace inference with verifiable evidence.
What Span AI Code Detector Promises
Span presents its detector as a developer-facing layer that identifies AI-generated code within repositories. Its publicly announced feature set includes:
- High-accuracy chunk-level classification of AI-generated versus human-written code segments
- Broad programming language support across common enterprise stacks
- Tool-agnostic tracing that claims to detect output regardless of which AI coding assistant produced it
- An actionable dashboard surfacing AI usage patterns across the codebase
- Governance and IP safety framing, positioning detection as a compliance aid
Each of these claims deserves scrutiny before an engineering leader commits budget or governance policy to them. The core issue sits in how ML-based detection works.
Why ML Detection Breaks in Real Engineering Environments
ML-based detectors operate on statistical patterns learned during training. When the code distribution shifts to a new language, a new model family, or a new domain, accuracy degrades sharply. AICD Bench evaluation of leading detectors found that performance drops substantially under distribution shifts to unseen domains or languages. That drop is not a minor accuracy penalty. It approaches random classification on the code distributions most likely to appear in a growing, multi-tool engineering organization.
Boilerplate code compounds this problem because detectors can exhibit failure patterns on human-written code that falls outside their training distribution. When engineering teams maintain codebases with scaffolding, generated ORM layers, or templated configuration files, these false-positive rates on human-written boilerplate become a practical governance liability, not an edge case.
ML detectors also cannot distinguish interaction modes. A developer who spends forty minutes in a deliberate back-and-forth with Claude Code, refining architecture and reviewing each suggestion, creates a fundamentally different risk profile than one who issues a single “build this for me” prompt. No chunk-level classifier captures that distinction. Microsoft’s ICSE 2008 study found organizational-complexity metrics, including management span, to be among the strongest predictors of defect-proneness. As manager-to-IC ratios stretch toward 1:8 or higher, bandwidth for code review shrinks at the exact moment when the need for reliable provenance grows. A detector that cannot tell a thoughtful session from a one-shot prompt gives managers no actionable signal.
ML detectors also lack longitudinal outcome tracking. Knowing that a chunk was probably AI-generated at merge time does not reveal whether that code caused incidents thirty or sixty days later, accumulated rework, or degraded test coverage over time. Without per-commit attestation anchored to a stable identifier, teams have no reliable substrate for that analysis.
Span AI Code Detector Accuracy in Context
Span claims approximately 95% accuracy for its chunk-level classifier. Independent parties cannot verify that figure, and the research record on ML-based AI code detection does not support confidence at that level under real-world distribution shift. By Exceeds’ own assessment, heuristic and watermark-based AI detection tops out around 20–25% accuracy across the range of tools and domains a mid-market engineering organization actually uses. The AICD Bench findings mentioned earlier apply directly here. Span’s claimed 95% accuracy cannot be independently validated under the distribution shifts that mid-market engineering organizations actually face.
The gap between a vendor’s in-distribution benchmark and production accuracy creates governance risk. A policy built on a 95% in-lab claim that degrades to 20–25% on the actual codebase does not function as governance. It functions as a liability.
Span’s Tool Coverage vs Real Multi-Tool Stacks
Span’s tool-agnostic claim rests on a single detection signal applied after code is committed. That architecture has a structural ceiling. It cannot distinguish which tool produced a given block, cannot capture the model version or session context, and cannot classify interaction mode. Many companies now use two or more AI coding tools in parallel, and engineers routinely switch between Cursor for feature work, Claude Code for large refactors, Codex for batch transforms, and GitHub Copilot for autocomplete within a single sprint. A single-signal detector that cannot differentiate those tools cannot support tool-by-tool outcome comparison, per-tool cost attribution, or model-level governance.
Exceeds Ink ships five first-class adapters for Claude Code, Cursor, Codex, GitHub Copilot, and Windsurf. Dedicated per-tool checkpoint materializers resolve edit evidence against the actual working tree at commit finalization. Lighter-weight detection extends across approximately 50 AI tools. Teams gain aggregate visibility across the entire toolchain, not a single-signal approximation.
Span vs Exceeds Ink: Client-Level Provenance Compared
The architectural difference between Span’s approach and Exceeds Ink represents a categorical distinction between inference and observation.
Span’s chunk-level ML approach:
- Classifies code after it has been committed, inferring probable AI origin from statistical patterns
- Cannot identify which specific tool, model, or session produced a given block
- Cannot capture the interaction-mode classification discussed earlier
- Produces no portable, auditable record that lives in the repository
- Degrades under distribution shift, new model families, and unseen languages
- Provides no substrate for longitudinal outcome tracking or AI technical debt analysis
Exceeds Ink’s client-level provenance approach:
- Observes what AI tools actually do on the developer’s machine at commit time, with no inference required
- Writes a structured Git Note at refs/notes/exceeds-ink with line-level attribution for tool, model, session, turn, interaction mode, and timestamp for every line
- Captures interaction-mode classification (plan, ask, agent, edit, headless), a signal no competitor publishes
- Produces a portable, machine-readable JSON attestation that lives in the repository and travels across forks and mirrors
- Records lines that cannot be confidently attributed as unknown_lines rather than silently assigning them to human or AI
- Fires deterministically from standard Git hooks at commit finalization, so the note is written before the commit is reachable for push
This distinction matters for governance. When a board, legal counsel, or auditor asks what percentage of the codebase was AI-produced, a classifier’s probabilistic estimate and a Git Notes attestation do not represent equivalent answers. One answer rests on inference. The other rests on evidence.
Capabilities Exceeds Ink Unlocks Beyond ML Detectors
Because Exceeds Ink writes a per-commit, per-tool, per-line attestation, the Exceeds AI platform can support capabilities that have no equivalent in any ML detector.
- Board-ready ROI proof: Commit and PR-level attribution connects AI tool usage directly to productivity and quality outcomes, including cycle time, rework rates, and incident rates thirty or more days post-merge. Leaders can answer “Is our AI investment paying off?” with hard numbers, not sentiment. For example, a mid-market customer with 300 engineers discovered that GitHub Copilot contributed to 58% of all commits and correlated with an 18% lift in overall team productivity within the first hour of deployment.
- AI technical debt tracking: Longitudinal outcome monitoring on Ink-attested code surfaces whether AI-touched lines accumulate rework, trigger incidents, or degrade test coverage over time. Teams can see the hidden risk that passes review today and fails in production ninety days later.
- ink-prompting-coach in-agent coaching: A SKILL.md and slash command installs directly into the developer’s own Claude Code or Cursor agent, delivering coaching where the work happens. No ML detector closes the loop back into the developer’s workflow.
- Policy and governance: Because the attestation is structured JSON in the repository, it becomes a natural input to policy engines. Teams can block deploys when AI authorship exceeds a threshold in sensitive paths and require additional review on commits where agent mode produced more than a defined percentage of the diff.
See Exceeds Ink’s deterministic provenance in your own codebase
Frequently Asked Questions
How Exceeds Ink Handles Access and Security
Exceeds Ink functions as an on-machine capture layer, not a server-side scanner. It installs as a lightweight Rust binary on the developer’s machine and fires from standard Git hooks at commit time. For the Exceeds AI platform, repository access is scoped read-only and code exists on Exceeds servers for seconds before permanent deletion. Only commit metadata and snippet information persists. The platform supports SSO/SAML, data residency options, HMAC-SHA256-signed remote ingest with revocable per-machine tokens, and an in-SCM deployment option for organizations with the highest security requirements. Exceeds has passed formal enterprise security reviews, including a Fortune 500 retailer’s two-month evaluation process.
How Exceeds Ink Treats False Positives and Unknown Lines
Exceeds Ink uses per-tool checkpoint materializers that resolve edit evidence against the actual working tree at commit finalization. Multi-edit Cursor sessions correctly retain human-typed lines, and Claude Code rewrites are attributed to Claude rather than overwriting human contributions. Lines that cannot be confidently attributed are recorded as unknown_lines in the Git Note. They are not silently rolled into either the human or AI bucket. This conservative approach keeps the attestation trustworthy for governance purposes because it does not overstate AI authorship to produce impressive-looking dashboards.
Setup Time and Time to First Insight
GitHub or GitLab OAuth authorization takes about five minutes. Repository scoping takes another fifteen minutes. First insights are available within sixty minutes of authorization. Complete historical analysis across twelve months of commit history completes within four hours. Real-time updates appear within five minutes of new commits. Exceeds Ink hooks and adapters for Claude Code, Cursor, and GitHub Copilot can be installed and wired up the same day. This compares to Jellyfish’s commonly reported nine-month average time to ROI and LinearB’s multi-week onboarding process.
How Exceeds AI Works with Existing Analytics Platforms
Exceeds AI is designed to complement existing metadata-based analytics tools, not replace them. Those tools track PR cycle times, commit volumes, and deployment frequency, which remain useful signals. Exceeds adds the AI-specific intelligence layer those tools cannot provide. Teams see which lines are AI-generated, by which tool, in which interaction mode, and what the long-term outcomes of that code are. Most customers run Exceeds alongside their existing stack. Exceeds integrates with GitHub, GitLab, Azure DevOps, JIRA, Linear, and Slack.
Interaction Modes Exceeds Ink Classifies and Why They Matter
Exceeds Ink classifies the five interaction modes mentioned earlier: plan, ask, agent, edit, and headless. This classification matters because a developer in plan mode, iterating deliberately with the AI before generating code, produces a different risk and quality profile than one in headless agent mode issuing a single batch prompt. No ML detector captures this distinction because it requires observing what happened on the developer’s machine, not inferring it from the committed output. Interaction-mode data forms the foundation for Exceeds’ coaching surfaces. When a team’s rework rate rises, Ink’s mode classification can reveal whether the pattern correlates with agent-mode sessions that skipped a plan phase, which represents a coachable and fixable behavior rather than a vague quality concern.
Conclusion: Replace Inference with Evidence
Span AI code detector features represent the current ceiling of ML-based, post-hoc classification. The approach delivers chunk-level inference that degrades under distribution shift, cannot distinguish tools or interaction modes, and produces no portable attestation for governance or longitudinal analysis. The research record on ML-based AI code detection confirms that current detection tasks remain far from solved, especially for hybrid human-LLM co-authored and adversarial code. Engineering leaders who build governance policy on unverifiable accuracy claims accept a liability rather than manage one.
Exceeds Ink takes a categorically different approach. It observes what AI tools actually do on the developer’s machine, writes a deterministic, line-level, tool-aware attestation as a Git Note alongside every commit, and uses that attestation as the substrate for ROI proof, technical debt tracking, and in-agent coaching. The attestation lives in the repository, travels across forks and mirrors, and is readable by any Git client. It does not depend on Exceeds’ platform to remain useful.
Engineering leaders at mid-market software companies must answer the board’s AI ROI question with evidence rather than estimates and often manage teams using Claude Code, Cursor, Codex, GitHub Copilot, and Windsurf simultaneously. Exceeds Ink is the only provenance layer built for that reality.