Lineage Lens
Track which lines of your code came from AI
Start new thread

The hardest question in AI code governance: how confident are you that code was actually AI-written?

by•

Something that keeps coming up when I talk to teams about AI code governance: everyone focuses on capturing records, but almost nobody asks how confident they are in those records.

There are two very different things you can have.
Record A: a file-watcher noticed 47 lines appeared in and Cursor was probably running.
Record B:a proxy intercepted the Anthropic API call, matched it to the editor insertion via request UUID, measured 1.4 seconds between the API response and thecode appearing, and computed 0.81 trigram similarity between the model output and what landed in the file.

Both produce a row in your audit database. The second is dramatically more defensible — but most governance tooling treats them identically.

In LineageLens, every record gets a confidence score from 0.0 to 1.0, broken into five independent evidence signals. Easy Mode captures (VS Code extension, no proxy) score around 0.27 — honest about what you know. Power Mode captures (proxy running, full request interception) score up to 1.0. The score is not about whether the record is useful. It is about how much you can defend it when someone asks.

I built this after realizing that "we have records" and "we have auditable records" are not the same claim. Curious how others are thinking about this: does capture quality differentiation matter for what your team needs, or is any record good enough right now?And what would you add as a confidence signal?

40 views

Add a comment

Replies

Best

Capture quality definitely matters if the record is ever going to be used outside the dev team.

One signal I’d add is edit distance after insertion, but bucketed by intent: exact paste, light cleanup, substantial rewrite, or AI-drafted then human-restructured. Raw similarity alone can underrate real AI influence when someone keeps the logic but rewrites the surface.

I’d also separate “evidence confidence” from “policy implication.” A low-confidence record can still be useful as a prompt for review, but it probably shouldn’t be treated like audit proof.

 I really like the distinction between “evidence confidence” and “policy implication.” A lot of systems implicitly treat those as the same thing, but they’re solving different questions.

The edit-distance-by-intent idea is interesting too because provenance gets much harder once humans substantially reshape AI-generated logic instead of simply pasting it directly. At that point similarity alone can underrepresent how much of the conceptual structure still originated from the model.

I also agree that low-confidence records can still be operationally useful — just not interchangeable with high-confidence audit evidence.

Yes — and the review layer is where this gets really actionable.

A useful report probably needs three separate states: what the system can prove, what the human reviewed, and what policy decision followed. Blending those together makes the audit trail feel cleaner than it really is.

I also like the idea of keeping weak signals visible instead of hiding them. A 0.27-confidence record can still say “look here,” as long as the UI prevents it from pretending to be proof.

What would be a real use-case of having this knowledge?

 A few practical ones come to mind immediately:

  • Enterprise audit/compliance reviews where teams need to explain whether sensitive code or documents were AI-generated and under what conditions.

  • Security incident response when organizations need to trace which prompts, tools, or agents influenced a production change.

  • PR review workflows where reviewers want provenance confidence instead of a simple “AI was used” label.

  • Vendor and contractor environments where organizations need evidence continuity after people leave or access is revoked.

  • Regulated industries where the question is not only “was AI involved?” but “can this history still be independently verified months later?”

I think the broader shift is that AI-generated artifacts are increasingly becoming operational infrastructure, so provenance stops being analytics and starts becoming accountability evidence.

Drop the questions below!!