Praveen

Show PH: I built a VS Code extension that scores AI code risk

Something I built led to a design decision I want to get feedback on.

LineageLens is a free VS Code extension that captures every AI code insertion and scores it for risk on a 0–100 scale. Works with Cursor, Copilot, ClaudeCode, Gemini CLI. Zero config on install — just start using your AI tools and your insertions start showing up in the sidebar.

The scoring is deterministic rules: +28 for credential-like material, +24 for eval/exec patterns, +22 for subprocess calls, +14 for landing in an auth or payments file, and so on. Fully traceable. No ML, no black box.

The design decision that surprised me: missing prompt capture — when the extension records a file insertion but has no record of what was asked — adds +24 to the risk score. Same weight as detecting an eval() call.

The reasoning was that a code record without a prompt is less auditable. You cannot reconstruct developer intent. You cannot detect whether the model exceeded its scope. You cannot spot prompt injection in an agentic session. All of those failure modes require the prompt to be visible.The practical consequence: if you run only the extension without the proxy backend, your risk scores are 24 points higher for the same code than they would be with full prompt capture. The architecture exposes that gap through the score rather than hiding it. I think this is the right call, but it is not obvious — it conflates "we are missing data" with "the code is risky," and those are not the same thing.

Worth discussing: is penalizing incomplete capture a reasonable design for a risk score? Or does it make the number misleading?

37 views

Add a comment

Replies

Best
Praveen

Drop your comments below!!