Why I made CodeGate fail-closed on security

Hey — building CodeGate here, launching August 11. Wanted to share a bit of the "why" behind one specific decision, since it's the one I went back and forth on the most.

Why fail-closed on security, specifically.

Most review tools (AI or static analysis) implicitly treat "no issues found" as "you're clean." But that's only true if the tool actually managed to analyze everything. If it hit a file it couldn't parse, a pattern it wasn't confident about, or context it didn't have — silence gets read as a pass.

That's the failure mode I cared most about avoiding. So the security agent is explicitly fail-closed: if it can't analyze something with confidence, it says so, instead of quietly rolling it into "all clean." A false negative disguised as a pass is worse than an honest "couldn't verify this part."

The trade-off is that it's less satisfying from a product-polish standpoint — "0 issues found ✅" is a cleaner UI moment than "3 issues found, 1 section not fully analyzable." But I'd rather ship the honest version.

The same instinct shaped the confidence-band approach across all four agents (correctness, security, architecture, performance): high-confidence findings get flagged as blocking, low-confidence ones get demoted to optional nitpicks instead of being silently dropped or shouted as loud as everything else. Cost of a full 4-agent run is about $0.01, which is what makes a genuinely useful free tier possible instead of a stripped-down demo version.

Curious if others building review/analysis tools have run into the same tension between "clean-looking output" and "honest-but-messier output.