Following up on yesterday's post about fail-closed security today I want to zoom out to the broader thing I think about a lot: what actually makes someone trust an AI reviewer's output, versus just tolerating it until they mute the bot.

I think the trust problem with AI code review isn't accuracy in the abstract. It's variance in what gets flagged. If a tool posts 15 comments on a PR and 12 are trivial style nits dressed up as "issues," the 3 real findings drown with them. The team's response isn't "let's read carefully" it's "let's turn this off."

So the thing that seems to actually build trust over time is consistency in what counts as worth interrupting someone for. Not "did the model find something," but "was this specific thing worth a human's attention right now." That's a different, harder question it means a tool has to be willing to say less, not more, to stay useful.

A few things I think help (built these into CodeGate, but curious if others have found different levers):

Hey building CodeGate here, launching August 11. Wanted to share a bit of the "why" behind one specific decision, since it's the one I went back and forth on the most.

Why fail-closed on security, specifically.

Most review tools (AI or static analysis) implicitly treat "no issues found" as "you're clean." But that's only true if the tool actually managed to analyze everything. If it hit a file it couldn't parse, a pattern it wasn't confident about, or context it didn't have silence gets read as a pass.

That's the failure mode I cared most about avoiding. So the security agent is explicitly fail-closed: if it can't analyze something with confidence, it says so, instead of quietly rolling it into "all clean." A false negative disguised as a pass is worse than an honest "couldn't verify this part."