The trust problem with AI code review

Following up on yesterday's post about fail-closed security — today I want to zoom out to the broader thing I think about a lot: what actually makes someone trust an AI reviewer's output, versus just tolerating it until they mute the bot.

I think the trust problem with AI code review isn't accuracy in the abstract. It's variance in what gets flagged. If a tool posts 15 comments on a PR and 12 are trivial style nits dressed up as "issues," the 3 real findings drown with them. The team's response isn't "let's read carefully" — it's "let's turn this off."

So the thing that seems to actually build trust over time is consistency in what counts as worth interrupting someone for. Not "did the model find something," but "was this specific thing worth a human's attention right now." That's a different, harder question — it means a tool has to be willing to say less, not more, to stay useful.

A few things I think help (built these into CodeGate, but curious if others have found different levers):

• Separating confidence from severity. A finding can be severe but uncertain, or trivial but certain. Only high-confidence + meaningfully-severe should ever interrupt someone as blocking.

• Being honest about what wasn't checked, instead of implying full coverage.

• Explaining the "why," not just the "what." A comment that says why something matters gets read differently than a bare rule violation.

None of this is solved — it's closer to "the thing I keep iterating on" than "the thing I've figured out."

Genuinely curious: for anyone here who's used AI code review tools (or built them) — what made you stop trusting one, or start?