Viktor.comAn AI coworker that actually does the work
AI coding agents end tasks with the same line: fixed it, tests pass, ready to merge. Often they never ran anything. cngx is an open source CLI that reads that claim, runs your real checks (pytest, npm test, go test, etc.), and blocks the merge when the story and output do not match. The verdict comes from command output, not regex. `pipx install cngx && cngx quickstart` gets you started in seconds. Built for anyone tired of merge-ready agent output that never touched the test suite.
this is a real problem, i've seen the "tests pass, ready to merge" line be flat out false more than once. one thing i'm curious about: what happens when the agent actually runs the tests and they genuinely pass, but the agent wrote the test itself and made it trivially green (asserts true, mocks away the actual behavior, etc.) to game the check? does cngx look at anything beyond exit code, like whether the test file changed alongside the source change, or is that intentionally out of scope for v1?
Hey @galdayan! Thanks for asking this.
cngx binds the agent's claim to a real run of your command. If pytest exits 0 and the agent said tests passed, that's VERIFIED. We don't inspect test quality or whether the agent wrote the tests.
That's by design. v1 targets the bigger daily lie: "tests pass" when nothing ran. Trivial green tests show up in the diff anyway, and cngx is meant to sit on a suite you already trust, not replace review.
that's a fair line to draw honestly, catching the "nothing ran at all" case is already most of the problem. would you ever add an optional stricter mode later (e.g. flag when the test file changed in the same commit as the source with no real assertions touching the new logic) or is that intentionally staying out of scope to keep the tool simple and fast
finally something that takes the agents at their word. ran it against a messy PR and it caught two "fixed it" claims that did not actually pass pytest, saved me from a bad merge.
Hey @hogultekin47153 ! This is exactly why we built it. Messy PRs are where agents sound most confident and verify least. Happy it saved you from a bad merge :)
How does it handle cases where the test suite itself is flaky or slow, like a long-running integration test that sometimes times out — does cngx retry, fail open, or just block the merge?
finally something that catches agents bullshitting about test results. ran it on a recent pr and it flagged a "passing" agent commit in seconds because pytest actually had two failures. genuinely useful.
Ran it against a few of my old agent PRs and it caught two that confidently claimed tests passed when half the suite had actually errored out. Love that it just shells out and reads real stdout instead of trusting the agent's word.
Finally someone forcing AI agents to actually run the tests instead of just claiming they passed. The command-output verdict idea is smart and the pipx install was painless.
Thank you @sunakphh ! That’s exactly the gap we built for where agents often just sound merge-ready without evidence they ran tests. Glad install was easy :)