Launching today
cngx
Catch AI agents that claim tests passed when they didn't
8 followers
Catch AI agents that claim tests passed when they didn't
8 followers
AI coding agents end tasks with the same line: fixed it, tests pass, ready to merge. Often they never ran anything. cngx is an open source CLI that reads that claim, runs your real checks (pytest, npm test, go test, etc.), and blocks the merge when the story and output do not match. The verdict comes from command output, not regex. `pipx install cngx && cngx quickstart` gets you started in seconds. Built for anyone tired of merge-ready agent output that never touched the test suite.
this is a real problem, i've seen the "tests pass, ready to merge" line be flat out false more than once. one thing i'm curious about: what happens when the agent actually runs the tests and they genuinely pass, but the agent wrote the test itself and made it trivially green (asserts true, mocks away the actual behavior, etc.) to game the check? does cngx look at anything beyond exit code, like whether the test file changed alongside the source change, or is that intentionally out of scope for v1?
Finally a tool that actually catches the "tests pass, ready to merge" lie. I tried it on a few of my recent PRs and it caught two agents claiming green when the suite was actually red. Easy setup too.