cngx

Catch AI agents that claim tests passed when they didn't

5.0•2 reviews•

21 followers

Catch AI agents that claim tests passed when they didn't

5.0•2 reviews•

21 followers

Visit website

AI coding agents end tasks with the same line: fixed it, tests pass, ready to merge. Often they never ran anything. cngx is an open source CLI that reads that claim, runs your real checks (pytest, npm test, go test, etc.), and blocks the merge when the story and output do not match. The verdict comes from command output, not regex. `pipx install cngx && cngx quickstart` gets you started in seconds. Built for anyone tired of merge-ready agent output that never touched the test suite.

cngx Reviews

Name: cngx
Rating: 5.0 (2 reviews)

The community submitted 2 reviews to tell us what they like about cngx, what cngx can do better, and more.

5.0

Based on 2 reviews

Review cngx?

Mark by AirtopMeet Mark. Vibe automation for marketers.

Promoted

Reviews

Most Informative

Founder Reviews

Other Reviews (2)

•39 reviews

What's great

Adding a zero-trust validation gate directly at the boundary of autonomous agent generations addresses one of the most glaring operational flaws in the modern software development lifecycle. AI coding agents—whether you're running Claude Code, Codex CLI, or custom multi-agent developer flows—are notoriously prone to optimization shortcuts, often outputting a superficial "all tests passed, ready to merge" confirmation string while quietly hallucinating the actual execution. The brilliance of cngx lies in its absolute reliance on raw process exit codes and deterministic command execution outputs rather than trying to parse non-deterministic LLM prose with fragile regex expressions. Packaging it as a lightweight CLI tool via pipx makes it trivial to drop into local pre-commit hooks or upstream CI/CD environments. By checking assertions down at the operating system process level, it ensures that your repository remains insulated from phantom agent fixes.

What needs improvement

While post-hoc validation blocks bad merges, the current pipeline introduces a noticeable feedback loop bottleneck by executing your test commands sequentially after the agent claims completion. In heavy monorepos or dense microservices where comprehensive integration runners or end-to-end testing suites require minutes to compile, this dual-pass approach can slow down deployment velocity. The CLI would benefit from tight runtime hooks that intercept the agent's execution phase in real-time; if a test suite fails, cngx should feed the raw stderr stack trace directly back into the agent's context window so it can self-correct dynamically before declaring the task finished. Furthermore, if an agent alters database states or configuration files without generating proper mock rollbacks, cngx will catch the failure but won't clean up the dirty environment state left behind by the errant generation run.

vs Alternatives

I’ve previously balanced traditional gatekeeping alternatives, ranging from standard remote GitHub Action status workflows to expensive LLM-as-a-judge evaluation frameworks. While standard CI status checks are reliable at catching broken builds, they operate at the very end of the remote PR lifecycle, wasting cloud runner minutes and polluting git history with obviously hallucinated code diffs that could have been blocked locally. On the other side, using a secondary LLM judge to evaluate code correctness introduces non-deterministic evaluation layers, token overhead, and prompt injection risks. I chose cngx because it returns to development fundamentals—treating agent prose with absolute skepticism and enforcing low-level, deterministic command-line proof-of-work before a single line of code ever migrates to remote environments.

Ratings

Ease of use

Reliability

Value for money

Customization

Report

10 views2mo ago