Launching today
nuhuh
Your AI agent said "Done." nuhuh runs the experiment
2 followers
Your AI agent said "Done." nuhuh runs the experiment
2 followers
Coding agents end most tasks with "Done! All tests pass." Sometimes it's false. nuhuh is a Stop hook that extracts every claim from the agent's final message and re-runs reality, fresh tests, real exit codes, actual files, then bounces a false Done back with the evidence. Local, deterministic, no API key, MIT.
Last week my agent told me "All tests pass." One test failed. It had reported the result of a run it never made. So I built a test runner wearing a Stop hook.
nuhuh treats the final message as a list of hypotheses and re-runs each one. The whole suite in a clean process, files checked on disk, localhost actually probed. A false Done gets rejected and the evidence goes straight back to the agent, which returns to work.
It ships with a benchmark whose ground truth doesn't know the tool exists, so it catches nuhuh's own mistakes too. Across 102 runs per model, Codex falsely declared Done 2.1% of the time, Haiku 6.0%, frontier Claude 0%. And every false Done contained zero checkable claims, just confident tone, which is exactly why the benchmark exists.
That finding became a feature. NUHUH_STRICT=1 bounces any completion declaration that carries no checkable claim. Replayed against all 306 runs it catches 7 of 7 false dones, and the honest price is one extra bounce on roughly half the true dones, which is why it's opt-in and aimed at unattended lanes like CI and batch runs.
Try it without installing anything. npx nuhuh demo stages a lie and catches it in ten seconds.
Ask me anything about the false accusations we caught our own tool making, there are six and each one is now a regression test.