I built a tool that won't let a coding agent mark its own work as done. What you want it to check ?

by

Coding agents often say "done" when the tests never ran or are still failing, and the summary sounds just as confident either way. I kept re-checking their work by hand, so I built foolscap to do the checking for me.

How it works: you give it a task. GPT-6 Astra plans it and hands the work to Claude Code, Codex or OpenCode running on your machine. Then foolscap reads what the test command actually printed and decides whether the run passed. A model doesn't make that call.

Two rules do most of the work:

  • Repairs are capped (two for implementation, one after review), so a run can't loop forever.

  • The review goes to a different agent, read-only. An agent can't approve its own work.

A real run: the first attempt failed a test, one repair fixed it, and Codex reviewed the change. It took 6 turns and cost $0.08.

Current limits: the desktop app is Linux-only for now. On macOS and Windows you run it with npx foolscap. There's no cloud version yet. It's open source (MIT) and needs no account:

What I'd like feedback on:

  1. When an agent says it's finished, what do you check first: test output, the diff, git status, or do you just run it again?

  2. Is "a different agent must review" a real safeguard, or does it just cost more?

  3. What would stop you from letting a tool like this run agents on your own code?

4 views

Add a comment

Replies

Be the first to comment