I'm Nathan, the builder of Plumb. Plumb started with a simple failure: a coding agent told me it was done, but it hadn't actually established that. My first instinct was to add more instructions. Then I realized I was assuming that would solve the problem, so I tested the assumption instead. That turned into months of work across real repositories, held-out evaluations, third-party research data, and quite a few experiments that failed. Some features were killed entirely because the evidence didn't support them. The result is Plumb: a deterministic layer that reads what a repository itself declares, finds the verification it can support, and checks the work independently of the coding agent. Plumb doesn't use another model to grade the first model. It can also refuse to claim success when the evidence isn't there. It runs locally, makes no model calls, doesn't upload your repository, and works alongside coding agents like Claude Code and Codex. Install it with: curl -fsSL https://plumbing.sh/install | sh I built Plumb around a simple idea: Agents can guess. Evidence shouldn't. This is the hardest I've worked on a software project, and at this point the most useful testing can't come from me anymore. It needs to come from repositories and developers I haven't seen. If you use coding agents, I'd love for you to try Plumb and tell me where it gets things wrong.

Plumb

Your coding agent says it's done. Plumb checks.

Your coding agent says it's done. Plumb checks.