Launching today
say-do-diff
Catch AI agent replies that claim more than the tools did
2 followers
Catch AI agent replies that claim more than the tools did
2 followers
Free MIT licensed Python checker for AI agents. It finds claims like sent or refunded in the final reply and grades each against the run tool log. Flags queued emails, failed refunds, partial batches and duplicates. CLI, GitHub Action, n8n, LangChain.
Free
Launch Team
CloudflareYour next app belongs on Cloudflare Workers. Deploy for free
Promoted
Hi Product Hunt. say-do-diff is a small checker for one agent failure: the final reply says "I sent it" or "refunded" or "all set," but the tool calls in that run do not back it up. Common causes are an email API that returned 202 (queued), a payment call that timed out, or 3 of 5 sends landing when the reply says all 5.
You give it the reply and the run tool log. It pulls out each completed action claim, matches it to calls by action and target, grades the evidence from S0 (no call) to S4 (verified afterwards), and flags claims whose wording needs more than the evidence shows. It also catches wrong targets, amount mismatches and duplicate sends. Python stdlib only, no model in the loop, MIT licensed. Exit codes work in CI. It also runs as a GitHub Action and can read logs from n8n, LangChain, planlock, and Microsoft Agent Governance Toolkit.
Important limits: English only with a fixed verb list, so a clean report is not proof the reply is honest. It trusts the log as written. It only reports and does not change the reply. The CLI checks the reply against tool logs. It does not open files or show an automatic file content diff.
If you need the check inline before the reply goes out, with HMAC signed receipts written by your tool layer using a key the model never sees, that is the Say-Do Gate ($49): https://solokit33.gumroad.com/l/...
Free 10 minute manual checklist: https://solokit33.gumroad.com/l/...
We would value feedback on two things: which APIs give you "accepted" responses that agents report as done, and whether requiring an object id for "done" is too strict.