AI agents can look reliable in a demo and still fail unpredictably across repeated runs. Agent Reliability Toolkit is an open-source developer tool for testing agent reliability at scale. Run your agent repeatedly, measure pass/fail rates, inspect failures, track latency, and detect regressions between versions, all from a simple dashboard. Instead of asking, “Did my agent work?” Ask: “How reliably does it work?” Built for developers shipping AI agents beyond the demo.