hi PH 👋 shipped this for the @AnthropicAI hackathon — falsify, a tiny python CLI that does one thing: it pre-registers your ML accuracy claim with SHA-256 before the experiment runs. if anyone silently edits the threshold afterwards, the next run exits 3 and CI refuses to produce a verdict. the honest amendment path is `lock --force` which writes a new audit entry. both paths are legible; only one is silent. three days, single file, 3925 LOC, 518 tests, MIT. uses claude code skills + an MCP server + 2 forked-context subagents. honesty score 1.00 (yes, falsify uses itself in CI on every push). design choices i'd love feedback on: — canonical YAML for hashing (vs JSON-LD / RFC 8785 JCS) — landed on sorted-key YAML for human readability — deterministic verdict, no LLM in the verdict step — if the model judges its own claim, integrity collapses — exit-code contract: 0 PASS, 10 FAIL, 3 tampered, 11 guard violation — CI gates on these directly why now: EU AI Act high-risk regime kicks in 2 august 2026. article 12 (logging) and article 18 (10-year tech-doc retention) require exactly this kind of cryptographically-anchored audit trail. happy to dig into the YAML canonicalization, the threat model (docs/ADVERSARIAL.md lists 8 defended attacks + 6 explicitly out of scope), or why this composes with W&B/MLflow rather than replacing them. ask me anything. — pip install falsify · falsify.dev · 90s demo: https://youtu.be/vVZTNeak5PA

falsify

Pre-register your ML accuracy claims

Pre-register your ML accuracy claims