Prism-Eval is open-source adversarial CI for AI extraction agents. Catch digit drops, prompt injections, and OCR drift before poisoned tool calls reach your engines. G4 false-accept invariant · JUnit/SARIF · Apache-2.0. pip install prism-eval.
Fin Startups get Fin free for a year + 93% off Intercom
Promoted
Maker
📌
Hey Product Hunt — maker here.
pytest assumes deterministic functions. AI extraction agents do not.
An agent can pass every golden fixture on Monday and still ship $45,000 instead of $450,000 on Tuesday — digit drops, OCR column shifts, PDF-footer injections. Happy-path tests never see those.
Prism-Eval is open-source adversarial CI for that gap: builtin G4 corpora → attack-aware scoring → fail the build when critical false accepts slip through (g4_invariant_held). JUnit/SARIF for GitHub Actions. Not a chatbot judge. Not a runtime blocker.
Try it:
pip install "prism-eval==0.3.0"
Live demo (no signup): https://insightitsgit.github.io/...
Landing: https://www.insightits.com/produ...
GitHub: https://github.com/insightitsGit...
If CI finds the blast radius and you need production enforce later, that’s Prism-Shield (separate) — soft pair, not required to try Eval.
Question: what’s the worst silent wrong-dollar / wrong-field your agent suite still “passed”?