trending

1d ago

agent-eval: Statistical regression testing for LLM agents

Hi Everyone!

@rudrendu_paul_ai and I are officially soft-launching a new OSS solution, agent-eval, today.

5h ago

Agent-Eval - Statistical regression testing for LLM agents

Statistical regression testing for LLM agents. Run versions A and B 50 times to get a p-value, Cohen's d, and a 95% CI proving whether behavior actually shifted. While DeepEval, Braintrust, and Promptfoo test single responses against a threshold, none measure distribution drift. It is completely self-hostable under an Apache 2.0 license and requires no SaaS subscriptions. Works natively with LangGraph, OpenAI Agents SDK, CrewAI, and LangChain LCEL. Start testing: `pip install agent-regress-cli`.