Launching today

AgentDiff

Launching today

Trajectory regression testing for AI agents

0 followers

Trajectory regression testing for AI agents

0 followers

Visit website

Traditional evals test outputs, missing when agents loop on tools or burn 50% more tokens. AgentDiff fixes this by recording agent runs as DAGs and diffing execution trajectories against golden baselines directly in CI. It catches tool loops, cost spikes, and latency regressions before merge evaluating how the model arrived at the answer, not just what it returned. Framework-agnostic with adapters for OpenAI Agents, Langfuse, LangSmith, and OpenInference.

Free

Launch tags:Open Source•Software Engineering•Artificial Intelligence

Launch Team

Framer AI AgentsDesign and publish professional sites with AI

Promoted

Maker

📌

The annoying bug that started this: I refactored an agent, all tests passed, and it silently started calling the DB tool in a loop. Cost went up ~50% and only the invoice caught it. The part I'm most proud of is that it posts the report onto the PR automatically. There's a real example in the repo - a Gemini agent that looped get_user_database_stats twice, AgentDiff flagged the culprit step, and the job failed the gate. No manual diffing, no reading raw traces.

Report

1d ago