Hey Product Hunt 👋 I built EvalTrim around a problem that became more obvious as my AI-agent eval suites grew: Writing another regression test is easy. Knowing which tests are still worth keeping is much harder. Over time, an eval suite can accumulate near-duplicates, stale cases, flaky tests, conflicting expectations, and many tests covering the same behavior. EvalTrim tries to solve the maintenance side of evaluation. The part I'm most interested in is the counterfactual removal workflow: Instead of saying “these two tests look similar,” EvalTrim asks what would actually be lost if one were removed. It tracks unique behavioral witnesses, critical coverage, requirements, historical failures, oracle conflicts, and other evidence before making a recommendation. The current version is 0.6.0 beta and runs locally by default. Some included constructed benchmarks, with no LLM and embeddings disabled, currently measure 1.00 precision, 1.00 recall, 1.00 retirement safety, and 1.00 critical coverage across the three included suites. I’d love feedback from people building AI agents: What would you need to see before trusting a tool to recommend merging or retiring an eval? And where does eval-suite maintenance hurt most in your current workflow? GitHub: https://github.com/lowjieseng181...

EvalTrim

Prove which AI-agent evals are worth keeping.

Prove which AI-agent evals are worth keeping.