Ahead of Product Hunt we have open sourced Prefactor Evals, No LLM Required. Apache 2.0, public repo.
Most agent failure is behavioural: a silent loop, a step that failed, a run that never finished, a task that took four times the usual work. All of that is checkable in code, deterministically, for nothing. There is no model client anywhere in the project and there never will be, so it cannot cost you a token by construction.
Dumb Phone
fantastic work guys, looking forward to weaving in these kinds of alerts!
Prefactor
@teeeeeegz thanks for the support! The video we put out is all the better from your early feedback! ♥️
Lets get your agents cooking on Prefactor ASAP!
Invent List
Prefactor
@naumanthanvi absolutely. We want to ensure that you don't even have to worry about what happens in Production because you've done all the hard work to build it.