Launching today
Prefactor
Evaluate your AI Agents in real-time
642 followers
Evaluate your AI Agents in real-time
642 followers
Most agents pass their evals and fail in production. Prefactor is the evaluation layer that closes the gap. We score every agent run in real time, surface quality regressions and drift as they happen, and show engineering teams exactly how their agents are performing at scale. Built for the teams shipping agents to customers.
Congrats on the launch.
When teams try Prefactor for the first time, how do teams catch the first bad run before users notice?
I really like the idea of tracking how the agent is working, as it is performing actions. Being more proactive is better than finding out something went wrong several days ago. It's not to be overlooked that the interruption isn't just killing the agent run, but handing it to a human too. I am going to have to play around with this and my own agents.
Prefactor
@philnash Thanks for the comment Phil. Would be happy to share some of the examples Ive built too, I'm building an eval triaging tool. 3 phases.
First phase: Tokenless Evals/ technical reviews
Second phase: pattern management against prior activity
Third phase: LLM as judge
All live. Let me know where you get to.
Intent by Upflowy
Fantastic product and team, very excited to see them launch here, congratulations on the successful launch of the product!
Prefactor
@matthew_browne1 Thanks Matt. It means a lot coming from you! Thanks for all of the support. To the moon!!!!!!
Voquill
I like that this is built around production instead of just benchmarks. That's where things usually get interesting. Congrats on the launch!
Prefactor
@henry_habib Thanks Henry, and thanks for the comment. Benchmarks are great in controlled envs. Agents don't operate in hermetically sealed containers. They exist in the wild. We needed to meet that challenge.
Prefactor
@henry_habib Thank you! Yes, production is where things tend to get real pretty quickly. Understanding how your benchmarks/evals translate to the real world is super important.
Prefactor
@henry_habib thanks for the comment, Henry! We agree, an agent demo on a developers laptop is one thing, but once it hits a production environment that where the fun really begins. Are you deploying anything into production currently?