... prod, you shouldn't get an alert and a spike on a dashboard. You should get the answer: what went wrong, why, and what to change. Not a debugging exercise. Not manual. Not broken by every model update. Today Prefactor lets you: Evaluate agents in development using production or synthetic data. Bring your own evals or use Prefactor

... Prefactor finished #1 Product of the Day, and the response has been ridiculous. To celebrate, anyone who signs up by Sunday and connects an agent will receive 1 million free spans. Prefactor

... consent-of-record? trying to name the shape of what enterprise buyers ask for that the industry hasnt named yet. Comment from Matt Doughty(@matt_doughty): Every agent should have an unique identifier, a human owner, timestamp and within Prefactor, we define an agent contract. Eg, what the agent is able to do, what the risk profile and quality profile of that agent is. Ultimately our contract says - What should an agent be doing in prod, and this ... ... failure' and the reviewer owns 'did i sign off on this specific instance.' two owners, different failure modes. thats not a get out of jail free, its specifying which human is answerable for which layer. the drift catching prefactor

... Matt Doughty(@matt_doughty): @artk   @alieksia  I think that if there is a way to reduce token cost and make, taking action even more valuable. Then there absolutely is value. Full disclosure, I am the founder of Prefactor, and we are focused on evaluating agents. The core of our first class functionality is tokenless evaluation - quality and risk. My broader focus is to create examples where tokenless evaluation can be layered so that you only bring ... ... judge. Comment from Anastasiia(@alieksia): @artk   @matt_doughty   That layering is the interesting part. To be honest I'm not sure our idea fits directly, but the approach is similar. I will take a look at Prefactor