Agnost AI
Catch agent failures your evals miss
839 followers
Catch agent failures your evals miss
839 followers
Agnost AI analyzes conversations between users and your production AI agents and discovers: silent failures, agent behavior drift, hallucinations, user frustration, hidden feature requests, and churn signals. It groups them into recurring patterns, shows the exact users and conversations behind each insight, and turns them into evals and fixes.
Free Options
Launch Team / Built With
World model AI : Sharm SEKAISimulate your idea on a living world.
Promoted
Hey Product Hunt! Shubham here 👋
Parth and I try almost every AI product we come across (we’re just young curious folks).
And we kept on doing the same thing: a product launched with an insane claim, their agent would feel magical for 10 minutes, then it claimed it completed something it hadn’t, invent a link, or make us repeat ourselves three times.
We’d then message the founders and hear: this is really useful feedback. we had no idea.
And we’d think: wait, you already have the entire conversation & traces. why did we have to tell you?
Turns out, their the AI observability dashboards showed a successful request: 200 OK, tool call succeeded, response generated.
The failure was only visible if someone actually read the conversation. So we built Agnost AI.
Agnost AI reads every production conversation across chat and voice agents. It groups them into recurring failures, behavior drift, hallucinated links, frustration, feature requests and churn signals, with the exact users and conversations behind each one.
From there, you can create an eval, or ask your coding agent to debug the problem & fix it.
Because evals test problems you already know about. You can’t write an eval for something you haven’t discovered yet.
Agnost AI connects in three lines of code or through OpenTelemetry and already analyzes more than one million messages every day.
If you’re running a user-facing agent, connect it. I’ll personally help you find three things happening in your conversations that you probably don’t know about.
Also, how do you currently discover failures your evals don’t cover: user complaints, manually reading traces, or something else?
@shubhampalriwala Congrats on launching...🙌 You mentioned turning discovered failures into evals does Agnost export synthetic test datasets directly to frameworks like DeepEval or Braintrust?
@priya_kushwaha1 Hey yes! You can integrate our MCP and it can plug into any of your frameworks. Happy to chat more, calendar link's on our website
Can I also save my model inference costs using the insights Agnost gives me?
@sarthak_aggarwal4 Yes, we train you an SLM based on where your agent fails today with frontier! And its actually more accurate, faster, & cheaper too!
Been waiting for a product like this. Congrats on the laucnh!
@kai_brokering haha thanks kai! onboarding you sooon.
Congrats on the launch team!!! I played around with the product and it felt super snappy
@frcbls means a lot coming from you
You two kept messaging founders about failures in their agents and kept hearing that they had no idea. Congrats on shipping the version where they find out first, without needing you to tell them.
@ben_kahan yessir!
@zachx0 thank zach! u too
congrats on the launch! Let's go team.
@haita thank hai ta