QAgent
Automated QA for AI agents. Stop shipping on vibes.
53 followers
Automated QA for AI agents. Stop shipping on vibes.
53 followers
Automate AI agent quality testing. Score correctness, detect hallucinations against ground truth, verify policy adherence, and benchmark RAG before bad responses reach real customers.
Free Options
Launch Team / Built With
AWS Activate CreditsCredits that fuel your startup’s next milestone
Promoted
Hey Product Hunt! 👋
I am Abhiram, the creator of QAgent.
I built QAgent after watching my own AI agents pass every single playground test, only to break in front of real customers.
The harsh reality of building LLM agents is that single-turn benchmark scores do not translate to multi-turn reliability. If your agent is 95% accurate per turn, the compounding failure rate means it has barely a 35% chance of finishing a 20-turn conversation without hallucinating or breaking rules.
Most solo developers cannot afford to write 200 lines of brittle Python evaluation scripts every time they tweak a system prompt.
QAgent is a zero-bloat automated QA and regression testing platform. You connect your agent through a simple webhook in two minutes without installing any SDK. You can run automated test suites across factual groundedness, policy compliance, RAG faithfulness, and multi-turn context retention, and get a clear 0-100 quality scorecard showing the exact turn where your bot drifted.
There is a free tier with 100 evals every month with no credit card required so any builder can test their bots today.
I would love to hear your feedback, questions, or feature requests!
The scariest failure mode in production AI isn't hallucination. It is politeness.
One of the best comments on our launch yesterday came from Gal Dayan at Dial. He pointed out that manual transcript spot-checking is fundamentally broken because humans naturally grade the tone instead of the actual claim.
If an agent sounds empathetic, structured, and polite, our brains instinctively give it a passing grade.
Meanwhile, that exact same polite agent just agreed to a 60-day refund window when your hard policy is 14 days, or handed out an unauthorized 30% discount because the user asked nicely three times.
Nothing crashed. Datadog logged a clean HTTP 200. Sentry shows zero errors. But business logic quietly broke in front of a real customer.
That is why we built QAgent around deterministic Policy Adherence scoring over HTTP webhooks. It ignores conversational pleasantries and tests whether your hard rules were actually obeyed.
If you are running customer-facing agents and want to see if your models are quietly leaking rules, the free tier includes 100 evaluations every month at qagent.in.
We just crossed 50 followers, 79 upvotes, and over 130 developers testing the platform since morning. Huge thank you to everyone who checked it out so far.
The biggest takeaway from the feedback and chats today is that almost everyone is struggling with the exact same failure mode: catching silent policy drift without getting buried in false alarms.
If anyone here is currently testing a customer-facing agent and wants help setting up custom evaluation webhooks, feel free to drop your use case below or ping me directly. Happy to help configure your ground-truth test sets today.
And remember, the free tier has 100 automated evaluations every month with zero SDK setup at qagent.in.
"Stop shipping on vibes" is uncomfortably accurate for where most of us are with agent QA right now - regression testing against ground truth instead of eyeballing transcripts is exactly the gap. Scoring policy adherence specifically is the part I'd actually pay for, since that's the failure mode that's hardest to catch by just reading a few sample outputs.
@galdayan Appreciate this so much Gal. You hit on the exact reason policy adherence was the hardest metric to dial in.
On the surface, an agent's answer can sound totally polite and reasonable, but underneath it quietly promised an unauthorized discount or return window. Early on, I was constantly fighting false positives where the evaluator would fail an agent just for phrasing things differently, or false negatives where a broken rule slipped through because the bot sounded confident.
I would genuinely love to see how it performs on your agent's policies and get your feedback. There is a free tier at qagent.in that takes two minutes to hook up via webhook, or feel free to message me directly if you want me to help you set up a policy test suite for your setup!
@abhiram_reddy_k Appreciate the offer, will keep the webhook option in mind once we have a policy set worth testing against. The false-positive-on-phrasing failure mode you described is exactly what worries me about eyeballing transcripts ourselves - too easy to grade the tone instead of the actual claim.
@galdayan Spot on, Gal — that "polite tone trap" is the silent killer of production agents. Human reviewers naturally give an agent an A+ for sounding friendly, even while it casually invents an unauthorized 40% discount or hallucinates a return window.
Quick tip that might save you time: you actually don't need a formal 20-page policy document to start with QAgent. Most builders start with just 3 or 4 single-line assertions in individual test cases (e.g., "Never discount >15%" or "Escalate if legal action is mentioned").
Whenever you have 2 minutes to stress-test your first few baseline rules, the webhook runner is ready for you. Would love to hear your thoughts when you try it!
@galdayan 100% — that booking time read-back is a textbook test case! That's exactly the kind of silent policy slip-up that human reviewers miss when an agent sounds otherwise polite.
Whenever you plug in your webhook this week, give me a shout if you run into any quirks with the payload format or auth headers. Happy to help take a look at your first run!