AgentX - Evaluate AI agent, pinpoint issues, and fix with one click.
by•
Evaluate AI agents before they fail. Create test suites, run evaluations, and pinpoint issues before they reach production.
AgentX provides full observability and traceability for your AI agents. AI analysis not only identifies problems but also suggests fixes-like an AI doctor for your agents.
Simulate run your agents across multiple LLM providers to compare performance, cost, and latency, helping you make better decisions about which LLM to go.
Run eval before deploy. Like CI/CD for AI agents.
Replies
Biggest one for me: the agent itself worked, but a second AI step that summarized the conversation afterward silently dropped fields the user had actually given (a specific detail mentioned mid-call just vanished from the summary). No error, no crash, just quietly incomplete output. That kind of failure is the hardest to catch because everything LOOKS fine until you diff the transcript against the summary by hand. Root-cause analysis on the analysis step itself, not just the main agent, seems like exactly the gap tools like this should close.
@david_marko A hundred percent! If we only compare input and output, the hidden issues will never surface!
@david_marko That’s a really good example of the kind of failure that standard observability often misses.
The main agent can complete the task correctly, but a later AI step - summary, extraction, classification, handoff note - can silently drop important information.
This is why evals need to cover the full pipeline, not only the primary agent response. The source conversation, intermediate steps, and final structured output should all be checked against each other to catch missing fields, dropped context, or mismatches.
“Looks fine” is often the most dangerous failure mode in production
The "CI/CD for agents" framing resonates — the hard part has always been defining what "passing" even means for a non-deterministic agent. How does AgentX handle the eval oracle: are test suites assertion-based, LLM-judged, or a mix, and how do you keep those judgments stable across runs? The multi-LLM cost/latency comparison is a genuinely useful addition — picking a model on vibes is still way too common. I'd just want the AI-suggested fix to show its reasoning before I trust it anywhere near production.
@codeamesh_consultancy Great observation. There are many metrics we offer for various of scenarios.
For example cosine score for the semantic closeness, Jaccard score for the text overlapping rate. And most useful is the overall score of the whole eval from the multiple LLM-as-a-judge.