AgentX - Evaluate AI agent, pinpoint issues, and fix with one click.

@easton_carter The root cause analysis with suggested fixes - consistently, and it's not close. People expect a score.
They don't expect to be told exactly which instruction caused the drift, why it caused it, and get a concrete rewrite suggestion applied directly to their agent. The first time someone sees that land on a failure they'd been chasing manually for hours, that's usually the moment it clicks.

The second one is tool usage validation. Teams assume if the output looks right the agent did the right thing - it often didn't. You can set tool usage as an explicit eval criterion, so a run only passes if the agent actually called the right tool, not just returned a plausible answer. That's caught a lot of silent failures nobody knew were happening.

Both tie together: observability shows you what happened, root cause analysis tells you what to do about it. One without the other is half the loop.

@easton_carter 
Great question!

The strongest feedback is usually around eval-before-deploy: teams want confidence before pushing prompt, model, tool, or workflow changes to production.

Observability helps you understand what happened after the fact, but evaluation gives you a way to catch regressions earlier.

The combination is important, but the “can I safely ship this agent version?” question seems to resonate the most.

@fletcher_oliver Thank you! Traditional monitoring tells you something broke. We tell you why, where in the chain it broke, and what to change to fix it.

Most monitoring tools are built around infrastructure: latency, error rates, uptime. Useful, but when an agent returns a confidently wrong answer with 200ms response time and zero errors, traditional monitoring gives you a green dashboard while your users get bad outputs.

The difference is we evaluate behavior, not just availability. Every run captures the full decision chain, thinking process between agents, tool calls, handoffs, routing decisions - scored by a panel of LLM judges against criteria you define. When something's off you get a specific diagnosis, not a spike on a graph.

And the loop closes automatically: suggested instruction changes applied directly to your agents, not a ticket for someone to investigate later. What does your current monitoring setup look like?

@fletcher_oliver 
Thank you!

Traditional monitoring is usually great at system signals: latency, errors, uptime, logs, costs.

Agent observability needs a different layer: what the agent decided, which tools it used, whether the steps made sense, whether it stayed aligned with the task, and whether the final outcome was actually useful.

For agents, “200 OK” does not mean the work was done correctly.

That’s where we focus: connecting traces, evaluation, and outcome quality so teams can debug and improve the agent itself.

@mikebrandswarm Hey! You've described exactly the gap we set out to close. You're right that most tools drop you after deploy.

Post-deploy every conversation is logged with full traces - per-agent steps, tool calls, the decision chain between agents. It's not sampled, it's everything, so you're not hoping a bad response happened to get captured.

Bad responses can be flagged and replayed step by step. You see the full thinking process, where the chain diverged from what you expected, and what specifically went wrong. From there the same AI analysis that runs in pre-deploy evals kicks in: root cause, suggested instruction changes, applied directly.

The feedback loop is the point. Pre-deploy evals catch known failure modes before they ship. Post-deploy monitoring catches what real users actually trigger. Both feed back into the same eval framework, so your agent gets measurably better over time rather than just getting patched when someone complains.

Demo to production-grade is exactly the gap we're targeting.

@mikebrandswarm 

Totally agree - the real challenge starts after launch!

We don’t see evaluation as something that should be left entirely to the builder. The feedback loop needs to include traces, bad response review, replay, and comparison against expected behavior so teams can understand what went wrong and whether a fix actually improved the agent.

The goal is to connect pre-deploy evals with post-deploy learning: catch regressions before release, then use real production failures to improve the eval suite over time.

That’s what turns agent QA into an ongoing system, not a one-time checklist.

@karimbenkeroum Thank you!

@karimbenkeroum Thank you Karim, really appreciate the support! 🙏

@jared_salois Thanks! And this is a sharp one - model sovereignty is easy to claim and harder to make survive a model bump.

What transfers cleanly is the eval definition, not the score. Your dataset: the cases, the acceptance/rejection criteria, the expected results - is model-agnostic. It describes what "good" means for your task, and that doesn't change when GPT or Claude ships a new version. So you don't rebuild the eval when you switch or update a model, you re-run the same one.

What moves is the result, and that's the point rather than a bug. Swap the LLM, re-run, and the score tells you whether the new model holds up against your existing bar — same cases, same judges, apples-to-apples on quality, cost and latency. Model sovereignty isn't "switch and assume it's fine," it's "switch and get a measured answer."

The one thing you occasionally re-tune is the judge criteria, if a new model's output style shifts enough to need tightening. But that's a small adjustment, not a rebuild - the cases stay.

@jared_salois 
Good point, and thank you!

We don’t assume evals transfer perfectly across models. That’s why this layer matters.

The eval suite should stay focused on the business task, expected behavior, required fields, tool usage, and failure cases - not one model’s wording.

Switching LLMs may still need tuning, but now you can see what changed, which scenarios still pass, and what needs fixing before deploy.

So it’s not magic transfer - it’s controlled comparison instead of guessing.

@luki_notlowkey Build AI agent is just the first step. Making sure it is good enough to use in production is the real hard part. Our goal is to provide the platform that you can confidently deploy your AI agent with peace in mind.

@luki_notlowkey 
Thank you, really appreciate it! 😊

That’s exactly what we’re hearing from agent builders too - the agent layer is moving fast, but the eval / QA layer is still missing.

For GTM, we’re starting with teams already building agents for real workflows: AI agencies, agent builders, and companies moving from prototypes to production. The focus is to make evaluation easy to plug into existing agent stacks, then expand through integrations, templates, and repeatable eval suites for common use cases.

Would love to learn more about the types of agent builders you work with too.

@galdayan Great question Gal. The real evaluation should always handle the complete chain of thoughts and sub-processes.

We analyze all the data retrievals, tool uses, multi-step, and multi-agent layer. Check each one and aggregate to summarize the report.

@galdayan Exactly - that’s where things get messy fast.

For multi-step tool chains, we track the full execution path, not just the final output: each tool call, inputs, outputs, step order, and where the chain started to drift.

So if the agent fails 3-4 calls deep, the eval should show whether it was caused by the wrong tool selection, bad parameters, incorrect interpretation of a tool result, or a later reasoning mistake.

That’s the real value of eval-before-deploy: catching hidden failures before they become production issues.

@renchu_song Thank you Richard, really appreciate it!

Yes - AgentX evaluation goes beyond the final output. We evaluate the full execution path: reasoning trajectory, tool / function calls, step selection, final response, and task completion.

For multi-turn interactions, we evaluate the whole conversation as one scenario, checking context retention, decision quality, tool usage, and final business outcome.

The final answer can look right even when the agent took the wrong path, so we want evals to catch that too.

@ozandag 
Great question - handoffs are usually where multi-agent systems get fragile.

We think evals need to look at the workflow boundary between agents, not just each agent in isolation: what context was passed, what task was delegated, whether the receiving agent understood it correctly, and whether the handoff produced the expected next action.

So the failure can be flagged as a coordination issue, not incorrectly blamed on the final agent response.

@crystalmei 
Thank you!

That’s exactly where suggestions need to be tied to the trace, not just the final output.

When a failure spans multiple agent calls, the goal is to identify where the chain started drifting: bad handoff, missing context, wrong tool use, weak instructions, or a downstream interpretation issue.

Then the fix can be targeted to that step, instead of rewriting the whole workflow!