AgentX - Evaluate AI agent, pinpoint issues, and fix with one click.

@gaius_loxley  Thank you!

All major providers out of the box: Claude, GPT, Gemini, Llama, Grok. And if you're running something more custom, you can point it at any LLM via a custom base URL, so you're not locked into the list.

The comparison runs the same agent and eval criteria across providers simultaneously, so you're getting apples-to-apples on quality, cost and latency - not just vibes.

Which providers are you currently working with?

@gaius_loxley 
Thank you!

Currently we’re focused on being provider-agnostic rather than tied to one model family. The goal is to let teams compare agent behavior across different LLMs and decide which model is best for a given workflow.

We support the major model/provider setups our customers use, and we’re expanding based on demand. The key is not only “can it run on provider X,” but “does the agent still complete the task reliably when provider X changes?”

Happy to share more details based on your current stack.

@jocky Great sense Jocky! Developers can setup the eval pipeline into their dev -> prod pipeline and use our eval score metrics to determine the pass gate.

@pradyumna6 Yes, I totally feel you. There are many cases the Agent hallucinated the output that happens to be correct. That's why our eval looks into the deep reasoning, tool use, data retrievals, etc to make the judgement, not just output.

@hehe6z LFG 💪💪💪

@ana_popescu2 In a hierarchical setup, the manager agent ultimately decides which conclusion to follow. But the real value is being able to test and validate these scenarios before they reach production.

That's exactly what AgentX is built for: evaluate AI agents before they fail. You can create test suites, run evaluations, and simulate multi-agent workflows to see how your team behaves when agents disagree. With full observability, traceability, and AI-powered analysis, AgentX helps pinpoint where conflicts occur and even suggests improvements to your manager prompt or individual agents.

Think of it as CI/CD for AI agents — run evals before deploy, continuously tune your agent team, and make sure it behaves reliably when real-world conflicts arise.

@andrew_paul11 Not today, but it's on our roadmap 😊

And honestly, that's a great idea for a future Product Hunt launch. As agent teams become more complex, we'll want AgentX to evaluate and trace behavior across nested hierarchies too. After all, the goal is to run evals before deploy and catch issues before they reach production.

@antonio_manuel1 That depends on how you've designed your agent workflow. In most hierarchical setups, a sub-agent can escalate back to the manager agent, which decides how to proceed. AgentX doesn't dictate the orchestration logic, it helps you evaluate it.

This is actually a great use case for AgentX evaluations. You can create test scenarios where agents fail, get stuck, or return low-confidence results, then verify that your escalation and recovery paths work as expected before deployment. With full observability, traceability, and AI-powered analysis, you can quickly identify weak points and improve your workflow before they impact users.

@maciej_jurewicz Manual testing only covers the scenarios you think of. Evaluations let you build repeatable test suites, run them automatically, and catch regressions before they reach production. AgentX not only identifies where an agent fails but also provides observability, traceability, and AI-powered analysis to help you understand why it failed and how to fix it. Think of it as CI/CD for AI agents—run evals before deploy, not after users find the bugs.

@carlos_leonardo1 Great question - short version: you set the team up once, but the routing is dynamic per task.

You define the structure once: a Manager (orchestrator) plus a few specialist agents, each with its own role, knowledge and permissions. After that you don't hardwire "this request goes to that agent." At runtime the Manager reads the intent of each task and delegates to the right specialist(s) - they work independently, hand off context, and escalate when needed, over multiple rounds. So it's genuine team orchestration, not a fixed flowchart.

For a small team that's the sweet spot: you're not wiring every branch by hand. Start with a 3-agent hub-and-spoke (Manager + 2 specialists) and add specialists as you grow - the architecture doesn't change.

If you do want deterministic, fixed logic for a specific step, that's available too via the workflow builder - so you can mix dynamic orchestration with hardcoded paths where it matters.

And since this is the eval launch: the nice part is you can evaluate the whole team against real cases, tune the routing and prompts, and ship once it behaves the way you want - instead of finding out in production.

What does your setup look like today? One do-it-all agent, or already splitting into roles?

@diego_joaquin1 Exactly - and the delegation problem compounds fast. One agent doing the wrong thing is a bug. Five agents doing the wrong thing because the orchestrator misrouted the task is a production incident.

The hierarchy holds because the Manager is the single point of delegation - agents don't self-assign or broadcast to each other, they receive scoped work and hand back results. That containment is what keeps it predictable as you add specialists.

Where it really shows up at scale is in evals: you can see exactly which agent in the chain introduced the failure, not just that the final output was wrong. That per-agent traceability is what makes debugging a 10-agent team feel manageable rather than like reading tea leaves.

What's the scale you're thinking about - a few specialists or something closer to a full department?

@diego_joaquin1 
I agree - coordination is where things usually break once agents move beyond simple tasks.

Hierarchy helps because it creates clear ownership: who manages the workflow, who handles each specialized step, where handoffs happen, and where failures should be traced.

Scaling agent teams needs structure, not just more agents running in parallel.