Multi-agent in production. How are you actually monitoring what runs?
by•
Running into a wall I haven't seen written about clearly.
Single agent in production is fine. Logs, traces, normal observability.
Multi-agent is where it gets weird:
- Which agent triggered which downstream call
- When agents disagree, who wins and why
- Token cost per agent vs per task vs per user
- The 'who broke production at 3am' question when 4 agents are in the loop
What's working for the folks who actually have this in production? Custom tooling, Datadog, Langfuse, something else? Or is everyone just running blind and praying?
Genuinely asking because we're hitting the limit of 'tail the logs and hope.'
38 views
Replies
The '' who wins and why'' question caught my attention. i'd want to see that decision in the trace rather than digging through a pile of logos afterward.
the thing that helped us most was refusing to treat "agent disagreement" as a runtime event to debug after the fact. we made one agent an explicit arbiter with the actual authority to decide, and every other agent's output is just a proposal that gets logged with a reason string attached, not an autonomous action. that turns "who won and why" into a single line in one trace instead of reconstructing intent from four separate logs.
for the triggering-chain question, a shared trace/run ID that every agent call inherits (even across process boundaries) is what actually made "which agent caused this downstream call" answerable in real time instead of after a postmortem. token cost per agent falls out almost for free once you're tagging spans that way, you're just grouping the same trace by agent role instead of building separate cost tracking.
the 3am one is still the hardest honestly. best we've got is treating a disagreement-without-resolution (arbiter times out or the proposals contradict) as its own alert category, separate from a normal error, because it's usually the multi-agent-specific failure mode that a single-agent alert setup won't catch.