Open-source AI agent monitoring platform. Latitude automatically detects all the ways your agents fail at scale, and gives your coding agent the tools to fix it.
Replies
Best
The issue abstraction is the important move. Raw traces are necessary, but small teams need a release gate: is this failure mode understood, reproducible, and covered by an eval before we ship again?
the "issues not logs" framing resonates. i've lost hours scrolling through agent execution traces trying to find why something broke, only to realize the actual failure happened 6 steps earlier. how do the evals work here, do you define failure criteria upfront or does it infer patterns from the traces?
Report
The phrase all the ways your agents fail is ambitious is failure detection here pattern based on known anti patterns or does it learn failure signatures from your own agent's history over time?
Report
While most agent tools stop at dumping logs, auto-building an eval from each failure cluster looks totally spot on! One thing I'd poke at - how do you stop those auto-evals from overfitting to the exact transcripts that triggered them instead of the general failure mode? Thanks!
Report
How does Latitude differentiate a genuine failure from an agent that's thinking out loud through a messy but ultimately correct reasoning path?
Report
For agent systems with non-deterministic outputs, how do you define failure in a way that's consistent enough to monitor reliably at scale?
Report
This is a super clean approach to agent observability! Triage is a nightmare when you're just staring at a massive, unorganized stream of logs. Grouping traces into auto-clustered issue datasets makes finding where a trajectory went wrong way faster.
How does Latitude handle automated regression testing once a fix for a specific trace issue is pushed?
Report
The MCP-into-the-coding-agent piece is the clever bit, and underrated in the thread so far. Most observability tools die at the dashboard — signals pile up where nobody looks, so failures just rot. Routing the signal to where the fix actually happens (the editor) is the real unlock; detection was never the bottleneck, action was.
One sharp question on that loop: when you auto-generate an eval per signal and hand it to the coding agent to fix against, how do you keep the agent from overfitting to the eval — patching the specific failing cases rather than the underlying behavior, so the cluster 'closes' but the real issue persists? Curious if there's a held-out/regression check or a human-in-the-loop on the generated evals. That's the failure mode I'd worry about most with auto-fix.
Congrats on shipping this — genuinely needed.
Report
Congrats on the launch, Cesar! The "cluster conversations into failure modes" piece is the part I'd get the most from. One question from running agents that deliberately hand off to a human: how does Latitude tell a real failure apart from a correct escalation? In our setup the agent is supposed to stop and route anything sensitive — refunds,
account changes — to a person, so a "drop-off" there is it doing its job, not breaking. Does it learn which escalations are intended vs the agent actually giving up?
Replies
The issue abstraction is the important move. Raw traces are necessary, but small teams need a release gate: is this failure mode understood, reproducible, and covered by an eval before we ship again?
Humalike
Congrats on the launch! <3 desde bcn
BetterClaw
The phrase all the ways your agents fail is ambitious is failure detection here pattern based on known anti patterns or does it learn failure signatures from your own agent's history over time?
While most agent tools stop at dumping logs, auto-building an eval from each failure cluster looks totally spot on! One thing I'd poke at - how do you stop those auto-evals from overfitting to the exact transcripts that triggered them instead of the general failure mode? Thanks!
How does Latitude differentiate a genuine failure from an agent that's thinking out loud through a messy but ultimately correct reasoning path?
For agent systems with non-deterministic outputs, how do you define failure in a way that's consistent enough to monitor reliably at scale?
This is a super clean approach to agent observability! Triage is a nightmare when you're just staring at a massive, unorganized stream of logs. Grouping traces into auto-clustered issue datasets makes finding where a trajectory went wrong way faster.
How does Latitude handle automated regression testing once a fix for a specific trace issue is pushed?
The MCP-into-the-coding-agent piece is the clever bit, and underrated in the thread so far. Most observability tools die at the dashboard — signals pile up where nobody looks, so failures just rot. Routing the signal to where the fix actually happens (the editor) is the real unlock; detection was never the bottleneck, action was.
One sharp question on that loop: when you auto-generate an eval per signal and hand it to the coding agent to fix against, how do you keep the agent from overfitting to the eval — patching the specific failing cases rather than the underlying behavior, so the cluster 'closes' but the real issue persists? Curious if there's a held-out/regression check or a human-in-the-loop on the generated evals. That's the failure mode I'd worry about most with auto-fix.
Congrats on shipping this — genuinely needed.
Congrats on the launch, Cesar! The "cluster conversations into failure modes" piece is the part I'd get the most from. One question from running agents that deliberately hand off to a human: how does Latitude tell a real failure apart from a correct escalation? In our setup the agent is supposed to stop and route anything sensitive — refunds,
account changes — to a person, so a "drop-off" there is it doing its job, not breaking. Does it learn which escalations are intended vs the agent actually giving up?