The split between tool-call telemetry and outcome visibility is compelling—especially the demo's `send_email` failing 94% of calls. How much of that workflow/outcome linkage is automatic from the one-line SDK, versus something a server needs to instrument explicitly?
@brooksswift Hi Brook, 100% of it will automatically come in from the SDK, no need to instrument explicitly.
Report
What I can't see is whether the result was any good. A tool returns 200 with an empty array, the agent carries on and answers anyway, so my success rate looks healthy while the answer is wrong. The signal I'd want is what happened in the next turn: did the model retry the same tool, switch to a different one, or quietly ignore what came back. That's the gap between a server that responds and one that works.
@asadmalik901 Yes, I hear you and that's exactly the problem we are solving. A 200 response only tells you the server replied. It actually doesn't tell you whether the tool helped the agent. Even an empty result can look like a success when the agent gives the wrong answer and the metric states healthy.
We are building the visibility to connect exactly these dots: what the tool returned, how the agent interpreted it, what happened next. It also identifies whether it retried, which tool asked for more information, or just quietly carried on with the bad result. That next turn behaviour is the real signal of whether the system worked and that's our focus. That's the gap we are actually trying to close with TrackMCP.
Report
@asadmalik901@krishnaa_ Was gonna ask the same thing, but asad beat me to it. I hope to see more visibility in the future versions and congrats on your PH Launch today!
Report
@krishnaa_@simon_liang_1 The breakdown I'd want is per tool, not per server. When we instrumented ours the ugly finding was that one badly worded tool description caused most of the re-asks, and the server itself returned the right thing every time. If the retry data can point at which description is confusing the model rather than just flagging that a retry happened, that's the bit I'd pay for.
Report
@krishnaa_@simon_liang_1 The hard part is that the server never sees the next turn. It knows it returned something, and whatever the model did with that happens inside a client it has no window into. So the retry-or-ignore signal has to be collected client side, which is a much bigger install ask than one line of code on the server. Worth being upfront about which half you're actually measuring today.
Report
Your site design and product UI are both exceptionally well done. The consistency goes beyond the visual styling; the discipline around hierarchy is especially tight. I used to lead Web UX and Visual Design at Neo4j, so I’m deeply familiar with how difficult it is to represent a complex technical product visually. The “How It Works” block on the homepage is especially strong, and the micro-interactions add a lot without distracting from the explanation. Kudos to the design team.
For the record, not AI generated. AI spellchecked. Im a terrible speller.
Report
Krishna, the part that stuck with me is catching the stuff that looks fine but quietly did nothing. That kind of quiet failure is the worst because you never think to go looking. Genuinely cool.
@robin_de_lacroix For a second I thought the product stopped working. Thanks for adding those last two words, lol. Jokes aside, thank you for your kind words.
Report
@krishnaa_ ahaha my bad for the wrong writing structure
"where does the job stop" is the question I'd want answered proactively rather than by staring at a dashboard. once you're tracking tool call sequences and outcomes, does TrackMCP ever flag a regression on its own - like a tool's success rate suddenly dropping after a deploy - or is it purely pull, where you have to go looking for the dip yourself?
The split between tool-call telemetry and outcome visibility is compelling—especially the demo's `send_email` failing 94% of calls. How much of that workflow/outcome linkage is automatic from the one-line SDK, versus something a server needs to instrument explicitly?
TrackMCP
@brooksswift Hi Brook, 100% of it will automatically come in from the SDK, no need to instrument explicitly.
What I can't see is whether the result was any good. A tool returns 200 with an empty array, the agent carries on and answers anyway, so my success rate looks healthy while the answer is wrong. The signal I'd want is what happened in the next turn: did the model retry the same tool, switch to a different one, or quietly ignore what came back. That's the gap between a server that responds and one that works.
TrackMCP
@asadmalik901 Yes, I hear you and that's exactly the problem we are solving. A 200 response only tells you the server replied. It actually doesn't tell you whether the tool helped the agent. Even an empty result can look like a success when the agent gives the wrong answer and the metric states healthy.
We are building the visibility to connect exactly these dots: what the tool returned, how the agent interpreted it, what happened next. It also identifies whether it retried, which tool asked for more information, or just quietly carried on with the bad result. That next turn behaviour is the real signal of whether the system worked and that's our focus. That's the gap we are actually trying to close with TrackMCP.
@asadmalik901 @krishnaa_ Was gonna ask the same thing, but asad beat me to it. I hope to see more visibility in the future versions and congrats on your PH Launch today!
@krishnaa_ @simon_liang_1 The breakdown I'd want is per tool, not per server. When we instrumented ours the ugly finding was that one badly worded tool description caused most of the re-asks, and the server itself returned the right thing every time. If the retry data can point at which description is confusing the model rather than just flagging that a retry happened, that's the bit I'd pay for.
@krishnaa_ @simon_liang_1 The hard part is that the server never sees the next turn. It knows it returned something, and whatever the model did with that happens inside a client it has no window into. So the retry-or-ignore signal has to be collected client side, which is a much bigger install ask than one line of code on the server. Worth being upfront about which half you're actually measuring today.
Your site design and product UI are both exceptionally well done. The consistency goes beyond the visual styling; the discipline around hierarchy is especially tight. I used to lead Web UX and Visual Design at Neo4j, so I’m deeply familiar with how difficult it is to represent a complex technical product visually. The “How It Works” block on the homepage is especially strong, and the micro-interactions add a lot without distracting from the explanation. Kudos to the design team.
For the record, not AI generated. AI spellchecked. Im a terrible speller.
Krishna, the part that stuck with me is catching the stuff that looks fine but quietly did nothing. That kind of quiet failure is the worst because you never think to go looking. Genuinely cool.
TrackMCP
@robin_de_lacroix For a second I thought the product stopped working. Thanks for adding those last two words, lol. Jokes aside, thank you for your kind words.
@krishnaa_ ahaha my bad for the wrong writing structure
Dial
"where does the job stop" is the question I'd want answered proactively rather than by staring at a dashboard. once you're tracking tool call sequences and outcomes, does TrackMCP ever flag a regression on its own - like a tool's success rate suddenly dropping after a deploy - or is it purely pull, where you have to go looking for the dip yourself?