Launching today
TraceLLM
OpenTelemetry for production AI applications
37 followers
OpenTelemetry for production AI applications
37 followers
Tracellm is an observability platform for production AI applications. Monitor prompt execution, token consumption, latency, spans, errors, and model calls across your LLM workflows. Export traces using OpenTelemetry (OTLP) and quickly identify bottlenecks before they impact users
Every question on that list is about a failure that announces itself. Failed request, slow call, token spike, all visible.
The one that costs most in production is the request that succeeded. No error, normal latency, sensible token count, and the answer was confidently wrong. To catch that afterwards the trace has to hold what the turn was grounded in, which retrieval actually landed and what the tool returned, attributed to that turn. Otherwise the worst incident of the year looks like a healthy span.
Does a span carry the retrieved content and tool results, or the call and the timing only?
The thing I keep wanting from LLM tracing is MCP tool calls as first class spans, not just the model call with a blob of tool JSON hanging off it. Half my latency is the tools, not the model. Does the OTLP export break tool calls out as child spans?