telemetry.dev - LLM observability for AI agents
by•
telemetry.dev shows each run of your AI agent as one trace. Each model call and tool call is a span with its duration, tokens, cost, and error. Prompts and evals use the same traces: change a prompt without a deploy, then run it on a dataset and compare the scores with production. Alerts tell you about errors, cost spikes, and agent loops. Connect with our TypeScript or Python SDK, or with the OpenTelemetry exporter that you already use.
Replies
Hi Product Hunt, I'm Ephraim, one of the makers of telemetry.dev.
When an agent breaks in production, you want to know which step did it and what the run cost. telemetry.dev shows each run as one trace, with every model call and tool call as a span with its tokens, cost, and error.
New in this launch:
- Prompt management: change prompts without a deploy
- Evals: test a new prompt version on a dataset before it goes live
- Alerts when errors pile up or a run gets expensive
- A CLI and an MCP server, so your coding agent can read your traces
It's built on OpenTelemetry, with TypeScript and Python SDKs for OpenAI, Anthropic, the Vercel AI SDK, LangChain and more. The free plan needs no card, and the code PHBU7A47FH gets you 1 month of Pro.
What's the hardest part of debugging your agents right now?