Launching today
telemetry.dev
LLM observability for AI agents
147 followers
LLM observability for AI agents
147 followers
telemetry.dev shows each run of your AI agent as one trace. Each model call and tool call is a span with its duration, tokens, cost, and error. Prompts and evals use the same traces: change a prompt without a deploy, then run it on a dataset and compare the scores with production. Alerts tell you about errors, cost spikes, and agent loops. Connect with our TypeScript or Python SDK, or with the OpenTelemetry exporter that you already use.
Free Options
Launch Team
CloudflareDeploy serverless functions globally in seconds
Promoted
Hi Product Hunt, I'm Ephraim, one of the makers of telemetry.dev.
When an agent breaks in production, you want to know which step did it and what the run cost. telemetry.dev shows each run as one trace, with every model call and tool call as a span with its tokens, cost, and error.
New in this launch:
- Prompt management: change prompts without a deploy
- Evals: test a new prompt version on a dataset before it goes live
- Alerts when errors pile up or a run gets expensive
- A CLI and an MCP server, so your coding agent can read your traces
It's built on OpenTelemetry, with TypeScript and Python SDKs for OpenAI, Anthropic, the Vercel AI SDK, LangChain and more. The free plan needs no card, and the code PHBU7A47FH gets you 1 month of Pro.
What's the hardest part of debugging your agents right now?
absolute fan of @ephraimduncan's work. the maker of @blocks.so keeps cooking!
@fmerian thank you for the support🫡❤️
Congrats Ephraim. We run a remote MCP server, so the agent loop lives in the user’s Claude, not on our side. All we see are separate tool calls coming in. Can telemetry.dev group the calls from one conversation into a single trace from the server side, or does it have to sit inside the agent?
ive built ai apps and always felt I needed something like this to debug and improve my ai responses. dope launch
@soham_bharambe thank you, let me know if you need any help onboarding
Impressive tracing. Questions: does pricing scale for high-volume inference? Self-hosting option? How does this compare to vanilla OpenTelemetry sampling?
Swapping a prompt without deploying and scoring it against the same traces that came from production is the loop I have been faking with spreadsheets, and alerting on agent loops alone would pay for itself.