All activity
Tracing + evaluation in one open-source tool. LangSmith is closed-source. Langfuse is overcomplicated. Most logging tools lack built-in eval. Auditi combines all three. 2-line auto-instrumentation captures all OpenAI, Anthropic & Google API calls. 7+ LLM-as-Judge evaluators run automatically on traces. Human annotation workflows when AI judges aren't enough. Real-time cost tracking. Turn production traces into fine-tuning datasets. Self-host with docker compose up. Python SDK, FastAPI, React.
AuditiOpen source AI agents observability and evaluation
Dedy Ariansyahleft a comment
Hey Product Hunt! 👋 I'm Dedy, the creator of Auditi. I built this while working on AI agents in production. The hardest part wasn't building the agents — it was knowing if they were actually working well and when/where they are not performing as expected. I tried the existing tools: Langfuse (too complex), LangSmith (closed source, vendor lock-in), and various logging tools (zero evaluation)....
AuditiOpen source AI agents observability and evaluation