LLMonitor

👋 Hi Product Hunt, My name is Vince, and together with my friend Hugh we've been building LLMonitor, an open-source observability toolkit for builders of AI apps. If you’re using OpenAI or LangChain, the integration takes seconds to start recording data. Main Features: ⚡ Log full queries & errors 💸 Metrics into costs, tokens & latency 👨‍👩‍👧‍👦 Track users and their usage 💬 Record chat conversations & user feedback 🕹️ Prompt playground to iterate on prompts 🐾 Traces to easily debug agents 🧪 Evaluations of outputs quality 🧑‍💻 You can Self-host it on your own servers. Or use our cloud-hosted version (free up to 1k events /day.) I started building LLMonitor as an internal tool at my previous startup. We had no insight into how users interacted with our chatbots and needed a way to monitor chats and LLM queries easily. 
If you’re building a conversational app powered by AI, we’d love to hear what you think and how we can make it better! Also, we'd be reaaally happy if you can support us with a ⭐ star on GitHub: https://github.com/llmonitor/llm... Thankss 🙏

Congrats!!!

No-nonsense OSS LLM Monitoring tool - one of the best out there! Congratulations on the launch!

Simple yet powerful. Exactly what I needed.

I think this is a must have if you're developing LLM products, it's so easy to integrate and gives a lot of value to devs and also other team members. congrats again guys!

We've been using LLMonitor in our production app for quite some time now and the experience has been great. The team has been quick in assisting for any issues as well as considerate in taking inputs for their roadmap. Keep up the great work and congrats on the launch @vincelwt !

Congrats on the launch!