We replaced a $5k/year LLM observability tool with our own 🙂

PH Community!

Last week we built us a tool to see exactly what our LLM calls cost.

We were paying ~$5k/year for a different tool that did this, but it was not working so well for us. So we just built it.

Now we can see:

→ cost per LLM call

→ which user costs the most

→ latency

→ where it fails

→ limit per user

→ down to a single request, up to any timeframe

We might open source it because it's super helpful. Basically I want to make sure no user will ever bankrupt us just by using our product. And I want to be able to debug any LLM issue a user had in a few minutes.

Who wants to try it??