Inferock
An independent receipt for every LLM API call
Start new thread

Inferock Bench - An independent receipt for every LLM API call

by
•
Inferock-bench is a local proxy that sits between your app and OpenAI, Anthropic, Gemini, or OpenRouter shaped calls. It captures per-call token usage, failures, and retries, then generates an independent receipt showing what you were billed and how much you're actually overpaying for.

Add a comment

Replies

Best

The bill dispute question is interesting. have you personally managed to get a provider to credit a charge after showing them one of these per call receipts or is that still something you are testing?

Maker

 honest answer: no credit to brag about yet, that's exactly why we threw the question to the community. What we can stand behind today is the receipt itself, knowing which call failed and what it cost, instead of arguing from a monthly total.

Does anyone know of any tools that can work with web-based logins (claud.ai etc)

Maker

 we don't touch that layer. inferock-bench works where there's an API key and a baseURL to point somewhere, and web app subscriptions don't expose the per call detail we'd need. If someone has cracked that curious to see it too.

auditing your own inference bill is such an obvious gap 🔍 nice one. seeing much overbilling in the wild yet?

Maker
it really is one of those gaps hiding in plain sight. And yes, we see it, and it scales with your spend, every cut off answer and retry gets billed like a success, and at production volume that's money leaking with zero visibility. What bothers us most is that without your own records you can't even know how much it's costing you.

Retries are the part I’d put in giant font. ‘$0.04/call’ means nothing if a successful job secretly takes 6 calls. I want cost per accepted outcome.

Maker
Cost per accepted outcome is a really good way to put it. Today the receipt shows all six of those calls and what each one cost, so the secret multiplier at least stops being secret. But the roll-up per outcome is a good suggestion. We’ll try to incorporate in the following revisions!

Congrats on the launch of Inferock Bench.

Love the idea of having an independent receipt for every LLM API call. Transparent usage, billing, and overpayment tracking can be a huge win for teams working with multiple LLM providers.

Maker
Thank you so much! Multi-provider teams are exactly where things get messy fastest, a different bill from everyone and no shared way to check any of them. Hope it takes some of that pain off your plate.
First
Previous
123