Launching today
Inferock Bench
An independent receipt for every LLM API call
117 followers
An independent receipt for every LLM API call
117 followers
Inferock-bench is a local proxy that sits between your app and OpenAI, Anthropic, Gemini, or OpenRouter shaped calls. It captures per-call token usage, failures, and retries, then generates an independent receipt showing what you were billed and how much you're actually overpaying for.
Inferock Bench
Hey PH
We built inferock-bench because we kept paying for AI answers that died mid sentence, and nobody could tell us where the money went.
Providers give you totals. They don't give you the per call receipt you'd need to prove which answer broke, which retry ran, or which token count changed. The company that charges you also decides what counts as a failure and keeps the only detailed records.
inferock-bench runs locally as a proxy in front of OpenAI, Anthropic, Gemini, OpenRouter shaped calls. Point your existing SDK at it (change two settings: apiKey and baseURL), and it captures every call as an independent, per call record. Your provider key never touches our servers, it's used locally only, attached to provider requests.
What it catches:
- Answers cut off mid stream that still got billed
- Empty replies with billed tokens attached
- Token counts that don't match visible output
- Retries that may have silently doubled a charge
- Cache discounts you may be missing on your invoice
Every run reports a receipt: spend observed, bill-bounded money loss, time loss, and a separate "invoice-check exposure" line that never gets summed into money loss, because we don't want a louder headline at the cost of a weaker claim.
Run it in about a minute: npx inferock-bench
Its open source (FSL-1.1-Apache-2.0, converts to Apache-2.0 in 2 years).
Question for this community: has anyone here actually disputed an AI provider bill and gotten a credit? What worked?
Overpaying for failed calls has quietly cost me more than I'd like to admit. I've had timeouts that still consumed tokens on the provider's side and no clean way to catch that pattern until my bill arrived. If this surfaces failure-related overspend specifically, not just total usage, I'd trust the numbers more than any provider dashboard.
This solves a problem I didn't know I could solve, I always assumed API billing discrepancies were just the cost of doing business. Turns out I was wrong.
Thanks @itohan_blessing_eigbadon , honestly, this is exactly what we told ourselves for months, until we actually looked at the per call receipts and couldn't unsee it. Really glad this clicked for you. Pls run it on your own traffic, come back and tell us what your receipts show, genuinely curious!
@etiennegarcia Thank you! The retries and weird token counts are the bits that never show up on the bill. Really appreciate the kind words on launch day!
Buffup.AI
The retry tracking caught my eye. silent retries are probably one of the easiest ways for API costs to creep up without anyone noticing.
@sansa_grey Right, retries are the sneakiest of the bunch: the answer still arrives, everything looks fine, and the bill just grows. Making those visible was one of the first things we wanted for ourselves.
I like that it works as a local proxy. Keeping billing and usage data on the developer's machine feels like a thoughtful design choice.
@awesome_america Thank you! That part was non negotiable for us, it's your spend and your keys, so the record should live on your machine, not on ours.
The independent receipt idea is genuinely useful. LLM costs are still surprisingly hard to audit at the individual request level.
@ashir_murtaza1 thank you! That's exactly the gap we kept running into, you can see the total, but never the audit at the individual request level. Glad it resonates.