Launching today
Inferock Bench
An independent receipt for every LLM API call
665 followers
An independent receipt for every LLM API call
665 followers
Inferock-bench is a local proxy that sits between your app and OpenAI, Anthropic, Gemini, or OpenRouter shaped calls. It captures per-call token usage, failures, and retries, then generates an independent receipt showing what you were billed and how much you're actually overpaying for.
Congrats on the launch @himashwetha_gowda. Good find @fmerian.
Question regarding accuracy, is it possible it might not be able to distinguish a genuinely billable provider failure from valid hidden token usage, such as reasoning, refusal, cache, or tool-call tokens?
@rohanrecommends thank you! Anything that could be legit hidden usage like reasoning, cache, or tool tokens doesn't count as loss, it just gets flagged for you to check. Only what we can prove makes the loss number.
This solves a problem I didn't know I could solve, I always assumed API billing discrepancies were just the cost of doing business. Turns out I was wrong.
Thanks @itohan_blessing_eigbadon , honestly, this is exactly what we told ourselves for months, until we actually looked at the per call receipts and couldn't unsee it. Really glad this clicked for you. Pls run it on your own traffic, come back and tell us what your receipts show, genuinely curious!
I like that it works as a local proxy. Keeping billing and usage data on the developer's machine feels like a thoughtful design choice.
@awesome_america Thank you! That part was non negotiable for us, it's your spend and your keys, so the record should live on your machine, not on ours.
I like the positioning here, it's not trying to replace the provider's billing system, just verify it. That's a smarter pitch than "cost optimization," which every tool claims.
Inferock Bench
@ayla_reynolds Thank you, that's how we think about it as well. Your bill already exists, you should just be able to check it.
I've been burned by silent retries inflating my OpenAI bill before. Having an independent receipt for that would've saved me a painful invoice conversation.
Inferock Bench
@ramish_saje Ugh, the invoice conversation where you both know something's off. Sorry you've lived that one. Next time at least you'd be the one holding the itemized version.
The independent receipt idea is genuinely useful. LLM costs are still surprisingly hard to audit at the individual request level.
@ashir_murtaza1 thank you! That's exactly the gap we kept running into, you can see the total, but never the audit at the individual request level. Glad it resonates.
@etiennegarcia Thank you! The retries and weird token counts are the bits that never show up on the bill. Really appreciate the kind words on launch day!