Launched this week
Auriko treats LLM providers as trading venues and arbitrages the spread. Built by ex-quant traders, Auriko’s cost-arbitrage engine calibrates to each user’s request patterns and selects optimized inference paths based on token price, cache behavior, latency, reliability, and request quality. Auriko benchmarks show average 30% cost reduction against industry peers and direct providers. See the source: https://www.auriko.ai/reports/llm-cost-arbitrage
How does Auriko handle providers with different caching rules? Some make caching easy to reason about, while others expose less detail. Does Auriko normalize all of that for developers?
Auriko
@withshawn Thanks for the comment! Our data engine tracks and tests provider-specific behavior like token thresholds, block granularity, read/ write pricing, expiration windows, and pricing tiers.
Auriko also handles the provider-specific steps needed to activate caching where possible, like cache directives, cache keys, or session affinity. Developers still see a normal OpenAI-compatible API and consistent cache usage/saving fields in the response.
The prompt caching focus sounds valuable. A lot of teams know caching exists but do not optimize around it.
Auriko
@yolo_xiao Exactly. Caching is easy to know about but hard to price correctly, because every provider handles prompt caching differently, and cost is also a function of the user’s request pattern.
If we had a crystal ball and knew the exact content and timing of a user’s future requests, as well as each provider’s exact caching mechanism, we could compute the cost of the same workload across providers and pick the cheapest path. In reality, the problem is much harder because the information is incomplete. The user’s request pattern has to be estimated, and each provider’s prompt caching mechanism has to be probed and modeled correctly. That is where Auriko’s quant-trading-inspired infrastructure comes in, and how we are able to drive around 30% cost reduction.
Agnes AI
@yolo_xiao Caching is exactly one of the most important factor in model routing - nice catch!
The quant-desk framing lands, and Michael's point that spend hides in cache pricing and routing more than headline token price matches what bit me. Where I'd push: Ridhwik already asked the quality-floor question, and "optimized without compromising quality" can't be the real answer. I run an LLM backend where the output is the product, and two models at the same token price diverge hardest on the one axis a price benchmark never sees — by the time a complaint tells me, the bad output already shipped. So the mechanism I want to understand: can I define what "request quality" means per route — my own golden set or judge — or is it one internal score you calibrate? Whose definition the router optimizes against is the whole risk surface for anything customer-facing.
This is amazing. Love the concept. Thinking of giving this a go but without signing I can't find quantisation of the models. Also a question for you: whats your process if a provider you have on there suddenly swaps to a different quantisation? Can they do it without notice and do you have fail safes for that? I got burnt a little on OpenRouter where a provider I was using swapped to a lower quantisation and I didn't know about it until things started failing. Now I just pin it 3 levels deep to different providers as a fail safe for me.
The trading desk analogy is compelling but trading desks also have slippage, the cost of execution diverging from the expected price. What's the equivalent for LLM routing, like how often does Auriko route to a provider that then has a latency spike or quality degradation that negates the cost saving, and is there a real-time feedback loop that reroutes mid-session or only adjusts for future requests?
A trading-desk framing for LLM calls makes sense. Once teams have more than one model and more than one workload, the real work becomes routing, cost control, and knowing why a call behaved the way it did. The audit trail matters as much as the cheaper token path.
Agnes AI
@krekeltronics Definitely - audit trail just assures you everything is transparent!
Auriko
Auriko’s core competence is our quant-trading-grade data pipeline, signal generation engine, and inference cost modeling. We track each inference provider’s prompt-caching mechanics, estimate users’ request patterns, and model inference cost with both provider and user signals in mind.
Our data pipeline also generates real-time signals on provider health, latency, and throughput.