Viktor.comAn AI coworker that actually does the work
Auriko treats LLM providers as trading venues and arbitrages the spread. Built by ex-quant traders, Auriko’s cost-arbitrage engine calibrates to each user’s request patterns and selects optimized inference paths based on token price, cache behavior, latency, reliability, and request quality. Auriko benchmarks show average 30% cost reduction against industry peers and direct providers. See the source: https://www.auriko.ai/reports/llm-cost-arbitrage
The quant-desk framing lands, and Michael's point that spend hides in cache pricing and routing more than headline token price matches what bit me. Where I'd push: Ridhwik already asked the quality-floor question, and "optimized without compromising quality" can't be the real answer. I run an LLM backend where the output is the product, and two models at the same token price diverge hardest on the one axis a price benchmark never sees — by the time a complaint tells me, the bad output already shipped. So the mechanism I want to understand: can I define what "request quality" means per route — my own golden set or judge — or is it one internal score you calibrate? Whose definition the router optimizes against is the whole risk surface for anything customer-facing.
This is amazing. Love the concept. Thinking of giving this a go but without signing I can't find quantisation of the models. Also a question for you: whats your process if a provider you have on there suddenly swaps to a different quantisation? Can they do it without notice and do you have fail safes for that? I got burnt a little on OpenRouter where a provider I was using swapped to a lower quantisation and I didn't know about it until things started failing. Now I just pin it 3 levels deep to different providers as a fail safe for me.
The trading desk analogy is compelling but trading desks also have slippage, the cost of execution diverging from the expected price. What's the equivalent for LLM routing, like how often does Auriko route to a provider that then has a latency spike or quality degradation that negates the cost saving, and is there a real-time feedback loop that reroutes mid-session or only adjusts for future requests?
A trading-desk framing for LLM calls makes sense. Once teams have more than one model and more than one workload, the real work becomes routing, cost control, and knowing why a call behaved the way it did. The audit trail matters as much as the cheaper token path.
Agnes AI
@krekeltronics Definitely - audit trail just assures you everything is transparent!
Auriko
Auriko’s core competence is our quant-trading-grade data pipeline, signal generation engine, and inference cost modeling. We track each inference provider’s prompt-caching mechanics, estimate users’ request patterns, and model inference cost with both provider and user signals in mind.
Our data pipeline also generates real-time signals on provider health, latency, and throughput.
This is a smart wedge most teams are eating unnecessary inference cost simply because provider selection is usually a one time decision baked into the code rather than something dynamic. A 30% reduction is meaningful at scale. Would love to know how request quality is scored in your benchmarks, and whether the savings hold up for latency sensitive production workloads or mainly batch use cases. Excited to see this evolve bookmarking for our team's eval.
Smart angle. LLM costs are getting complex fast when you're routing between multiple providers and models. How do you handle latency tradeoffs when optimizing for cost? Sometimes the cheapest call isn't fast enough for real-time use cases.