Launched this week
Auriko treats LLM providers as trading venues and arbitrages the spread. Built by ex-quant traders, Auriko’s cost-arbitrage engine calibrates to each user’s request patterns and selects optimized inference paths based on token price, cache behavior, latency, reliability, and request quality. Auriko benchmarks show average 30% cost reduction against industry peers and direct providers. See the source: https://www.auriko.ai/reports/llm-cost-arbitrage
Love the “trading desk for LLM calls” framing. Cost optimization across providers is becoming a real pain point as AI apps scale.
How do you balance cost savings with output quality and latency, especially for production workloads?
Auriko
@longway1 Thanks for the question Kevin! Our data engine actively tracks model performance - latency, throughput, reliability, etc. The routing engine computes a composite score for each available route.
Users can choose preset routing strategies depending on what matters most for a workflow, like cost, TTFT, or throughput. Power users can also set custom routing weights for more fine-grained control.
Can developer set their own priorities, like preferring lower latency over lower cost, or is the routing fully automatic?
Auriko
@lakeesha_weatherwax Definitely! Power users can fine-tune the routing strategy to their specific objective.
Auriko
@thys_beesman Thanks and great question! It's both.
The routing engine decides where the llm request should go before it is sent. For each request, Auriko builds the eligible provider set, applies hard constraints like capabilities, data policy, budget rules, parameter support, and availability, then scores the viable routes accounting for expected cost, latency, throughput, capacity headroom, and cache economics. The cost model accounts for provider-specific prompt caching mechanics and the workload’s reuse patterns, so routing is based on expected request-level economics, not just static model prices.
We then feed the request metadata back into our signal generation engine for future routing decisions. This helps us maintain a dynamic and effective awareness of the routing candidates.
How do you measure quality when routing across models? Do you use developer feedback, user behavior, retries, or some kind of evaluation layer?
Auriko
@phoenixhu We measure metadata such as latency, throughput, error rate, fallback, cache-hit and use that to inform our routing engine. And user can check all information in detailed request logs
Congrats on the launch! , Doesn't cheapest-per-request routing fight with caching though? If you hop providers to save on one call you lose the warm cache you built at the last one, and the next 20 calls cost more. Curious if the router accounts for that or just prices each call on its own.
Auriko
@irahimiam Very sharp question! In a nutshell, we model expected cost across the full session, not just the first request.
For example, a heavy coding-agent user with rapid-fire requests, large context, and long sessions will usually have a very different cache-hit profile from someone running periodic deep research. Auriko calibrates routing against each user’s usage pattern, and that pattern becomes an input to the expected cost computation. So in some cases, our routing engine may choose a provider with a deeper cache discount instead of choosing a provider with the lowest headline token price.
Triforce Todos
Congrats on the launch!
For teams running agents that have really strict latency requirements, can you set a hard ceiling on response time and let Auriko optimize cost within that constraint, or is it more of a balance between the two?
Auriko
@abod_rehman Definitely. There are two ways to control this in Auriko.
1. you can set hard constraints, such as a TTFT ceiling. 2. you can use softer control by increasing the weight of response time (TTFT) in the composite routing score. Both affect the routing decision!
The 30% cost reduction number, is that on top of what you'd already save by using OpenRouter or similar, or is that the comparison baseline?
Auriko
@boyuan_deng1 Yes if you are using another model aggregator or router, you will mostly likely see the cost reduction against it! It should be self-explanatory with some A/B test. Happy to help you set up a test to see the difference!