Launching today
Auriko
Trading desk for LLM calls
226 followers
Trading desk for LLM calls
226 followers
Auriko treats LLM providers as trading venues and arbitrages the spread. Built by ex-quant traders, Auriko’s cost-arbitrage engine calibrates to each user’s request patterns and selects optimized inference paths based on token price, cache behavior, latency, reliability, and request quality. Auriko benchmarks show average 30% cost reduction against industry peers and direct providers. See the source: https://www.auriko.ai/reports/llm-cost-arbitrage
Auriko
Is Auriko mainly about monitoring and comparing LLM calls after the fact, or does it help decide where a call should go before it is sent? For developer teams, that distinction matters a lot, especially if they’re juggling quality, latency, and spend across different AI workflows.
Auriko
@crystalmei Great question! Definitely pre-request.
When a request hits Auriko, we build the available routing candidate set, apply hard constraints like capabilities, budget, data policy, parameter support, and availability. The routing engine scores every available candidate i across cost, latency, throughput, and success rate, then picks the best one based on your strategy
The request performance data feeds back into routing: we use it to generate provider health and performance signals, then use those signals to calibrate future routing decisions. So the main value is real-time routing, with observability data used to make the router smarter over time.
The prompt caching focus sounds valuable. A lot of teams know caching exists but do not optimize around it.
Auriko
@yolo_xiao Exactly. Caching is easy to know about but hard to price correctly, because every provider handles prompt caching differently, and cost is also a function of the user’s request pattern.
If we had a crystal ball and knew the exact content and timing of a user’s future requests, as well as each provider’s exact caching mechanism, we could compute the cost of the same workload across providers and pick the cheapest path. In reality, the problem is much harder because the information is incomplete. The user’s request pattern has to be estimated, and each provider’s prompt caching mechanism has to be probed and modeled correctly. That is where Auriko’s quant-trading-inspired infrastructure comes in, and how we are able to drive around 30% cost reduction.
quant background makes sense for this, arbitrage is fundamentally about finding mispriced spreads and providers pricing caching differently is exactly that. the tension I'd want to understand: prompt caching usually rewards staying on the same provider for a session so the cache stays warm, but a router optimizing per-request could bounce a session across providers chasing the best price each time and never let any single cache warm up. does the routing engine account for cache-state as its own signal, like "this provider already has a warm cache for this context, don't move away from it even if a competitor is nominally cheaper this instant"
Love the “trading desk for LLM calls” framing. Cost optimization across providers is becoming a real pain point as AI apps scale.
How do you balance cost savings with output quality and latency, especially for production workloads?
Auriko
Auriko’s core competence is our quant-trading-grade data pipeline, signal generation engine, and inference cost modeling. We track each inference provider’s prompt-caching mechanics, estimate users’ request patterns, and model inference cost with both provider and user signals in mind.
Our data pipeline also generates real-time signals on provider health, latency, and throughput.
Triforce Todos
Congrats on the launch!
For teams running agents that have really strict latency requirements, can you set a hard ceiling on response time and let Auriko optimize cost within that constraint, or is it more of a balance between the two?