ReplayAutonomous app testing for teams who ship at AI-speed
Promoted
Launched this week
ARBR
Control Every AI Request
86 followers
Control Every AI Request
86 followers
AI stacks are getting more complex with more models, providers, costs, and decisions. ARBR gives your applications one control layer across your AI stack. Connect once through an OpenAI-compatible endpoint to route, govern, observe, evaluate, and deploy across AI models. Open source, MIT-licensed, provider-neutral, and self-hosted.
ARBR
Cost + performance is going to be a big challenge as AI usage grows for us. Routing different
tasks to different models seems like a pretty sensible approach.
ARBR
@shivani_bakhetia Exactly! That’s the problem we built ARBR around. Different tasks need different models, and routing them intelligently can make a big difference to both cost and performance.
I like the idea of having one layer to handle model routing instead of building all of this logic into every application.
ARBR
@amit_gyde Yes, Amit, having a dedicated layer gives you room to test and switch things around without making the app itself more complex.
Congrats! I like that ARBR focuses on the workload rather than pushing you toward a particular model. Different tasks obviously need different things.
ARBR
@nancy_dalal16 Thank you! We are glad you like our product.
shadow test, guarded canary, then rollback if quality drops. thats a lot of gates for one model swap. what actually trips the rollback, an eval score or a person
ARBR
@niveditha_patluri1
Automatic. Nobody gets paged to make the call.
The canary monitor recomputes candidate-vs-baseline metrics over a trailing 60-minute window every 5 minutes, and rolls back on the first guardrail breach. Four gates, with defaults:
error rate more than 2 points above baseline
p95 latency regressed over 25%
cost saving fell under 10%, so the swap stopped paying for itself
shadow worse-rate over 10%, from LLM-judged pairs on mirrored traffic
Only the last is an eval score. The other three are production telemetry. A 20-request minimum gates all of it, so a handful of bad requests won't trigger anything.
On breach the experiment flips to rolled_back, traffic returns to baseline on the next routing decision, and a webhook fires. You find out from the webhook.
Promotion works the other way. Going to 100% takes an operator hitting promote, recorded against their email.
It's one pipeline, configured once per swap. Shadow gives you the quality signal before any user sees the candidate, and the canary rechecks it on live traffic.
There is something great about pulling a scattered pile of AI tools back under one roof where you can genuinely keep an eye on the whole thing.