ARA - Give AI systems a memory of every decision they make
by•
AI systems make decisions, then forget everything: the exact inputs, the model version, the context. ARA is an infrastructure layer that records every decision your ML models make with the exact state that produced it, and can replay any of them months later. Incident debugging becomes a query. Drift shows up per entity. Training data joins without label leakage. Free Community Edition, runs entirely on your own infrastructure in under 10 minutes. SDKs on PyPI and Maven Central.
Replies
Set this up in our staging env over lunch and the replay query on a flaky batch job pointed straight at the offending model version. Self-hosted install was painless and the PyPI SDK felt like a normal client, not a framework I had to wrap my head around.
@elanurince86376 Glad the install was smooth.
the EU AI Act timing detail is a nice touch, that's the kind of context that makes a launch post credible instead of generic. question on the mechanics: for a high-QPS serving path (say fraud scoring or ad ranking at thousands of req/s), what's the actual latency/throughput hit of that inference-time write, and is it synchronous or does it get queued off the hot path?
@galdayan Great question, and it splits by mode.
Ledger mode (ARA alongside your stack): ARA is only on the write side of the hot path, reads still come from your cache. The write is in the request path but fire and forget: serialize, hand off to an already open connection, no ack wait. Tens of microseconds client side. The part I care most about is the failure mode: if ARA slows or dies, the client logs and drops, it never blocks scoring. Worst case under duress is a gap in the record, never added latency. The honest cost: a hard kill can lose the last unflushed writes.
Serving mode (ARA as the feature store): now reads are also on ARA, so it is latency critical, which is why I tell people to stage into it with a cache fallback rather than start there. Reads hit an append only in-memory temporal index, sub microsecond at the index, your network hop dominates end to end.
On sustained throughput, the attached dashboard is from the published benchmark run: Apple silicon laptop, five workers, the Java SDK's throughput test client, methodology in the docs. Write bursts peak above 1.2M ops/sec, reads past 500K, error panel flat at zero. I quote that as burst, not sustained: on a single append only node, sustained is really a provisioning question, volume times retention sizes your memory, and the server exposes history depth controls for exactly that. Against thousands of req/s, that envelope leaves roughly three orders of magnitude of headroom, which is why I am comfortable with the hot path claim while still making no ad ranking scale promises until someone proves them.
One disclosure: the benchmark run used an enterprise license because community throughput quotas would trip at stress test rates. The free tier (2M recorded decisions/month, single node) is sized for real workloads, not benchmarks.
Download is free if you want to check any of this on your own hardware.
@tushar_haldar This clears it up, I was picturing every write as latency critical, but the ledger vs serving distinction makes it obvious that's only true once you actually flip to using it as the feature store. The fire and forget failure mode (gap in the record, never added latency) is exactly the tradeoff I'd want. Also respect you disclosing the enterprise license behind the benchmark instead of just quoting the number, most launch posts don't bother with that caveat.
@galdayan Thanks Gal. That staging is the designed adoption path: ledger mode first, so the record earns trust before it earns the read path.
@galdayan That motivated me further, now the website hosts full benchmarks. https://aralabs.ai/benchmarks.html
Replay is cool, but a built-in diff view comparing two specific model versions side by side on the same input would save so much time during debugging. Right now I'd have to script that comparison myself across snapshots.
@erva914472 The Diff tab does part of this today: pin two moments on an entity's timeline and it diffs the exact served values, with model version changes marked. Version-vs-version on the same input is a good extension, added to the list. Thanks.
the replay-any-decision-months-later thing is genuinely clever, especially pairing it with per-entity drift tracking. most observability tools stop at aggregate metrics so this feels like it was built by people who actually debugged a model at 2am.
@turgayunlub2ws Thanks
The "forgot everything I told it yesterday" pain is real, I work with coding agents across long-running projects and end up maintaining markdown context files by hand just so each session doesn't start from zero. Curious how recall works in practice: does the agent decide what to remember/retrieve on its own via the MCP tools, or do I control what gets stored? The failure mode I'd worry about is it confidently recalling stale info after the project has moved on.
@cannetjam Small correction so I don't oversell: ARA is not agent memory in the context-recall sense. It records the decisions an ML system makes, with the exact state that produced each one, so they can be replayed and audited later. Nothing is 'recalled' into future decisions unless you query it, which also means nothing stale leaks in.
The replay feature sounds like a lifesaver for debugging those weird edge cases that pop up months later. One thing that would really round this out for me: a visual diff tool to compare two recorded decisions side by side, showing exactly which inputs or model state differed. Would make root-causing regressions way faster than reading through logs.