Viktor.comAn AI coworker that actually does the work
Promoted
Zro
Private inference for coding agents
687 followers
Private inference for coding agents
687 followers
Fast and optimized open-model inference on multi-region infrastructure with zero request retention.
@Emir Soytürk makes sense, TTFT and TPS are the two numbers that actually matter for an agent loop, not just raw throughput. curious if you've published any benchmarks against the big hosted providers or if that's still coming
Zro
@omri_ben_shoham1 As Moonmath AI we share each of our works transparently and cleanly! You can check the blogs out here: https://moonmath.ai/posts/
We are also exploring the ways to share real-time performance metrics of ZRO on the website. Stay tuned!
Streaming support for token-by-token responses would be huge for chat use cases, even if it means slightly more work on the caching layer. Right now waiting for full completions before anything renders feels dated for an inference API in 2026.
Zro
@frat8d8j Totally agree. ZRO already supports streaming through the OpenAI-compatible API.
If you saw a buffered response, let us know which client or integration you used!
private + coding agents, finally. how much latency do you give up to keep it private?
Zro
@doganakbulut You don’t have to trade one for the other! ZRO is optimized for fast, long-context coding-agent workloads while keeping zero data retention. Which model would you like to see on ZRO next?
Do you also support Claude? Would be interested if you supported their models.
Zro
@sai_xiong We are only serving open-weight models like GLM 5.2 and Minimax M3. Addition to that we have built-in support for Claude Code.
check integration page out for more information: https://zro.moonmath.ai/integrations
you can also add ZRO as custom provider next to your existing claude models so you can easily switch between them
If I specifically need Live Search, is that supported? On average, how much does a single search cost?
Zro
@natalia_iankovych We have Web Search MCP. Each Web Search costs you $0.02. You can see more about that on Pricing Page: https://zro.moonmath.ai/pricing
"Zero request retention" as a verifiable claim is the hard part of private inference, since the whole value proposition depends on users trusting that it's true without being able to audit it themselves. Is there a technical mechanism that makes this verifiable, like a transparency report, third-party audit, or something architectural like processing in a TEE that provides cryptographic guarantees, or is it currently a policy commitment rather than a technical one?
Congrats on #2. I want to poke at the intersection of two answers you gave in this thread. Zero request retention is the headline, but the cache discussion implies prompts are held somewhere at least briefly, since a prompt cache only speeds things up by remembering what was already sent. What is the actual lifetime and scope of that cache: per API key, per region, per tenant? And does zero retention mean zero after cache expiry? Not a gotcha, I think the honest version of this answer would make the privacy page stronger, because a security reviewer will ask exactly this.