ZenMux
An enterprise-grade LLM gateway with automatic compensation
760 followers
An enterprise-grade LLM gateway with automatic compensation
760 followers
ZenMux is an enterprise-grade LLM gateway that makes AI simple and assured for developers through a unified API, smart routing, and an industry-first automatic compensation mechanism.
Free Options
Launch Team / Built With
Render: Get up to $100K in startup creditsThe fastest way to launch and scale your startup in the cloud
Promoted
Automatic compensation is a bold promise. Love this angle. Congrats on launch 👏
@sandy_liusy Thanks! 🙏 The auto-compensation is the hook, but the real gold is — every payout is an edge case from your own business. Take those insights back as Context to reverse-optimize your product experience. That’s the data flywheel.
@sandy_liusy Thanks! We felt that infra providers shouldn’t only optimize throughput — they should stand behind output quality and reliability. That’s the bet we’re making.
Congrats on the launch! The model insurance angle is interesting, especially for production use cases where reliability matters more than raw capability. How do you objectively determine when an output qualifies as poor versus just subjective dissatisfaction?
@vik_sh Thanks for the great question! 🙏 We've built our own detection algorithm, and right now we have two dimensions live:
Unexpected content generation
High latency
How do we detect "unexpected content generation"? One example: if a user asks two consecutive questions with the same intent, we treat that as a signal that they weren't satisfied with the first response. That's one of the ways we identify bad cases.
The payout is just the outcome. The real value is: every flagged bad case is an edge case from your own business, ready to be used as context to improve your product experience.
That's where the data flywheel starts turning.
The insurance mechanism is a genuinely novel idea in the LLM gateway space. Most aggregators (OpenRouter, LiteLLM) treat themselves as dumb pipes.. you get your tokens, and if the model hallucinates or latency spikes, that's your problem.
I'm curious about the implementation like how does ZenMux detect "degraded quality" automatically? Is it running a lightweight evaluation model on every response, or is it based on heuristics like response length, latency thresholds, and known failure patterns?
The line between a genuine hallucination and a subtly wrong answer seems really hard to draw programmatically. Also, does the insurance payout data feed back into routing decisions? That would create a really interesting flywheel like the more claims you process, the smarter your routing gets
@haize_yu really cool! thank you for the answer! Good luck
Congrats on the launch, ZenMux team! A unified LLM gateway with smart routing is already valuable, but the “automatic compensation” angle is especially interesting — it’s rare to see reliability/quality guarantees treated as a first-class product feature. Curious how you define and measure “subpar results” (latency, hallucination rate, eval score, user feedback?) and what the compensation workflow looks like in practice.
@31xira Great question — and yes, we have algorithms to detect both latency spikes and content quality issues. When the conditions are met, compensation is triggered automatically.
That said, we're still iterating on the thresholds — this is our first version, and we're learning fast. Would love for you to stay tuned and share any feedback if you give it a try! 😊
@31xira Thanks, Angie! You asked the million-dollar question regarding our workflow.
We actually designed two core mechanisms: Passive and Active compensation.
Passive Compensation (Live now): This happens in the background without user intervention. We use our internal 'insurance mining' algorithms to detect qualifying failure patterns and issue credits automatically. However, purely automated detection naturally has a technical ceiling.
Active Compensation (Coming H1 this year): This is where we see the most potential. Developers can use the ZenMux SDK to embed feedback controls directly into their AI products. When an end-user flags a poor result (via a standard interaction flow), we evaluate it and route the compensation through you (the developer) directly to that end-user.
We believe this 'Active' approach will be the real game-changer for building trust!
Model insurance for AI infra? That’s new. Curious to try it.
@luke_pioneero Appreciate it! 🙏 You hit it — the model insurance is new, but honestly the best part is what comes with the payout: real edge cases from your own usage, ready to plug back in and make your product smarter.
Curious to hear what you think once you try it! 🚀
@luke_pioneero Thank you! We built it because we felt infra shouldn’t shift all risk to builders.
Multiple suppliers for the same model + auto failover = no more "our model provider is down" incidents.