Relay
Reliable delivery layer for LLM APIs
Start new thread
Alikhan Kenzhebaev

Relay - Reliable delivery layer for LLM APIs

by•
Drop-in SDK that adds auto retry, provider failover (Anthropic ↔ OpenAI), and smart caching to every LLM call. One line keeps your AI agents online even when providers don't. BYOK, edge-native, $0 to start.

Add a comment

Replies

Best
Alikhan Kenzhebaev
Maker
📌
Hey Product Hunt! Alikhan here, solo founder of Relay. I've shipped four AI products over the last two years and each one needed the same reliability code on top of the LLM API - retry, backoff, provider failover when Anthropic 529s, caching for repeat prompts. I rewrote that loop four times, each time with a different bug. The fourth time, I admitted it: this is a product, not a yak. Relay is that code, but reliable and importable: Auto retry with decorrelated jitter Provider failover - Anthropic ↔ OpenAI with model mapping Smart caching (exact + semantic) Live dashboard for every request — latency, cost, retries BYOK - your keys, encrypted, your bill stays with the provider Think Resend, but for LLM APIs. One line replaces @anthropic-ai/sdk, and your agent stays up when providers don't. SDK is open-source (MIT) at github.com/relay-llm/sdk. Early waitlist signups get 6 months of Pro free. Would love your feedback: 1. Does the "one-line failover" positioning resonate? 2. Hand-rolling retry code in your AI projects today? 3. What providers should I add after Anthropic + OpenAI? Roast me - better here than in prod)