Launching today
ngrok AI Gateway
One private gateway for every AI model
158 followers
One private gateway for every AI model
158 followers
ngrok AI Gateway provides one hosted gateway for every model: public providers, custom endpoints, and the models you run yourself. Use one key and one URL to route across OpenAI, Anthropic, and self-hosted models with observability, access control, and fallbacks built in. Your private models connect through ngrok’s network, so they sit beside hosted providers without being exposed to the public internet.
Payment Required
Launch Team / Built With
Ngrok
Hey Product Hunt 👋
I'm Niji, a product manager at ngrok. Today we're launching ngrok.ai, ngrok's AI Gateway.
For years, ngrok has helped developers connect their applications and services in minutes instead of days.
As I started building with AI, I ran into similar infrastructure problems at the application layer.
An application might start with OpenAI, then Claude for another use case. As newer, faster, or more affordable models became available, I would create more accounts and update my code just to try them. Eventually, more specialized needs would lead me to run fine-tuned or task-specific models on my laptop, private GPUs, or internal cloud infrastructure.
Before long, I was managing multiple gateways and SDKs, sharing provider keys across configuration files and vaults, checking usage in several dashboards, maintaining complicated fallback logic, and accidentally exposing models that were supposed to remain private.
If any of this sounds familiar, it is why we built ngrok.ai. It gives you one hosted gateway for managing models across providers, private infrastructure, and your own hardware.
One URL for every model
Getting started is simple. Point your SDK at https://gateway.ngrok.ai with your ngrok.ai access key, and begin routing requests to public providers, custom endpoints, and models you run yourself.
It works with all popular SDKs like OpenAI, Anthropic, and Vercel AI, so you can easily swap models and providers without rebuilding your entire application.
Aside from being a hosted AI Gateway, we enable you to:
Connect self-hosted models privately
Route to a model running on your laptop, local GPU, or private network without complex networking, opening inbound ports or dealing with IPs.
Build fallbacks into the gateway
Define a list of models and when a model or key fails, we will make another attempt or route the request to a healthy alternative.
Use credits to make requests
Leverage ngrok.ai to make requests against OpenAI, Anthropic, z.ai and more without having to create your own accounts with each provider.
Use your existing provider keys
Don't want to use our accounts? No worries, you can bring your own OpenAI, Anthropic, or custom provider keys that you already.
Control access by application or developer
Create separate access keys and decide which providers and models each one is allowed to call, and which keys each model should use, whether ours or yours.
See usage across your entire model stack
Track tokens, latency, errors, models, providers, and estimated cost in one place instead of piecing together several provider dashboards.
Manage everything through the dashboard or API
Set up gateways, keys, providers, access rules, and routing from your own tooling using our API or directly in the ngrok.ai dashboard.
Who we're building this for
ngrok.ai is for developers and platform teams that want the freedom to use the right model for each job without worrying about how to scale and maintain an ai gateway themselves and or taking on another infrastructure project every time their model strategy changes.
We're especially interested in hearing:
How are you routing between models today?
Are you running any models on your own infrastructure?
Which gateway features would make your AI stack easier to manage?
We'll be here throughout the launch to answer questions and hear what you think. Thanks for checking it out.
@nijikokun How do you see teams deciding when to route a request to a hosted model versus a self-hosted one, especially as cost, privacy, and latency tradeoffs change?
Tabstack by Mozilla
@ngrok is so back.
If you've ever needed to share a local app or demo something live, there's one thing that strikes about this product. Developers love it. I mean millions of developers - look at this wall of love - including people at @GitHub, @OpenAI, @Vercel, and much, much more.
@cassidoo put it simply: "ngrok saves me so much time during development. What a tool."
pumped to see them back [on Product Hunt]. S/O ?makers, already looking for your future launches.
Vamah.ai
@cassidoo @fmerian ngrok has been core in my dev setup for a very long time to test real world app scenarios during development. one gateway for every hosted model sounds interesting.
Tabstack by Mozilla
@gokuljd lfg! if you get a chance to give ngrok.ai a spin, make sure to add your review here: producthunt.com/products/ngrok-ai-gateway/reviews/new
Serand
How easy is it to switch traffic between providers if one model has highest latency or goes offline?
Tabstack by Mozilla
good q. @ngrok AI Gateway is designed for automatic failover. if one provider/model fails, times out, returns an HTTP error, or has a connection issue, the gateway automatically tries the next candidate instead of requiring your app to switch providers itself.
read the docs for more details on how it works: ngrok.com/docs/ai-gateway/how-it-works
hope it clarifies!
Managing different AI providers can get messy pretty quickly. A single endpoint with build in routing makes a lot of sense.
Premarket Bell
Having one gate way for both hosted and self hosted models sounds really useful. It feels like it could simplify a lot of infrastructure work.
Hedy AI
I like that private models can stay off the public internet while still working alongside hosted ones. That's a thoughtful approach.
How does ngrok AI Gateway help developers handle these issues compared to building their own layer? congrats team!