Launching today
ngrok AI Gateway
One private gateway for every AI model
653 followers
One private gateway for every AI model
653 followers
ngrok AI Gateway provides one hosted gateway for every model: public providers, custom endpoints, and the models you run yourself. Use one key and one URL to route across OpenAI, Anthropic, and self-hosted models with observability, access control, and fallbacks built in. Your private models connect through ngrok’s network, so they sit beside hosted providers without being exposed to the public internet.
Payment Required
Launch Team / Built With
Voquill
I like that you can start with your own provider keys and still get a single place to track usage, latency, and costs. Jumping between multiple dashboards gets old fast, so having that consolidated is a practical improvement.
ngrok AI Gateway
@henry_habib Yep yep, you have complete control over whether you bring your existing providers keys (BYOK) or use ngrok-managed ones. Either way, it all ends up on the same request/usage logs.
We actually just added more detailed breakdowns of usage data we were already collecting, so now it's really easy to see things like the average cost per request or per-provider failure rate. This is me just playing around with a few requests this morning (and having forgotten to bring my local Ollama model online before sending the request):
I've been looking for a better way to experiment across OpenAI and Anthropic without constantly rewriting integrations. This feels like it could make testing significantly faster.
ngrok AI Gateway
@sitara_ismail Totally, even a "simple" switch like Open AI<>Anthropic also means using different SDKs and dealing with different API shapes, and none of that's trivial to just make happen. It's all cognitive load and more code. There's also a playground in the dashboard that would let you do some freeform testing without having to edit your app at all until you're ready.
ngrok AI Gateway
@sitara_ismail Thanks for the support! Would love to hear what your testing for and how we could layer in some new features to make that dead simple.
Tabstack by Mozilla
yes! make sure to add your review here: producthunt.com/products/ngrok-ai-gateway/reviews/new
Hedy AI
I like that private models can stay off the public internet while still working alongside hosted ones. That's a thoughtful approach.
ngrok AI Gateway
@kate_sleeman Yup yup. Just knowing that no one random can access a self-hosted model on a URL like https://my-precious-model.internal is a huge W, and then you still get to configure access a bunch of different ways.
ngrok AI Gateway
@kate_sleeman as a bonus, you can monitor usage and requests of a self-hosted model using the same ui as the hosted alternative, providing a one-stop shop for all your models and inference!
I can definitely see this being for teams experimenting with different models. Have one plaace to manage everything sounds much easier. 👍
ngrok AI Gateway
@zeeshan_aslam2 Yeah, you can definitely experiment by changing the `model` key in your app where you're calling the inference providers, checking the output, and seeing what feels right.
We also have a playground where you can send the same prompt to multiple models and see the differences right away (plus status on TTFT and overall speed!): https://app.ngrok.ai/playground
TrackerJam
I'm curious how smooth the setup is for someone already using multiple AI providers. Is migration pretty straightforward?
ngrok AI Gateway
@maklyen_may we worked hard to make the setup as seamless as possible. we support the inference apis from anthropic and openai, so it should be a drop in replacement for most sdks.
here's our docs page on how to set that up: https://ngrok.com/docs/ai-gateway/overview#quick-example
ngrok AI Gateway
@maklyen_may @brenden_ehlers
native gemini API support and cross-API translation are some other things on our near term roadmap, so stay tuned for those features :)
Tabstack by Mozilla
yes, and it's worth mentioning this quick, 5-step migration guide to move from a good ol' endpoint-based setup to app.ngrok.ai and gateway.ngrok.ai.
read this: ngrok.com/docs/ai-gateway/guides/migrating-from-endpoints
hope it helps!
How does ngrok AI Gateway help developers handle these issues compared to building their own layer? congrats team!
Tabstack by Mozilla
ngrok AI Gateway
@hamza_afzal_butt
as with most software products, if you know how to write software and have the budget to host/distribute it, you can totally roll your own solution if your needs are simple.
similar to a traditional API gateway, if all you need to do is proxy requests, that's really easy to build your own solution. once you start getting into solving for things like rate limiting, observability & reporting, request manipulation, authentication, authorization, and role-based permissions, caching, building all these things in-house compared to picking an off the shelf solution is a decision that most teams have to decide if they'd rather pay the cost of a product or the cost of maintaining their own solution. we do the same thing internally with other tools. sometimes we write our own, other times we pay for a solution.
our goal with ngrok.ai is to provide one unified interface for any kind of LLM inference need and make it super easy to use while offering the depth to solve as many of the pain/friction points as possible. we hope that the benefits that people get from using it and the problems they don't have to think about solving themselves make it worth the cost to them. we're always on the lookout though for ways to keep improving it each week :)
What I appreciate is that this isn't just about hosted providers. I've worked on internal tools where the model had to stay inside our network for compliance reasons and every gateway I tried assumed everything sat behind a public API.
ngrok AI Gateway
@morgan_nabors We fully expect some kind of AI compliance reckoning, especially in Europe, as people start to hear more about the open models and neoclouds like RunPod or Lambda AI get more appealing. We hope to be right there with folks as they search for simple ways to try those out without changing how their apps behave!