Hey all! We're excited to finally launch llmgateway.io on Product Hunt 🚀
A fully open-source Ai Gateway for your LLM needs, route, manage, and analyze your LLM requests across multiple providers with a unified API interface.
Since launch, LLM Gateway continued to grow, mostly through subreddits, X and LinkedIn. Now it's time to launch it on Product Hunt.
Luca and I (Ismail) are excited to launch LLM Gateway with the Product Hunt community with an exceptional offer of 50% off the pro plan forever! use Promo code: "PRODUCTHUNT"
◆ BYOK (Bring your own keys), use credits or go Hybrid mode (we support up to 11+ providers and over 60+ models)
◆ Configure caching to your preferences
◆ Activity stats: Keep track of each model usage, charts showcasing cost estimates per provider, request volume etc... ◆ Advanced activity: Detailed overview of each prompt, model, cost, provider, time, response metrics and cost information etc...) ◆ Self-hostable: Checkout the Docs
We've put a lot of time, effort & care into building LLM Gateway, and we honestly hope you like the product! Let us know if you have any feedback, we'll take your feedback and work on it! Thank you for your support!
i saw it is open sourced, so does it mean we can combine it with our own product and let users charge LLMs to your product to use or let them bring their own keys to use our own product without charging. And in both ways, they can manage the token usage in Gateway dashboard right??
@joey_zhu1 not sure I understand to be honest, right now you’re free to do whatever you want. we might restrict reselling llmgateway directly by charging users credits since that’s not the intention. but using the gateway as an end user will always stay free and open source
Report
Congrats on the launch! 🚀 This is interesting, does it have cost tracking as well to measure how much I'm spending?
@budb Thanks! the huge plus over openrouter is that LLM Gateway is open-source and can be self hosted, you can read more about the comparison here: https://llmgateway.io/compare/open-router
@smakosh This looks promising. Congrats on the launch! Quick question, how does the cost estimation work across different providers? Is it real-time or based on historical data?
@rachitmagon Thanks! The estimation works by the token count responses from the providers, and it updates in real time on the dashboard as every log comes with exact cost details. If a provider does not return token counts, we estimate it using the gpt tokenizer which gives cost estimates (so it may vary slightly and you will see for each log if it has been estimated or if it's exact).
LLM Gateway
Hey all! We're excited to finally launch llmgateway.io on Product Hunt 🚀
A fully open-source Ai Gateway for your LLM needs, route, manage, and analyze your LLM requests across multiple providers with a unified API interface.
Since launch, LLM Gateway continued to grow, mostly through subreddits, X and LinkedIn. Now it's time to launch it on Product Hunt.
Luca and I (Ismail) are excited to launch LLM Gateway with the Product Hunt community with an exceptional offer of 50% off the pro plan forever! use Promo code: "PRODUCTHUNT"
Here is a list of our top features:
◆ Simple usage overview dashboard (total requests, tokens used, cost estimate, organization credits, etc.)
◆ BYOK (Bring your own keys), use credits or go Hybrid mode (we support up to 11+ providers and over 60+ models)
◆ Configure caching to your preferences
◆ Activity stats: Keep track of each model usage, charts showcasing cost estimates per provider, request volume etc...
◆ Advanced activity: Detailed overview of each prompt, model, cost, provider, time, response metrics and cost information etc...)
◆ Self-hostable: Checkout the Docs
We've put a lot of time, effort & care into building LLM Gateway, and we honestly hope you like the product! Let us know if you have any feedback, we'll take your feedback and work on it! Thank you for your support!
AltPage.ai
i saw it is open sourced, so does it mean we can combine it with our own product and let users charge LLMs to your product to use or let them bring their own keys to use our own product without charging. And in both ways, they can manage the token usage in Gateway dashboard right??
LLM Gateway
@joey_zhu1 not sure I understand to be honest, right now you’re free to do whatever you want. we might restrict reselling llmgateway directly by charging users credits since that’s not the intention. but using the gateway as an end user will always stay free and open source
Congrats on the launch! 🚀 This is interesting, does it have cost tracking as well to measure how much I'm spending?
LLM Gateway
@ler_ws thanks! yep it tracks cost and shows total spending over time for each project you create, in the future this will be even more fine grained
LLM Gateway
@german_merlo1 thanks!
This looks great! It's something I wanted to exist to make integrating LLMs in my product easier.
Congrats on the launch!
Also, wanted to let you know that your Docs link in the main menu and in the footer point to http://localhost:3005/
LLM Gateway
@harund Issue has been fixed! Thanks a lot for the support and feedback
3Common
LLM Gateway
@budb Thanks! the huge plus over openrouter is that LLM Gateway is open-source and can be self hosted, you can read more about the comparison here: https://llmgateway.io/compare/open-router
Smoopit
@smakosh This looks promising. Congrats on the launch! Quick question, how does the cost estimation work across different providers? Is it real-time or based on historical data?
LLM Gateway
@rachitmagon Thanks! The estimation works by the token count responses from the providers, and it updates in real time on the dashboard as every log comes with exact cost details. If a provider does not return token counts, we estimate it using the gpt tokenizer which gives cost estimates (so it may vary slightly and you will see for each log if it has been estimated or if it's exact).
Smoopit
@steebchen Smart approach, Real-time cost tracking is crucial.