Launched this week
infai
Run local AI with profiles, not flags
1 follower
Run local AI with profiles, not flags
1 follower
Its an open-source terminal control plane for local AI inference. Find or import models, save reusable launch profiles for llama.cpp and vLLM, check hardware fit, and run multiple models with live logs, ports, tokens-per-second, CPU, RAM, and VRAM telemetry. One Go binary, one SQLite file, no cloud or telemetry.
Free
Launch Team
AWS Startup AdvisorPersonalized AWS guidance that meets you where you’re already building
Promoted
👋 Hey Product Hunt! I'm @dipankardas011 the maker of infai.
The problem: Every local inference engine — llama.cpp, vLLM — has its own dialect of flags. You end up scrolling through shell history, copying flags from READMEs, and praying you didn't forget --flash-attn . Your shell history is not a configuration system.
The fix: infai gives every engine one shape: the profile (model + engine + config). Define it once, launch it with one keypress, run multiple models concurrently, and watch live telemetry — all from a single terminal.
What's already shipped:
Support popular inference engine: llama.cpp and vLLM backends under the same workflow
Runs dashboard: with per-model status, uptime, and live TPS
Resource Telemetry: System + per-model CPU, RAM, VRAM telemetry
Hugging Face model imports: model search & download with resumable transfers
Hardware-fit checks before launch
11 terminal themes
One SQLite file, one Go binary, Apache-2.0
It's fully open source: github.com/dipankardas011/infai https://github.com/dipankardas011/infai
I'd love to hear what engines or features you'd want next. Fire away! 🔥