Launched this week
Byte
Your local AI model or API key in a customizable llm chatbox
22 followers
Your local AI model or API key in a customizable llm chatbox
22 followers
Run Llama, Mistral, and other AI models locally for free, right on your machine. Or bring your own API key for GPT-5, Claude, and Gemini. One app, no subscription, no limits.
Would love to see a built-in model benchmarking tool so I can compare inference speed and memory usage across different models on my hardware before committing to one.
@emeltanaslan That is actually such a good idea! We will implement that in our next version.
Finally gave it a spin on my laptop and it pulled down a 7B Llama model way faster than I expected, ran completions smoothly without choking my fans. Nice to have a single place for local models and API keys without juggling five different apps.
@tahsin1326739 Thank you so much for trying it out. We are expecting updates soon with things like MCP Servers (Connections)
Finally a clean way to run Llama locally without fighting config files, and swapping in my own OpenAI key took like two seconds. Solid little app.
@leventkodaman Thank you so much!
A nice no-nonsense wrapper. One thing that would really help me is a built-in model benchmark panel that shows tokens/sec, VRAM usage, and first-token latency right in the chat window, so I can compare Llama and Mistral runs without alt-tabbing to the terminal.
@kriyexqxx That is a good idea. That will be on our mind when building the next version.
Love how simple this is for running local models without juggling terminals. One thing that would make it even better is built-in benchmark scoring, so you can see tokens per second and memory usage across different models side by side and know which one actually fits your hardware best.
@sezerkarakx9y4 A lot of people are asking for this so it will be in our next version!!
A model comparison view would be super useful, like a split screen where I can run the same prompt through Llama and Mistral side by side and see the responses next to each other. Would make picking the right model way easier.
@saadetsalerzv Quite a lot of people are asking for this so we will add this in our next update!
Local model support that just works without the usual Python venv nightmare is a real gift. Love that you didn’t bury it behind a subscription wall.
@yunusdemirsoy Github opensource projects have done so much, giving back to the community!