BaseRT
Fastest LLM runtime on Apple Silicon
269 followers
Fastest LLM runtime on Apple Silicon
269 followers
Running AI locally means no per token cost + full privacy. BaseRT is the fastest LLM runtime on Apple Silicon. Install it with one command and run local models on your own device.
This is the 2nd launch from BaseRT. View more
BaseRT
Launched this week
BaseRT is the fastest LLM runtime on Apple Silicon. Install it with one command and run local models on your own device.
Free
Launch Team
BaseRT
Tindog
curious how much of this gain is specific to M5's tensor cores vs the general Metal 4 tensor API. I'm running an M4 Pro, plenty of people doing local inference haven't upgraded yet - does BaseRT still beat llama.cpp/MLX meaningfully on M4, just with a smaller multiplier, or is most of the win locked to the new hardware?
BaseRT
@galdayan we’ve optimised for all M series chips. Check out our technical reports, the first one has the benchmarks for previous M series chips you’re looking for. Here https://www.basecompute.co/research
@lukas_base will do, thanks - checking the report now. good to know the mini isn't left behind on this one, appreciate the quick answer
BaseRT
@galdayan you're welcome! We running a discord for on-device and open-source AI if you have more questions https://discord.gg/wQF5vQ3jpe
@lukas_base will hop in there, thanks. good way to keep the questions out of the comment section for anything more detailed
Love how simple the install is and the speed on my M2 is genuinely impressive. One thing that would make this even better is a built-in model benchmark tool so you can see tokens/sec for different models on your specific hardware right from the CLI.
BaseRT
@ag_rbas9241 it's already included, check out https://docs.basecompute.co/
One thing I'd love to see is a simple built-in model downloader so I can grab and switch between popular open weights from the CLI without hunting for GGUF files manually. That would make testing new models way smoother.
BaseRT
@adem1343093 it's already included! have a look at the download section in the docs https://docs.basecompute.co/
the no-per-token part matters as much as the speed imo, it's what makes always-on background agents affordable to run. is that the direction you're headed?
BaseRT
@andrewzakonov yes! All you can eat tokens :)
Very interesting work. It feels like we're finally seeing software catch up to what Apple Silicon has been capable of for a while.
One thing I'm curious about: if an application needs to switch between different models depending on latency or quality requirements, does BaseRT help manage that orchestration, or is that intentionally left to the application layer?
BaseRT
@nextmark that feature is in the making :) But you can already load multiple models