BaseRT
Fastest LLM runtime on Apple Silicon
264 followers
Fastest LLM runtime on Apple Silicon
264 followers
Running AI locally means no per token cost + full privacy. BaseRT is the fastest LLM runtime on Apple Silicon. Install it with one command and run local models on your own device.
This is the 2nd launch from BaseRT. View more
BaseRT
Launched this week
BaseRT is the fastest LLM runtime on Apple Silicon. Install it with one command and run local models on your own device.
Free
Launch Team
BaseRT
curious how much of this gain is specific to M5's tensor cores vs the general Metal 4 tensor API. I'm running an M4 Pro, plenty of people doing local inference haven't upgraded yet - does BaseRT still beat llama.cpp/MLX meaningfully on M4, just with a smaller multiplier, or is most of the win locked to the new hardware?
BaseRT
@galdayan we’ve optimised for all M series chips. Check out our technical reports, the first one has the benchmarks for previous M series chips you’re looking for. Here https://www.basecompute.co/research
@lukas_base will do, thanks - checking the report now. good to know the mini isn't left behind on this one, appreciate the quick answer
BaseRT
@galdayan you're welcome! We running a discord for on-device and open-source AI if you have more questions https://discord.gg/wQF5vQ3jpe
@lukas_base will hop in there, thanks. good way to keep the questions out of the comment section for anything more detailed
Love how simple the install is and the speed on my M2 is genuinely impressive. One thing that would make this even better is a built-in model benchmark tool so you can see tokens/sec for different models on your specific hardware right from the CLI.
BaseRT
@ag_rbas9241 it's already included, check out https://docs.basecompute.co/
One thing I'd love to see is a simple built-in model downloader so I can grab and switch between popular open weights from the CLI without hunting for GGUF files manually. That would make testing new models way smoother.
BaseRT
@adem1343093 it's already included! have a look at the download section in the docs https://docs.basecompute.co/
the no-per-token part matters as much as the speed imo, it's what makes always-on background agents affordable to run. is that the direction you're headed?
BaseRT
@andrewzakonov yes! All you can eat tokens :)
Very interesting work. It feels like we're finally seeing software catch up to what Apple Silicon has been capable of for a while.
One thing I'm curious about: if an application needs to switch between different models depending on latency or quality requirements, does BaseRT help manage that orchestration, or is that intentionally left to the application layer?
BaseRT
@nextmark that feature is in the making :) But you can already load multiple models
6.4x faster than llama.cpp is a big claim — what's the tradeoff? Curious whether it's fast enough to run a real-time voice pipeline on-device (where every 100ms of latency is felt), or if the win is mainly on batch/throughput. Running local with no token cost + full privacy is a huge deal for small-business use cases. Following.
BaseRT
@david_marko Check out our technical papers, we compare for all M series chips and a whole suite of models https://www.basecompute.co/research