VeloxQuant-MLX
Run bigger local LLMs in less memory, on Mac
6 followers
Run bigger local LLMs in less memory, on Mac
6 followers
VeloxQuant lets you run bigger local AI models in less memory, fully private with no cloud required. One simple API compresses memory usage up to 16x while keeping generation fast on Apple Silicon.
Free
Launch Team
Render: Get up to $100K in startup creditsThe fastest way to deploy and scale apps and agents
Promoted
@rajveer_rathod This looks amazing. I noticed that most of the models in your benchmarks are smaller/older models. Will this show improvement with much larger models like the Qwen 3.8 line?
@clickbrain Great question. That’s actually one of the biggest things I want to validate next. Most of the current VeloxQuant benchmarks are on smaller models because they’re easier to test consistently across local Apple Silicon hardware.
For much larger/newer models like the Qwen 3.8 line, KV-cache memory and memory bandwidth become even more important, so there’s definitely potential for compression to have a bigger practical impact. But I don’t want to claim that until we have solid numbers.
I’m working on expanding the benchmarks to newer and larger models now. I’d also love to get more community contributors testing VeloxQuant across different Macs and model sizes so we can publish much more comprehensive results.