New support for Bonsai 27B models (perfect for 24GB RAM!)
Quick follow-up from the Osaurus team.
On Tuesday, PrismML released Bonsai 27B, a big open source AI model compressed to a fraction of its normal size. A model like this usually needs 54 GB of memory, which means expensive cloud servers or a maxed-out workstation. This one fits in under 8 GB. That's small enough for a regular MacBook.
Why this matters: the models keep getting smaller while staying smart. Every time that happens, "AI that runs on your own computer" gets more real. No subscription, no account, your conversations never leave your machine. A year ago that meant settling for noticeably worse AI. That gap is closing fast.
The catch was that the standard tools on Mac can't actually run models compressed this way yet. So we did the conversion work ourselves and shipped it. If you tried Osaurus during our launch this week, Bonsai is in the model picker right now, running entirely on your Mac.
Osaurus: github.com/osaurus-ai/osaurus
Honest question for this community: if local AI catches up to cloud AI in quality, does anything keep you on a subscription?


Replies
Osaurus
Speed?
Osaurus
@chrismessina Super fast. 31 tok/s, and sub 1 sec TTFT. I'm on my 48GB MacBook Pro M4
I'm beginning to think some are a waste of money. Where's the best place to look at benchmarks and how well it will operate on my machine? I like the name. How did you make it work better locally too? What exactly do you change?