Launching today

oMLX

Launching today

Mac LLM server that cuts agent wait times from 90s to 5s

24 followers

Mac LLM server that cuts agent wait times from 90s to 5s

24 followers

Visit website

AI Infrastructure Tools

•

LLM Developer Tools

oMLX turns your Mac into a full LLM inference server, run from the menu bar. It serves text, vision, OCR, embedding and reranker models with continuous batching, plus a RAM+SSD tiered KV cache that survives restarts, so Claude Code and Cursor respond in about 5s instead of 90s. OpenAI and Anthropic compatible APIs drop straight in. Native Swift, not Electron. Apache 2.0, open source.

Free

Launch tags:Open Source•Developer Tools•Artificial Intelligence

Launch Team

Viktor.comAn AI coworker that actually does the work

Promoted

Hunter

📌

Kept hitting the same wall with local models: the agent loops back, the whole conversation recomputes, 90 seconds of nothing. oMLX writes the KV cache to SSD. Old context comes back in milliseconds, even after a restart. Claude Code on a local model stops feeling like dial-up. Jun has been shipping this almost daily since February. 21k stars, and his Show HN still barely got seen. Felt wrong. If you run models on a Mac: what does your stack look like? Curious what people pair this with.

Report

14h ago

RunEvr

@rabnoor_s I guess some of the more tech descriptions are hard for me to fully grasp, but I definitely caught the idea of a local AI server and cutting response time down to 5s instead of 90s =D

Sounds great!

Congrats on the launch, and lots of success.

Report

31m ago