Launching today
mlx-serve
Local AI on Apple Silicon: LLMs, image/video gen, agents
5 followers
Local AI on Apple Silicon: LLMs, image/video gen, agents
5 followers
Native Zig inference server for Apple Silicon — no Python, no conda. One binary. 35%+ faster decode than LM Studio on Gemma 4 E4B 4-bit. Drop-in replacement for Ollama (/api/chat, /api/generate), plus full OpenAI and Anthropic APIs on the same port. Beyond chat: local image gen (FLUX.2 + Krea-2), video gen with audio (LTX-Video), voice cloning (Qwen3-TTS + ECAPA-TDNN), and an agent that runs in an isolated Linux VM on Virtualization.framework. Free macOS menu-bar app included.
Hi Product Hunt! I'm David, the solo developer behind mlx-serve.
I built this because I wanted local AI that just works on Apple Silicon without Python environments, conda, or 10-step setup guides. One binary, brew install, done.
What mlx-serve does:
LLM inference: 35%+ faster decode than LM Studio on Gemma 4 E4B 4-bit
- Drop-in for Ollama (/api/chat, /api/generate) and OpenAI + Anthropic APIs on the same port
- Local image gen: FLUX.2 and Krea-2 Turbo
- Video gen with audio: LTX-Video 2.3
- Voice cloning: Qwen3-TTS + ECAPA-TDNN speaker encoder
- Agent mode with an isolated Linux VM sandbox (Virtualization.framework, no libcontainer)
- Free macOS menu-bar app included
Claude Code, Raycast, Open WebUI, Cursor, and anything that speaks OpenAI or Ollama APIs works out of the box.
The whole server is written in Zig (no Python, no Go, no Node). Happy to answer any questions about the architecture, the performance work, or why on earth I chose Zig for this. AMA!