A fast, pure-Rust edge inference engine for small language models — chat, Whisper speech-to-text and real-time text-to-speech, fully on-device. From a laptop to a Raspberry Pi. No Python, no Docker, no CUDA — one command to install, one line to run.

Hey Guys! 👋 I'm Saidev, founder of OpenHorizon Labs. Excited to launch Sapient today! I built this because running LLMs on edge devices was way harder than it should be — juggling Python environments, CUDA drivers, and Docker containers just to get a small model talking on something like a Raspberry Pi. That never felt right for real edge/embedded use cases. Sapient is a fast, pure-Rust inference engine for small language models that runs fully on-device — chat, Whisper speech-to-text, and real-time text-to-speech. No Python, no Docker, no CUDA. One command to install, one line to run — from a laptop down to a Raspberry Pi. It's fully open source, check it out here: https://sapient.openhorizon.so Would love to hear your thoughts and feedback, and what hardware or use cases you'd want to see it run on next!

Ran it on a Raspberry Pi 5 and the Whisper transcription was impressively quick for a pure Rust build, no setup headaches. The one-line install lived up to the promise, which honestly surprised me for an edge inference tool.

Sapient

Make your hardware think, see, listen and speak.

Make your hardware think, see, listen and speak.