Framer AI AgentsDesign and publish professional sites with AI
Launching today
NobodyWho
Run AI models on any device
65 followers
Run AI models on any device
65 followers
NobodyWho is an inference engine for running LLMs fully on-device, built on llama.cpp. Open-source, free, no API keys, no cloud calls. We support Swift, Kotlin, Flutter, React Native, Python, and Godot. Includes type-safe tool calling with automatic grammar generation, multimodal input, Text-to-Speech & Speech-to-Text, GPU acceleration via Vulkan & Metal, and Hugging Face model downloads.
NobodyWho
Hey, I'm Pierre from NobodyWho 👋
We've spent the last months getting local inference to be production-ready across six platforms and frameworks, not just a cool demo that works on one device.
With NobodyWho you can:
- Get answers from any open-weight AI models: Gemma, Qwen, LFM...
- Analyse images and audio through multimodal input
- Transcribe speech to text with any Whisper models
- Generate natural-sounding speech with Supertonic, Pocket TTS and Kokoro models
- Tool calling with guaranteed schema-valid output, the grammar is built from your function signature so the model can't return malformed JSON
- Run long conversations without hitting a hard message-length wall, thanks to preemptive context shifting
Wanna try our work on your device? We've built a few demo apps: iOS, Android, Apple Watch & Vision Pro.
We've also built starter examples to get started in 5 minutes and a model selection page.
NobodyWho inference engine is open-source & free, please leave a star to support us on Github 👈
Happy to answer any questions :)
Cluing
NobodyWho
@ralic Thanks ! 😊
We are also planning a rust release soon, stay tuned ;)
@pierre_nobodywho Curious, how much does the grammar constraint affect latency on smaller models?
NobodyWho
@yashekbote it has a big impact on SLM. TFTT on Qwen3 0.6B is 240ms instead of 20ms without grammar constraint, where for the 4B variant it's 250ms instead of 140ms.
@pierre_nobodywho sounds interesting!
NobodyWho
@rom_hunt thanks ! 👋
Six platforms including Godot is a wild spread - most on-device inference projects stop at one and call it a day.
We went the other way on a consumer app I'm building: on-device only for the narrow stuff (Apple Vision for photo classification, SFSpeechRecognizer for voice, both tiny and purpose-built) and kept anything needing real context on the server. The deciding factor wasn't output quality - it was that the AI features need months of user history in the context window, and on a phone that's either impossible or unbearably slow.
So the honest question: at what tokens/sec and what context length does local actually replace a cloud call for you? Not for a demo - for a feature someone hits ten times a day without thinking about it. That's the number I keep failing to find in these projects.
And half-joking, half-not: if this takes off, the memory story gets interesting fast. We'll end up with phones shipping 32–64GB of RAM because a photo app wants a 12B-35B model resident. :)
NobodyWho
@roman_koropets_ actually, you won't need an 32GB to run a 30B model, it's already possible on iPhone 17 Pro Max with Bonsai 27B 🙂
For the inference speed, it depends on the model and the device, I get 27 tokens/sec on iPhone 15 Pro with Granite 4 (1B) for example, which is faster than I can read ;)
@pierre_nobodywho
Ha, but that's the requantise-it-hard branch though :) 27B at 1-bit is about as hard as
requantising gets — so the memory question just turns into a quality question.
And their own numbers put the 1-bit build at ~90% of full precision, which isn't nothing.
That kind of drop compounds once you're generating something long.
What I'd actually want to see is that 27B at 1-bit against a 6B or so at Q4 — same space
on disk, which one wins? Never seen anyone run that head-to-head.
And out of curiosity — who's the ideal user here?
NobodyWho
@roman_koropets_ yep agreed, I'm also curious to see the difference with that specific model and check if the claim is correct. We've done the analysis with gemma/qwen/llama models, and the sweet spot is Q4. Well, on-device AI in general (not specific to NobodyWho) is great for users who want privacy, no per-token cost, and offline capabilities.
@pierre_nobodywho Same conclusion here — Q4 is where it lands for me too. Home server runs Qwen3.6 35B, and it took a few sleepless nights to get there: everything below Q4 came apart in ways that weren't worth the memory it saved (for pc).
Great product, and thanks for the fast replies. Following.
LifeOS
niceeee, Godot support is unusual. is the intended use NPCs that stay local or tools that game clients aren’t supposed to phone home?
NobodyWho
@alkinoos_sarioglou yeapp, one of the use case for Godot is NPCs dialogs :)
BetterClaw
Running AI models fully on device is becoming more important than ever. Love the focus on privacy, offline capability, and broad platform support instead of relying on cloud APIs. Great launch!
NobodyWho
@better_shaya Thanks! 🙏