LocalClicky
Control your Mac with your voice locally
187 followers
Control your Mac with your voice locally
187 followers
LocalClicky is a Mac menubar app that lets you have a real conversation with your computer - completely offline. Say "Computer" to start a session. It stays listening. You chain commands back to back. Say "goodbye" when you're done. Everything runs on your machine: voice transcription, LLM multi models, VAD, macOS say No API keys. No subscription. No data leaving your Mac. MIT licensed.
NeighborHoodShare
Product Hunt
NeighborHoodShare
@curiouskitty Good question, Tbh there's no guardrails as such being implemented yet. There's no confirmation step before clicking. We have retries for tool calls that uses shell commands. So it runs, checks the output and will try a different approach if it get non desired output. Retries several times then provide feedback if still not able to perform the action.
Refocus
The fully local stack is the part that resonates with me. I work on voice AI for aging-in-place, and privacy comes up in nearly every conversation with families, because the data involved (health, daily routines, who stopped by) is about as sensitive as it gets. Keeping Whisper and the LLM on-device the way you have here removes a whole category of worry. One thing I am curious about: on older or lower-spec Macs, what is the realistic gap from end of speech to spoken response with the local models, and which model sizes stay usable? That latency is usually where local voice either feels like a real conversation or starts to break it.
NeighborHoodShare
@igorgurovich Oh, thats a solid use case.
To answer your question "on older or lower-spec Macs", the gap is noticeable, I haven't tested myself but on M1 with 16 gb ram with nothing else running, the latency should be under 3-5s for a normal command, but for complex commands like generating some health report, it can take a while. Yes latency has been a concern but I was earlier looking into possibility of distributed computing just like how torrent works, offloading compute, but couldn't make much progress there :)
the whisper + ollama stack being fully local makes the google wake-word the last phone-home piece — funny that the smallest model in the chain is the hardest one to cut the cord on. what's actually blocking the offline swap?
NeighborHoodShare
@qifengzheng tbh, I was more leaning towards hotkeys but then at the last moment swapped with wake word before publishing here :)
Goldfish
Love the local-first approach here. The session model also feels like the right UX for voice control, much closer to an actual conversation than one-shot commands.
Curious how you’re thinking about the safety layer before clicks or destructive actions, especially since everything can see/control the Mac locally. Is the plan to add confirmations for certain action types, or keep it lightweight and rely on retries/feedback?
NeighborHoodShare
@kar_re I want to first switch the approach from coordinates click to something more solid as that's not very much accurate :)
But right now it depends on retries on failures
I modified this to Jarvis voice (openwakeord + whipser + piper).
Wake word — the weakest link. It's ironic that an "offline" app phones Google to hear the wake word (needs internet, and it's fuzzy). I'd swap to Picovoice Porcupine — runs 100% offline, very accurate, and it literally ships a ready-made "Jarvis" keyword. (Free alternative: openWakeWord, which you can train.) This single change fixes reliability and gives a proper Jarvis trigger.
Voice — for a real cinematic Jarvis. Daniel (done) is the free quick win. Next levels: Piper (local, free, far more natural than say) or ElevenLabs (cloud, paid — a British male there sounds genuinely like the movies). Tradeoff: Piper stays fully offline; ElevenLabs is best-sounding but cloud.
Clicking — uses a vision model to guess pixel coordinates (that's why "click X" is hit-or-miss). A sturdier design reads the actual on-screen buttons via macOS's Accessibility API.
NeighborHoodShare
@devdutt_parikh yeah that was the only missing piece. btw contributions are welcome if you would like to