Experimenting with On-device AI: What cloud-based writing tools quietly cost you in privacy

Most AI writing tools (Grammarly, AI-rewrite extensions, etc.) send your text to a server to process it — even with a good privacy policy, that round-trip is the actual risk for anything sensitive.

Chrome quietly shipped a real on-device model (Gemini Nano) directly into the browser. I've been building on it with Quill —rewrite/proofread/summarize that never leaves your machine, because there's no server call to make. Not "we promise not to look," but "there's nothing to look at."

Tradeoff: On-device models are smaller than frontier cloud LLMs, but for everyday writing tasks the gap's closing faster than I expected.

Anyone else experimenting with on-device AI (Prompt API, WebLLM, in-browser ONNX)? Curious how the privacy/quality tradeoff has felt for you.

Launching Quill🪶 on PH July 14th if curious: https://www.producthunt.com/products/quill-15