What we run on the phone, and the one thing we didn't

Building Sunny Seed we had to decide this feature by feature, and the answers turned out not to be consistent.

On the phone: WHO growth percentiles - the LMS tables ship inside the app, it's arithmetic, and a server round-trip to divide two numbers would be absurd. Photo tagging for smart albums, via Apple Vision on iOS and on-device ML Kit on Android. And dictation on iPhone, which is transcribed locally and never becomes a file.

In the cloud: the AI assistant. Not because of model quality - because it needs months of a child's history in the context window to say anything useful, and that's the part a phone can't hold at a speed anyone would tolerate. Decode speed gets all the attention in these discussions, but prefill on tens of thousands of tokens is where local inference quietly stops being practical.

The pattern that emerged for us: narrow, purpose-built models go local. Anything that has to read your whole history stays on a server.

Curious where other people draw this line - especially anyone who's tried to keep a long-context feature on-device. Did you find a way to make prefill bearable, or did you end up cutting the context instead?