The AI default is one big model in the cloud. I think that's wrong for real products.

I have spent just over a year building a gardening intelligence and plant identification app, and it turned into a test of a hypothesis I'd been forming across years of running production AI systems.

The default now seems to be to reach for one large model, run it in the cloud, and treat it as a general solution. I've come to think that's the wrong default for anything working under real constraints. The bigger the model, the more it abstracts away from the specific job. The more cloud-dependent it is, the more it inherits the cloud's downsides: latency, cost, privacy exposure, vendor risk.

So I built the opposite. The whole identification model ships inside the app and runs on-device. There are no server calls ever. Under the hood that's not one big model but smaller, specialised ones composed into a system, the kind of architecture software has used for fifty years and the current AI moment has partly forgotten.

The economics are something I think people maybe don't think about enough. Cloud inference costs per request, so for a subscription app the cost scales with engagement and your best users become your most expensive. On-device inverts that. Once the model ships, inference is effectively free, and heavy users become your cheapest instead of your priciest.

The trade-offs however, present a significant engineering challenge. On-device means hard limits on compute, memory, and model size. You can't rent a bigger GPU to escape a problem. It removes the safety net, which is uncomfortable, and forces a kind of careful design that, honestly, tends to produce better systems.

For those who've shipped ML models in production: where's the real line for you between what belongs on-device and what genuinely needs the cloud? And do we reach for cloud-first out of necessity, or just habit?