Local vs cloud AI in SaaS: where do you draw the line?

I've been building a product that runs its AI models fully on-device. There are no API calls and it works offline. It started as a privacy decision, but it changed a lot more than I expected.

What got better:

• No per-request API costs, so I can offer generous usage without worrying about margins.

• Latency is predictable and there's no dependency on someone else's uptime.

• Users trust it more when their files never leave their device.

What got harder:

• Model size versus quality is a constant trade-off, especially on older devices.

• Shipping model updates is slower than swapping an endpoint.

• Debugging is harder because I can't see what's happening on the user's machine.

Most SaaS products I see default to cloud APIs for everything, and I understand why: it's faster to ship. But I'm starting to think some features, like transcription, search, classification and simple generation, belong on-device.

For those of you building SaaS with AI:

1) Have you tried running any models locally or on the edge? What pushed you there, or what stopped you?

2) Do your users actually care about data staying local, or is it only a concern for enterprise buyers?

3) Would you use a hybrid setup, with local for fast, simple tasks and cloud for the heavy ones?

I'd love to hear what's worked for you.