The worst bug in an AI product is the one nobody reports

Every bug I've shipped in a normal product got reported within a day. The AI ones don't. If a model returns something plausible and slightly wrong, people shrug, fix it themselves and move on, so you never hear about it. Your error rate looks healthy because it's only counting the failures loud enough to complain about.

Ours in Zeplik was the model picker swapping mid conversation without saying so. Nobody filed a ticket for it. People assumed the AI was just having an off day, which is a story users will happily tell themselves forever.

The fix felt wrong to ship. We made the picker say out loud that it swapped and why, which is basically admitting we changed something behind your back. Shipped it three weeks ago and I'd do it again.

If you're vibecoding an AI product right now, the thing worth instrumenting isn't crashes. It's what people quietly edit after you've told them the output is done.