I audited an AI-written billing service. 22 findings, 10 critical. None of them threw.
I audited an AI-written billing service this week. 22 findings, 10 critical.
None of them crashed. All returned 200. Four that stuck with me: an auth check that silently lets anyone in when one env var is missing, SQL injection in the one query that wasn't parameterized, a broken permission boundary so anyone can attach a charge to a workspace that isn't theirs, and a Stripe retry that double-grants seats because there's no idempotency check anywhere.
The part I keep thinking about is what happened during the fixes. When the agent patched the SQL injection, its first fix didn't actually work. The check reproduced the bug, applied the patch, and it still failed. So instead of shipping a test that said "fixed" when it wasn't, the agent tried again with a smaller, cleaner change and re-proved it. That one held.
That's what I'm building FetchSandbox around: reproduce the real failure, apply the fix, then prove it holds on your actual code before you ship.
For those shipping AI-written integrations, are you catching this stuff before prod, or mostly finding out when something quietly breaks?
Replies
“Everything return 200" is becoming the AI era version of " but it works on my machine. " 😅
The fact that everything returned200 is the scary part. Passing requests don't necessarily mean passing security checks, especially when AI generated code handles billing or permissions.
The silent failures worry me more than obvious crashes. How are you deciding which real use flows to resproduce first? I would prioritize permissions and billing paths since small mistakes there can become expensive quickly.
WebCurate.co
The scary part is that none of these failures look like failures from the outside. Returning 200 and passing the happy path can hide a lot. Reproducing the actual failure before shipping feels much more worthy than just adding more tests.
The most useful part here is the failed first fix. I would trust an agent much more if it had to reproduce the vulnerability, patch it, and independently verify the original exploit no longer works. That feels closer to real engineering review than static code inspection.
Mostly catching it before prod now, but only after finding one the hard way. Early on I had an AI written Stripe webhook handler that looked complete and passed every test I threw at it. What it did not have was idempotency on the fulfilment step. A retry from Stripe during a slow response window double granted access on one account. Nobody complained, I found it by accident scanning the database for duplicate grants months later.
The fix that stuck was not better prompting. It was writing the failure case first, replay the same event twice and assert only one grant happens, before accepting any webhook code as done. Same idea as your SQL injection example, the first pass often looks right because it handles the happy path convincingly. It is the deliberately adversarial retest that catches the rest.
Curious what your workspace boundary check looked like before the fix. That one worries me most across a multi tenant setup, since it fails silently and looks exactly like normal traffic.