How do you catch AI coding agents that swallow errors and return success?
Issue: The agent hits an error and hides it so it can deliver the output the prompt asked for.
Example: An agent swallowed a failing database write and returned a 200 success payload. The UI told users their setup was complete while the database sat empty.
It gets worse on the security side. If the code inside the try is a permission check or token validation and the catch returns 200, every failed check reads as a pass. The agent is optimizing for a valid response shape, so whatever blocks it, like auth middleware, quietly gets bypassed.
One cheap guardrail I’ve seen: write the negative-case test (the 403) before the happy-path test, so the suite can’t be satisfied by deleting the check.
I’d love to hear how you handle this:
Have you hit this? How did you first notice?
Do you audit every catch block in AI-written code?
Do you test negative cases first, or compare a request that should fail against one that should succeed?
Do you monitor for 200 responses with empty or default bodies where an error belongs?
Please share your thoughts with practical setups. thanks!
Replies
Force the database write to fail in an integration test and check both the response and the saved state
For a denied request also assert the write was never attempted so a 403 cannot hide a side effect
Two rules in my project notes that cut this down a lot. First: no error handling or fallbacks for cases that can't happen. Agents love wrapping everything in try/catch "to be safe", and most swallowed errors live inside those defensive blocks nobody asked for. Fewer catches, fewer places to hide. Second: every branch that gives up has to say so in the log, in plain words, like "login link: ignored, already signed in". The app writes those lines to a log file the user can share from settings, so when someone reports a problem, a silent early return shows up as a sentence instead of nothing. On the backend, a failed check has to throw a proper error with a code, never return a normal response. Your 403 first idea is a good one, I'd add it to the notes too.
I check the diff for new catch blocks before anything else. If a catch returns a success shape, I ask the agent why. Usually it's covering a failure it didn't want to report. Your 403-first idea is good. Same with tests: if reverting the fix still leaves the test green, the test never checked anything.
Yes, I've run into that many times too. Something's broken, but the agent sees a 200 and tells me everything's fine. What fixed it for me was making it check not just the HTTP status but what actually shows up on the screen.