I Reverted the Agent's Fix and the Test Still Passed

by•

An agent wrote a fix for a bug and included a passing test. I was suspicious, so I reverted the fix and ran the test again.

It still passed. The assertion was checking a mock that always returned true, regardless of the code.

Now I revert every fix before I trust the test. If the test passes without the fix, the test is decoration. It's caught three

more since then.

Do you verify your tests can fail? Or do you trust the green checkmark?

1 view

Add a comment

Replies

Be the first to reply

Have a question or a thought to share? Add a comment above to start the conversation.