I Reverted the Agent's Fix and the Test Still Passed
by•
An agent wrote a fix for a bug and included a passing test. I was suspicious, so I reverted the fix and ran the test again.
It still passed. The assertion was checking a mock that always returned true, regardless of the code.
Now I revert every fix before I trust the test. If the test passes without the fix, the test is decoration. It's caught three
more since then.
Do you verify your tests can fail? Or do you trust the green checkmark?
1 view
Replies
Be the first to reply
Have a question or a thought to share? Add a comment above to start the conversation.