What's the one AI coding habit you had to unlearn the hard way?
Mine was trusting the diff summary instead of actually reading the diff.
I used to just glance at "renamed X, updated Y files, tests passing" and move on. Then one day the agent quietly changed a timeout value in a config file while it was "cleaning up," totally unrelated to what I asked for. Nothing broke locally, so it shipped. Took three days in prod before anyone noticed requests were timing out way faster than before.
Now I read every diff like I'm reviewing a junior dev's PR, even the boring stuff that feels too small to matter. Slower, but I haven't gotten burned since.
What habit did you have to build or break after getting burned like this? Was it a review process, a specific way you prompt now, running things in a sandbox first, something else entirely? And did it actually stick, or did it slide after a few weeks of nothing going wrong?
Replies
Similar to yours, actually, trusting that the agent understood the spec just because it produced something that looked plausible. I described a feature in detail, it built something, and on the surface it looked right. It wasn't until I actually walked through it screen by screen, checking what happened at each step, not just whether the UI rendered, that I realized it missed a piece of the logic I actually cared about.
Now I try to describe the outcome I want, not just the feature, and verify against that specifically instead of just checking "does this look like what I described." Still working on making that stick every time, especially when I'm moving fast.
@webinteg That "verify against the outcome, not just checking it looks like what I described" distinction is a good one, plausible-looking and actually correct are two different bars, and it's easy to only check the first when you're moving fast.
mine was trusting a green test suite without checking whether the agent had touched the tests themselves. I had it fix a bug, tests passed, shipped it. turned out it had quietly loosened an assertion in the test that was catching the actual bug, so of course it went green. now I diff the test files with the same suspicion as the source files, if a fix touches both I read the test change first before i even look at the fix. still catches me off guard sometimes when it's buried in an unrelated refactor commit though
@galdayan That's a nasty one because a green suite is supposed to be the thing you trust more when you're moving fast, not less. Once the tests themselves are in scope for edits, "passing" stops meaning "correct" and starts meaning "consistent with whatever it just changed, which is a completely different guarantee.
Mine was staying in the same chat too long.
When the context fills up you trust it to compact and keep going, and that is the mistake. The compaction quietly drops the parts that mattered, and from there the answers get worse. It still sounds confident, but it is working from a worse and worse picture of what you are actually doing, and you do not notice because it happens slowly.
What I learned is to just start fresh from time to time. New context, paste back only the few things that actually matter, and it is sharp again. A long session feels efficient but it is often the resaon the output degraded, not the model.
To your last question, it half sticks. When things go well I stay in one chat way too long again, until it starts making dumb mistakes and I remember why.