Vibecode is instant legacy
I keep coming back to this idea: vibe code is basically instant legacy code.
Not because AI-generated code is necessarily bad, but because it often puts us in a familiar situation: the system works, yet no one fully understands how it works. That makes me wonder whether some of the most useful techniques for vibe coding are actually techniques we already use for legacy systems. For example:
Snapshot / approval tests: capture the current output and require explicit approval when it changes.
Characterization tests: describe what the system actually does today, regardless of what it was originally intended to do.
Golden master tests: run known inputs through the system and compare the results with a trusted baseline.
Contract test: protect the boundaries between components, services, APIs, events, and data stores.
Differential testing: compare two implementations and detect where their behavior diverges.
Test carving: observe a larger execution and extract smaller, focused regression tests from it.
The common idea is simple: you do not need to understand every line of code before you can start protecting its behavior. That feels especially relevant for AI-generated systems. Before asking an agent to refactor, regenerate, or extend something, first capture the behavior that already works. Then use those tests as constraints for the next change.
I’m particularly interested in test carving here. It is also the focus of my PhD work, and vibe coding seems like a natural use case: take a successful execution of an AI-generated system and turn parts of it into focused regression tests automatically.
Curious whether others are already using legacy-code testing techniques for vibe-coded projects.
Replies
Characterization tests feel especially practical here. My first instinct with AI-generated code would be to protect what already works before touching the implementation.
@kimberly_west The idea of capturing behavior before asking an agent to refactor is something I’d probably adopt myself. It gives the agent a boundary instead of letting it freely redefine things.
Golden master testing seems like a good vit for vibe-coded apps because the expected behavior can matter more than understanding every implementation detail.
The legacy toolkit fits, but it only covers half of what an agent breaks
A lot of my rules aren't about behavior though, they're bans: a name that was retired and must never come back, a claim in the copy we're no longer allowed to make. A snapshot happily freezes the wrong thing, because at capture time the wrong thing was already there.
What works for me for those is: a test that `greps` the source. One walks every file under src, fails on the retired parameter names, and carries a hardcoded allowlist of five files where the old names are still legal because they're storage and wire formats. Another fails if banned marketing phrases show up anywhere in the app copy.
Where it stops: anything needing judgement, like whether an abstraction sits at the right level or whether an error message is actually helpful. My attempts were worse than nothing
Contract test tend to ensure that the behavior of the functionalities is according to expectation. This keeps any further divergence in check. Vibe coders should employ continuous testing as a regular practice to ensure product quality.
I recently found that a pile of obsolete pytest tests had accumulated in a project I've been vibe-coding. Keeping tests properly linted seems just as important.