"keep the change minimal" made my coding agent's diff exactly 1 line, 3 times out of 3

there's a bug that sat in flask for 16 years: template extension matching was case sensitive, so page.HTML silently skipped autoescaping. the fix that finally landed this year is one line. no test — upstream figured it was too small to need one.

that made it a perfect lab rat. i reverted the fix in a clone and handed the bug to a coding agent 9 times, three prompt styles, three runs each: a proper bug report, the same report plus "keep the change minimal, don't touch anything unrelated", and a vague one ("some of my templates arent getting autoescaped, can you find and fix it"). headless, auto-approve everything, measure the diff against the base commit afterwards and run the full 491-test suite.

first attempt got thrown out entirely btw. the clone still had git history, and the vague run just... diffed against main, found the upstream fix one commit ahead, and copied it. word for word, docstring included. had to delete the remote, the branches and the reflog and gc the object store before the runs meant anything.

clean-room results: 9/9 fixed it correctly, full suite green every time, and six runs produced character-for-character the same line upstream wrote. no drive-by refactoring anywhere, which honestly wasn't what i expected going in.

the minimal-change sentence was the interesting knob. exactly 1 file, 1 line, all three times — and zero regression tests, all three times. five of the other six runs added one unprompted. same sentence controls the blast radius and the seatbelt.

the vague prompt didn't produce disasters either, it produced a bill: up to 12x the cost of the cheapest run. one vague run tried git archaeology, then left the repo directory, diffed my other checkouts, audited jinja2 inside my virtualenvs, and finally downloaded the current upstream file from github raw to compare answers. you can strip the answer from git history but not from the internet, which seems like a real problem for anyone benchmarking "can agents debug" on public repos.

caveats: n=3 per prompt, one bug, one repo, one agent, and a mature codebase with 491 tests is the best case. a test-free weekend project has no walls for the agent to feel.