What an agent does when you tell it which testing technique to use

Dan Luu ran an experiment we wish we had run. One task, a Zstd implementation in Rust from the spec, given to a coding agent 160 times per condition, where a condition is, in most cases, one line appended to the prompt: use test-driven development, use QuickCheck, use fuzzing, use TLA+, audit first. Then he scored each run on a hidden test suite. That was the whole design.

No instruction wildly beat the run with none. He had written down six guesses beforehand about which conditions would not outperform, and all six came true. What interested us was how the instructed runs failed, because the failures share a shape. Told to use QuickCheck, all of them used the library, and 63 of 160 checked a single property, mostly with random inputs that hit the same rejection path. Told to do differential testing, 135 of 160 did something resembling it, and none built a second full implementation; where it might have caught something, the agent wrote the same code twice and put the same bug in both. Told to use TLA+, 159 of 160 wrote a model, and he could not find one case where the model changed the code. Told to fuzz, 10 of 160 built structured inputs, and about half of those found a real bug, which he counts among the more effective uses of any technique he saw.

Our reading is that every verification technique earns its value from one place: a check that did not come from the implementation being tested, such as a second implementation the agent did not write, or a property that existed before the code did. Naming the technique gives the agent the motions of the technique. The check still comes from the agent's own reading of the spec, so the new framework fills up with the tests it would have written anyway, and the suite it generates tells you what the agent already believed about the code. Our own data point is smaller: on a function with a sixteen-year-old bug, the agent-written suites that saw the broken code all passed, and two of them wrote the bug in as the expected result.

The rule we took away is to ask, before naming a technique, where its independent check will come from in this task, and then to supply that, or set up the structure that makes the agent go and get it.