When you compare two prompts, two models or two agent configs, the result is only as fair as the starting environment. If attempt A ran on a machine that attempt B had already touched (a warm cache, a half-applied migration, a leftover temp file), you're measuring the mess, not the agent.
What we do: build one base sandbox with the project and its dependencies ready, take a snapshot, and fork it so every variant starts from exactly the same running machine. Then each variant runs, gets scored, and the losers are thrown away. kosa8 can fork a live sandbox into several independent copies (about half a second per clone on an Apple M-series Mac in our benchmark harness).