We got our tool to write its own bug-reproduction tests

The gap most auto-fix tools have

They tell you the bug is fixed. They don't show you. Our rule has always been: make the bug happen on real code, apply the fix, show it stops happening. The gap was we could only do that for bugs we'd scripted in advance. Anything unusual and the honest answer was "found it, fixed it, can't prove this one."

What we built

We taught FetchSandbox to write the reproduction itself, no pre-scripted test required.

The safety check that makes it trustworthy

A generated test could be wrong. It might pass broken code and hand you a fake green. So before we trust it, the test has to actually catch the bug on the broken code first. If it can't, we throw it away.

Real example

Ran it end-to-end on a billing bug: negative seat count slipping through and shrinking a customer's plan. It found the code path, drove the real handler, stubbed only the db as a passive recorder so the app's own logic decided the verdict. Confirmed the reproduction before trusting it. No hand-holding.

26 views

Add a comment

Replies

Best

I like that the generated test has to fail befor it's trusted. That extra check makes the whole workflow feel much more reliable.

How does FetchSandbox handle bugs that depend on timing or race conditions? Those are usually the hardest ones stop reproduce.