"Walk away and let the agent cook" is a lie (RIP my company's GitHub repo)
by•
I planned out an implementation, reviewed it, and asked Claude to create an epic and break it down into tasks.
I stepped away and came back 3 hours later to find out it had written a script, gotten stuck in an infinite loop, and created 1,000+ issues on GitHub.
P.S. It was our company repo :(
Are you actually comfortable leaving agents running unattended for hours yet, or are we still strictly in human-in-the-loop territory?
46 views
Replies
An agent wrote a reporting job. The output carried a confident metric everyone believed, until a full day went into tracing discrepancies. It turned out to be a hardcoded placeholder. It didn't fail loudly, it just faked the data to finish the script.
@konstantin_tikhaev Honestly, that’s way scarier than my 1,000 GitHub issues. At least my agent failed loudly enough to blow up my inbox, yours basically gaslit the whole team into believing fake metrics.
Did you end up adding tests specifically to catch hardcoded stubs, or are you manually reviewing every line of generated scripts now?
@rakhijha Writing tests for stubs did not help. The agent generated unit tests that asserted its own fake values, so every line has to be reviewed manually now.
Hours unattended, yes, but only for things that can be undone. Anything that leaves my machine, like creating issues, sending email or deploying, stops and waits for my OK. A thousand issues is exactly what that rule is for. Even so, over 90% of my work runs through agents now, and I'm not going back.