What's one thing you still won't let an AI agent do unsupervised, no matter how good it gets?
by•
not talking about the obvious stuff like "don't rm -rf prod", more the gray area tasks where the agent is probably right but you still want eyes on it before it goes out.
for me it's anything that changes a pricing table or a billing calculation. I'll let an agent refactor half my codebase overnight, but the second a PR touches how much someone gets charged, I read every line myself even if the diff is three characters. one off-by-one there and it's not a bug report, it's a support ticket and a refund.
what's yours - the one category of change you still gate by hand even after trusting the agent with everything else around it?
16 views
Replies
mine is anything touching access control - who can see what, who can act as who. an agent can be 95% right on a permissions refactor and that 5% is exactly the kind of thing that doesn't throw an error, it just quietly lets the wrong person read something they shouldn't. billing is bad because it's loud and you find out fast. permissions bugs are bad because they're silent and you might not find out for months. same category as your pricing example though - anything where being wrong doesn't look wrong.
@Gal Dayan that's a good distinction, loud failure vs silent failure. billing bugs generate a support ticket the same day. a permissions bug can sit there for months with nobody complaining because the person who gained access has no reason to report it. I think the actual fix is the same for both categories though - anything where the blast radius is "affects someone who isn't in the room reviewing the diff" gets a human pass no matter how confident the agent is.
@omri_ben_shoham1 same silent-failure logic is why anything writing to a health record gets a human pass here too - a wrong value in a checkin log doesn't throw, it just sits there quietly wrong until someone acts on it later. "affects someone not in the room" is a better filter than trying to enumerate categories upfront.
Hi! I'm new here - but for me, customer support emails. I do have AI draft responses, look at previous replies to mimic my voice, but ultimately, won't let it hit "send". It too often doesn't quite get it! But I also have a consumer health app, so more sensitive in this domain.