The hidden problem with AI agents: they don't just answer, they act
For a long time, most AI conversations were about answers.
Could a chatbot write a better email? Summarize a document? Explain a concept? Even when the output was wrong, the result was usually a suggestion that a person could accept, reject, or ignore.
Agents change that dynamic a bit.
An AI agent might update a CRM, approve a workflow, modify a database record, send a message to a customer, or trigger actions across multiple systems. At that point, the conversation is no longer just about the quality of the answer. It's also about the consequences of the action.
What's interesting is that many of the controls that felt optional with chatbots start to feel much more important with agents. Permissions, approval paths, monitoring, ownership, and the ability to understand what happened later all become part of the discussion. The governance problem changes when AI moves from generating suggestions to taking actions on behalf of people.
I suspect this is one reason agent deployments often feel different from chatbot deployments, even when they're built on similar models. The technical foundation may be familiar, but the operational questions are not.
What actions should an AI agent never be allowed to take without approval?
Replies
I'm finding that the real value isn't just smarter responses but smarter decision boundaries. I've started asking whether an AI agent knows when not to act, because that's often more important than acting quickly
@bill_frad I'm seeing more teams focus on what AI agents can do, but not enough attention goes into what they should never do.
@bill_frad @dipmala_kumari The second question often appears only after the first incident. Ideally, teams should answer both before deployment imo
@bill_frad Exactly. “Is it smart enough?” gets less interesting once it can take action. “Does it know where to stop?” is the real question.
My approach is that an agent should get human approval for anything that's
irreversible,
spends money or creates liability,
leaves your system,
access related.
In practice that's no deleting from the DB, nothing touching payments, nothing sent to a customer, and no changes to credentials or access - those either can't be reversed or recalled once they're out. Another underrated thing is logs and backups: an agent that can delete those can also erase the evidence of everything else it got wrong.
Agreed, @alieksia the point about logs is especially important. An agent should never control both the action and the evidence used to investigate that action.
It's a difficult issue. "Leaves your system" gets tricky with agents that are actually supposed to send stuff, like a support bot replying to tickets. The real fix there isn't blocking egress, it's tiering: known/templated actions auto-fire, anything novel queues for a human.
@aidan_codefox Tiering makes sense, though I’d also evaluate the content and destination. A familiar action can still become risky if the message contains sensitive data or goes to the wrong recipient.
Dial
@Anastasiia the irreversibility angle is the one I'd anchor on too, but it gets fuzzy fast in practice. is force-pushing to a branch irreversible? not exactly, but it can overwrite someone else's work with no easy undo, so it lands in the same bucket even though nothing was deleted. I think the actual rule ends up being narrower than a category list: would a reasonable person want to be asked first, given what they know about this specific action right now. that's harder to encode than "no deletes, no payments" but it's closer to how humans actually judge risk.
@omri_ben_shoham1 Permission systems love neat categories. Reality keeps refusing to cooperate with them.