The hidden problem with AI agents: they don't just answer, they act
For a long time, most AI conversations were about answers.
Could a chatbot write a better email? Summarize a document? Explain a concept? Even when the output was wrong, the result was usually a suggestion that a person could accept, reject, or ignore.
Agents change that dynamic a bit.
An AI agent might update a CRM, approve a workflow, modify a database record, send a message to a customer, or trigger actions across multiple systems. At that point, the conversation is no longer just about the quality of the answer. It's also about the consequences of the action.
What's interesting is that many of the controls that felt optional with chatbots start to feel much more important with agents. Permissions, approval paths, monitoring, ownership, and the ability to understand what happened later all become part of the discussion. The governance problem changes when AI moves from generating suggestions to taking actions on behalf of people.
I suspect this is one reason agent deployments often feel different from chatbot deployments, even when they're built on similar models. The technical foundation may be familiar, but the operational questions are not.
What actions should an AI agent never be allowed to take without approval?
Replies
I'm finding that the real value isn't just smarter responses but smarter decision boundaries. I've started asking whether an AI agent knows when not to act, because that's often more important than acting quickly
@bill_frad I'm seeing more teams focus on what AI agents can do, but not enough attention goes into what they should never do.
My approach is that an agent should get human approval for anything that's
irreversible,
spends money or creates liability,
leaves your system,
access related.
In practice that's no deleting from the DB, nothing touching payments, nothing sent to a customer, and no changes to credentials or access - those either can't be reversed or recalled once they're out. Another underrated thing is logs and backups: an agent that can delete those can also erase the evidence of everything else it got wrong.
I believe permission levels deserves as much attention as model quality. An agent should earn access gradually through proven reliability. What framework helps teams measure trust before expanding autonomous responsibilities across critical systems today?
@donna_gerrard That's the real gap. If "reliability" is scored by the same team shipping the agent, you've just reinvented self-attestation. You'd want an independent audit trail, not the agent's own success metrics deciding its next permission tier.
It's a difficult issue. "Leaves your system" gets tricky with agents that are actually supposed to send stuff, like a support bot replying to tickets. The real fix there isn't blocking egress, it's tiering: known/templated actions auto-fire, anything novel queues for a human.
@Anastasiia the irreversibility angle is the one I'd anchor on too, but it gets fuzzy fast in practice. is force-pushing to a branch irreversible? not exactly, but it can overwrite someone else's work with no easy undo, so it lands in the same bucket even though nothing was deleted. I think the actual rule ends up being narrower than a category list: would a reasonable person want to be asked first, given what they know about this specific action right now. that's harder to encode than "no deletes, no payments" but it's closer to how humans actually judge risk.