Should AI agents have an authority budget?
I’ve been thinking about another problem with AI autonomy.
An individual action can look completely reasonable, while the cumulative behavior becomes risky.
For example, maybe an agent is allowed to:
issue refunds under $50
give small account credits
send customer emails
update customer records
Any one of those actions might be low risk.
But what if it issues 200 small refunds in an hour?
Or gives $5,000 in credits across hundreds of customers?
Nothing necessarily violated the rule for a single action, but the overall impact is very different.
So perhaps agents need more than permission limits.
They may also need an authority budget.
Something like:
up to $X in refunds per day
up to Y sensitive actions per hour
tighter limits when unusual patterns appear
human approval once cumulative exposure crosses a threshold
That would make autonomy depend not only on what the agent is allowed to do, but also on how much authority it has already used.
For teams running agents in production, are you putting cumulative limits around agent actions, or mostly evaluating each action independently?
Replies