Should AI agents have an authority budget?

by

I’ve been thinking about another problem with AI autonomy.

An individual action can look completely reasonable, while the cumulative behavior becomes risky.

For example, maybe an agent is allowed to:

  • issue refunds under $50

  • give small account credits

  • send customer emails

  • update customer records

Any one of those actions might be low risk.

But what if it issues 200 small refunds in an hour?

Or gives $5,000 in credits across hundreds of customers?

Nothing necessarily violated the rule for a single action, but the overall impact is very different.

So perhaps agents need more than permission limits.

They may also need an authority budget.

Something like:

  • up to $X in refunds per day

  • up to Y sensitive actions per hour

  • tighter limits when unusual patterns appear

  • human approval once cumulative exposure crosses a threshold

That would make autonomy depend not only on what the agent is allowed to do, but also on how much authority it has already used.

For teams running agents in production, are you putting cumulative limits around agent actions, or mostly evaluating each action independently?

18 views

Add a comment

Replies

Be the first to comment