Should AI agents be able to lose authority automatically
I have been thinking about AI autonomy as something that can increase over time based on good performance.
But the reverse is probably just as important.
An agent that has been reliable for months could still start behaving differently because of:
a model update
a prompt or workflow change
new customer data
an integration behaving unexpectedly
a completely new edge case
So maybe authority shouldn’t just be earned. It should be continuously re-evaluated.
For example:
Normal performance: existing permissions stay
Error rate increases: tighter limits
Unusual action pattern: approval required
Major policy violation: autonomy temporarily removed
That starts to look less like static permissions and more like a dynamic trust system.
Curious how teams are handling this in production. What would make you reduce an agent’s autonomy after it has already earned your trust?
Replies
The idea is right but I think it skips the hard part, which is detection rather than the response ladder.
Every trigger in your list assumes you can tell that behaviour degraded. In my experience the failures that mattered didn't look like failures. An agent I ran started returning incomplete results because a search provider had degraded and its tool call was failing. The agent carried on and produced something plausible. No crash, no exception, no error rate to threshold on. Error rates were flat. The only person who noticed was the user.
So if you tighten permissions on error rate you'd never have caught it, and if you tighten on output quality you need a judge, which is another model that can also be confidently wrong.
The signal that would have caught it wasn't in the model output at all. It was two spans away: a tool call returning a non-200 that nothing was watching, because the agent's own reasoning about it looked fine.
Which makes me think the useful version of what you're describing is less "reduce autonomy when the agent looks wrong" and more "reduce autonomy when the things the agent depends on start behaving differently". Those are much easier to measure and much harder to fool.
Full disclosure, I build in this space, and I've deliberately stayed out of runtime enforcement. Sitting in the request path and revoking permissions live is a real distributed-systems problem and I don't think anyone has earned the right to fail closed in production yet. So I gate at release time instead, where being wrong costs a red check rather than a broken customer session. Genuinely unsure that's the right call long-term though, which is why your question is interesting.