Every agent safety gate I have seen fires on one action. The damage arrives from four hundred.

by

There is a good discussion running elsewhere on this platform about when an agent should stop and ask permission. The consensus has moved from "can this be undone" to "how far does this reach", which is a real improvement. Irreversibility over-fires on writing a log line and under-fires on a migration you can only roll back with downtime. Reach is the better axis. But every gate in that conversation, and every gate I have seen shipped, evaluates a single action in isolation. Sending one email to one person is low reach. It passes. Four hundred of them inside twenty minutes is a mass email to your entire list, assembled one approved step at a time, and not one individual step ever looked like the thing you were guarding against. The reach was never in the action. It was in the rate. So the gate wants a running total inside a window, not a verdict on the action in front of it. And approvals need a clock on them, because right now an approval is treated as a standing licence. You approved "send this email". The agent reads that as permission to send email. If it comes back to the same signed-off action ten minutes later it should have to ask again, because what it is doing now is a different act from the one you looked at. The uncomfortable version: most of the agent incidents I have heard described were not one bad decision. They were one acceptable decision executed a few hundred times while everybody watched the individual approvals go past and felt fine about each one. Has anyone here actually shipped a rate-aware gate rather than a per-action one? Curious what you keyed the window on, and whether it survived contact with a legitimately bursty workload.

6 views

Add a comment

Replies

Best

"an approval is treated as a standing licence" is the exact phrase for it. I've set up recurring agent tasks on a daily approval and never once asked whether today's run is actually the same shape as the one I approved.

 The recurring task is the sharpest version of it, because the approval and the run drift apart silently. Nothing tells you the shape changed. The cheap fix I keep coming back to is approving a shape rather than a task: roughly this many items, roughly these targets, this blast radius. When today's run falls outside the shape it re-asks, and when it does not it just runs. That way the gate fires on the thing that actually changed instead of on the calendar. What does your daily approval actually show you at approval time, the plan or just the go button?