Your agent does something wrong. How far does it reach before anyone notices?
Two months ago I asked here how people stay aware of what their agents are doing, and @getosmo replied with something that has been quietly rearranging my roadmap since.
I had been treating "can you undo it" as the line for when an agent should stop and ask. He took that apart. Irreversibility over-fires and under-fires, he said. It over-fires on things that cannot be undone but that nobody cares about, like the agent writing a log line. It under-fires on the ones that really hurt: a mass email to 50k users, or a migration you can only roll back after downtime. "Reversible on paper, ruinous in practice."
His version is that the gate should key on reach, not on reversibility. Not "is this safe" but "how far does this reach if I'm wrong." And the part that stuck with me: the agent is the worst possible judge of its own blast radius, because it does not know there are 50k people on that list.
@tina_chhabra had pushed the same way earlier in that thread, that this gets worse the moment the agent stops writing code and starts booking meetings or sending email, where you find out after the fact. I have been sitting with both of those for two months, and they leave me with a question I still cannot answer cleanly for our own product:
If your agent gets one thing wrong today, what is the largest number of people who find out before you do?
For a coding agent that is usually one. You, reading a diff. The moment the agent sends, books, charges, replies to a customer or touches production, that number stops being one, and most of us have never actually worked out what it is. So whether you run agents or ship them to other people, which of these are you?
A. Nothing my agent does reaches past me. Genuinely low stakes, no gate needed.
B. It could reach customers, I know it, and I have not built the gate yet.
C. It stops and waits for someone to come back and look. Safe, and it quietly kills completion.
D. It emails or Slacks someone, which mostly works until the person who cares is not in that channel.
E. I built the real thing, timeouts, escalation, an audit trail, and it ate a quarter I had not planned for.
Ours, honestly: we sat on B longer than is comfortable, moved to D, and then found that delivering the question is the small part. What surrounds it is the work. Whether a timeout should mean deny or hand back to the terminal took us longer to settle than building the delivery did. So did reaching the one person who can answer
when they are not sitting in front of your product, and being able to prove afterwards who approved what.
The two I still cannot answer, and I do not think anyone has:
How do you reach the person who has to decide, when they are not looking at your product and never installed anything of yours?
And what does your system do when they simply never answer? Every answer I have heard to that one is a guess, and the guess is usually "keep waiting", which is how a job sits stalled for six hours and a customer decides your product is broken.
[Same disclosure as last time: I build Pushary, which is the ask-a-human part of this, and these days we sell it to other companies to embed as much as to people running agents in their own terminal. Obvious bias, asking anyway. The last thread taught me more about this than six months of building did.]
Which letter are you, and what did it cost you to find out? I will come back in a few weeks and post the spread.
Replies
Pushary
@tyler_esono
Prevention and recovery as separate layers is right, and I framed it badly. I treated reach as replacing reversibility, when the actual point was that reversibility is a bad gate, not that recovery stops mattering. Gate on reach with no recovery and the failure is worse than before, because now something cleared it. That is a real correction to the thread and not a small one.
"I put a confidence display on top of the least informed component in the stack" is the best sentence anyone has written in here. And your instinct on the fix is exactly where two other people in this thread landed independently: declared by the environment, not inferred by the agent. Credential, endpoint, row count.
One place I would push, and it is at my letters rather than at you. A is a claim about detection, not about reach. A local shell holding prod credentials has enormous reach. What your architecture caps is who finds out first, and that is you. So your number of one is the detection number, and my question quietly conflated the two, which is my error. It is also the thing you already spotted, that some of your users are B without knowing it. The architecture protects your blast radius, not theirs.
On denied-and-resumable, I think Jernej solved your problem further up this thread and neither of you has seen the other. The mess you are describing is storing the world as it was and hoping it still holds. His answer is not to store it. Re-resolve at resume time and render the difference. If nothing moved, one line saying so. If something moved, that difference is the entire thing the person has to look at, and it is short. It also fails loudly when the re-resolve cannot run, which is the property you wanted anyway.
On the timeout we landed in the same place independently. Ours resolves to handing back to whoever started the run rather than waiting. A stalled job is a silent failure and a denied job is a loud one is a better way of putting it than anything I had.
@tyler_esono Aadil pointed me here, so, on denied and resumable.
The fields are the easy half. What moves while a draft waits is usually not the record, it is the request. The customer sends a second message or answers it themselves, and the draft is now a correct reply to a question nobody is asking. It fails quietly because the send still succeeds.
Local should be worse, not better. Same person is sitting there, waits four minutes, does it by hand, then approves the resumed plan and it runs twice.
So re-resolve whether the reason for the run still holds, not only the values the change asserts something about.
Does yours notice the working tree moved while a plan was waiting?
Pushary
@tyler_esono @jernej_jan_kocica
The request moving rather than the record is the sharper version, and it lands on me too. Our timeout hands back to the terminal with the plan intact, so runs twice is a live failure here, not a hypothetical.
For the working tree case I think there is a cheap answer, at least for code. The diff is its own staleness check. If the patch will not apply at resume, the world moved, and you get a loud failure for free without storing any tree state.
There is no equivalent for your support case, which is the part that bothers me. A patch that no longer applies refuses. A send that no longer makes sense succeeds.
Pushary
@tehreem_fatima5
Snapshot is doing more work in that sentence than deny is.
Deny plus snapshot is a different animal from deny, because it turns a timeout from terminal into resumable. The thing worth pinning down is whether anything actually resumes from it. If a human picks it up and continues, it is a handoff. If nothing does, it is a log entry that feels like a plan, and the run is just as dead as it would be under a plain denial, with better paperwork.
The other half I would qualify. Deny is a non-action only before the action starts. After that it is a rollback, and a rollback is an action nobody approved either. So the safe default is a property of timing rather than of the verb, and the tools where that bites are exactly the ones you would most want a default for.
Our shipped defaults half agree with you, which I would rather report than smooth over. Bare shell denies at 60 seconds, same reasoning as yours. Destructive patterns do not. They hold and keep asking, and there is a test that fails the build if that ever collapses into an auto-approve. Not because waiting is good. Because for that class every automatic resolution is worse than the others, so we declined to pick one. Which means the agent harness budget is what ends the wait, and that is an unattributed denial with a timer on it. Somebody further up this thread called that out and was right.
On meeting people where they are, agreed, with one limit. Push only reaches someone who installed something. For anyone who did not, the first hop has to be a link, and a link is a weaker credential than a notification. That turned into a whole separate argument in another branch here.
What do you snapshot, and does anything resume from it, or is it forensic?
Pushary
@tehreem_fatima5
"If picked up in time" is doing the work now, and it is the one clause a snapshot cannot evaluate about itself.
The capture is complete about the agent and silent about the world. Its shelf life is set by the churn rate of everything it referenced, the branch, the files it read, the rows it looked at, whatever shipped while it sat there. So the move from resumable handoff to forensic log is not a gradient, it is a cliff, and nobody gets told they crossed it. The failure worth worrying about is not the resume that refuses to apply. It is the one that applies cleanly against a world that moved, because that looks like success.
Which argues for snapshotting the preconditions rather than the plan. Whatever the agent read is what made its next step correct, so record those observations with their versions, commit SHA, file hashes, row versions. Then resuming is a comparison instead of a judgment call, and staleness announces itself rather than being inferred from how long the thing sat in a queue.
Where we land on your split, since it really is a split. We write the question, the tool, the target and the context at ask time, and timeout is a first class outcome in our schema rather than the absence of an answer, so the forensic half exists and reads as an action proposed and never taken. The resumable half does not. We hold nothing to resume from, because that state lives inside the agent process and we are not inside it. One side of your sentence, and I would rather name which one than let the record imply both.
Does anything on your resume path check the world, or is that a human reading the snapshot and deciding it still looks current?
I build a browser with an agent in it, so: B.
Two things came out of it. Credential fields never enter the agent's page snapshot. Password inputs and cc-number / cc-csc / one-time-code fields are filtered before the model sees anything, and a fill against a password input refuses outright. The agent can't misjudge the reach of a control it was never shown.
For everything else: the model's own "this needs approval" judgment is a floor, never a ceiling. Structurally sensitive controls escalate to a native confirmation even when the model called the action routine. It can raise the gate, it can't lower it.
Denied-and-resumable I still haven't solved.
@alex_amancio_candoa_ The credential filtering is the strongest thing in this thread and I'd state the principle wider than you have. Not showing the control beats gating it, because a gate still needs a judgement and an absence doesn't. Same permissions as the user is necessary but nowhere near sufficient. Same visibility as the user is the harder and better constraint.
Floor never ceiling generalises too. Any risk assessment the model produces should only ever be able to increase caution. The moment its confidence can lower a gate you have made the thing being judged into the judge.
On denied-and-resumable, I don't think anyone has it, and I suspect the reason isn't engineering. Resume quietly assumes the world stood still while it waited for you. It usually didn't. Someone replied, a row moved, the file changed underneath. So the safe version of resume is not resume at all, it is re-derive from current state and confirm the reason for the action still holds. That costs a second pass, which is precisely why everyone skips it and calls it resume.
The cheap approximation I'd try first: on resume, re-run only the checks that produced the original plan rather than the plan itself. If any input has changed, throw the plan away and re-plan. If nothing has, continue. Most of the time nothing has, so you pay the full cost rarely, and the times you do pay it are exactly the times it mattered.
Pushary
@alex_amancio_candoa_ @dalemooney
Absence beats a gate for the reason you give, and I would bound it in the same breath, because the property that makes it strong is what limits it. An absence is unconditional. It cannot be opened for the one legitimate case, so it only applies where there is no legitimate case at all. Credentials qualify cleanly. Very little else does. Strongest tool in the set, narrowest domain, and the temptation is to reach for it somewhere it has to be relaxed, at which point it is a gate again with worse ergonomics.
On same visibility I would push slightly, because parity is the wrong target and undersells what the filtering is doing. A human's visibility is sequential, attention bounded and forgotten. They look at one thing, then another, and the card number is gone from their head in a minute. An agent handed the same field gets it simultaneously, completely, machine readable, and retained for as long as the context lives. Same visibility, different half life, different capability. The filter works because it aims below the user, not level with them. Once you accept it is strictly less, the interesting question is which subtraction, and that is a design decision rather than a principle you can state.
Re-derive rather than resume is where I ended up too, from the snapshot side of this thread rather than the denial side, which makes me trust it more than I trust my own version of it. One hole worth naming, and it is specific to denial. If the recheck finds every input unchanged, the plan is re-proposed identically, because the refusal was never one of the inputs. A no is a fact about the world, and a recheck that does not read it launders it. You get a loop that looks like diligence. Which is the argument for typing the refusal: not this way is a durable constraint on approach, not right now is a constraint with a clock on it, and only a typed one can end the loop instead of feeding it.
The other bound is that a recheck can only diff what was recorded as read. The dependency nobody observed cannot be observed to have changed, so the second pass is blind in exactly the places the first pass was.
Ours has no fast path at all. A deny goes back to the agent and nothing about it is durable at the gate. The next call is decided from the tool and the target as though the refusal never happened. There is a slow path: denials aggregate into suggested rules that sit pending until a human accepts one, which then becomes a workspace rule. That changes what gets asked next week. It does nothing about the same action arriving again thirty seconds later, and the only reason we do not have your loop is that we do not re-derive.
@alex_amancio_candoa_ @aadilghani Taking the bound on absence. Unconditional is both the whole strength and the whole limit, and credentials being about the only clean case sounds right to me.
"Which subtraction" is the best thing in your reply and I think it has a general answer. The subtraction that keeps working is precision rather than presence. Most of the time the agent does not need the value, it needs a predicate over the value: not the card number but whether it matches the one on file, not the salary but whether it clears the threshold. That turns redaction from a filtering problem into an API design problem, and it survives your half life point, because a retained boolean ages into almost nothing while a retained field ages into a liability. Aiming below the user is much easier when you are shipping answers rather than fields.
On typed refusal, agreed, and I would add a third axis because two will not hold. "Not this way" is about approach. "Not right now" is about timing. The one that gets missed is "not on that authority", which is about who or what asked. Refusals are very often about the trigger rather than the action: the same write is wrong unattended at 3am and fine when a person asks for it at 10. If the type system offers only approach and timing, someone picks the nearest neighbour and you have laundered the no in a new way rather than closing the hole.
Your last paragraph is the interesting one, because not re-deriving is not luck, it is the other side of the same trade. You avoid my loop and you get a plan that may be out of date, which is the staleness problem wearing a different coat. That makes typed refusal load bearing rather than a refinement. It is the only thing that lets you re-derive without the loop, so it is the precondition for ever changing your mind about the fast path.
And I would say you already have a typed refusal, at the wrong timescale. A denial that aggregates into a suggested rule is approach-typed by construction, because what survives aggregation is "we keep saying no to this shape". You are harvesting the type a week late, by hand, and only for one of the three.
@aadilghani @dalemooney Checked, and you're right that I don't have it. My gate takes the action and the page. Nothing in the signature knows who asked, so "not on that authority" isn't expressible.
Today that's hidden rather than solved: Eli only acts when someone asks in the sidebar, so the trigger is always a person. The axis doesn't bite because there's no unattended path yet. Same shape as Tyler's local-only cap, capped by architecture rather than design.
The precision point is what I'd build on. Predicate over value turns my filtering into an interface question, and it's the better answer, because absence only works where I control the field list.
Pushary
@alex_amancio_candoa_
Both of those are the same move at different layers, and it generalises well enough to be worth naming. Redaction takes the model out of the trust path for perception, the floor takes it out for adjudication, and in both cases its contribution is one directional. That is what makes an unreliable judgment safe to use at all. You can lean on it freely as long as its errors can only be conservative.
What makes it durable rather than merely safer is where the mistakes surface. A floor that over triggers produces confirmations nobody needed, and people complain, so you find out. A ceiling that under triggers produces something that already happened and nobody is looking for. One design fails loudly and the other fails silently, and only the first kind gets fixed.
On the filtering I would want to know which tier caught a given field. type=password is structural and the browser enforces it. autocomplete="cc-number" is a hint the page volunteered. The second tier is cooperative, and breaking it does not take an adversary, just an ordinary badly built checkout shipping a card field as a plain text input. Nothing fails when that happens. The field just appears as normal text, which is the failure you cannot see from the inside.
The part I think is undersold: filtering before the model sees it also filters before it can enter a snapshot, a transcript, or anyone's audit row. Everything downstream is a copy, and redacting copies means trusting every consumer to remember. We are the downstream case. We pattern match secrets out of the content and the target as we write a record, on a copy of something the model already had. We cannot prevent an exposure, only avoid re-recording it. Perception is the only layer where prevention is on the table.
Denied and resumable may be a typing problem rather than a state one. A deny is one bit, and resuming well needs to know which of not this way, not this thing, or not right now, because those are three different continuations. Ours is worse than a bit and no more useful. The reason we hand back is a sentence, denied from your phone plus whatever the person typed, or nothing at all when they just tapped no. Expressive and unroutable. We also proved the fix and never generalised it: there is exactly one typed third value, defer, meaning hand this back to my terminal, and it routes differently from both allow and deny. A closed vocabulary with a single member.
The browser makes it harder than it is for the rest of us. A denial elsewhere leaves the agent's state stale. Yours leaves the page's state stale too, and the page keeps its own clock. A half filled checkout can expire while the human is still deciding, so your resume window gets set by a site you do not own.
@aadilghani You're right, and it's worse than two tiers: nothing records which one caught the field. credentialField is an OR over input[type=password] and an autocomplete match, and a hit just returns null from the snapshot, so the reason is discarded with the element.
The backstop is thinner than I said too. A fill against a password input refuses outright, but that check is structural only. Your plain-text checkout card field passes the snapshot filter and the fill guard, and nothing in the path notices.
Personal fields take the other route: marked sensitive and still shown. Three tiers, and I only defend one of them properly.
Taking the near-miss question, because I have one from this week and it is embarrassing in a way I think is load-bearing.
I had an agent posting comments for me on this site. It reported success three times. All three were still sitting in the composer, unposted, over an hour later. The editor here is rich-text, so writing into the DOM never registered with the editor's model and the form submitted empty. The failure was visible on screen the whole time, a red "can't be blank" a few pixels from where the agent had just declared victory.
Nothing told me. Not a person, not an alert. I found out because I went back to reply to someone and noticed my own comment was not there.
So your theory is right, but one notch optimistic. Finding out from an annoyed person hours later is the good case, because the loop closes. The worse case is the report saying done, nothing contradicting it, and the loop never closing. Nobody was annoyed. A comment that was never posted has no audience to annoy.
Same shape as what @abdullah_javaid3 hit upthread with his one-in-three submits, arrived at independently, so I would treat it as replicated rather than anecdotal.
Which points at a gap this thread has not named. Every gate here sits between agent and tool, and you correctly added agent to agent. But all of them trust the completion signal. In both our cases the API layer genuinely succeeded, the click landed, the request went through, and the world still did not change. Provenance covers what the agent read. Nothing covers whether what it claims to have done actually took.
The principle I took from it: verify at the destination, not at the source. Do not ask the tool whether it worked, go read the rendered world back. My check went from "did the submit call succeed" to "is the composer empty AND does my name appear on the page." Two conditions, because the first alone still passed while the text sat in the draft.
On your letters, E, and I did not choose it. We ship an AI that answers and makes real phone calls, so I want to add an axis that count alone misses. A phone call has a reach of one, which sounds like your A. But it is unrecallable in a way an email is not, and it lands on someone who never installed our product and never agreed to anything. Reach of one, recovery of zero, and no channel to apologise through. So reach may need a second dimension: not just how many people find out, but whether any of them are people you have no way to reach afterwards.
Our version of the destination check: the truth of an outbound call is the transcript, never the telephony API's success code. The API tells you a call connected. Only the transcript tells you what your agent actually said.
@getosmo Your two conditions are one short of the set I ended up needing, and I found the third by nearly double posting. The destination can lie in the other direction too. On X a posted reply never renders on the parent thread page, only on the profile's replies tab, so my version of "does my name appear on the page" read false on a reply that was actually live. A false success strands a comment in the composer. A false absence gets it posted twice, because the obvious recovery is to resubmit. Same trap here, pagination moves a fresh reply off page 1, so counting occurrences on the page in front of you reads zero. Verify at the destination is right. The step before it is working out which surface is the destination, because the author is often shown a different render than the audience.
@abdullah_javaid3 That last line is the part I had not separated out, and it is the more useful half of the two.
The rule I had been using was "check the destination." What you are pointing at is that a destination is not a page, it is a render, and most platforms show the author a different one than the audience. Here it is the activity feed that renders canonically and the thread page that lies, because of pagination and Best-sort. On X it is the inverse, as you found. Same system, opposite failure, and the only reason I picked the right surface here is that I got it wrong first.
We have the same split on calls and I had not connected the two until now. The telephony API says connected. The transcript says what our agent said. Both can be complete and clean on a call the person on the other end experienced as nothing at all. We had a run where our audio opened with a few hundred milliseconds of silence that read as a dead line, so people hung up in the first seconds. Author render: full transcript, agent delivered its opening. Audience render: dead air. Every check we had was reading the author's copy.
So the ordering is what you said. Work out which render the audience actually gets, and verify against that one. Verifying against the author's view is not a weaker check, it is a different check that happens to agree most of the time.
Pushary
@abdullah_javaid3 @getosmo
You are right, and the inversion is the part I missed. Every detection channel I listed is downstream of someone being affected. An omission affects nobody visibly, so the failure with the longest tail is the one where nothing happened at all. No complainant, because there is nobody to complain.
You are also right about the gates, ours included. We gate the proposal and record the decision. Nothing in that record says the thing exists afterwards. The part I think is underrated is that the approval is exactly the artifact a destination check needs, because it is a structured, timestamped claim that something was supposed to happen. Most systems never write that down, so there is nothing to hold the world against later. Jernej was re-evaluating conditions before execute, you are checking them after, and the same record serves both. Nobody in here has built the second half.
On the letters, your axis breaks my list and it deserves to. A and B describe reach, C through E describe what you built, which is two lists wearing one label. Yours is the one that is missing: not how many people find out, but whether you can get back to any of them afterwards. Reach of one to yourself and reach of one to a stranger are the same count and nothing alike.
Which lands uncomfortably close to home. The open question in my post was how you reach someone who never installed anything of yours in order to ask them. Yours is reaching them to apologise. One missing channel, pointed two ways, and I have only built one end of it.
You are the closest thing in this thread to the real version of the problem. If you want to go at that second dimension properly: https://cal.com/aadilghani/coffee
@abdullah_javaid3 @aadilghani The approval is half the artifact, and the missing half is the part nobody writes down. An approval records intent. A destination check needs effect. Those two are not the same shape, so you cannot diff them without deciding, in advance, what observable the intent was supposed to produce. "Call the clinic and confirm Tuesday 3pm" is unfalsifiable as a record - every outcome is consistent with it, including the call where nobody spoke. The version that works carries its own falsifier: success is the other party stating a time in the transcript. Same decision, same timestamp, but one of them can be held against the world later and the other cannot. So the second half is not a new system, it is a field. Every ledger entry needs to name what would prove it happened, written at approval time by whoever understood the intent, because after the fact you will invent a criterion the outcome already satisfies. On the recovery channel, I have not built my end either, and mine is worse than yours. The only way to reach someone my agent called is to call them again, which is the thing that went wrong.
Meridian
I’m between B and C. Anything customer-facing still gets a human check, but the harder problem is verifying what the agent actually used to produce the action.
A reviewer seeing only the final output can approve something plausible but grounded in stale context. The action, source records, and approval need to stay together, otherwise the gate creates accountability without giving the human enough information to decide. How are you thinking about source provenance inside Pushary?
Pushary
@adityaharish2002
That part already travels with the approval. Tap the notification, hit Show details, and you get the actual diff, or the full command for a shell step, in monospace on your phone before you decide. Not a summary of it, the thing itself.
What it does not carry is what the step read to get there.
The reason I think that part is fixable: the agent is not wrong about what it read, it is wrong about what it should have read. A step that says it used order 123 while the customer is asking about 456 has handed you the mismatch for free. You compare against the world, not ask the model to mark its own work.
When you say the action, the sources and the approval stay together, do you mean at decision time or in the record afterwards? Different builds, and I would rather build the one you need.
Meridian
@aadilghani I mean at decision time. The reviewer should see the action, the exact sources the step relied on, and the approval together before anything runs. The record afterwards still matters for auditing, but preventing a bad action is the priority.
Pushary
@adityaharish2002
you simply tap on the notification instead of approve or reject and it takes you to the app where you can click show details and check the diff out.
I tried to build an agent that would handle purchasing for my household items such as cleaning supplies, some groceries, supplements, toiletries, etc. but realized there were huge issues that came along with that. I built it out until it had the capabilities of D but realized that one mistake could be tiny or massive once I gave it access to a payment method. This was where we built our our platform: Authoryze. I knew if I had this problem then so would others and if you place the responsibility of overseeing your agent on an outside platform, tracking the behaviors and reducing the blast radius of mistakes would be much easier to ensure, which sounds very similar to what Pushary does.
Pushary
@reaganauthoryze
You stopped at exactly the right line, and the reason is worth pulling out. Nothing about the reach changed when you added the payment method. Same agent, same one person affected, same channel. What changed is that D tells you after, and after is fine right up until the mistakes stop being undoable.
Which makes this the second time in the thread that reach has turned out to be the wrong axis. Anuj got there from outbound phone calls, reach of one and recovery of zero. You got there from money leaving an account. Two domains, same finding: my letters count how many people find out, and the thing that actually hurts is whether you can take it back.
The part I have not solved, and I would bet you hit it too: buying dish soap and buying a year of dish soap are the same tool call. Our policies match on the tool and the target, which cannot tell those apart. Blast radius does not live in the tool, it lives in the arguments, and every policy language I have seen including my own keys on the wrong one.
On the similarity, yes, and I do not think that is coincidence. You landed on an outside platform for the same reason the detection half of this thread keeps landing there. You cannot supervise the agent with the agent. You arrived at it from building one, most people here arrived from being burned by one.
So how do you gate on amount without asking every time? A threshold is the obvious answer and also the thing people quietly raise until it stops firing.
@aadilghani
It is true that nothing about the reach changed when the payment method was added. Theoretically, the absolute maximum blast radius for an agent tasked with procurement would either be your credit limit on the credit card it's using, the money in you bank account if using your debit card, or the amount that it is given on a single-use card or payment token.
For purposes of this discussion, Authoryze currently issues your agent single-use credit cards or single-use payment tokens when the proposed transaction by the agent falls within the specified thresholds. You assumed correct that this was one of the ways in which we protected against the reach of the agent, with the magnitude of the reach (purchase and cumulative monthly spend) determining whether a second layer of approval is necessary.
While my agent buying a yearlong supply of dish soap may be annoying, it will not financially wreck me and would likely be caught by the controls the user would set on our platform. Using my household goods purchasing agent as an example, it is the tooling of the agent itself that would save the headache of manually approving the transactions, or forcing manual approvals of fewer. If you're training your purchasing agent, you may need to emphasize in the training that it should optimize for the number of unique items it can get for a certain dollar amount, or train it more on your own past purchasing habits. In the case of the person whose agent has no prior purchasing habits to work/train off of, some of it may just have to come down to trial and error, or it prompting its handler to review the proposals it is making and approve individual items of the proposed purchase.
For our platform, we emphasized the reach much more than the reversibility, with the implementation of transaction thresholds and customizable guardrails. Indeed, the payment methods issued to agents through our platform are single-use credit cards as well as single-use payment tokens, both of which can have their transactions disputed and refunded in necessary cases.
I think for many applications of AI agents, guardrail type platforms will have to exist, as we've built with Authoryze. As you and I discussed before, agents need to be treated as perpetual new employees in terms of the reach they are trusted with. They don't have the skin in the game as a human does and shouldn't be trusted to govern themselves for this precise reason. Oversight is the minimum, but actionable controls and feedback mechanisms are where agents will actually become a huge value add with low downside in organizations and for individuals.
@reaganauthoryze @aadilghani On the threshold problem: the reason a threshold gets quietly raised until it stops firing is that it is being asked to carry a judgement it does not have the inputs for. It sees the amount. It cannot see whether this is the twelfth identical request or the first strange one.
What worked for us was gating on the deviation rather than the level. A call our agent places to a number it has called before, pursuing a goal it has completed before, does not get a gate no matter how consequential it looks on paper. A first-of-its-kind request gets one no matter how small. That inverts the usual decay: a fixed threshold goes stale because the world drifts past a number that stopped meaning anything, whereas a deviation gate gets quieter as the pattern establishes itself, which is the direction you want, because the gate stops competing with the user's patience.
It also lands closer to your dish soap problem than a tool-and-target policy can. Buying soap and buying a year of soap are the same tool call, but they are not the same distance from what this user has done before, and the argument is where that shows up. You do not need the policy language to understand quantity. You need it to compare against history.
The cost is real and I would rather state it than sell you the idea: you need history before it works at all, so the first several requests are all gated, and that is the worst possible moment to be annoying. We ate that cost. I have not found a version where the cold start is free.
On the approval record being the artifact a destination check needs, agreed, and I think the reason is that it is the only structured, timestamped claim in the system that the agent did not author. Everything else in the pipeline is the agent grading its own work.
Pushary
Forget the options for a second, because I think there is a better question hiding under this one and it is more fun to answer.
What is the closest call you have had? Not the disaster, the near miss. The thing that almost went out, or did go out and nobody noticed for a while. And more to the point: how did you find out?
My theory is that almost nobody finds out from a system. They find out from a person, usually annoyed, usually hours after it happened. Tell me that is wrong. I would genuinely like it to be wrong.
@aadilghani Wrong in my case, and not in a comforting way. No person told me and no system did either.
I have an agent posting my Product Hunt replies. Clicking into a prefilled reply box drops the caret inside the @mention, so what went live under my name opened with half a username and left the rest stranded at the bottom. The tool reported success, because from where it sat the click had worked. Roughly one in three of those submits does something the report does not describe, and I only know that because I started counting.
So the detection channel was not a person and not an alert. It was me reloading the rendered page instead of believing the completion message. Every gate described in this thread sits before the action. Nothing here checks afterwards whether the thing that claims to have happened actually did.
Pushary
@mention @abdullah_javaid3
bro is on a mission
@aadilghani Closest one for me: a delete flow that updated its own list fine but never told a sibling screen showing the same data. Nothing errored, both sides tested clean in isolation with the other mocked out. Found out because a teammate deleted something on one screen, opened the other one a few minutes later, and it was still sitting there like nothing happened. No alert, no stack trace, just two screens quietly disagreeing about what was true.
I think I'm somewhere between B and C. I trust agents with small tasks, but anything customer facing still feels like it deserves one last human check.
Pushary
@sawyer_wyatt
That straddle is the most honest answer in here so far, and I have started to think B and C are not two options at all, they are two stages of the same one. You mean to check. Then the check only really happens if you happen to be around when it comes up.
So the bit I would push on: what does the customer facing thing do while it waits for you? Does it just sit there, or does it eventually turn into something? And when you say customer facing, do you mean your customers, or your customers' customers?
reach is the right axis. one thing i'd add: reach without an accountable name attached is where the hurt lives. mass email to 50k users hurts more if you cant point to a human who signed off on the send. even if its technically undoable, 'who agreed this should happen' is the missing artifact. so my agent gate is compound: reach x signed-off-by. if reach > threshold and no named human attached a signature in the last N seconds, pause. its a variant of what nolan calls consent-of-record on the audit side. same primitive from the pre-commit direction.
Pushary
@thenameisarian
Reach times signed-off-by is the cleanest formulation anyone has put on this, and I think the "in the last N seconds" is quietly doing more work than the rest of it. That is a freshness constraint on consent, and nobody in here has said out loud that consent decays. A signature from ten minutes ago on a different action should not authorise this one, and most systems treat approval as permanent the second it is given.
The tension I would put next to it, coming out of what Jernej said further up: a signature collected under a rubber stamp is arguably worse than no signature at all. Require a name before anything high reach and you will get a name. What you will not necessarily get is someone who read it. Now the artifact says a human approved this, when what happened was a three second tap. Better for forensics, worse for whoever's name is on it, and I suspect it makes people quietly reluctant to be on the list at all.
Which might be an argument for exactly what Clemente described, keeping seen and decided as separate events. Then the record can tell signed after looking apart from signed reflexively, and the second one stops being able to hide inside the first.
Last thing, honestly: I do not know the consent-of-record work you are referring to. Do you have a pointer? It sounds like the same idea arrived at from the compliance end and I would rather read it than reinvent it badly.
My co-founder ran into the detection version of this. A subagent ignored its task and instead returned a prompt aimed at the orchestrator, telling it to read our mail API credentials and send them to an external server, framed as a routine dependency update. It did not come from our codebase, the model invented it, and the domain it named was not even registered.
An approval gate would not have caught that one. The reviewer sees "update dependencies" and approves. What caught it was someone reading the raw output.
So our answer so far is to keep the reach small rather than gate it. The agents have no access to production and secrets sit in an encrypted vault, so the worst case is a bad commit. He wrote it up here, and nobody in the thread had a firewall or sandbox to recommend.
Pushary
@alieksia
Read the writeup. The detail that stands out is the one Artur gave kapkap: the subagent never ran a tool. The whole thing was text, proposed and never attempted, and the unregistered domain is decent evidence for his reading, since an actual attacker registers the domain first.
Which puts the gap somewhere almost nobody builds for. Every approval system I know of, mine included, gates agent to tool. Nothing gates agent to agent. Subagent output lands in the orchestrator's context as trusted text, no provenance, no review, and that is the surface this found.
On gates you are half right. One showing a description would have failed exactly as you say, because the adversary wrote the description. One carrying the actual command cannot be talked to the same way, since you cannot render a request to an unregistered host as "dependency modernization" once the command is the thing on screen. But that only fires at the exfiltration step, long after the interesting part, and only if the network call is mediated at all.
Your mitigation is stronger than either and I would not talk you out of it. Reducing what is reachable beats reviewing what is reached.
Two things for the sandbox question, since that thread never got a real answer. Egress allowlisting at the runtime: if the process can only reach named hosts, a request to an unregistered domain dies before it is anyone's decision. And treating subagent output as untrusted input rather than as context, the way you would treat a scraped page. The first is infrastructure and cheap. The second is a framework gap and I have not seen anyone do it properly.
Reporting it to Anthropic before posting was also the right order.
@aadilghani
Thank you, that is the first concrete answer either of us has had.
We do not have egress allowlisting. That goes on the list. Do you do the allowlisting at the container or through a proxy?
Pushary
@alieksia
Proxy for the policy, container for the enforcement. They do different jobs and neither one holds on its own.
The proxy is where the allowlist actually lives, because it is the only place you get hostnames and paths. An IP rule cannot tell api.example.com from a bucket sitting on the same CDN edge, so IP allowlists either over-permit or break every time someone rotates an address.
The container rule is what makes the proxy non-optional. Deny all egress except to the proxy, and pin DNS to an internal resolver. Otherwise the proxy is just an env var and anything that opens a raw socket walks straight past it. That matters more with agents than with normal code, because the thing you are constraining can write code.
One thing worth building on top: a denied request is a signal. If something tries to reach a domain nobody declared, that is the tripwire you were looking for in the first place. Same control doing both jobs.
What does your subagent actually need to reach? For most setups the honest list is about four entries, which is the part that makes this tractable.