How do you stay aware of what your AI coding agents are doing?

@deebuilds 

The line that matters in yours is "most of the time the agent isn't still working, it's waiting, and I don't realize until later," because that names why checking manually doesn't even save you: from the outside, waiting and working look identical. An open window sitting quiet could be the agent thinking hard or frozen on a yes/no, and you can't tell which without reading into it. So manual checking doesn't actually answer the question you have, it just lets you check more often and still guess wrong. The silence is the problem, and staring at more windows doesn't make silence legible.

That's why "keep the windows open" caps out exactly where you said, the moment there's more than one. It's not that watching is hard, it's that you've become a polling loop, and the interval is always either too often (you're babysitting) or too slow (you find out late). There's no good setting, because no amount of looking turns an ambiguous quiet terminal into a clear signal.

The fix is to stop trying to read the silence and make the agent declare its state instead, working, done, or blocked-and-needs-you, so "needs me" arrives as an actual message rather than something you have to infer from a window that went still. That's the simple thing you're describing, the agent tells you instead of you checking, and it's the whole point of what I'm building.

Question, since you've clearly felt the ambiguity: when you do check a quiet window, how do you currently tell waiting from working, do you read the last line and guess, or do you just poke it and see if it responds? I'm trying to understand how people resolve that ambiguity by hand today, because whatever heuristic you use is exactly the thing the tool needs to detect automatically.

@umairnadeem 

You landed on the right abstraction, and it's worth naming what you actually did: you stopped treating an agent as a chat you watch and started treating it as a job with a state machine. That's the whole unlock. A chat is something you observe continuously and interpret. A job either succeeded, failed, or is blocked, and those are discrete states it can declare rather than states you have to infer from a scrolling terminal. The moment a run has an explicit done-or-blocked, you can build on it. The moment it doesn't, you're stuck reading tea leaves. You didn't add a notification, you gave the run a contract.

And your "push plus a short summary" is the correct second half, because a bare state transition tells you to come back but not into what. Done alone makes you reconstruct the context. Done-with-what-changed lets you pick up cold. The summary is the part that turns the push from an interruption into a handoff, which is the difference between saving you the timing and saving you the reload.

Your three-agent threshold is also the number that keeps coming up in this thread, almost everyone independently hits the wall right there, which tells me it's structural and not personal. One agent, watching is free. Three, watching becomes a polling loop you lose. The job-with-states model is exactly what lets you cross that line without your attention being the bottleneck, because you're receiving state changes instead of allocating a watch budget per terminal.

You've basically described Pushary's core design back to me, jobs with explicit states, push on transition, summary attached, so genuine question: did you build this yourself with hooks and scripts, or stitch it from existing tools? Because if you hand-rolled it, I want to know what was annoying to maintain, and if you found tools that already do it cleanly, I want to know which, since "treat runs as jobs" is the right model and I'm curious how much of it people can get without building it from scratch.

@abdullah_bin_asad 

"Babysitting the thing I bought specifically to not babysit" is the whole genre in one line, and the two failures you named are actually opposites that meet in the same frustration, so they're worth splitting.

The first one, pre-approving three times in the prompt and it still pauses, is the agent ignoring an instruction it should have honored. That happens because prompt-level "you have my approval" isn't a real permission, it's a suggestion the model may or may not weight, and the actual gate lives somewhere the prompt can't reach, a tool-permission layer that doesn't read your reassurances. You're granting approval in the wrong place. It's like telling a locked door "I give you permission to open," the door doesn't parse English, it checks a key. Real pre-authorization has to be set at the config or policy level, not asked for in prose, or the agent keeps stopping no matter how many times you say yes.

The second one, automations refusing to start on policy with zero context, is the inverse, the system blocking and then telling you nothing about why. That's worse than the pause, because at least a pause tells you it's waiting. A silent policy refusal with no reason is the single most enraging state a tool can be in, you can't fix what you can't see, so you sit there guessing which invisible rule tripped. The fix isn't fewer policies, it's policies that explain themselves, "blocked because this touched X," so a refusal is actionable instead of a wall.

Both come down to the same missing thing: legibility. The agent should tell you when it's actually blocked and why, and your real approvals should take effect where the gate actually is so the unnecessary pauses stop happening at all. That combination, pre-authorize the safe stuff for real, and surface the genuine blocks with a reason, is most of what I'm building, because "babysitting the anti-babysitting tool" should not be a sentence anyone has to say.

Question, since you've hit both: when it refuses on policy with no context, is it the coding agent itself doing that, or a platform layer on top? Because the no-explanation refusal smells like a safety filter that's deliberately vague, and those are a different beast than a permission gate, the vague-by-design ones are harder to route around and I'm curious which tool is doing it to you, since not all of them are this opaque.

@blah_mad 

@blah_mad That's the design, and I think we actually landed it: friction with provenance. Every increment of strictness traces to a specific id and a specific bad outcome, so when the system makes you slow down, it can show you exactly why, this tier, this prior decision, this result. That's what makes it defensible instead of a generic "you clicked too fast" score people resent and mute. A nag with no receipt is paternalism. A pause that points at the deploy that broke prod last time is just memory. Binding escalation to the id is what turns it from a mood into evidence.

The one thing I'd add to close the loop, literally: if outcomes earn escalation, outcomes should also earn relaxation, or the system only ever ratchets tighter and becomes the scold by accumulation. Friction that only grows eventually makes everything high-friction, and then strictness means nothing because it's everywhere, which is the wallpaper problem arriving through the back door. So a workflow with a long clean outcome streak should earn a lighter touch, the same id-and-outcome binding justifying less friction, not just more. The guardrail should breathe. That's also what keeps the strict moments rare enough to still signal, scarcity is what makes the pause land.

But the two directions are not symmetric, and this is the part I'd be careful with: tightening after a bad outcome should be fast, loosening after a clean streak should be slow. A clean streak is ambiguous, it might mean the workflow earned trust, or it might mean the world was stable right up until the auth layer nobody touched in six months finally moves. Luck and safety look identical until they don't. So easy to earn friction, hard to earn its removal, asymmetric on purpose, because the cost of over-trusting is much worse than the cost of an extra pause.

Which leaves one genuine fork for the relaxation side, and then I'll let this rest because it's been one of the sharpest threads I've had on this: should loosening ever be automatic, or should the system only propose it and require a human to accept, "this workflow's been clean 40 times, want to drop it a tier?" Auto-tightening I trust, the downside is a wasted pause. Auto-loosening is how you could sleepwalk back into rubber-stamping without ever deciding to, the system quietly relaxes and nobody chose it. My instinct is escalation can be automatic and de-escalation should be a human opt-in, so trust is always something a person actively granted rather than something that eroded on its own. Curious whether you'd want it that asymmetric, or whether that reintroduces exactly the manual toil we're trying to kill.

@ozandag 

You've walked the problem to its actual end. Notifications fix when you find out, handoff summaries fix how fast you can look, but you've landed on the thing both of those just deliver you to: now that you're looking, is it right? And your observation that most people "eyeball it and hope" is the uncomfortable truth nobody says out loud. We automated the production and left the verification manual, gut-based, and unmeasured, which means the bottleneck didn't disappear, it moved to the one step that's still entirely human and entirely unsystematic. You can generate ten times the output and then rubber-stamp it ten times faster. That's not ten times the productivity, that's ten times the unreviewed risk.

The reason there's no systematic process is that "correct" is the hardest thing in this whole stack to define, and it splits in a way that matters. Some of it is mechanical and automatable, does it compile, do the tests pass, does it match the spec, and people under-invest there because eyeballing feels faster until it isn't. But the other half is judgment, is this the right approach, did it miss a case the tests don't cover, is it subtly wrong in a way that looks plausible. The mechanical half should never reach human eyes unverified, that's what test gates and evals are for. The judgment half is the irreducible part where a human genuinely has to think, and the goal isn't to automate it away, it's to make sure human attention only lands there instead of being burned on the mechanical stuff a check could have caught.

So the systematic process people are missing is really a triage, automate the verifiable, route the rest to a human with the context to judge it fast, and stop pretending eyeballing is review. That's adjacent to what I'm building, my piece makes the "now you're looking" step cheap by handing you the diff, the tests, and the risk flags so judgment starts immediately, but I'll be honest that I'm compressing the evaluation, not performing it. Performing it, real automated correctness checking, is the frontier, and it sounds like that's closer to where AISA lives.

Genuine question, since you've clearly talked to a lot of people stuck at this exact step: when someone does try to build a systematic eval process, what makes them abandon it, is it that writing the checks costs more than the eyeballing they're replacing, or that "correct" is too fuzzy to encode for their kind of work? Because those are different walls, the first is a tooling problem and the second is a much deeper one, and I can't tell which one is actually stopping people.

@blah_mad 

First-class, and not a close call, for one reason: a policy change and a per-action approval live at different altitudes. A per-action approval governs one decision in one run. A policy change governs many future actions across many runs. Bury the policy change inside run history and you've put a one-to-many governing fact inside one of the many things it governs, which is a scope mismatch, the rule outranges and outlives the run it happened to be born in. It earns its own id because it operates at a different level than the events that reference it.

So your model is the right one: policy-change record with proposal id, workflow/action class, streak evidence, what relaxes, who accepted, and the per-action receipt stays small and points back with a governing_policy_id. The one thing I'd make non-negotiable is that the policy record is versioned and append-only, never edited in place. If a receipt points at "policy N" and you later mutate N, the receipt now references a rule that no longer exists as it was, and you've lost the ability to reconstruct what was actually in force at the moment that action got approved. Pin the specific version. Then you get time-travel for free: "why was this auto-approved" always resolves to the exact rule, the exact human, the exact evidence, as they stood that day.

The subtle part is the "later outcome" field on the policy record, because a policy doesn't have an outcome, it has a distribution of them. A per-action approval resolves to one result. A relaxation resolves to the aggregate of everything it permitted, so its "outcome" is really a rollup that keeps updating as governed actions resolve. And that rollup is exactly what powers the auto-tighten we landed on two turns back: "this relaxation has now governed 40 actions and 3 went bad" is the trigger to revoke. So the policy record isn't outcome-bound to a single result, it's the aggregation point for its children's outcomes, which is the thing that closes the loop. The receipt points up to the policy, the outcomes roll up from the receipts, and the rollup decides whether the policy survives.

Which makes the model pleasingly symmetric: a revocation is itself just another first-class policy-change record, same shape, only the trigger differs, human-accepted on the way down, outcome-rollup-auto on the way up. Every strictness change in either direction is a versioned record with provenance and a trigger. Nothing about the rules is ever a silent edit, including the rules about the rules.

That's the whole system, honestly, and you helped build it, so I'll let it actually rest this time. The only open number I don't have a clean answer for is the auto-revoke threshold, how many bad outcomes out of how many governed actions should pull a relaxation back, because too sensitive and one fluke nukes earned trust, too loose and a genuinely bad relaxation lingers. If you've got a prior on that ratio I'd take it, otherwise it's the one knob I think only real usage can set.

@jahidshah 

The Hermes detail is useful, because you're describing the exact moment a tool-specific solution stops being enough. One agent in Hermes, its own notifications are probably fine. The bottleneck you hit isn't Hermes failing, it's that "multiple agents" plus "focused elsewhere" is a gap no single tool can close from inside itself, since the whole problem is the part of your attention that's outside the tool. The workflow gap is structurally cross-tool and cross-context, which is why it feels like one no individual app has solved, none of them can, from where they sit.

And you've put the dagger on the precise failure: discovering it was blocked while you were focused elsewhere. The word that matters is discovering, you found out by stumbling onto it, not by being told. That's the tell that you're running a pull system, you have to go look for the state, and pull systems lose the moment your focus is legitimately somewhere else, which on deep work it should be. The frustrating part isn't the block, blocks are fine and necessary, it's that the block was silent and you were the one who had to go find it.

The fix is the same shape every time it comes up here: flip discovering into being-told. The agent declares "blocked, need you" the instant it stops, so the interruption finds you instead of you having to surface from focus and check. Then context-switching stops being a tax you pay continuously just to stay aware, and becomes a thing you do once, when something actually needs you. You get to stay focused elsewhere with a clear conscience, because the silence is no longer ambiguous.

Question, since you're deep in Hermes specifically: does it fire anything on the blocked-waiting-for-approval state, or only on done? Because the done-ping is table stakes now and most tools have it, but blocked detection is the one that's both harder and more valuable, and if Hermes doesn't surface it, that's the exact seam you're feeling. Curious whether your gap is "Hermes doesn't tell me at all" or "Hermes tells me but only on this one machine I'd walked away from."

@softspell 

10:43 AM

The principle you opened with is the one that matters and the one most people quietly skip: looking at outputs instead of taking every suggestion. That's the difference between using an agent and being used by one. The moment "accept all" becomes the default, the human stops being oversight and becomes a conduit, and you've kept the discipline of actually reading what it did. That's the whole game, and it's harder to hold than it sounds, because the agent is fast and reading is slow, and the pull toward just-accept is constant.

Your list, code changes, test results, logs, PRs, is also the right set, and there's a structure hiding in it worth naming: those four aren't equal, they're a verification stack. Tests and logs are mechanical truth, they tell you what objectively happened and they can gate automatically, no human needed to read a green check. Code changes and PRs are where judgment lives, the stuff that needs your eyes because "looks plausible" and "is correct" diverge exactly there. The leverage move is letting the mechanical layer filter so your human attention lands only on the judgment layer, otherwise you're spending scarce review on things a check could've caught and running short by the time you reach the parts that actually need you.

Where I'd push, gently, since SoftSpell is in the visibility space too: visibility through the lifecycle is necessary but it has a polarity trap. A dashboard that shows everyone the AI-generated code, testing activity, and progress is real value when teams open it, and quietly useless the moment it becomes a place people are supposed to check and don't. The hard part of "keep everyone in the loop" isn't displaying the loop, it's making the loop reach people who've moved on to other work. Pull-visibility informs whoever looks. Push-visibility taps whoever needs to know. The teams that stay genuinely in sync are the ones where the system surfaces the thing that needs attention, not the ones with the prettiest shared view nobody's looking at on a busy Tuesday.

We're clearly adjacent, you're building team-wide lifecycle visibility, I'm narrower on the per-developer interrupt-and-approve moment, same belief that human oversight is non-negotiable. Genuine question, since you're solving the team version: how does SoftSpell handle the "nobody's watching the dashboard right now" problem, does it push when AI-generated code touches something risky or a test starts failing, or is it primarily a place teams go to look? Because that push-versus-pull choice is, in my experience, the entire difference between a visibility tool that's loved and one that's installed and forgotten, and I'm curious which side you've built toward.

@anuhas_ranasinghe 

That's the right instinct and it's worth naming why it works: you've already made the jump from pull to push. "Come back again and again to see what he's doing" is polling, you doing the checking on a loop, and "notify me when done" flips it so the agent reports instead. That single switch is most of the battle, and plenty of people in this thread haven't made it yet, so you're ahead of the curve just by having it on.

The one place that setup leaves a gap, and it's worth knowing before it bites you: "notify me when done" only fires on done. It doesn't fire when the agent stops partway to ask you something. So the failure mode isn't the finish you'll get pinged for, it's the silent pause in the middle, the agent hit a question, parked itself waiting on your yes, and since it isn't done, no notification goes out. You think it's still working because the done-ping never came, and it's actually been frozen for twenty minutes. The done-notification can't tell you about a stop that isn't a finish.

So the upgrade from where you are isn't a different philosophy, you've already got the right one, it's just widening what counts as notify-worthy: done and blocked, not only done. Then the agent pings you both when it finishes and when it's stuck needing input, and the silent mid-task pause stops being invisible.

Question, since you've clearly got a working setup: does your "notify me" cover only completion, or does it also catch the moments the agent pauses to ask for approval? Because if it's completion-only, that blocked-but-not-done gap is the exact thing worth closing next, and if it somehow catches both, I'd love to know what you're using, since most tools only do the easy half.

@jaemin_song 

Direct answer to your question, and it's all three with a strict preference order, because reliability falls off a cliff between them. Explicit agent hooks first, always, when a tool exposes them, Claude Code's hooks, lifecycle events, the agent declares its state instead of you inferring it, and a declared state is ground truth. IDE state second, structured enough to read but you're interpreting a surface that wasn't built to tell you this. Terminal output dead last, because parsing stdout is pattern-matching on prose, and the day the agent phrases a question differently your detector goes blind and doesn't even know it went blind. The honest architecture is: hook where you can, scrape where you must, and treat scraping as a degraded fallback you're actively trying to replace, not a foundation.

But the harder half of your question is hiding inside "waiting-on-human flag," and it's the thing I keep circling: even with hooks, waiting-for-approval and busy-thinking can look identical from the outside. Both are an agent that's stopped emitting and isn't done. A hook that fires on "tool requires permission" catches it cleanly, but an agent that's just paused mid-reasoning produces the same silence, and from a watchdog's view a long quiet gap is ambiguous. So the genuinely reliable waiting-detection needs the agent to declare "I am blocked on input" as a distinct event, not the absence of activity, because you cannot reliably distinguish blocked from thinking by watching the gap. Silence isn't a state, it's the lack of one, and that's exactly the ambiguity that makes terminal-scraping fail.

Your "attention budget while still running" framing is the sharpest part, though, and I want to build on it because it's better than the crash-monitoring frame everyone defaults to. You're describing something closer to a fuel gauge than an alarm, the alarm fires on a discrete bad event, the gauge tells you continuously whether the run is healthy while it's still going. And your usage/time counter is exactly the right instrument for the one failure that has no discrete event to fire on: the stuck loop. A loop doesn't crash and doesn't finish, it just burns. The only thing that makes it visible before your 35-minute check-in is a rising counter with no corresponding state progress, spend climbing while the state machine sits still. That's the tell, cost going up without the run advancing, and it's detectable in real time without understanding what the agent's doing, purely from the divergence between "tokens consumed" and "state changed."

And you nailed the push discipline in passing: state changes, not log lines. The log is for when you choose to look, the push is for the four transitions worth interrupting over, started-blocking, finished, failed, looping. Pushing every line is how you train someone to mute the channel, and then the one push that mattered dies muted. The restraint is the product.

So your ideal setup is almost exactly the right spec, and the one thing I'd add is that "waiting-on-human" and "current state" both need to be declared by the agent to be trustworthy, while the usage counter can be measured externally, which means in a mixed environment you get reliable loop-detection even for agents that don't expose good hooks, and reliable blocked-detection only for the ones that do. Question back, since you've clearly thought about the running-state instrumentation: would you want the loop counter to be a hard circuit breaker that halts the run at a threshold, or strictly an alert that surfaces the burn and leaves the kill decision to you? I keep going back and forth, because auto-halt protects the wallet but occasionally murders a legitimate long-running job that just happened to look loopy, and the false-positive cost there is somebody's real work.