Where does the scope "live"?
I’ve run into this with longer agent runs and I’m curious how others handle it.
The original issue might contain the scope, constraints and evidence required. But when another model or session picks up the work later, those conditions often get reduced to a summary.
The work continues, but the original spec is no longer necessarily what the agent is working against.
I’ve stopped treating chat history as the source of truth. The original spec stays outside the agent, and the output has to be checked against that same version before I accept it.
For people running work across several agents or sessions, where do you keep the authoritative scope? In the issue, a state file, the orchestrator or somewhere else? How do you notice when the work has drifted?
Replies
Same problem here. I moved scope into a versioned YAML file the agent has to reload every session instead of trusting summaries.
I treat the original issue like a contract and never let it get compressed. My orchestrator pulls it fresh each handoff. The moment I let a summary stand in for the real thing, scope started slipping in ways I didn't notice until review.
For me the orchestrator owns scope, not the model. Each session gets a fresh pull of the original issue plus constraints, never a summary of a summary. I've seen too many runs where "reduced to essentials" quietly dropped an edge case that mattered. Re-fetching the source every time fixed most of it.
What changed things for me was moving the small number of rules that actually hurt out of prose and into tests. My codebase has a naming convention that reads backwards if you get it wrong, and agents reintroduced the wrong version constantly no matter how loudly the spec said not to. Now a test asserts the old names appear nowhere outside five allowlisted files. Same for a handful of marketing claims we can't back up. The rules didn't get better written, they got executable, and a drifted run fails CI instead of depending on whoever is paying attention that day.
The spec file is still there and I still pin the revision. I just stopped expecting it to enforce anything. And it only works for rules you can state as a check, which is maybe a fifth of what's in the spec...
I keep the immutable request and acceptance criteria in a versioned spec, then give each run a small mutable state file that points to the exact spec revision. Every handoff should record the revision, completed evidence, unresolved decisions, and any approved deviation; summaries are context, not authority. Drift becomes detectable when the final checker compares both the output and the claimed evidence against that pinned revision, and refuses completion if the pointer changed or a criterion has no proof. For long jobs, I would also hash the spec at start and recheck it before irreversible actions so a stale worker cannot silently finish against yesterday's scope.
My go to choice is the versioned SPEC md or a state file. And yes chat history is unrealiable because context compression will always alter the original spec. On top of that, the agent starts treating its own incorrections as truth and goes off track completely.
Thus, the best approach is to store the original spec in a versioned spec md or state file. To check if your work has deviated, simply run a lightweight evaluator agent or automated test to check output against that spec before accepting.
Note: It is important that you run it in a seperate session becasue running the evaluation in the same session will bloat the context and may also lead to a confirmation bias.
@siarheihamanovich 's shift from prose rules to executable tests is the part that actually holds up. A spec sitting next to the agent only works if someone rereads it every session, and that discipline erodes exactly when you need it most. Turning the rules that keep breaking into a check that fails the build removes the dependency on anyone's attention that day. Most of a spec still can't be phrased as a test though, so the two approaches end up needing each other.
@alex_gidirim The line between the two turned out sharper than I expected, and it isn't really about difficulty. Almost every rule I managed to make executable is a rule that forbids something. This name appears nowhere outside this allowlist. This claim appears in no user-facing string. Grep does all of it, and a negative rule is cheap precisely because you never have to describe the right answer, only the wrong one.
Everything I failed to test was a rule requiring judgement about a value. Is this error message actually useful to the person reading it, is this abstraction at the right level, does this copy read clearly. There's no assertion for those, and the couple I tried were worse than nothing, because they kept passing while the thing they were supposed to guard quietly got worse