RunAI Coder

Two hours into a refactor yesterday, my agent wrote a helper function the exact same one it wrote 90 minutes earlier in a file it created itself. Then it apologized. It always apologizes.

The easy diagnosis is "it ran out of context". Except my session was sitting at 120K in a 200K window. Nothing overflowed. The context didn't run out it rotted.

You have two budgets, not one:

• Hard Budget: Token limit. You notice it when the API errors out.

• Soft Budget: Attention quality. It drains silently long before you hit the limit.

Analyzed a day of token logs across an autonomous coding agent setup running on internal codebases. The raw count: 769M input tokens vs 7.4M output tokens (~104:1).

Because long agent runs re-read session history (files, AST diffs, test outputs) every turn, input costs accounted for ~95% of total spend. Optimizing output length turns out to be looking at the wrong variable.

Three things actually saved us money:

1. Routing: Shifted non-interactive workloads (evals, background analysis) to batch/flex channels. Billed at 0.5x list price with zero code logic changes.

2. Cache Discipline: Kept system prompts strictly byte-stable (no top-level timestamps). Achieved a 94.9% prompt cache share, driving input costs from $10/M down to ~$1.46/M blended.

3. Context Compression: Built a pipeline sending compact session representations instead of verbatim transcripts. Achieved 2.83x median compression (fitting ~500K session history into a 200K window).

Hey Makers!

Handing over repo access to autonomous coding agents is awesome, but honestly, it's also a great way to break things if you aren't careful.

My team recently set up a strict "Zero Trust" baseline to keep things under control. Here are the 5 rules we actually enforce right now:

1. Bound the blast radius before the first run

2. Assume it reads everything it can reach

3. Treat everything the agent reads as a potential instruction

4. Verify like it's a stranger's pull request

5. Be ready to do forensics