We were optimizing output tokens to save money. Turns out 95% of our bill was input.

Analyzed a day of token logs across an autonomous coding agent setup running on internal codebases. The raw count: 769M input tokens vs 7.4M output tokens (~104:1).

Because long agent runs re-read session history (files, AST diffs, test outputs) every turn, input costs accounted for ~95% of total spend. Optimizing output length turns out to be looking at the wrong variable.

Three things actually saved us money:

1. Routing: Shifted non-interactive workloads (evals, background analysis) to batch/flex channels. Billed at 0.5x list price with zero code logic changes.

2. Cache Discipline: Kept system prompts strictly byte-stable (no top-level timestamps). Achieved a 94.9% prompt cache share, driving input costs from $10/M down to ~$1.46/M blended.

3. Context Compression: Built a pipeline sending compact session representations instead of verbatim transcripts. Achieved 2.83x median compression (fitting ~500K session history into a 200K window).

What’s currently the biggest bottleneck in your API spend—input history, output length, or model hallucination loops?