"Did the model get worse?" Usually the wrong question. Your agent is two version numbers

Here's a debugging pattern worth stealing: when your AI coding agent suddenly feels different, before blaming the model, ask which of its two version numbers actually moved.

Because what you call "the agent" is a pair. The model itself never executes anything — it reads what it's given and proposes. The program wrapped around it (the harness) does everything else: assembles what the model sees each turn (its own system prompt, every tool definition, your rules files, the trimmed conversation), runs whatever the model proposes, handles the errors and retries, manages what stays in the context window as the session grows, and decides when the loop stops. One message from you can be forty laps around that loop, and every design decision inside it was made by the tool, before you typed a word.

That layer moves real outcomes. One major provider's blog on agent harnesses says plainly that even a frontier model, run in a loop with nothing but a high-level prompt, falls short of production-quality work; the difference is the structure around it. And academic work on agent scaffolds keeps finding the same thing: identical model weights land at meaningfully different resolve rates and wildly different token bills depending on the loop they're wearing.

Two habits that follow. First, log both versions: harnesses ship far more often than models, so when behavior drifts, the tool is usually the prime suspect: a changelog line like "improved tool descriptions" quietly rewrote what your model reads every turn. Second, only compare models inside the same tool. A model that looks smarter elsewhere may just be wearing a better loop, and cross-tool comparisons measure the pair.