A bigger context window doesn't mean better answers

It's tempting to send everything now that models accept huge inputs. But Chroma's context-rot study tested 18 models, and every one got less reliable as the input grew, long before the window was full.

On LongMemEval, models answered better with about 300 relevant tokens than with the full 113K history that contained the same answer.

So trimming isn't only about the bill. Less noise often means better focus.

Our approach with Contextwaise:

• Keep what's relevant to the task, drop the rest

• Keep your stable prefix untouched, so prompt caching still hits

• Works with any LLM provider (TypeScript + Python SDK)

Do you send full history by default, or do you already filter? What's worked for you?

Source: trychroma.com/research/context-rot