Bigger context windows aren’t a retrieval strategy

When an AI application misses something, the tempting fix is to retrieve more. More chunks, more documents, maybe the entire attachment. Congratulations, you’ve turned your context window into an expensive filing cabinet.

A Graphwise experiment illustrates the alternative. Across 11 questions over a dataset covering 6,000+ companies, graph-grounded retrieval used roughly 75,000 context tokens versus over four million for context stuffing. About a 54× reduction. Their write-up

That’s a focused experiment, not a universal benchmark or a claim of 54× lower total costs. Building and maintaining the graph costs something too.

But the architectural lesson is useful. Relationships resolved during ingestion don’t need to be rediscovered from raw text on every request. Retrieval can follow explicit connections and return targeted evidence rather than asking the model to reconstruct the connections itself.

We’re hearing the same pattern from yggy customers and partners: substantial token savings alongside more accurate, complete answers. The magnitude varies by workload.

The catch is that building this structure traditionally takes considerable modelling and integration work. That’s the part we’re automating with yggy:

• Discover the ontology from existing data and documents.

• Resolve entities across sources and surface conflicting claims.

• Preserve source references so retrieved facts remain inspectable.

• Expose the graph through REST API and MCP for applications and agents.

The goal isn’t the smallest possible prompt. It’s the smallest sufficient context for a correct, complete, evidence-backed answer.

For anyone evaluating retrieval architectures, are you measuring context tokens per successful answer, or mostly looking at model pricing? We’d love to compare notes.