Wispr Flow: Dictation That Works EverywhereStop typing. Start speaking. 4x faster.
Promoted
Launched this week
IQ Routing
Trajectory-aware LLM routing that cuts agent cost
75 followers
Trajectory-aware LLM routing that cuts agent cost
75 followers
IQ Routing is a drop-in gateway. We classify each request, serve from cache, and route to the cheapest model that clears the quality bar and completes the task. The insight others miss: an agent is a trajectory, not a stream of independent calls. We route each step based on where it sits in the run: a cheap model on boilerplate, the strongest model on what matters.
Does the cache reuse cross runs? That is the piece I can't place under your own trajectory framing. If it does, a step gets handed an answer produced under a different set of earlier steps, so a request that reads identically can sit at a very different point in the run.
Haven't tried it yet so this may well be handled. It's the first thing I'd look for in the docs.
@dimhold The cache key for any request with prior conversation turns is built from the full history up to that point, not just the current message. A cache hit only occurs when the entire trajectory leading into that step matches as well, so a change anywhere earlier in the run changes the key and produces a miss instead of a hit. That’s the mechanism that rules out a step being served an answer computed under a different set of earlier steps.
The one exception is the first call of a run, before any history exists to hash. That request is matched on content alone. Since there’s no prior-step context at that point, it falls outside the scenario you’re describing.
@george_avila That closes it, thanks. Full-history keying was the part I could not see from outside.
The first call is the one I would still watch. My own runs open with a byte-identical first message every time, same system prompt and same task template, while the repo underneath has moved. A content-only match hits there. What comes back was computed against a different working tree.
I have not run IQ, so I may be reading that exception too broadly. Do you scope the first-call key by anything outside the message?
@dimhold A cached response is matched only on the content of the request itself, meaning the instructions and the message text, rather than on anything about your environment or repository state. Given that, an identical opening message can return a cached response even after what’s actually on disk has changed since that response was generated, and the same holds for near-duplicate opening messages, not just exact ones.
To limit that exposure, cached responses expire on a schedule rather than staying valid indefinitely, and responses involving tool use are given a much shorter shelf life than plain text responses, since they typically depend more on something outside the request itself. That narrows the exposure window, but it does not close it entirely.
If a live call is ever required for a specific request, sending a request header (x-iqr-cache: off) opts that call out of caching entirely, and nothing else about it changes.
@george_avila That is the answer I was after, thanks. The shorter shelf life for responses that involve tool use is the part I would not have guessed.
For my case the header is the simple fix: set x-iqr-cache: off on the first call of a run and leave everything after it cached. The opening message is the one place where identical text does not mean identical world.