Hey Product Hunt 👋

I'm Mohamed ABidi, an engineer. I build products on top of LLM APIs, and for a while my bills kept going up faster than my usage.

So I started looking at what we were actually sending. Most of it was context the model never needed: ten search results when two had the answer, raw tool output, logs, old messages. In agents it was worse, because every step resends everything from the steps before.

The clearest example: our demo agent ran a test suite, and the tool returned 7,209 tokens of passing tests and progress bars. The model needed the 2 failures, which were 156 tokens, and the rest was billed again on every turn after that.

I tried the usual fixes first: → Fewer search results: cheaper, but sometimes the answer got dropped → A cheaper model: cheaper, but quality dropped → Prompt caching: it helped, but it only discounts the repeated part. The noise is still there, just billed at a lower price. → Trimming by hand: it worked, but it broke the cache, and I had to rewrite it for every agent

Contextwaise is the tool I wanted instead. Before each call, it scores every item against the task and removes what the model doesn't need. It works on RAG results, tool output, web pages and memory.

The parts I cared most about getting right:

• Your prompt cache keeps hitting. Each item is decided once and replayed byte for byte, so trimming doesn't cancel out your caching savings.
• Nothing is silently lost. Every removal is logged with its reason, and the model can call expand() to get trimmed parts back.
• Easy to add. About 100 ms per call, any LLM provider, SDKs for TypeScript and Python. Wrap your model and tools, with no new model and no migration.

On our own demo agent, tools returned 52.6K tokens and the model read 9.1K. Cost went from ~$0.129 to $0.057. That's our test setup, so I'd love to see what it does on yours.

Start free with 1M tokens at contextwaise.com.