LLMSlim
Contract-safe context tooling for LLM tool schemas.
Start new thread

LLMSlim - Semantic prompt compression that never drops instructions.

byβ€’
Open-source Python package cutting LLM token costs by 40–70% in 1 line of code. Surgically compresses prompts, RAG document contexts, and multi-turn chat logs with guaranteed 100% instruction fidelity.

Add a comment

Replies

Best
Maker
πŸ“Œ
Hey Product Hunt! I'm Yash, a CS/AI undergrad currently doing an AI research internship, and LLMSlim is something I built because I kept hitting the same problem in my own RAG pipelines: most of what I was sending to the API was filler. LLMSlim runs a deterministic, offline pipeline, no LLM calls, no telemetry, that scores every sentence on centrality, entity density, and instruction presence, then hard-locks anything matching an imperative ("must," "never"), a role marker, or a code fence before it's ever eligible to be cut. Everything else gets ranked and trimmed to hit your target ratio. It's early v0.2.0, MIT-licensed, 93%+ test coverage, and the benchmarks on the site are the actual numbers from the CI-gated benchmark suite in the repo, not rounded up. Latency ranges from ~24ms on short prompts to ~340ms on 12K-token documents, I'd rather you see the real range than a single best-case number. What's next: streaming compression (compress_stream), native ONNX embeddings for sub-5ms scoring, and async batch helpers, all tracked on the roadmap in the repo. I'd genuinely like feedback on where the compression gets it wrong. If you paste a prompt into the playground and it drops something it shouldn't have, that's exactly the kind of report that makes v0.3.0 better. Thanks for taking a look.
πŸ”Œ Plugged in

honestly i'm pretty impressed with how simple it was to drop in, basically one line and my prompts got way more compact without breaking anything. the fact that it holds onto the actual instructions matters a lot for my stuff.

Maker

Β Thanks so much for trying it out! Really glad the drop-in experience was smooth and that the instruction-locking held up for your use case β€” that's exactly the guarantee we designed around. Let me know if you run into any edge cases, always looking to improve.

πŸ”Œ Plugged in

love that it's genuinely one line of code with a fidelity guarantee, the kind of overconfident claim that usually falls apart under testing but seems to hold up here based on the benchmarks. the chat log compression is especially clever, that's where most token waste happens.

Maker

Β Appreciate you calling that out! I know a "never drops instructions" claim sounds like the kind of thing that breaks under real-world testing, so I wanted the benchmarks to be verifiable rather than just marketing copy. Chat log compression was actually the hardest part to get right, glad it's landing well.

πŸ”Œ Plugged in

got 50% fewer tokens on my rag pipeline with no quality drop on my eval set, super easy drop-in replacement

Maker

Β That's a great result, thank you for sharing it! A 50% reduction with no quality drop on your eval set is exactly the outcome I built LLMSlim for. If you're open to it, I'd love to hear more about your RAG setup, always looking for real-world benchmarks to learn from.