LLMSlim
Semantic prompt compression that never drops instructions.
Start new thread

LLMSlim - Semantic prompt compression that never drops instructions.

byβ€’
Open-source Python package cutting LLM token costs by 40–70% in 1 line of code. Surgically compresses prompts, RAG document contexts, and multi-turn chat logs with guaranteed 100% instruction fidelity.

Add a comment

Replies

Best
Maker
πŸ“Œ
Hey Product Hunt! I'm Yash, a CS/AI undergrad currently doing an AI research internship, and LLMSlim is something I built because I kept hitting the same problem in my own RAG pipelines: most of what I was sending to the API was filler. LLMSlim runs a deterministic, offline pipeline, no LLM calls, no telemetry, that scores every sentence on centrality, entity density, and instruction presence, then hard-locks anything matching an imperative ("must," "never"), a role marker, or a code fence before it's ever eligible to be cut. Everything else gets ranked and trimmed to hit your target ratio. It's early v0.2.0, MIT-licensed, 93%+ test coverage, and the benchmarks on the site are the actual numbers from the CI-gated benchmark suite in the repo, not rounded up. Latency ranges from ~24ms on short prompts to ~340ms on 12K-token documents, I'd rather you see the real range than a single best-case number. What's next: streaming compression (compress_stream), native ONNX embeddings for sub-5ms scoring, and async batch helpers, all tracked on the roadmap in the repo. I'd genuinely like feedback on where the compression gets it wrong. If you paste a prompt into the playground and it drops something it shouldn't have, that's exactly the kind of report that makes v0.3.0 better. Thanks for taking a look.

honestly i'm pretty impressed with how simple it was to drop in, basically one line and my prompts got way more compact without breaking anything. the fact that it holds onto the actual instructions matters a lot for my stuff.