LLMSlim
Contract-safe context tooling for LLM tool schemas.
Start new thread

LLMSlim v0.3.0 - Cut LLM token costs 40-70% with offline prompt compression

by•
v0.3.0 ships hybrid prompt compression: offline TF-IDF extraction pre-prunes context, then an optional LLM rewrite pass semantically optimizes what remains. Result: 40-70% fewer tokens billed, sub-30ms CPU latency, and 100% retention of system directives, code blocks, and JSON schemas. Works with OpenAI, Anthropic, Gemini, LangChain, LlamaIndex, Ollama and more. Zero dependencies. Pure Python. pip install llmslim

Add a comment

Replies

Best
Maker
📌
Hey PH! I'm Yashvardhan, the builder behind LLMSlim. The idea started from a real pain point: I was running a RAG pipeline that passed massive retrieved contexts to Claude, and watching my API bill climb even when 60% of those tokens were filler sentences and redundant background prose that the model just ignored anyway. Truncation killed instruction fidelity. Summarization added latency and cost. I wanted something that could surgically remove noise while guaranteeing system directives and code blocks survive untouched. So I built LLMSlim. v0.3.0 is the biggest release yet: a full hybrid strategy engine. The extractive pipeline (offline TF-IDF + LexRank centrality) now feeds into an optional generative rewrite pass that uses any LLM you already have to semantically compress what's left. The result is cleaner context, not just shorter context. Happy to answer questions about the architecture, the Priority Tier system, or anything about building open-source Python libraries. Would genuinely love feedback from folks running production LLM pipelines.