Viktor.comAn AI coworker that actually does the work
SAGE - CLI
85-98% token compression and save your money
10 followers
85-98% token compression and save your money
10 followers
SAGE — Save AI coding tokens by compressing terminal output, keeping raw logs local, and tracking real savings. 🧠S.A.G.E - Smart Agent Guidance Engine for AI coding assistants. by⚡ PsYc+GoD AI&ML 🤖 - PsYcGoD/sage
How does it actually measure savings, does it count tokens before and after compression or estimate based on output size? Curious how accurate the tracking is across different terminals and shells.
@englhsrevz8oe SAGE uses actual token counting, not estimation. Here's the technical stack:
1. Real tokenizer
2. Befoer/after measurement
3. Fallback mode
4. Shell-agnostic
@englhsrevz8oe Accuracy proof: The repo includes provider-confirmed A/B tests where they sent the same prompt to Claude with raw vs compressed output. Claude's own token counter showed 64,833 → 91 tokens (99.86% reduction). That's not SAGE's estimate...- that's Anthropic's API response.
Please check the Readme in github repo to know more!!
Does it play nice with Claude Code out of the box, or do I need to wire up some custom hooks to actually see the token savings show up in real time?
@sailerxycz Out of the box, with one-time auto-setup. Here's what happens:
1. First run magic: When you run sage run -- or sage init for the first time, SAGE automatically -
- Writes enforcement instructions to ~/.claude/CLAUDE.md (Claude Code reads this globally)
- Registers itself as an MCP server in ~/claude_desktop_config.json...
- Creates enforcement hooks in .claude/hooks/enforce_sage.py that block bare shell commands and direct file tools
2. Real-time integration: Once installed, Claude Code:
- Reads the MCP tools (mcp__sage__sage_read_file, mcp__sage__sage_grep, etc.) automatically
- Uses those instead of direct Read, Grep, Edit tools
- Routes all shell commands through sage run -- (the hook blocks any attempt to bypass this)
3. Live feedback: After every sage run -- command, you see:..
[sage] saved run #42 exit=0 time=1180ms
[sage] context: saved 8,214 tokens (91.2% compression)
3. This isn't a dashboard you open later - it prints right in the terminal, in real time.
4. No manual wiring: The hooks are Python scripts that Claude Code executes automatically via its PretoolUse hook
system... You don't write YAML configs or edit settings files manually - sage install does it all.
Love this, terminal output bloat is such a real pain when working with coding assistants. One thing that would help me a lot is a per-session summary view that shows what got compressed and why, so I can trust I'm not losing anything important when I ask SAGE to squash the logs.
@eyllyaskb3pm This is already built in - here's how to access it:
1. sage context report - Full transparency report:
sage context report
1. Shows:
- Total commands processed
- Original tokens vs compressed tokens
- Compression rate (%), broken down by strategy (test output, package logs, build logs, diffs, progress bars)
- Which tokenizer was used (tiktoken vs fallback)
2. Per-command metadata: Every run stores:
- Raw output (local SQLite + artifact files, never sent anywhere)
- Compressed output (what the AI saw)
- Compression strategy used (test, logs, package_manager, build, diff, auto)
- Token counts for both
3. sage show --raw <run_id> - Recover exact output:
sage history --limit 10 # See recent run IDs
sage show --raw 42 # Get the EXACT original output from run #42
3. Nothing is lost - compression is a presentation transform, not a deletion. The raw logs stay on your machine.
4. Compression strategies explained (from src/sage/context/compression.py):
- Test output: Keeps failures, errors, warnings, final summary. Drops DEBUG, TRACE, repeated "OK" lines.
- Package logs: Keeps errors, important version notes. Drops progress bars, download percentages,
spinners.
- Build logs: Keeps errors, final status. Drops verbose compiler output, linking steps, timestamps.
- Diffs: Uses smart diffing - only shows changed hunks with 3 lines of context.
- Auto-detect: SAGE pattern-matches the output to choose the right strategy.
5. Safety floor: Even aggressive compression won't drop below a minimum - if the compressed output would be empty or lose critical error signals, SAGE keeps enough text to preserve correctness anchors (stack traces, error codes, exit
status).
@eyllyaskb3pm Example:
$ sage run -- python -m pytest
[sage] saved run #42 exit=0 time=1180ms
[sage] context: saved 8,214 tokens (91.2% compression)
[sage] summary:
144 passed
$ sage context report
SAGE context compression report
Original tokens: 120,450
Compressed tokens: 12,831
Saved tokens: 107,619 (89.3%)
Strategy breakdown:
- test: 42 runs, 87,212 tokens saved (avg 92.4%)
- logs: 18 runs, 15,407 tokens saved (avg 78.1%)
- auto: 5 runs, 5,000 tokens saved (avg 60.2%)
You can audit any run - the full raw output is tagged with a SHA256 hash and stored in ~/.sage/artifacts/. SAGE never
deletes it unless you manually expire old runs.
Do give me a star and PR if any on github only if u profit from it!
the fact that raw logs stay local while still giving you token savings tracking is such a thoughtful call. love that it solves the real friction point instead of just slapping on another wrapper.
@nazmiyebaysytm Thank you - this was the core design constraint. Here's why it works this way:
1. Privacy-first architecture:
- SQLite local: All command history, raw outputs, token counts → ~/.sage/sage.db on your machine.
- Artifact storage: Raw stdout/stderr → ~/.sage/artifacts/<run_id>.txt, never uploaded.
- Telemetry opt-in: Default mode (telemetry_level=0) sends nothing. If you opt in to connected mode (sage connect),
only aggregate counters go to the dashboard (total tokens saved, compression rate %, success rate). No commands, no
file paths, no logs.
2. Why this matters:
- You can use SAGE in corporate repos, NDA projects, private codebases - it never leaks context to a third party.
- The token savings proof is locally verifiable - you can audit the SQLite DB yourself, run the A/B test scripts
(claude_sage_ab_proof.py), and confirm the math without trusting a dashboard.
3. Real friction point solved:
- Before SAGE: AI agents would read 6,000-line npm install logs, 10,000-line test failures, repeated stack traces →
context window fills up fast, you hit token limits, conversation dies.
- After SAGE: Agent sees "npm install succeeded, 42 packages added" (15 tokens instead of 8,000). Raw logs are local
if you need them (sage show --raw 123), but the agent doesn't burn context on noise.
4. Thoughtful details:
- Redaction layer: SAGE auto-detects secrets (API keys, tokens, passwords) and redacts them before storing. The
stored logs are already cleaned.
- Retention expiry: You can set retention_days=30 and SAGE auto-deletes old runs. No manual cleanup scripts.
- ML prediction: SAGE trains a local ML model on your command history to predict failures before running the command
(76% accuracy with the optional neural model). This isn't a cloud service - it's a local daemon that loads once and
serves predictions in ~5ms.
Example of the "not just a wrapper" philosophy:
- The compression strategies (test, logs, build, diff) are heuristic-based, not LLM-generated. They run locally,
instantly, deterministically. No API calls, no latency, no cost.
- The token counter uses tiktoken (the real BPE tokenizer), not len(text) / 4. When you see "saved 8,214 tokens",
that's the actual token count the AI provider would charge you for.
- The enforcement hooks (enforce_sage.py) block any attempt by the AI to bypass SAGE - even if Claude tries to run a
bare python test.py, the hook rejects it and forces sage run -- python test.py. This is mandatory by default in the
repo's CLAUDE.md.
The result: SAGE is infrastructure, not a dashboard. The savings are real, the privacy guarantees are auditable, and
the integration is zero-config after the first sage init.
Do check my repo and give me a star if u liked it! only after using it!
Check the Live Dashboard -
https://sage.api.marketingstudios.in/dashboard