README.md

Cut the tokens you waste. Keep the work you'd lose.

It runs automatically in the background. You keep working as usual. Run the audit when you want the full picture.

The 30-Second Version

Token Optimizer cuts the tokens your AI coding assistant wastes, keeps your work alive across sessions and compactions, and shows you where every dollar went on a live dashboard. Most of it runs automatically. You install it, run the audit once, and the hooks do the rest.

Why not just use Headroom or RTK? They compress command output, which covers 15-25% of your context. Token Optimizer covers that plus the other 75%: bloated configs, unused skills, stale memory, compaction loss, model misrouting, behavioral waste. Every saving is cache-safe and measured. The dashboard updates after every session, automatically.

Works on Claude Code (CLI and VS Code), OpenCode, OpenClaw, Codex, Hermes, and GitHub Copilot (beta). Windsurf and Cursor are next on the roadmap.

Install

Claude Code (recommended):

/plugin marketplace add alexgreensh/token-optimizer
/plugin install token-optimizer@alexgreensh-token-optimizer

Then in Claude Code: /token-optimizer

Enable auto-update after installing. Claude Code ships third-party marketplaces with auto-update off by default. /plugin → Marketplaces tab → select alexgreensh-token-optimizer → Enable auto-update. One-time, 10 seconds.

After install, run /token-optimizer once to set up hooks. From there, everything runs automatically: compression, checkpoints, quality scoring, dashboard updates. You don't need to run any command again unless you want an audit.

codex plugin marketplace add alexgreensh/token-optimizer

opencode plugin token-optimizer-opencode

openclaw plugins install github:alexgreensh/token-optimizer

git clone https://github.com/alexgreensh/token-optimizer.git
token-optimizer/install.sh --hermes

git clone --depth 1 https://github.com/alexgreensh/token-optimizer.git
cd token-optimizer
bash install.sh --copilot

tmp="$(mktemp -d)"
release_json="$(curl -fsSL https://api.github.com/repos/alexgreensh/token-optimizer/releases/latest)"
tag="$(python3 -c 'import json,sys; print(json.load(sys.stdin)["tag_name"])' <<<"$release_json")"
git clone --branch "$tag" --depth 1 https://github.com/alexgreensh/token-optimizer.git ~/.claude/token-optimizer
bash ~/.claude/token-optimizer/install.sh
rm -rf "$tmp"

What You Get

Runs automatically, every session, you do nothing:

• 🔄 Smart Compaction: checkpoints before auto-compact, restores after
• 🗄️ Session Continuity: cross-session hints, cold-resume, checkpoint scoring
• 📦 Active Compression: 9 features, all on by default (delta diffs, skeletons, bash/search compression, lean-output nudges, quality nudges, loop detection, activity mode, decision extraction)
• 📊 Quality Scoring: 7 signals, real-time, letter grades S–F
• 🗃️ Session Database: SQLite, 15 tables, full audit trail, zero network
• 🔍 Progressive Disclosure: large outputs archived, expand on demand
• 🧠 Context Intel Digest: post-compaction re-orientation without re-reads
• 🔀 Model Routing Nudges: steers to the right tier for the task

When you ask for it:

• 🩺 /token-optimizer: full audit with guided fixes
• 📈 /token-coach: 30-day trend analysis with specific fixes
• ⚡ quick: 10-second health check
• 🔧 doctor: installation check
• 💰 savings: dollar savings report
• 📋 report: per-component token breakdown
• 🌐 dashboard: open the full dashboard
• 📝 memory-review: MEMORY.md structural audit
• 📂 expand: retrieve archived tool result
• 🔙 resume-lean: reopen a cold session

Install, run /token-optimizer once, everything else runs automatically.

How It's Different

Most token tools compress command output. That covers 15-25% of your context. The other 75% goes untouched.

Compression coverage. Headroom and RTK compress bash and command output. Token Optimizer compresses eight surfaces of the output stack. Status: 🟢 supported, 🟡 partial, 🔴 not supported.

RTK reaches the first surface. Headroom reaches the first and the third. Token Optimizer covers all eight, then keeps going into what happens around compression:

• Three kinds of waste, not one. Structural (bloated configs, unused skills, stale memory), runtime (verbose output, re-reads), and behavioral (model misrouting, cache expiry, retry loops). How each works →
• Savings survive compaction. Checkpoints before auto-compact, restores after. Without this, compression savings vanish the moment compaction fires.
• Measures whether it helped. Before/after token deltas, dollar savings across four pricing tiers, quality scores that track degradation. Not just "we compressed stuff."
• Zero baseline overhead. External process, no always-on instructions in your context, no MCP server, no dependencies, no telemetry.

Every claim is tested against real sessions and a 57-fixture compression suite you can run yourself. Full benchmark methodology and results →

The Dashboard

One HTML page, auto-regenerates after every session via the SessionEnd hook, no manual trigger needed. Bookmark http://localhost:24842/token-optimizer and it's always current.

Per-turn token breakdowns, cost across four pricing tiers, cache analysis with TTL mix and hit rate, quality scores overlaid on every session, subagent cost breakdown, savings tracker with four non-overlapping pools. Zero setup after install. Full dashboard docs →

What It Saves

Savings come from four non-overlapping pools, tracked in two tiers:

Two numbers, kept separate:

• Counted (~$313/mo), logged action by action. Every time Token Optimizer swapped in a lighter model, trimmed a bulky result, or skipped a repeat read, it added it up: smarter habits ~$260/mo, while-you-work compression ~$53/mo. This is the slice metered event by event, so it is smaller and exact.
• Big picture (~$1,877/mo, ~18%), the full counterfactual. Had you worked the way you did before Token Optimizer (~95% Opus), you would have paid about ~$10,585/mo versus $8,708/mo now. The gap is mostly a lighter model mix (95% Opus down to 60%, $1,076/mo for main routing + caching), plus cheaper subagents ($741/mo) and the metered compression add-back ($60/mo).

These numbers are never summed. Counted is the floor with hard receipts. Big picture is a model priced against your frozen pre-Token-Optimizer baseline. See the full methodology →

Based on 684 sessions over 30 days (snapshot ending 2026-06-15), priced against a frozen pre-Token-Optimizer baseline (~95% Opus). Your number is your own. See the methodology →

Active Compression

Nine features that actively reduce context, all on by default, all automatic, all toggleable from the dashboard or CLI.

Under the hood, PreToolUse hooks intercept every Read and Bash call before it enters your context. If the file was already read, only the diff comes back. If it's a code file re-read, a structural skeleton replaces the full content. If it's a CLI command, the output is compressed. PostToolUse hooks archive the full original to disk and log a compression event to SQLite. Nothing is lost, and everything is retrievable. You do nothing. The hooks handle it all.

Toggle from the dashboard Manage tab, CLI (measure.py v5 enable|disable <feature>), or env vars. The v5 verb is a legacy command name that controls current features.

Read how each feature works →

python3 measure.py compression-stats --days 30

The Session Database

Everything Token Optimizer does is backed by two local SQLite databases. Nothing leaves your machine. Zero network calls.

Per-session DB (~/.claude/token-optimizer/snapshots/session-store/<session>.db) holds 8 tables tracking file reads, tool outputs, command outputs, cached content, context intel events, activity log, decision log, and hint serves. WAL mode for concurrent read/write from hook processes. 50MB cap per session.

Trends DB (~/.claude/token-optimizer/snapshots/trends.db) holds 7 tables tracking session history, daily aggregates, skill/model/subagent usage, savings events, and compression events. Indexed by session UUID for O(log n) joins. This is what powers the dashboard, coach mode, and 30-day trend analysis.

Every compression event, every saving, every quality measurement is a row you can query. The dashboard is a read-only view of this data. measure.py compression-stats is a SQL query. Your data is your data.

Progressive Disclosure

Large tool results (>4KB) are archived to disk and replaced with a short preview plus a retrieval pointer. The full output survives compaction. When the model needs it, it pulls the original via expand, with no command re-run and no lost output.

This isn't just storage. The system tracks how many results were archived vs re-expanded, so you can see the net tokens that stayed collapsed. Re-expansions are netted out of the savings total, so you only count what actually stayed compressed.

python3 measure.py expand --list          # List archived tool results
python3 measure.py expand <tool-use-id>   # Retrieve a specific result

Session Continuity

Compression matters, but the most important thing Token Optimizer does is keep your work alive across sessions and compactions, automatically.

When you end a session, Token Optimizer checkpoints your state to SQLite: active task, key decisions, modified files, git branch, recent reads. When you start a new session, it scores all recent checkpoints against your prompt and surfaces the most relevant one as a hint. You resume with context, not from zero.

What happens automatically:

• Cross-session hints: relevance-scored checkpoints surface when you start a new session. The hint includes the prior session's task, decisions, files, and branch. All fenced as RECOVERED DATA, never instructions.
• Cold-resume-lean: reopen a stale session without paying the full transcript cost. Token Optimizer reconstructs a lean context from its checkpoint. No LLM call, no full-transcript cold-resume. Token-free reconstruction from SQLite.
• Hint-follow measurement: when a continuity hint surfaces file paths and the model subsequently reads one, Token Optimizer credits an avoided exploratory search. Measured causality, not a guess.

python3 measure.py resume-lean                    # list reopenable cold sessions
python3 measure.py resume-lean <#|session_id> --print  # emit lean context block

Compression savings only stick if your session survives compaction. Session continuity is what makes that happen.

Smart Compaction

When auto-compact fires, 60-70% of your conversation vanishes. Decisions, error-fix sequences, agent state, all gone.

Token Optimizer checkpoints your session before compaction and restores what the summary dropped, automatically, via hooks. It also injects a context intel digest: heuristic summaries of large tool outputs the model already processed (file paths touched, errors seen, line counts). After compaction, the model knows what it saw without re-reading everything.

Activity mode detection classifies your session in real time (code, debug, review, infra, general) using the last 10 tool calls. The mode feeds into compaction guidance so PRESERVE/DROP priorities adapt to what you're doing right now, not a generic heuristic.

Decision extraction captures decision statements from tool outputs as they happen and stores them in the session DB. At compaction time, these are injected as CRITICAL DECISIONS the summary must preserve verbatim. Capped at 10 per session.

Compression savings only stick if your session survives compaction. Saving tokens on git status doesn't help if the next auto-compact wipes out the decision that made you run it.

python3 measure.py setup-smart-compact    # checkpoint + restore hooks

Output Tokens: Lean vs Verbose

Output tokens are the most expensive part of your session. They cost 5x more than input tokens on Opus and are billed per generation, not per cache read. A verbose response to a simple question burns dollars you never needed to spend.

Token Optimizer handles this automatically with lean-output nudges. When your context fills past 25% and quality starts dropping, a short nudge tells the model to reason deeply but keep visible output lean. Live A/B testing showed a 10-15% typical reduction in output tokens, up to 30-41%, on real prompts.

How it works:

• The nudge is injected as additionalContext, never modifying the existing prefix, so your cache stays intact
• It only fires when context is filling up, not when you have plenty of room
• The model still thinks through the problem; it just produces a more concise visible answer
• Disable any time: TOKEN_OPTIMIZER_VERBOSITY_STEER=0

This is one of the 9 active compression features, and it's the one that saves on the output side, where tokens cost the most.

Quality Scoring

The quality score tracks two things: Resource Health (how close you are to the degradation cliff) and Session Efficiency (whether your tokens are doing useful work). Letter grades from S to F make triage instant.

As context fills, quality drops. MRCR falls from 93% to 76% between 256K and 1M context. Your AI gets measurably dumber as the window fills. The quality score shows you exactly when that happens.

The status bar shifts color as quality degrades: green, yellow, orange, red. When quality drops below 75, session duration appears as a warning. Running subagents show with their model and elapsed time.

python3 measure.py setup-quality-bar      # one-time install

Read how scoring works →

python3 measure.py setup-quality-bar --uninstall

Coach Mode

> /token-coach

Tell it your goal. Get back specific, prioritized fixes with exact token savings. It reads 30 days of your session data and surfaces what no single session can show: quality drifting down, sessions getting longer, cache hit rates falling, cost per session climbing.

Every insight is grounded in your actual numbers. "Your short sessions score 68 vs 60 for long ones" hits differently than "consider shorter sessions." Coach Mode also identifies project-level optimization opportunities (skills you never use, MCP servers that load eagerly, CLAUDE.md patterns that break your cache) and teaches you how to fix them so future sessions start leaner.

Read about Coach Mode →

python3 measure.py keepwarm-enable          # opt in (API billing only)
python3 measure.py keepwarm-report            # net savings, spend, tripwire state
python3 measure.py keepwarm-disable           # opt out any time

python3 measure.py inject-routing --dry-run   # Preview
python3 measure.py inject-routing              # Inject

FAQ

All Commands

License

PolyForm Noncommercial 1.0.0. Source-available. Personal, research, educational, and non-commercial use requires no license purchase.

Personal / hobby / research / education?

Go for it. Full source, runs locally, no license purchase needed.

Small team (under 5 people OR under $20k/month revenue)?

Small teams get a no-cost commercial license automatically. Just use it.

Started personal, now it's turning into a business?

Your past use is totally fine. The license has a built-in 32-day grace period after any written notice. Reach out for a commercial license when you're ready.

Larger company / commercial use?

Contact Alex Greenshpun or me@alexgreenshpun.com.

Created by Alex Greenshpun.