API and CLI reference

Every HTTP endpoint, every CLI command, and the MCP tools. The endpoint table is checked against the route decorators by tests/unit/test_version_consistency.py, in both directions — a documented endpoint that does not exist fails the build, and so does a live endpoint missing from the table.

API Reference

Tool calling

Send the OpenAI shape and get it back:

resp = client.chat.completions.create(
    model="claude-sonnet-4-6",                       # any supported provider's model
    messages=[{"role": "user", "content": "weather in Pune?"}],
    tools=[{"type": "function", "function": {"name": "get_weather",
            "parameters": {"type": "object", "properties": {"city": {"type": "string"}}}}}],
    extra_body={"session_id": "my-project"},
)
call = resp.choices[0].message.tool_calls[0]          # finish_reason == "tool_calls"
# run the tool, then continue the loop:
messages += [resp.choices[0].message,
             {"role": "tool", "tool_call_id": call.id, "content": '{"temp_c": 31}'}]

Every provider's deltas come out in the OpenAI chunk shape, indexed by tool call — so a client assembles them the same way whatever is behind the proxy. Anthropic indexes content blocks rather than calls and Gemini numbers nothing at all; both are mapped here rather than passed on.

What the pipeline does with tool traffic: tool turns are redacted like any other message and otherwise passed through untouched (no compression, no history pruning); windowing keeps a tool call and its results together; a tool-call answer is never cached or output-trimmed; a request that ends on a tool result is never served from the cache. The deprecated functions / function_call fields are ignored with a warning.

CLI

tokenmizer serve [--port 8000]
tokenmizer checkpoint [session-id]      # default: the working directory name
tokenmizer resume [session-id] [--level standard|full|critical]
tokenmizer stats

File analysis, three ways. FileIntelligence turns a large CSV / JSON / PDF / Excel / log / code file into a token-budgeted digest, and is reachable from all three surfaces:

Surface	Use it when
tokenmizer analyze <file>	A plain shell, a script, CI. Runs locally — no server, no API key.
POST /api/analyze	Another tool, curl, a remote client. Content is sent inline, never a server-side path.
/tokenmizer:analyze	Inside Claude Code (plugin skill).

tokenmizer analyze data.csv --token-budget 300
tokenmizer analyze big.json --raw > digest.txt

The endpoint takes content inline rather than a path on purpose: the server is often a container or a remote host, so a client-side path means nothing to it — and accepting one would be an arbitrary-file-read primitive against the server.

Claude Code Integration

Option A — Plugin (recommended)

# Add TokenMizer as a plugin marketplace
/plugin marketplace add Shweta-Mishra-ai/tokenmizer

# Install
/plugin install tokenmizer@Shweta-Mishra-ai/tokenmizer

Then use skills directly:

/tokenmizer:checkpoint my-project      → save session to graph memory
/tokenmizer:resume my-project          → load previous session (300 tokens)
/tokenmizer:resume my-project full     → full 600-token context
/tokenmizer:analyze /data/sales.csv    → analyze file (99% token savings)
/tokenmizer:stats                      → token savings report

Option B — MCP server (Claude Desktop, Claude Code, Cursor, VS Code, Zed)

Add this mcpServers block to your client's MCP config file:

{
  "mcpServers": {
    "tokenmizer": {
      "command": "tokenmizer-mcp",
      "env": { "TOKENMIZER_URL": "http://localhost:8000" }
    }
  }
}

Where the config file lives:

[mcp_servers.tokenmizer]
command = "tokenmizer-mcp"
env = { TOKENMIZER_URL = "http://localhost:8000" }

Then restart the client.

The proxy is optional for most tools. checkpoint_session, resume_session, get_graph_stats, why_decision and analyze_file fall back to reading and writing the SQLite store directly when nothing answers at TOKENMIZER_URL, and say so in the reply. Point TOKENMIZER_STORAGE_DIR at the same directory as graph_checkpoint.storage_dir if you have moved it (default: ./checkpoints). Only get_savings_stats needs the proxy, because savings are measured on requests that pass through it.

Only a transport failure falls back. A 401, 403 or 404 means the proxy is running and refused, and answering from local storage would bypass the session-ownership boundary it was enforcing — so those stay errors.

If tokenmizer-mcp isn't on your PATH, use "command": "python", "args": ["-m", "tokenmizer.mcp.server"] instead.

Tools exposed (6): checkpoint_session, resume_session, get_graph_stats, analyze_file, get_savings_stats, and why_decision — ask your agent "why did we pick X?" and it traces the decision's supersession chain with reasons and evidence.

Other Tools

Cursor / Continue.dev / any OpenAI-compatible tool:

API Base URL:  http://localhost:8000/v1

← Back to the README