Crypcodile

A deterministic engine for the whole crypto market.
Pull order books, trades, funding and on-chain DEX events from 100+ venues into one Parquet lake — then replay any slice of it byte-for-byte.

_{Python 3.12+ · Apache-2.0 · public market data needs no API keys}

---

Every market-data tool can fetch a price. Two things make this one different.

One schema for everything. A hand-written Deribit connector, a ccxt Kraken venue, and a Uniswap V3 pool on Base all emit the same record types. So a question like "show me every BTC trade across all my sources" is one SQL statement, not three codebases.

The lake is deterministic. What lands on disk is normalized, validated, and replayable — replay a window today or next year and you get identical bytes. No hidden clocks, no re-fetching, no drift.

On top of that sits an options and microstructure analytics library, FlowMap (a GPU order-flow visualizer), and an MCP server so LLM agents read real prices instead of inventing them.

Install

uv pip install crypcodile          # or: pip install crypcodile

The base install is the whole streaming core: every native connector, the Parquet lake, replay, and the 47-command CLI. Heavier surfaces are opt-in extras so you never pay for a dependency tree you don't use:

uv pip install 'crypcodile[market]'   # +100 exchanges via the universal ccxt connector
uv pip install 'crypcodile[gui]'      # FlowMap visualizer + gas tracker (PyQt6)
uv pip install 'crypcodile[ml]'       # funding prediction + Black-Scholes (xgboost/scipy)
uv pip install 'crypcodile[web]'      # FastAPI server + Streamlit examples
uv pip install 'crypcodile[onchain]'  # Base L2 / GMX / Superchain connectors (web3)
uv pip install 'crypcodile[full]'     # everything

Prefer one command? install.sh (macOS/Linux) and install.ps1 (Windows) install crypcodile[full].

Five minutes

# stream Deribit BTC-perp trades + book deltas into a local lake
crypcodile collect --exchange deribit --symbols BTC-PERPETUAL \
    --channels trade --channels book_delta --data-dir data

# the lake is just partitioned Parquet — ask it anything in DuckDB SQL
crypcodile query "SELECT count(*) FROM records WHERE channel = 'trade'"

# replay that window later — identical bytes, every run
crypcodile replay --channels trade --symbols deribit:BTC-PERPETUAL

# open the order-flow visualizer on live Binance data  ([gui] extra)
crypcodile flowmap --symbol binance-spot:BTCUSDT --historical-hours 2.0

No lake yet? replay and query fall back to the bundled sample in test_data/, so a fresh clone works offline. There's also an interactive shell — crypcodile shell — with history and tab-completion, and every command runs inside it.

Reaching the whole market

Crypcodile speaks to 108 venues: ten native connectors, hand-written for fidelity, plus the entire ccxt family (104 exchanges) behind one universal connector. When a name exists in both, the native connector wins.

	Venues
Native	Binance · Bybit · Coinbase · Deribit · OKX · Base on-chain (Uniswap V3, Aerodrome) · GMX/Synthetix · Derive · Superchain · CoinGecko
Universal	any of ccxt's 104 exchanges — Kraken, KuCoin, MEXC, Gate, HTX, Bitget, …

You don't have to name symbols. Name a slice of the market and Crypcodile resolves the concrete list from the live universe:

# the 200 most-liquid pairs on Binance, streamed over a single WebSocket
crypcodile collect-market --exchange binance --top 200 --use-ws \
    --channels trade --channels book_ticker

# every USDT perpetual across three venues at once, order books included
crypcodile collect-market --exchange bybit,okx,mexc --all \
    --quote USDT --kind perpetual --channels book_snapshot --limit 400

# the whole coin universe — 17k+ coins, including the long tail no CEX lists
crypcodile collect --exchange coingecko --symbols _ --channels ohlcv

Two design choices make that scale honestly. The ccxt path is REST-poll-first (works on every venue) but upgrades to a single multi-symbol WebSocket per channel where the exchange supports it (watchTradesForSymbols / watchTickers) — the difference between one socket for three symbols and one socket for a whole exchange's book. And universe ranks any venue's markets by live 24h volume, so --top N covers the liquid core instead of ten thousand dead pairs.

The whole market, on one screen

crypcodile census measures the market live and writes a self-contained HTML dashboard — venue market counts (ccxt), the coin universe + market cap + dominance (CoinGecko), and total value locked (DeFiLlama). Every figure comes from a keyless public feed; a recent run:

108 reachable venues · 34,171 markets across the majors · 17,657 active coins · $2.29T market cap · $75.8B DeFi TVL

crypcodile census                  # → census.html + a terminal summary

The 47 commands

Cluster	Commands
Lake	collect · collect-market · backfill · replay · query · export
Discovery	census · markets · universe · search · resolve-symbols · data-coverage · catalog* (7) · list-exchanges
Options & funding	iv-surface · term-structure · vol-skew · risk-reversal · funding-apr · funding-predict · basis · open-interest
Microstructure	ofi · slippage · whale-alerts · liquidity-depth · indicators
On-chain / L2 risk	sequencer-latency · peg-deviation · chaos-score · lending-stress · gas-vol · smart-money · label-transfers · mev-sandwich
Desktop	flowmap · gas-tracker
Servers	mcp · api
Shell	shell · update

Ingest survives disconnects with sequence-gap bridging and a dead-letter queue (src/crypcodile/ingest/); whatever reaches disk is normalized against the 16-record schema and replayable.

FlowMap

FlowMap paints resting book depth over time as a liquidity heatmap and layers the tape on top: aggressor-colored trade bubbles, VWAP and BBO tags, COB/CVP/SVP volume profiles, a cumulative-delta strip, a DOM ladder, and iceberg / large-lot trackers. Feed it live data, a lake replay, or a built-in synthetic market for poking at the UI offline.

crypcodile flowmap --symbol binance-spot:BTCUSDT --historical-hours 2.0

It renders on QOpenGLWidget with a pure-NumPy density engine behind it (pin a backend with FLOWMAP_RENDERER=opengl|cpu). The uncapped offscreen benchmark clears 100 FPS at 1920×1080 on Apple Silicon; the window itself sits at vsync.

Analytics

The lake feeds an options + microstructure library, reachable three ways — CLI, MCP tool, or plain Python over a Catalog. Two runnable examples in examples/:

from crypcodile.analytics.funding import funding_apr
from crypcodile.analytics.volsurface import iv_surface
from crypcodile.store.catalog import Catalog

catalog = Catalog(data_dir="data")
apr     = funding_apr(catalog, "binance:BTCUSDT", from_ns, to_ns)  # Polars DataFrame
surface = iv_surface(catalog, "BTC", at_ns, rate=0.0)             # strike × expiry × IV

The full set spans OFI, slippage, whale alerts, term structure, vol skew, risk reversal, spot–perp / spot–future basis, open interest, and an L2/DeFi-risk family (sequencer latency, peg deviation, lending stress, MEV-sandwich detection). Each reads the same normalized records — native venue or ccxt, it can't tell the difference.

For agents (MCP)

crypcodile mcp --data-dir data     # Model Context Protocol server over stdio

Every tool is read-only and deterministic — answers come from the lake and the chain, never the model's imagination. Tools cover market-data reads (bounded DuckDB SQL, on-chain prices), catalog discovery (search_symbols, list_all_exchanges, coverage, inventory), and the analytics library. Works with Claude, Cursor, or anything that speaks MCP.

REST API

crypcodile api serves the same lake over FastAPI at /api/v1/* — ops and catalog discovery, a bounded read-only POST /query, the derivatives and microstructure analytics, and a payment-gated demo route over the x402 protocol.

Base L2

BaseOnchainConnector reads Uniswap V3 and Aerodrome swap/reserve events from Base RPC logs and emits the same record types as the CEX connectors, so a cross-venue query is one SQL statement instead of two codebases. Start with docs/base_quickstart.md; there's a Streamlit dashboard and a Farcaster frame server under examples/. Public data needs no keys; on-chain reads use a default Base RPC you can override.

Tests

uv sync --all-extras
pytest tests/

1,760 test functions across 141 files: a local mock-RPC server for degraded-network E2E runs (tests/e2e/), adversarial payload suites, and a regression file seeded by real exchange API anomalies. mypy --strict and Ruff gate src/. CI-friendly — Qt and Matplotlib run headless, and BLAS thread caps keep imports fast on Apple Silicon.

What it isn't

• Not a trading bot. There is no order-execution path, by design.
• Not a hosted service. Everything runs on your machine, against your lake.
• Not magic. Options analytics need options data — point iv-surface at a lake with Deribit snapshots in it, not an empty directory.
• FlowMap is a desktop app; it needs a display. The data pipeline doesn't.

Contributing

PRs welcome. Skim CHANGELOG.md for direction, keep the mypy --strict and Ruff gates green, and make sure the E2E and adversarial suites pass before opening one.

Apache-2.0 — see LICENSE.