Architecture

Overview

CANarchy is a CLI-first CAN analysis toolkit built around one core rule:

The CLI is the contract. The shell and TUI are views over the same command and event path.

The current implementation is Python-based and uses uv for dependency management, virtual environments, and packaging workflows.

The codebase is organized around four practical layers:

1. core models and protocol helpers
2. transport and backend selection
3. command execution and output shaping
4. front ends and tool surfaces that reuse the same command path

Important current-state note: live bus integration currently builds on python-can. CANarchy does not try to replace hardware abstraction itself; it adds a higher-level workflow, protocol, and structured-output layer on top.

Important boundary note: the deterministic scaffold backend is a transport backend for tests, CI, and offline demos. It is now distinct from explicit sample/reference protocol providers used by commands that are not yet truly transport-backed.

System View

flowchart TD
    U[Operator / Script / Agent] --> CLI[CLI]
    U --> SH[Shell]
    U --> TUI[TUI]
    U --> MCP[MCP server]

    CLI --> CMD[Shared command parser and executor]
    SH --> CMD
    TUI --> CMD
    MCP --> CMD

    CMD --> TP[Transport facade]
    CMD --> DBCP[DBC provider resolution / cache]
    CMD --> DBC[DBC encode / decode / inspect]
    CMD --> J[J1939 helpers]
    CMD --> UDS[UDS helpers]
    CMD --> RE[Reverse-engineering helpers]
    CMD --> SES[Session store]
    CMD --> REP[Replay planner]

    TP --> PY[python-can live backend]
    TP --> SC[Deterministic scaffold backend]
    CMD --> REF[Sample / reference providers]

    DBCP --> DBC
    DBC --> EV[Structured event model]
    J --> EV
    UDS --> EV
    RE --> EV
    TP --> EV
    REP --> EV
    REF --> EV

    EV --> OUT[JSON / JSONL / text output]
    DBC --> PROV[dbc_source provenance]
    PROV --> OUT

Layering

1. Core Model And Protocol Layer

This layer is responsible for the typed data model that everything else builds on.

Primary responsibilities:

• CanFrame validation and serialization
• typed event objects such as frame, decoded_message, signal, j1939_pgn, uds_transaction, replay_event, and alert
• J1939 arbitration ID decomposition and higher-level observation helpers
• DBC encode, decode, and schema-inspection helpers
• reverse-engineering analysis helpers such as counter and entropy ranking, plus DBC candidate scoring against captures
• replay planning from captured timestamps
• session context and persistence helpers

Relevant modules:

• src/canarchy/models.py
• src/canarchy/j1939.py
• src/canarchy/j1939_decoder.py
• src/canarchy/j1939_metadata.py — bundled SAE PGN / SPN / FMI / source-address catalogs
• src/canarchy/dbc.py
• src/canarchy/dbc_provider.py
• src/canarchy/dbc_provider_local.py
• src/canarchy/dbc_opendbc.py
• src/canarchy/dbc_cache.py
• src/canarchy/dbc_runtime.py
• src/canarchy/dbc_types.py
• src/canarchy/reverse_engineering.py
• src/canarchy/re_processors.py — plugin-registered RE processors
• src/canarchy/corpus.py — cross-capture corpus analysis (re corpus)
• src/canarchy/fuzzing.py
• src/canarchy/replay.py
• src/canarchy/sequence.py — coordinated multi-message sequence replay
• src/canarchy/simulate.py — data-driven bus simulator (simulate)
• src/canarchy/session.py
• src/canarchy/uds.py
• src/canarchy/scapy_uds.py — optional Scapy-based UDS enrichment
• src/canarchy/pcap_reader.py — PCAP / PCAPng capture ingestion
• src/canarchy/plot.py — signal time-series plotting (plot, optional extra)
• src/canarchy/web.py — read-only HTTP + WebSocket dashboard (web serve)
• src/canarchy/plugins.py — plugin discovery and registry
• src/canarchy/dataset_provider.py, src/canarchy/dataset_cache.py, src/canarchy/comma_segments.py — dataset provider, cache, and commaCarSegments support

DBC architecture (current):

• CANarchy-owned facade at src/canarchy/dbc.py keeps command handlers insulated from third-party runtime objects
• src/canarchy/dbc_provider.py resolves DBC refs across local paths and provider-prefixed refs such as opendbc:<name> and comma:<name>
• src/canarchy/dbc_provider_local.py handles explicit local file resolution
• src/canarchy/dbc_opendbc.py provides catalog search, cache refresh, and pinned-file resolution against the configured opendbc source
• src/canarchy/dbc_cache.py persists provider manifests and cached DBC snapshots under ~/.canarchy/cache/dbc
• cantools remains the runtime codec layer for encode, decode, and inspection through src/canarchy/dbc_runtime.py
• DBC-backed command results now expose dbc_source provenance so callers can see which provider, logical DBC name, version, and local resolved path were used
• third-party library details stay out of CLI, shell, TUI, and MCP command handlers even when provider-backed resolution is used

This layer should remain reusable without depending on any specific front end.

Parser self-fuzzing (current):

• tests/fuzz/ holds atheris coverage-guided harnesses that fuzz this layer's input parsers: the candump line parser (transport.parse_candump_line), the cantools-backed DBC parse path, the ISO-TP reassembler (uds.reassemble_uds_pdus), and the J1939 TP CM/DT reassembler.
• atheris is an opt-in dev dependency (pip install .[fuzz]); seed inputs live under tests/fuzz/corpora/<harness>/, and any crashing input is preserved under tests/fuzz/regressions/ with a paired regression test.
• The fuzz CI workflow runs each harness on a bounded budget per pull request, while tests/test_fuzz_harnesses.py exercises the harness logic over the corpora in the regular (atheris-free) test suite.

Skills provider architecture (current):

• src/canarchy/skills_provider.py defines the provider registry, descriptors, resolutions, and ref parsing for repository-backed skills
• src/canarchy/skills_github.py provides the first provider implementation by building a catalog from repository-hosted .skill.yaml manifests
• src/canarchy/skills_cache.py persists provider manifests and cached skill files under ~/.canarchy/cache/skills
• src/canarchy/skills.py provides structured provider/cache errors without coupling the command layer to provider-specific exceptions
• manifest parsing follows the versioned schema documented in docs/design/skill-manifest-schema.md

As with the DBC provider path, third-party repository details stay behind a CANarchy-owned facade so future CLI and MCP workflows can consume manifest-derived metadata rather than provider runtime objects.

2. Transport And Backend Layer

This layer is responsible for moving raw CAN frames in and out of the system.

Primary responsibilities:

• selecting the active backend from environment or config
• reading and writing live CAN frames
• parsing file-backed capture input such as candump
• exposing a stable local transport facade to the command layer
• keeping live and deterministic transport behavior behind one interface

The DBC provider/cache path is intentionally separate from the transport backend. Provider refresh and cache resolution may involve local filesystem or network access, but they do not change the live CAN transport abstraction.

Relevant module:

• src/canarchy/transport.py

Current backend model:

• python-can backend for live bus access
• scaffold backend for deterministic development and testing flows

Separate sample/reference providers currently exist for some protocol-oriented commands whose behavior is not yet fully transport-backed.

3. Command Layer

This layer is the main application surface.

Primary responsibilities:

• command definitions and subcommands
• argument parsing and validation
• dispatch to transport, DBC provider resolution, protocol, replay, reverse-engineering, export, and session helpers
• structured error handling and exit codes
• shaping output for --json, --jsonl, --text,

Relevant module:

• src/canarchy/cli.py

The command layer is the authoritative behavior contract for the project.

4. Front Ends And Tool Surfaces

The project currently ships four user- or agent-facing entry styles:

• CLI: non-interactive and authoritative
• shell: interactive loop that reuses the same parser and command executor
• TUI: full-screen Textual app that reuses the same executor, streams the bus live via CaptureSession, and renders interactive panes
• MCP server: stdio RPC surface that reuses the same command executor and result envelope

Relevant modules:

• src/canarchy/cli.py
• src/canarchy/tui.py
• src/canarchy/completion.py
• src/canarchy/mcp_server.py

The shell, TUI, and MCP server do not define their own business logic. They call back into the same execution path that powers the CLI.

Front-End Reuse

flowchart LR
    CLI[canarchy <command>] --> EXEC[execute_command]
    SHELL[canarchy shell] --> RS[run_shell]
    TUI[canarchy tui] --> RT[run_tui]
    MCP[canarchy mcp serve] --> MS[mcp_server.call_tool]

    RS --> EXEC
    RT --> EXEC
    MS --> EXEC

    EXEC --> RESULT[CommandResult]
    EXEC --> DBCRES[DBC ref resolution]
    DBCRES --> RESULT
    RESULT --> EMIT[Output rendering or TUI state update]

Current behavior:

• canarchy shell --command ... routes a one-shot shell command back through main()
• interactive shell mode uses shlex parsing and then calls the same executor used by the CLI
• canarchy tui renders a minimal status view and updates it from shared command results
• canarchy mcp serve exposes implemented commands as MCP tools and delegates tool calls to the same execute_command() path
• nested interactive front ends are rejected to preserve a single clear execution boundary
• DBC-backed commands resolve provider refs before handing local files to the runtime codec layer, then attach dbc_source metadata to the final command result

This is deliberate. The shell, TUI, and MCP server are convenience or integration surfaces, not separate applications.

Transport Boundary

flowchart TD
    CFG[Env vars / ~/.canarchy/config.toml] --> SEL[transport_backend_config]
    SEL --> BLD[build_live_backend]
    BLD --> PCB[PythonCanBackend]
    BLD --> SCB[ScaffoldCanBackend]

    PCB --> LT[LocalTransport]
    SCB --> LT

    LT --> CAP[capture / capture_stream]
    LT --> SND[send]
    LT --> GW[gateway]
    LT --> FILE[file-backed analysis helpers]

Current transport behavior:

• the default backend configuration is python-can
• the default python-can interface type is socketcan
• the scaffold backend remains important for deterministic tests and development flows
• gateway mode requires the python-can backend because it bridges live buses

Why this matters:

• CANarchy can stay focused on workflows and structured output
• live hardware support can grow through python-can without forcing CANarchy to own every device integration directly
• deterministic transport behavior remains available through the scaffold backend when tests or demos should not depend on live hardware
• sample/reference protocol data can stay explicit rather than hiding behind the transport backend abstraction

Event Model

The event model is the internal and external glue of the project.

Currently modeled event types:

• frame
• decoded_message
• signal
• j1939_pgn
• uds_transaction
• replay_event
• alert

These events are produced from typed Python dataclasses and then serialized deterministically for command output.

Not every command returns event streams. Some command families, including DBC inspection, cache/provider management, and reverse-engineering ranking helpers, return structured result objects under data instead. For DBC-backed decode, encode, and inspect commands, data also includes dbc_source provenance so downstream automation can distinguish a local file from a provider-backed cached schema.

Representative event flow:

flowchart LR
    RAW[Raw transport frame] --> FRAME[CanFrame]
    FRAME --> FE[FrameEvent]
    FRAME --> DE[DecodedMessageEvent]
    FRAME --> JE[J1939ObservationEvent]
    FRAME --> RE[ReplayActionEvent]
    FRAME --> UE[UdsTransactionEvent]

    FE --> SER[serialize_events]
    DE --> SER
    JE --> SER
    RE --> SER
    UE --> SER

    SER --> JSON[JSON]
    SER --> JSONL[JSONL]
    SER --> TABLE[Table]
    SER --> RAWOUT[Raw text]

Why the event model matters:

• it keeps transport, decode, and protocol logic from collapsing into free-form text
• it gives shell and TUI a stable state input
• it gives scripts and coding agents a predictable machine-readable output surface
• it enables command composition through JSONL event streams

Command Execution Flow

The command layer follows one main pattern:

1. parse argv into a canonical command name and arguments
2. validate command-specific constraints
3. dispatch to transport, protocol, replay, export, or session helpers
4. resolve provider-backed dependencies such as DBC refs when required by the command
5. normalize results into CommandResult
6. render through one output mode or serialize the canonical result envelope for MCP

This centralization is what allows the CLI, shell, and TUI to stay aligned.

Data Flow

For a typical live or file-backed workflow, the path looks like this:

sequenceDiagram
    participant User
    participant FrontEnd as CLI/Shell/TUI
    participant Command as Command layer
    participant Transport as LocalTransport
    participant Engine as Decode/Protocol helpers
    participant Output as Structured output

    User->>FrontEnd: run command
    FrontEnd->>Command: argv
    Command->>Transport: capture/read/send/filter
    Transport-->>Command: CanFrame objects
    Command->>Engine: decode / classify / replay / summarize
    Engine-->>Command: typed events and payloads
    Command->>Output: CommandResult
    Output-->>User: json / jsonl / text

Current Strengths

The current architecture is strongest in these areas:

• one shared command execution path
• structured event outputs as a stable contract
• protocol-aware workflows layered above raw transport
• clear boundary between transport integration and workflow logic
• provider-backed DBC workflows that preserve local reproducibility through cache and provenance metadata
• ability to reuse the same core behavior across CLI, shell, and TUI

Current Gaps And Boundaries

The architecture is intentionally ahead of some implementations. These are the main current gaps:

• live transport coverage is currently limited by the python-can integration and configured interfaces
• some protocol commands still rely on explicit sample/reference data providers instead of true transport-backed execution, although j1939 monitor, uds scan, and uds trace now have initial real backend paths when python-can is selected
• the TUI is a full-screen Textual dashboard with background live capture and interactive panes, as described in TUI plan; a remaining follow-up is a finite-timeout capture loop so live capture stops instantly on real hardware
• reverse-engineering now has a shared analysis subsystem for heuristic ranking (re signals, re counters, re entropy), reference-series correlation (re correlate), and provider-backed schema matching (re match-dbc, re shortlist-dbc)
• plugin command registration remains deferred; the registry and inspection/toggle CLI surface are implemented

Plugin Model

The plugin registry (src/canarchy/plugins.py) provides three stable extension points that let third-party packages add analysis processors, output sinks, and input adapters without forking.

Plugins are discovered at startup via Python entry points and must declare a compatible api_version. The registry follows the same lazy-singleton pattern as the DBC and skills provider registries. Operators can inspect and toggle discovered plugins with canarchy plugins list|info|enable|disable; read-only inspection is also mirrored through MCP as plugins_list and plugins_info.

Extension points:

• ProcessorPlugin — analysis processors: consume frames, return ranked candidates
• SinkPlugin — output sinks: write command payloads to custom destinations
• InputAdapterPlugin — input adapters: yield frames from custom file formats

Entry point groups:

Group	Extension point
canarchy.processors	ProcessorPlugin
canarchy.sinks	SinkPlugin
canarchy.input_adapters	InputAdapterPlugin

Built-in processors for the three heuristic reverse-engineering commands (re counters, re entropy, re signals) are registered by the default registry build and serve as the proof-of-contract migration. The reverse_engineering_payload() command handler routes these commands through get_registry().get_processor(name) rather than calling the underlying analysis functions directly.

See docs/design/plugin-model.md for the full design spec and docs/plugin-guide.md for the third-party author guide.

Non-goal:

• UI-only behavior that cannot also be reached through the canonical CLI surface
• Command registration via plugins (deferred to a later phase)

Design Summary

The architecture is best understood as:

• python-can plus scaffold backend for transport access
• provider-backed DBC catalog, cache, and provenance handling above the transport layer and below command output shaping
• explicit sample/reference providers for commands that are not yet truly transport-backed
• typed frames and events as the internal contract
• one command layer as the behavioral contract
• shell, TUI, and MCP as reusable views or integration surfaces over that same contract

That structure is what makes CANarchy suitable for both human operators and coding agents: the live bus boundary stays below the workflow layer, and the workflow layer stays above any one front end.