Distributed C2 via Meshtastic

April 19, 2026 · View on GitHub

The central node (sdr.py server) can optionally coordinate remote sensor nodes over a Meshtastic mesh. Remotes run sdr.py agent, get tasked by the server, and stream detections back.

Setup

Per-radio provisioning and systemd install are covered in service-setup.md. This doc focuses on the C2 protocol itself.

Prerequisites on both sides:

A Meshtastic-compatible radio on USB (Heltec V3, RAK4631, T-Echo, … see hardware.md).
Both radios must share the same primary channel PSK. Easiest way: venv/bin/meshtastic --port <device> --seturl '<URL>' on each.
Each agent radio's role should be CLIENT (the firmware default), not CLIENT_MUTE. See Mesh relay between agents below.
Server has "meshlink": { ... } in configs/server.json (configuration.md).
Agent has the radio port in configs/agent.json.

On first boot the agent broadcasts HELLO|<agent_id>|<version>|<hw> every 30 s while unadopted. On the server's web UI → Agents tab, the node shows up as pending. Click Approve → server sends APPROVE|<agent_id> → agent persists adopted: true in state.json and stops beaconing HELLO. Right after adoption the agent also fires a CFGINFO + SCANINFO snapshot (and again every 10 STATs thereafter) so the dashboard can render the agent's static config and running scanner under its row in the Agents tab.

Approvals persist: output/agents_state/agents.json on the server and state.json on the agent. A server restart keeps the approval; an agent restart resumes where it left off — including re-launching the last-tasked scanner.

If an already-approved agent sends HELLO — it only does that when unadopted — the server infers the agent's state.json got wiped (fresh service install, manual reset, …) and auto-re-emits APPROVE. No hand-edit required to recover.

Tasking

From the Agents tab (or POST /api/agents/cmd):

Start… — pick a scanner (pmr, ism, wifi, …) and optional args; sends CMD|<agent>|<seq>|START|<scanner>…
Stop — sends CMD|<agent>|<seq>|STOP
Status — sends CMD|<agent>|<seq>|STATUS

Rate-limit and interval settings propagate via CFG messages (same seq-carrying format).

Each CMD / CFG gets a server-allocated seq. The server's ServerOutbox retries unacked CMDs with exponential backoff (6 s → 120 s, 5 attempts max) until the agent sends ACK|<agent>|<seq>|ok. The agent ACKs immediately on receipt — before running the command — so a retry fired during a slow scanner start can't double-execute.

Operator click-Start-once is reliable now. pending_outbox on the server exposes in-flight CMDs for diagnostics.

Detection forwarding

Once tasked, the agent spawns sdr.py <scanner> as a subprocess (with --gps --gps-port <path> if configured). The scanner writes to its own session SQLite in state_dir/scanner/. The agent's DBTailer polls that DB for new rows and forwards each one as a DET message.

The tailer picks the live session by parsing the scanner-stamped timestamp out of the filename (<signal_type>_YYYYMMDD_HHMMSS.db), not by filesystem mtime — WAL sidecar touches on old .db files (e.g. the web dashboard opening a historical session read-only) can otherwise push a stale file's mtime above the live one and silently redirect the tailer to a dead session while STAT heartbeats keep flowing.

Server-side, the AgentManager ingests DET into a single output/agents_YYYYMMDD_HHMMSS.db. The standard web dashboard and triangulation tools work on these rows transparently — device_id holds the agent ID.

Resilience

Agents retry queued messages with exponential backoff until ACKed by the server.
Agents resume the last-tasked scanner after reboot.
No data loss on mesh outage — the persistent outbox.db drains when contact resumes.

Wire protocol

Plain UTF-8 text, pipe-delimited. Source: src/comms/protocol.py.

Tag	Direction	Meaning
`HELLO`	agent → server	Adoption beacon (`HELLO\|<id>\|<version>\|<hw>`). Repeats every 30 s while unadopted.
`APPROVE`	server → agent	Finalise adoption (`APPROVE\|<id>`). Auto-reissued on HELLO from an already-approved agent.
`CMD`	server → agent	Start/stop scanner, set param, request status. Verbs: `START`, `STOP`, `STATUS`, `SET`. Wire: `CMD\|<id>\|<seq>\|<verb>\|<args...>`.
`CFG`	server → agent	Update agent/comms config. Wire: `CFG\|<id>\|<seq>\|<key>\|<value>`.
`RES`	agent → server	Reply to CMD/CFG/APPROVE (`RES\|<id>\|<verb>\|ok\|err\|<msg>`). Informational — the reliability channel is ACK.
`DET`	agent → server	Detection with `agent_id`, `seq`, `type`, `freq_mhz`, `rssi`, `lat`, `lon`, `ts_unix`, `summary`, optional `snr`.
`STAT`	agent → server	Periodic health: scanner, state, GPS, sats, CPU, uptime. Default 60 s.
`LOG`	agent → server	Diagnostic text.
`CFGINFO`	agent → server	Snapshot of the agent's static config (`mesh_channel_index`, `meshtastic_port`, `gps_port`, `state_dir`, `version`, `hw`). Sent at startup, on APPROVE, and every 10 STATs thereafter.
`SCANINFO`	agent → server	Snapshot of the currently running scanner (`scanner_type`, `center_mhz`, `bw_mhz`, `channels`, `hopping`, `parsers`). Sent on START/STOP and every 10 STATs. Empty `scanner_type` = idle.
`ACK`	both	agent → server ACKs `DET`/`STAT`/`LOG`/`CFGINFO`/`SCANINFO` by seq; server → agent ACK happens inside the agent (pre-work) to clear the server's ServerOutbox for CMD/CFG.

Reliability

Agent → server. DET, STAT, LOG, CFGINFO, SCANINFO all go through the agent's persistent outbox.db. The drainer thread sends each row, marks it tried, and waits for an ACK to flip acked=1. Unacknowledged rows get re-sent with exponential backoff (capped by retry_max_sec in state.json, default 900 s) until acked or the outbox is wiped. Server-side, _on_det dedups by (agent_id, seq) so retries don't double-insert; _on_cfginfo / _on_scaninfo overwrite idempotently. Outbox prioritises control traffic (STAT/RES/CFGINFO/SCANINFO) over the bulk DET backlog so a busy WiFi/BT scanner can't starve heartbeats.

Server → agent. CMD and CFG go through a parallel ServerOutbox (src/server/outbox.py) that allocates a per-message seq, holds the wire frame until the agent acks by that seq, and retries with exponential backoff (6 s → 120 s, 5 tries max) on a 3 s tick. Broadcast targets (*) are fire-and-forget since no single agent ACK can clear them. APPROVE isn't retried by the outbox — instead, it's reissued on every HELLO from an already-approved agent, which is itself a robust signal that the agent needs re-adoption.

Comms log. The server keeps a 500-entry ring of every tx/rx frame keyed by (ts, direction, tag, agent_id, raw), served at GET /api/agents/comms and rendered under the Agents tab's C2 Logs sub-tab. Outbox retries show up there so operators can see the pressure without diving into journalctl.

Limitations at the transport layer

Meshtastic radios only forward decryptable packets to the serial API. PKC-encrypted DMs between other nodes are dropped by firmware before they hit the serial bus. For C2, use channel-broadcast messages (shared PSK) or DMs addressed to the agent's own node.
Text packets are truncated to fit within Meshtastic's frame budget (~200 B). DET encodes lat/lon at ~4 decimal places (~11 m) to stay under the limit.

Mesh relay between agents

Meshtastic does multi-hop forwarding by default — a far-away N02 can reach the server via N01 with no extra code. For that to work each intermediate node has to be willing to relay:

Set the radio's device.role to CLIENT (the firmware default), not CLIENT_MUTE. CLIENT_MUTE nodes still send their own traffic but drop everything else before it can be rebroadcast — a CLIENT_MUTE agent in the middle of a chain will silently strip the mesh of any node behind it. Check with meshtastic --port <dev> --info | grep '"role"'; flip with meshtastic --port <dev> --set device.role CLIENT.
Hop limit (lora.hop_limit, default 3) is the TTL on each packet. After three hops the packet is dropped silently. Bump it with meshtastic --set lora.hop_limit 5 on every node if your deployment is deeper than three hops.
Latency adds up: ~1–2 s of LoRa airtime per hop, so a three-hop CMD round-trip can take ~10 s. The web UI's single-shot CMD already requires manual retries on loss; deeper meshes make that more common.
Duty cycle is shared: EU 868 MHz limits each radio to 1% airtime. A node that relays for two others spends roughly 3× the airtime of a leaf node. Keep that in mind when sizing how many agents share one bottleneck radio.
High-volume scanners overwhelm the mesh. The agent enqueues one DET per detection. BT and WiFi produce dozens of detections per second in a populated area; LoRa drains at ~6 s per send. Don't run bt or wifi over the mesh in such environments — the outbox grows unboundedly. It's a physics limit of the radio, not a software gap. See troubleshooting.md.

Storage layout

Server

output/
├── agents_YYYYMMDD_HHMMSS.db     # one per server run — DETs from all agents
├── agents_state/
│   └── agents.json                # approved-agent registry (persistent)
├── server_YYYYMMDD_HHMMSS.db     # local captures (HackRF, RTL-SDR…)
└── server_info.json               # live status snapshot

Agent

/var/lib/sigint/                  # state_dir (or wherever agent.json points)
├── state.json                     # {agent_id, adopted, current_scanner, config, last_seq_*}
├── outbox.db                      # persistent retry queue
└── scanner/
    └── <type>_YYYYMMDD_HHMMSS.db  # raw detections from the scanner subprocess

HTTP API (on the server)

Method / path	Purpose
`GET /api/agents`	Return approved / pending / info maps. `info[id]` carries the latest STAT fields, `last_position` (newest geo-tagged DET), `config` (latest CFGINFO), and `scanner_info` (latest SCANINFO).
`GET /api/agents/detections`	Paginated list of mesh-forwarded DETs; `?limit` / `?offset`.
`GET /api/agents/comms`	Paginated 500-entry ring of every C2 frame tx/rx; `?limit` / `?offset`.
`POST /api/agents/approve`	`{"agent_id": "..."}`
`POST /api/agents/cmd`	`{"agent_id": "...", "verb": "START", "args": ["pmr"]}` — returns the allocated seq so callers can correlate with ACK.
`POST /api/agents/cfg`	`{"agent_id": "...", "key": "...", "value": "..."}`

Replay a recorded capture

sdr.py replay-c2 <db> --agent-id N99 [--rate 1.0] reads a saved detection .db and pumps DET messages over the configured Meshtastic link as if from a live agent. No Agent runtime is spun up — the driver encodes DETs directly with comms.protocol.encode_det_truncated and sends them. Useful for exercising the server ingest path without live RF and for deterministic triangulation / calibration benchmarks. --require-position / --require-power filter rows for those benchmark modes; --dry-run prints wire frames without opening a link.