Feature Roadmap (ROADMAP.md)

August 31, 2026 · View on GitHub

harness × adb synergy feature roadmap. Each feature notes: use case, why this capability on a harness plugin adds value (vs. a standalone CLI / Android Studio), harness mechanisms leveraged, rough cost, status.

Common thread: a single adb command can also run in a standalone CLI; the harness plugin's unique value is getting device data into model reasoning and letting model reasoning drive device operations, fully auditable, schedulable, and rollback-able — the "observe → analyze → decide → act → leave a trace" loop. Scope edges are reflected in REQUIREMENTS.md's "explicit non-goals".

Shipped

① One-click device health report — v1.2.0 ✅

Use case: QA gets an anomalous device and wants to know "what's wrong with this device right now". Click "one-click checkup" in the panel: collects device info (model/version/resolution/memory), top-RSS processes, crash buffer, W/E/F logcat window, storage usage, persists to the report store, and one-click sends to the conversation for the agent (with the dsh-adb-crash-analysis skill) to produce a structured diagnosis. Also supports comparing "morning checkup vs afternoon checkup" changes.
Why valuable on a harness plugin: Android Studio only gives raw dumps; the model reads them into readable conclusions inside the harness, and the report persists as a file for replay.
Mechanisms: existing RPC collection + crash-analysis skill + send-to-chat + new report store (~/.dsh/storages/dsh-adb/reports).
Implementation: adb_device_report tool + RPC deviceReport endpoint + panel "one-click checkup" button; each section degrades independently (failures go to errors, no whole-report failure). Evidence → Signal: crashes classified by signature (real crash + same-pid stack chains / MediaTek boot markers / other), W/E/F aggregated by tag (count + sample line), plugin-authored health summary (verdict/lines/issues) — the agent reasons from conclusions, not 17k raw lines. 56 unit tests green, real-device verified.

⑤ Wait / condition primitives — model→device ✅ (v1.3.0)

Use case: the agent orchestrates multi-step pipelines "install APK → wait for device online → wait for MainActivity in logcat → take a snapshot" instead of blindly sleeping fixed seconds.
Why valuable on a harness plugin: the agent loop + background jobs make "observe→judge→continue" a first-class primitive; a single adb command can't orchestrate across commands — this is the core need for agent-driven devices.
Mechanisms: adb_wait_for tool (device-online / boot-complete / process appeared / logcat keyword), polls to a budget cap, returns matched:false on timeout instead of throwing.
Implementation: pure host tool; 68 unit tests green (4 condition pure functions + 8 poll sequences); real-device verified (K50 Pro: device-online / process both matched). The foundation primitive for bench automation tests ④, crash attribution ②, and scheduled patrol ⑧.

⑨ Operation rollback ledger — persistence & reuse ✅ (v1.4.0)

Use case: the agent installed a new build and it's worse; the user says "go back to the previous version". The plugin records every push/install/uninstall/APK-replace operation, and the agent rolls back accordingly.
Why valuable on a harness plugin: the harness has persistent storage; making device operations auditable, rollback-able transactions is the trust base for "agent modifies devices on its own" — without a ledger the agent wouldn't dare touch the device.
Mechanisms: adb_operation_ledger tool (record/list/rollback), persisted to <baselineDir>/operations.json (append-only version-1 ledger).
Implementation: record validates (install/replace need an existing apk → LOCAL_FILE_NOT_FOUND; push/rm need remotePath); list filters by device/package; rollback reinstalls the last good install/replace APK (adb install -r, missing APK → APK_MISSING). 77 unit tests green (+9); headless verified record→persist→list; ⚠️ rollback real-device path pending device-online. Companion to the bench test ④ "install→test→rollback" loop.

Backlog

Ordering adjustment (2026-08-26): split by LLM dependency into two batches. First do the LLM-free ⑥⑧ and ②-orchestration parts — pure tool layer can be fully developed and verified; the LLM-dependent parts of ②③④ wait for the DeepSeek quota to be restored.

Batch 1: LLM-free, ready to develop now

P0 ⑧ Scheduled patrol/watchdog — time dimension ✅ (v1.7.0)

Use case: check the conclusion after an overnight soak test: periodically run perf_baseline compare + crash scan, and on regression automatically capture the scene (logcat window / screenshot / memory), persist as a report and trigger a session turn.
Why valuable on a harness plugin: adb goes from "query tool" to "watchdog": the harness's schedule/automation lets unattended detection carry a capture loop, not a number sent at midnight.
Mechanisms: adb_patrol_check (crash scan + baseline diff + battery/storage → verdict; compareToLast trend delta) + dsh-automation timed tasks (route A, docs/SCHEDULED-PATROL.md recipe with a self-contained prompt template) or schedule_create (route B).
LLM dependency: patrol orchestration and scene capture need no LLM; automatic attribution does (deferred — the scheduled agent drills in with the existing crash-analysis skill when the quota is back).
Cost: medium (host orchestration tool + scheduled prompt template). Status: shipped.

P1 ⑥ Dangerous-operation approval — model→device 🔧

Use case: the agent wants to install a test APK onto someone else's in-use phone, or reboot / wipe data on a bench. Destructive commands (install/uninstall/reboot) go through harness approval; the user clicks "allow once" to proceed.
Why valuable on a harness plugin: Android Studio has no approval concept; the harness's interaction capability gives adb a safe human-collaboration layer — a capability only available "inside the harness", and it lets the agent use destructive commands without fear of collateral damage.
Mechanisms: ctx.approval dispatch service (packages/interaction/user-approval, approval waterfall + ask/never policy).
LLM dependency: none. Pure policy layer + tool guards.
Cost: medium (tool approval policy layer + tests). Status: todo.

Batch 2: needs LLM, develop after the quota is restored

P2 ② Crash-attribution loop — data→model (first step shipped)

Use case: after reproducing a bug, the agent watches the crash buffer; when a new crash signature appears it automatically pulls the 30s logcat before the crash + memory at that time, and the model locates the root-cause line: "this OOM caused it".
Why valuable on a harness plugin: the tool-collection + model-reasoning combination is something standalone toolchains don't have — anyone can capture crash logs, but "attribute right after capture" needs an LLM.
Mechanisms: adb_watch_crash (v1.6.0 ✅, new-crash monitor, foreground/background) + crash-report tool + adb_screenshot (v1.5.0 ✅) + model context.
LLM dependency: required. Attribution prompt orchestration + model reasoning are the core value. Can test orchestration first with dsh-llm-replay or dsh-llm-mock-server; real attribution effect needs a real LLM.
Cost: medium (watch-crash shipped; remaining: auto-pull context after a crash + attribution prompt orchestration). Status: watchdog shipped, attribution orchestration todo.

P3 ③ Screenshot vision interpretation — data→model

Use case: during UI testing / crash reproduction, the agent auto-screencaps and has a multimodal model describe the current screen state: "black screen + unresponsive status bar at crash moment".
Why valuable on a harness plugin: hooking up a multimodal model turns adb into a semantic UI-state interpreter; Android Studio only gives you an image.
Mechanisms: adb_screenshot tool (v1.5.0 ✅, screenshot persistence + dimension parsing) + ctx.llm.stream({provider, model}) with a vision model (the pi-ai adapter supports image blocks; the main deepseek text-only session is unaffected).
LLM dependency: required, with an image-capable vision model (GPT-4o / Claude / Gemini). DeepSeek text-only models are insufficient.
Cost: small-medium. Status: tool shipped, vision routing pending a vision model. Remaining: hand the screenshot to a vision model for semantic judgment.

P4 ④ Bench automation tests (script execution + AI image comparison) — model↔device

Use case: the harness executes configured scripts on the bench — launch app (am start), kill app (am force-stop), tap/input (input tap/swipe/text) — and per user-configured detection conditions does AI-level image comparison (screenshot → vision model judges "did the login-success page appear"), producing a structured test report (per-step ✓/✗ + AI rationale + screenshot attachment) → send to chat + persist.
Why valuable on a harness plugin: bench action orchestration (Appium/UIAutomator can also do it) is only the first half; "AI image comparison" is harness-unique — screenshots route through ctx.llm.stream to a vision model for semantic judgment, replacing hard-coded pixel/coordinate assertions, so results go from "a pile of logcat lines" to "pass/fail + reason + screenshot evidence". This is the full "observe→analyze→decide→act→feedback" loop.
Mechanisms: adb execution core (am/input ready) + screenshot endpoint + ctx.llm.stream vision routing + image attachment service (attachment-local saveImage/readImage) + send-to-chat. Detection condition types: image (vision compare) / text (uiautomator dump text search) / process (alive) / property (getprop value).
LLM dependency: required, with a vision model. The script engine and text/process detection conditions can be built first (no LLM); AI image comparison is deferred.
Cost: medium-high (script engine + detection layer + result feedback + docs). Status: todo.

P5 ⑦ Multi-device comparison analysis — model→device

Use case: "this build is fine on the phone but the bench stutters". The agent runs the same snapshot/command in parallel on bench + phone + emulator, compares, and explains the differences (memory usage, frame rate, process differences).
Why valuable on a harness plugin: parallel multi-tool orchestration + comparative reasoning is a model strength; single-device tools only give you two piles of numbers — the model gives "why they differ".
Mechanisms: subagent parallelism / workflow orchestration + existing snapshot parsers.
LLM dependency: diff needs no LLM; difference explanation does. User has no multi-device need — lowest priority.
Cost: medium. Status: todo (low priority).

Extended (parked)

Debug-pipeline export: export a successful adb operation sequence from the session tool-call history as an automation task — reproduce interactively during the day, rerun unattended at night.
Agent preset packaging: a "device QA agent" preset (adb tools + crash skill + panel) — one command gives a new colleague complete device debugging capability.
Reverse annotation: after analyzing logcat, the agent highlights root-cause lines back in the panel so users can visually see "which lines the model thinks are the problem".

Implementation discipline

Ship only what is tested: every feature lands with unit tests (fake adb backend injection), all green before commit.
Release cadence: publish npm after GUI acceptance.
Order: do the LLM-free ⑥⑧ first (pure tool layer, fully verifiable); ②③④ join after the DeepSeek quota is restored; ⑦ is lowest priority (no multi-device need).
LLM-free verification: pure tools use unit tests + E2E scripts; LLM-dependent orchestration can be validated first with dsh-llm-replay (replays recorded sessions) or dsh-llm-mock-server (scriptable OpenAI-compatible server), with real reasoning effect pending a real LLM.