Testing and Diagnostics

Use focused checks for the boundary being changed.

yarn test:e2e
yarn test:e2e:online
yarn tycli:typecheck
yarn tycli:diagnostics:list
yarn tycli:diagnostics --filter documentation --details

The browser diagnostics UI is available at /diagnostics. Shared diagnostics must remain runnable from both browser diagnostics and the Node/tycli harness unless an unsupported runtime is explicitly modeled.

The browser page can filter tests, abort a run, and copy or download its YAML report. Add shared tests in locations already discovered by the diagnostics runner, such as packages/taskyon/src/tests/test*.ts or common modules exporting named test* functions.

Diagnostics that use an LLM must reuse the active browser/CLI profile or explicit harness override. Do not construct a hidden second provider configuration in the test.

Desktop diagnostics use the tauri:dev:diagnostics* scripts. Modelica has separate compare and baseline commands documented in Modelica.

Useful desktop paths include:

yarn tauri:dev:diagnostics
yarn tauri:dev:diagnostics:devserver
yarn tauri:dev:diagnostics:devserver:xvfb

The first command builds the diagnostics frontend. The devserver variants reuse an existing Quasar server, and the xvfb variants support Linux environments without a desktop session. Add :all-logs only when the default tagged diagnostic output hides information needed for debugging.

Playwright writes reports to playwright-report/ and run artifacts to test-results/. Use yarn test:e2e:ui for an interactive runner and yarn test:e2e:headed when a visible browser is enough. External-service tests read playwright.env.json. Use yarn test:e2e:online when those tests are required: the command fails instead of skipping them when the credential file is absent or incomplete.

Diagnostics distinguish deterministic checks from model-based capability evaluations. A model-based evaluation keeps its full assertions, but a miss is reported as MODEL MISS and contributes to the selected model's capability score rather than failing the diagnostics process. Mark these exported test functions with testFunction.modelBased = true; do not use this marker for deterministic runtime, protocol, storage, or integration failures.

For tycli, list and filter diagnostics before running broad sets:

yarn tycli:diagnostics:list
yarn tycli:diagnostics --filter '<test name>' --details

Normal CLI diagnostics output is concise: it prints one PASS, FAIL, SKIP, MODEL PASS, or MODEL MISS line per selected test, followed by the summary, every failed deterministic test by name, and the diagnostics log path. Runtime and test output is appended synchronously to that log while the run is active, so completed output remains available when a later test or the runner fails. Logs use TYCLI_LOG_DIR when configured and otherwise go to /tmp/tycli.

Pass --verbose to restore the full live console output, including runtime messages, test start lines, and the structured summary. --details includes failure details and full results without enabling all live runtime output.

Online diagnostics are opt-in. They reuse the selected CLI provider/model unless the command explicitly overrides them.

The tycli General Agent E2E catalog defines the separate interactive proof-run protocol, intervention policy, task-tree export, and trace audit used to evaluate long-running agent behavior.

Documentation changes must run yarn docs:check. Source changes should format only edited files. Run the full yarn lint before committing when the broader repository check is required.