Smoke Failure Classification Ledger

August 21, 2026 ยท View on GitHub

This ledger records public smoke-suite failures before changing runtime code. Its job is to separate product bugs from stale smoke assumptions, packaging gaps, and runner ergonomics issues.

Use this workflow when loopx canary smoke-suite --suite full-public turns red:

  1. Reproduce with the default installed loopx release snapshot.
  2. Reproduce with loopx-canary from the source worktree.
  3. Classify each failing public script by owner surface and suspected root.
  4. Fix product code only after the script is classified.
  5. Keep benchmark-runner execution and raw benchmark evidence out of this ledger; record only public script names, commands, and compact failure categories.

2026-07-03 High-Signal Control-Plane Batch

Source of truth for the run selection was the canary smoke-suite runner. A full full-public run was started but exceeded a heartbeat-sized bounded batch, so this entry uses module-scoped runs for the known high-signal red cluster:

loopx canary smoke-suite --suite full-public --module active-state --timeout-seconds 45 --no-progress
loopx canary smoke-suite --suite full-public --module blocker --timeout-seconds 45 --no-progress
loopx canary smoke-suite --suite full-public --module derived-state --timeout-seconds 45 --no-progress
loopx canary smoke-suite --suite full-public --module protocol-action --timeout-seconds 45 --no-progress
loopx canary smoke-suite --suite full-public --module session-runtime --timeout-seconds 45 --no-progress

The same module set was rerun with loopx-canary from the source worktree to distinguish release-snapshot packaging failures from source failures.

ScriptDefault ReleaseSource WorktreeOwner SurfaceClassificationPriorityNext Fix
examples/control_plane/active-state-interface-budget-smoke.pyRedGreenrelease snapshot smoke contextPackaging/runner-context gap. The installed release is not a git checkout, but the smoke shells out to git ls-files.P2Make the smoke tolerate release snapshots or run the git-only assertion only from a checkout.
examples/session_runtime/session-runtime-control-plane-adapter-doc-smoke.pyRedGreenrelease packaging / docs dependencyPackaging gap. The installed release did not contain docs/development/contributor-tasks.md, while the source checkout did.P2Either install the contributor task doc with release snapshots or make the smoke use a shipped public doc.
examples/blocker-push-runtime-smoke.pyRedRedquota/status blocker-push runtimeProduct/contract regression. The blocker-push path now includes public-safe omission warnings and projection metadata that the fixture did not accept.P1Inspect whether the new warning fields are intended; then update the runtime contract or fixture expectation.
examples/derived-state-boundary-smoke.pyRedRedstatus/project-asset todo projectionProduct/contract regression. Bounded derived-state visibility counts and claimed todo lanes changed relative to the documented projection budget.P1Reconcile project_asset_todo_summary with the bounded derived-state contract before broad quota/status refactors.
examples/protocol/protocol-action-packet-router-comparison-smoke.pyRedRedprotocol action packet / router comparisonContract or stale-fixture mismatch. The deterministic comparison no longer preserves all required facts/action clarity across scenarios.P2Decide whether this cold-path comparison is still a current contract; if yes, fix the router fixture or packet projection.

Runner Finding

The full full-public suite currently has no practical heartbeat-sized wall clock cap. A module-scoped run is usable for development, but the full suite needs one of these before it becomes the default autonomous gate:

  • a global suite timeout with partial-result JSON;
  • a resumable module/script batch plan;
  • or a persisted failure ledger generated by the runner itself.

Until then, autonomous product-capability turns should use module-scoped full-public runs for classification and reserve full sweeps for explicit release-readiness checks.