Release E2E Checklist

July 9, 2026 · View on GitHub

Use this checklist before tagging a release to verify the local gate and release-smoke paths.

0) Pre-tag Triage

Use this before you tag. The release process in RELEASING is the source of truth; this checklist only narrows the local checks that prove the release is ready. If a gate fails after a fast pass, do not tag. Use Incident Runbook when a failure needs cleanup or recovery after local validation.

QuestionInspect
"Is the release process itself current?"RELEASING and its pre-release checklist
"Did the local gate produce the expected evidence?".agents/releases/local-ci/<timestamp>/ for SBOM, security report, eval reports, readiness, HIL evidence, digital-twin/VIL evidence, and the manifest
"Did the bootstrap smoke actually run?"Fast gate marker printed as ao init + ao rpi smoke — but the ao rpi sub-checks were removed in 3.0 with the RPI engine, so only the ao init/bootstrap part is live (3.0 is hookless — no hook-install smoke)
"Did the release smoke path fail after a fast pass?"Re-run bash scripts/ci-local-release.sh and inspect the failing section before tagging
"Was the release already tagged or partially published?"RELEASING and Incident Runbook

1) Fast local gate (quick confidence)

Run:

bash scripts/ci-local-release.sh --fast --jobs 4

Expect:

  • Exit code 0
  • Output contains these markers:
    • Codex runtime sections
    • Codex artifact metadata
    • Skill runtime parity
    • ao init + ao rpi smoke (the ao rpi sub-smoke was removed in 3.0 — only the ao init/bootstrap check is live)

Reference test: tests/integration/test-release-e2e-validation.sh.

2) Full local gate (pre-tag requirement)

Run:

bash scripts/ci-local-release.sh \
  --release-version X.Y.Z \
  --hil-target 'local:bushido:ao version && ao init --help && ao doctor --quick && ao session bootstrap >/dev/null'

Expect:

  • Exit code 0
  • Final summary contains LOCAL CI PASSED
  • .agents/releases/local-ci/<timestamp>/release-artifacts.json links the full proof bundle:
    • sbom_cyclonedx and sbom_spdx
    • security_report
    • eval_fast_report and eval_baseline_audit
    • release_readiness
    • hil_evidence
    • vil_evidence and digital_twin_evidence
  • .agents/releases/local-ci/<timestamp>/release-readiness.json has release_status: pass and release_readiness_score >= 8
  • .agents/releases/local-ci/<timestamp>/hil-evidence.json has status: pass or status: waived; passing targets record OS/architecture/runtime identity, workflow checks, command fingerprint, and release-version verification
  • .agents/releases/local-ci/<timestamp>/digital-twin-evidence.json has status: pass and dimensions.vil.status: pass
  • .agents/releases/local-ci/<timestamp>/eval-agentops-fast.json has status: pass, and eval-baseline-audit.json has no stale_suite_hashes

If the fast gate passes but this full gate fails, stop. The release is not ready to tag. Use the failing section in the full-gate output, rerun the official local gate until it passes, and if a tag or publish already happened, follow the failure-mode steps in RELEASING before retrying anything else.

3) Codex runtime lint (focused check)

Run:

bash scripts/validate-codex-runtime-sections.sh

Use this when editing Codex runtime guidance or AGENTS runtime sections.

4) init + RPI smoke expectations

AgentOps 3.0 is hookless — it ships zero hooks, so there is no hook-install smoke check. The local gate includes this release E2E smoke check:

  • ao init + ao pawl smoke validates (in a fresh git repo):
    • ao init (creates .agents/; no hooks installed)
    • ao pawl smoke
    • ao session bootstrap
    • /rpi --help

If the fast gate passes but this smoke path fails, treat the release as blocked. Inspect the corresponding gate output, then rerun the full local gate so the fix is validated end to end before tagging.

5) Parity checks when workflow/docs contracts change

When changing CI workflow policy, hook/runtime docs, or required gate wording, run:

bash scripts/validate-ci-policy-parity.sh
bash scripts/validate-skill-runtime-parity.sh
bash scripts/validate-codex-runtime-sections.sh
bash scripts/validate-codex-generated-artifacts.sh --scope worktree
bash scripts/validate-codex-backbone-prompts.sh
bash tests/docs/validate-doc-release.sh