Roadmap Archive
August 8, 2026 · View on GitHub
Fully resolved (DONE/KILLED/SUPERSEDED) history moved out of ROADMAP.md on 2026-07-06 per #236(f). Rows are verbatim from the pre-cut file. If an archived item is reopened, move its row back to ROADMAP.md — do not edit it here.
Pre-Distribution (ordered) — completed
| # | Item | Status |
|---|---|---|
| 1 | Weekly workflow consolidation | DONE (PR #70) |
| 2 | Verify all-findings self-heal | DONE (PR #70) |
| 3 | "Prove It's Better" CI automation | DONE (v1.10.0) |
| 4 | Tier 2 E2E full suite audit | DONE (13 scripts, 3 bugs fixed) |
| 5 | Full system audit | DONE (4 bugs, 6 tests, CC overlap audited) |
| 6 | Package self-update for users | DONE (PR #76) |
| 7 | Post-update audit | DONE (PR #75) |
| 8 | Competitive audit | DONE (PR #77) |
| 9 | --bare for non-E2E workflows | DONE (PR #81) |
| 10 | Trigger weekly/monthly + audit | DONE (PRs #87, #88) |
| 11 | Re-run wizard on ourselves | DONE (PR #89) |
| 12 | CI efficiency audit | DONE (PR #92) |
| 13 | Cross-model full repo audit | DONE — pass 7 closed (PRs #97-100). No substantive open findings |
| 13.5 | Live-fire CI job audit | DONE — all CI paths verified (PR #102 fixed Tier 2 git init, 5-trial eval passed) |
| 13.6 | Wire cross-model review into own SDLC | DONE — cross-model review is now a first-class SDLC step |
| 14 | Distribution | DONE — npx agentic-sdlc-wizard init (PR #103). Zero-dep CLI, 16 tests |
Post-Distribution — archived rows (#19 remains in ROADMAP.md)
| # | Item | Description |
|---|---|---|
| 15 | E2E Scenario Coverage Audit | DONE — added ## Fixture: test-repo to all 16 scenarios, get_fixture_for_scenario() in scenario-selector.sh, 3 new scenarios targeting real fixture gaps (searchTasks, batchComplete, persistence). 5 new tests. Documented 7 existing scenario-fixture mismatches |
| 16 | Scoring System Review | DONE — enforce_tdd_consistency guard, tightened self_review + clean_code prompts, EVAL_PROMPT_VERSION v4→v5. 3 new tests (41 total in eval-validation) |
| 17 | Setup Drift Prevention | DONE — sdlc-wizard check [--json] subcommand. SHA-256 content hash, executable permission check, .gitignore line-aware matching, npm update detection. Exits non-zero on MISSING/DRIFT for CI gating. 6 tests |
| 18 | Tool/Plugin Discovery Automation | DONE — existing weekly-update release analysis + prove-it pipeline covers it. No marketplace.json registry exists for CC |
| 193 | PR #196 (a2f8726). Extracted scripts/persist-score-history.sh with fetch → rebase → retry loop (up to 3 attempts), explicit error-class detection (non-fast-forward vs branch-protection), and fallback to remote-take + re-append when score-history.jsonl conflicts. continue-on-error removed from the workflow step — persistent failures now surface in CI logs instead of silently vanishing | |
| 20 | Setup-Path E2E Proof | DONE — test-setup-path.sh validates init across 7 fixture types (47 tests). Wired into CI |
| 21 | Scan Routing Refactor | DONE — origin-typed routing (external vs internal-friction) replaces count-only findings_count gating. Full scan payload handoff between jobs. Friction-only weeks now create digest issues without triggering E2E. 6 new routing tests (176 total workflow tests) |
| 22 | Added /setup-wizard skill (.claude/skills/setup/SKILL.md) — 11-step mechanical checklist covering auto-scan, all 16 Q&A questions, file generation, and verification. Updated instructions-loaded-check.sh to explicitly invoke setup-wizard skill. CLI distributes skill as 8th file. 23 CLI tests, 25 hook tests, 47 setup-path tests passing | |
| 23 | Fixed settings.json merge bug — init now merges wizard hooks into existing config instead of skip/overwrite. New complex-existing-config fixture (custom hooks, skills, commands, settings.local.json, CLAUDE.md). 13 new tests (8 setup-path + 5 CLI merge), 113 total passing. PR #107 | |
| 24 | Harness Design for Long-Running Tasks | DONE (Tier 1) — Few-shot calibrated evaluator (7 criterion prompts), critical criteria enforcement (tdd_red + self_review must-pass), scoring rubric shared with generator (SDLC skill). EVAL_PROMPT_VERSION v5→v6. 8 new tests (46+25 total). PR #109. Codex cross-model review caught unenforced critical_miss — fixed. Tier 2 (evaluator tuning loop, contract negotiation) and Tier 3 (file-based handoffs, planner agent) deferred |
| 25 | Codex README/Docs Audit | DONE — Codex audit on all user-facing docs. Fixed install UX (first section, copy-pasteable), honest claims across 6 docs, added Playwright MCP vs Tests section, docs usability regression test (10 checks). Setup wizard auto-invoke UX fix (sdlc-prompt-check.sh redirects to setup-wizard when SDLC files missing) |
| 26 | npm Registry Publish | DONE — published agentic-sdlc-wizard@1.15.0 to npm. v1.16.0 includes setup-wizard auto-invoke hook fix |
| 27 | Created AGENTS.md (Codex review guidelines mirroring Claude PR review prompt), .reviews/experiment-tracking.md (tracking table for 10-20 PR comparison). 3 tests. Infrastructure ready — next step: install Codex GitHub App and start tracking | |
| 28 | Consolidate /testing into /sdlc | DONE — PR #113. Moved mocking table, unit test criteria, TDD Must PROVE, After Session into /sdlc skill. Deleted /testing skill + hook routing. Added upgrade cleanup (OBSOLETE_PATHS). 4 consolidation tests, 4 effort tests. Zero content loss verified |
| 29 | Added ## Recommended Effort Level section to wizard. high default via skill frontmatter, suggest max for LOW confidence / FAILED 2x / architecture decisions. Confidence table gets Effort column. 4 tests | |
| 30 | Added Post-Deploy Verification section to ARCHITECTURE.md template (health checks, log commands, smoke tests per environment, monitoring guidance). SDLC skill deployment section now includes post-deploy verification steps. 3 tests | |
| 39 | SDLC Enforcement Gap Audit | DONE — Audited all documented SDLC sections vs TodoWrite/hook/E2E enforcement. Fixed 5 gaps: capture learnings, scope guard, deploy tasks, new pattern approval, legacy delete check. Enforcement coverage 7/12 → 12/12. 6 new tests. Future: add E2E scoring criteria for scope_guard, after_session, deploy |
| 41 | PR #119. Added gh workflow run ci.yml dispatch after all 3 peter-evans/create-pull-request calls. Added actions: write to weekly-update + monthly-research. 4 tests |
Previous Release (v1.20.0)
| # | Item | Description |
|---|---|---|
| 36 | Formalized local shepherd CI fix model (in-session). Bot fallback (ci-self-heal.yml) was later deprecated — local shepherd provides higher quality fixes at lower cost. PR #124 | |
| 35 | /claude-automation-recommender | Wizard = enforcement engine, recommender = suggestion engine. Complementary, not competitive. Updated Step 0.3 with comparison table, added Complementary Tools section, setup skill recommends post-setup. 4 new tests. PR #125 |
| 38 | /clear vs /compact Guidance | Added Context Management section to wizard: comparison table, rules (/compact for continuing, /clear between tasks, /clear after 2+ failures). Documented auto-compact (~95% capacity). Updated SKILL.md with context management section. 4 new tests. PR #126 |
| 42 | Added Token Efficiency section to wizard: /cost monitoring, reduction techniques (compact/clear/subagents/effort), CI cost control (--max-budget-usd, --max-turns), OpenTelemetry for org-wide tracking. Updated CI_CD.md token tracking section. 4 new tests. PR #127 | |
| 31 | /init for Blank Repos | Verified wizard installs cleanly on blank repos. Added blank-repo fixture, 10 new E2E tests (68 total setup-path). Added guidance: no need for /init first, setup wizard generates all docs. PR #128 |
| 43 | Added ADR pattern guidance (docs/decisions/), claude-md-improver recommendation for CLAUDE.md health, "Documentation Sync" section in SDLC skill (enforce doc updates when code contradicts/extends documented behavior), docs-in-sync detection guidance. Strengthened transition step and After Session routing. 6 new tests. PR #129 | |
| 46 | weekly-update.yml version-test now passes path_to_claude_code_executable to all 3 claude-code-action calls, ensuring E2E actually tests the specific new CC version. Added id: install-cc + which claude path capture. CI_CD.md verdict table, wizard "How We Apply This" updated. 4 new tests (60 total in test-self-update). PR #131 | |
| 47 | Regression threshold 1.5→3.0, absorbs ±2-3 point LLM variance (rare extremes ±4 caught by Tier 2). Flaky test prevention guidance + external reference in wizard, SKILL.md. 2 new release consistency tests (64 total in test-self-update). PR #132 |
Previous Release (v1.22.0)
| Priority | # | Item | Description |
|---|---|---|---|
| 1 | 48 | Shepherd opt-in question (Q18) added to setup wizard. ci-analyzer skill was also added but deleted — violated Prove It philosophy (existence-only tests, no quality validation, overlap with third-party /claude-automation-recommender). Deletion led to Prove It Gate enforcement in SDLC skill | |
| 2 | 49 | Added "releases/publishes" as explicit cross-model review trigger in wizard + SKILL. Release Review Checklist subsection (CHANGELOG consistency, version parity, stale examples, docs accuracy, template parity) with v1.20.0 evidence. Triaged monthly research #84: 4 already done, 2 absorbed into existing items, 1 new unprioritized (#54), 2 skipped. 6 new tests | |
| 3 | 50 | Audited all 4 skills: /sdlc (core), /setup (core), /update (core), /ci-analyzer (deleted — unvalidated). Added Prove It Gate enforcement to SDLC skill + wizard doc. Internal consistency test catches stale references across all skills. 3 skills remain, each proven necessary | |
| 4 | 52 | Killed the fixed 18 questions. Setup wizard now scans repo, builds confidence per data point, only asks what it can't infer. Question count is DYNAMIC (0-2 for well-configured projects, 10+ for bare repos). 95% aggregate confidence threshold — if scan resolves enough, bulk confirm and generate. Wizard doc updated: Q-numbered questions → data point descriptions with detection hints. 6 new tests replace 2 old. PR #138 | |
| 5 | 53 | Skip plan approval when confidence >= 95% AND single-file/trivial task. Added to SDLC skill + wizard doc. Still announces approach, just doesn't wait for approval. "When in doubt, wait for approval" as safety valve | |
| 6 | 55 | Added systematic Debugging Workflow section: Reproduce → Isolate → Root Cause → Fix → Regression Test. git bisect for regressions, environment-specific debugging, "after 2 failed attempts, STOP and ASK USER" | |
| 7 | 37 | /feedback — Community Contribution Loop | Privacy-first /feedback skill: never scans without explicit consent. 4 feedback types (bug, feature, pattern, improvement). Creates GH issues on wizard repo. Distributed via CLI (9 template files) |
| 8 | 44 | Setup wizard detects branding assets (brand/, logos/, style-guide.md, brand-voice.md). BRANDING.md generated conditionally only when assets found. Template added to wizard doc | |
| 9 | 32 | Added Multiple Reviewers section to SDLC skill + wizard doc. Per-reviewer response pattern, conflict resolution (pick stronger argument), max 3 iterations per reviewer, escalate to user | |
| 10 | 45 | /agents Subagent Exploration | Documented .claude/agents/ pattern in SDLC skill + wizard doc. Example agents: sdlc-reviewer, ci-debug, test-writer. Skills vs agents comparison. Agents for parallel work and fresh context windows |
Previous Release (v1.24.0)
| Priority | # | Item | Description |
|---|---|---|---|
| 1 | 86 | PR #150. Three bugs: test-only scored 0, golden outputs were .txt not JSON, golden-scores encoded bug. Codex review caught regex false-positive (contest/ substring) + missing JSON pairing — both fixed. 29 deterministic + 9 regression tests passing | |
| 2 | 68 | if Conditionals | PR #151. Added CC v2.1.85+ if field to PreToolUse hook — TDD check only spawns for source files (repo: .github/workflows/*, template: src/**). Documented in wizard CC features section. 6 new tests (52 total hook tests) |
| 3 | 88 | PR #152. Added autocompact env var guidance (CLAUDE_AUTOCOMPACT_PCT_OVERRIDE, CLAUDE_CODE_AUTO_COMPACT_WINDOW) with community-recommended thresholds (75% for 200K, 30% for 1M). 1M vs 200K context window comparison table. Setup wizard Step 9.5 for context window configuration. Codex cross-model review caught setup skill parity miss + overclaimed env var documentation status — both fixed. 5 new tests (70 total self-update tests) |
Previous Release (v1.23.0)
| Priority | # | Item | Description |
|---|---|---|---|
| 1 | 64 | instructions-loaded-check.sh checks npm each session. 6 quality tests (fake npm, version comparison, failure modes). Non-blocking, graceful on network failure | |
| 2 | 59 | 7-topic deep research (hooks, skills, plugins, settings, upcoming features, Agent SDK, CLI). Key findings: 25 hook events (we use 3), plugin format is the official distribution path, KAIROS/Coordinator Mode coming, --bare bypasses wizard entirely. Spawned items #66-71 | |
| 3 | 72 | Audited 4 repos + external research (14 repos, 7 papers). Rewrote Cross-Model Review section: mission-first handoff, preflight self-review doc, verification checklist, adversarial framing, domain template guidance, convergence 2-3 rounds. 6 quality tests | |
| 4 | 73 | Added as section in SDLC skill (Prove It absorption check). Batch planning for releases. 3 quality tests | |
| 5 | 57 | Moved critical instructions to top 11% of SKILL.md. 3 quality tests | |
| 6 | 56 | Merged into #72 | |
| 7 | 65 | Explicit E2E vs Integration vs Unit boundary. 2 quality tests | |
| 8 | 69 | Full frontmatter field table. 2 quality tests | |
| 9 | 70 | --bare Docs | --bare bypass warning in SKILL.md + wizard. 2 quality tests |
Earlier Cycle Items (pre-v1.32.0 batch)
| Priority | # | Item | Description |
|---|---|---|---|
| 1 | 85 | weekly-update.yml fetches CC releases, analyzes with Claude (analyze-release.md), produces relevance/impact JSON, creates PRs with auto-update label. Session-start nudge added in instructions-loaded-check.sh (mirrors api-review-needed pattern) so open CC-update PRs surface at session start and don't bit-rot | |
| 2 | 91 | BaseInfinity/codex-sdlc-wizard PR #1. 3 hook scripts (bash-guard hard-blocks git commit/push, sdlc-prompt-check, session-start), non-destructive install.sh (6-case config merge, comment-aware), AGENTS.md, upstream-sync workflow. 15 behavioral tests. Codex review caught 2 bugs (commented config lines, macOS sed TOML corruption) — both fixed. ~70% CC parity | |
| 3 | 58 | Studied claw-code (168K stars), OmO (48K), OmX (16K). 16 candidate patterns identified. Codex certified 8/10 round 3. All candidates require Prove It Gate. Research doc: RESEARCH_58_CLAW_OMO_OMX.md | |
| 4 | 103 | Root cause: simulation prompt said "self-review" without explaining HOW (Read/Grep on modified files) or marking it scored. Golden output had text-only review (the exact NO example from the evaluator). Fix: all 5 simulation prompts now explain self-review = Read back modified files + marked scored in IMPORTANT section. Golden output/scores updated. 4 new tests |
This Release (v1.34.0, shipped 2026-04-17)
| Priority | # | Item | Description |
|---|---|---|---|
| 1 | 100 | PRs #184, #186, #187. Shepherd pattern: LLM-free weekly detector (.github/workflows/weekly-api-update.yml) fetches platform.claude.com/docs/en/release-notes/api.md, parses ATX date headers with ordinal normalizer + bullet-summary capture (scripts/parse-api-changelog.py), opens/updates single api-review-needed tracking issue. State persisted via scripts/persist-api-state.sh with non-blocking push (branch-protection safe). Session-time hook (instructions-loaded-check.sh) nudges when open issues exist — gated on LOCAL workflow presence so consumer/forks only see their own detector's issues. E2E verified in prod 2026-04-17 — issue body renders feature text like "We've launched Claude Opus 4.7..." not just the date. 33 tests incl. 8 fixture-based parser tests (bullet capture, subheader boundary, tab scrub, truncation, ordinal dates) + 2 integration tests. Codex xhigh 5 rounds across 2 PRs: 9/10 CERTIFIED. Found-in-prod P0 (gh api writes JSON error to stdout not stderr — label-create already_exists check broken after first run) hotfixed in #187 | |
| 2 | 189 | PR #189. /sdlc subsection defines three-bucket classifier (promote/keep/manual-review) with rule-based privacy denylist (user/reference → keep, project/feedback → manual-review). Hardened YAML parser normalizes quotes/comments/whitespace. SDLC.md seeded with 7 verified technical gotchas (each repro-verified and citing PR# or incident date). 10-fixture corpus (6/2/2) + 12-test suite covers structure, denylist hardening, corpus consistency. Protocol's first dogfood run caught 2 false lessons in private memory (${3:-{}} brace-default and --argjson result jq-conflict) that Codex verified as wrong — retracted with dated strikethrough rather than shipped. 3 Codex code-review rounds: 4→8→10/10 CERTIFIED. Meta-lesson: plan-CERTIFIED ≠ code-CERTIFIED; always run fresh Codex cycle on actual implementation | |
| 3 | 183 | /less-permission-prompts native skill | PR #183. Added to wizard + setup docs |
Previous Release (v1.33.0)
| Priority | # | Item | Description |
|---|---|---|---|
| 1 | 179 | Session-start hook nudges when effortLevel is below recommended xhigh (wording superseded by #217 — now max preferred, xhigh floor). Reads .claude/settings.local.json → .claude/settings.json → $HOME/.claude/settings.json. Non-blocking (exit 0). Bumped claude-opus-4-6 defaults to claude-opus-4-7 in pr-review.yml, evaluate.sh, sdp-score.sh, pairwise-compare.sh. Hook wired into SDLC.md table + CLI distributes model-effort-check.sh | |
| 2 | 180 | PR #180. SessionStart hook input doesn't expose model field — switched to effort-only detection; asks Claude to compare recommended model against its own system prompt | |
| 3 | 181 | PR #181. cli/init.js detects plugin install paths (~/.claude/plugins-local/sdlc-wizard-wrap/, ~/.claude/plugins/cache/sdlc-wizard-local/) and blocks with typed err.pluginPaths; --force bypass. instructions-loaded-check.sh non-blocking nudge when CLI+plugin both present. HOME isolation across test files (mktemp -d + trap), path.isAbsolute(home) guard, run_init_split helper. Codex xhigh 4-round: 9/10 CERTIFIED | |
| 4 | 182 | CLI template ships "model": "opus[1m]" + CLAUDE_AUTOCOMPACT_PCT_OVERRIDE=30 (1M-tuned, compacts at ~300K). cli/init.js merges top-level model when absent, respects user's explicit choice, --force overwrites. Wizard doc + /sdlc skill + /setup skill + SDLC.md all flipped to recommend 1M default. Hooks now recommend opus[1m] alias. Baseline bumped to CC v2.1.111+. 9 new tests, 6 existing updated. Codex xhigh 2-round: 9/10 CERTIFIED |
Previous Release (v1.32.0)
| Priority | # | Item | Description |
|---|---|---|---|
| 1 | 178 | Benchmark workflow adds opus-4-7 model choice + effort input (high/xhigh/max). Hard-fail on xhigh+non-4.7. Effort in artifacts + summaries. Wizard docs updated with xhigh guidance. 39 model-comparison tests | |
| 2 | — | Cross-model audit (Codex GPT-5.4 xhigh) rated benchmark 2/10 NOT CERTIFIED. 4 P0s: fake trials, answer key leaked, no independent verification, binary rubric. Documented in wizard with 3 concrete fixes. Tracked under #96 | |
| 3 | — | Automation Station link in README (sdlc-wizard + codex-sdlc-wizard + project-tracker) |
Previous Release (v1.31.0)
| Priority | # | Item | Description |
|---|---|---|---|
| 1 | 173 | PR #175. Silent exit when walk-up finds neither SDLC.md nor TESTING.md. Partial setups (one file) still warn. Added find_partial_sdlc_root helper. 2 new tests (60 total hook tests) | |
| 2 | 174 | CLI check scans ~/.claude/settings.json extraKnownMarketplaces for ephemeral (/tmp/, /private/tmp/, /var/folders/) and dangling directory sources. EPHEMERAL warns (exit 0), DANGLING errors (exit 1). Suggests ~/.claude/plugins-local/. 10 new tests (51 total CLI tests) |
Deferred: #83 (Local Model — needs real demand signal), #82 (Domain DLCs — Stefan's separate track), #71 (KAIROS — watch-only until CC feature exits experimental), #67 (Agent Team Hooks — same)
Previous Release (v1.30.0) — Domain + Detection
| Priority | # | Item | Description |
|---|---|---|---|
| 1 | 78 | Fleshed out firmware-embedded fixture: Python SD card overlay manager, 3 device configs, SIL + config validation tests within fixture. 12 quality tests prove domain indicators, multi-device differentiation, no-web misclassification. Completes the #79 domain-adaptive proof | |
| 2 | 94 | PRs #164, #165. workflow_dispatch benchmark: Opus vs Sonnet on 16 E2E scenarios with 95% CI. Codex GPT-5.4 xhigh reviewed (3 rounds plan, 2 rounds impl). P0 shell injection fix, wizard install verification, jq artifact construction. 37 quality tests. 7 workflows total | |
| 3 | 96 | Score persistence (ci.yml git-commits JSONL to PR branch) + wizard hardening (adaptive thinking context, CLAUDE_CODE_DISABLE_ADAPTIVE_THINKING, anti-laziness guidance). 14 behavioral tests |
Previous Release (v1.29.0) — Infra + Setup
| Priority | # | Item | Description |
|---|---|---|---|
| 1 | 93 | PR #160. 5 action version bumps across 6 workflows, 2 third-party actions replaced with gh CLI, 4 node-version bumps 20→22. 13 compliance tests. Expression injection P0 caught by CI reviewer and fixed | |
| 2 | 88 | CLI sets CLAUDE_AUTOCOMPACT_PCT_OVERRIDE=75 in settings.json env field by default. Smart merge preserves existing user env vars on upgrade, --force resets to defaults. Setup wizard Step 9.5 customizes for 1M models (30%). 7 new tests (39 total CLI tests) | |
| 3 | 80 | 52 historical catches seeded in .metrics/catches.jsonl. catch-analytics.sh computes DDE per layer, escape rates, severity breakdown. Results: cross-model-review (48%) and self-review (46%) nearly tied, self-review missed 28 bugs caught downstream. 14 tests. Log automation deferred until analytics proven useful |
Previous Release (v1.28.0)
| Priority | # | Item | Description |
|---|---|---|---|
| 1 | 79 | PR #157. Setup wizard auto-detects domain (firmware/data-science/CLI/web) and generates domain-specific TESTING.md. 4 domains with distinct testing layers. 3 new fixtures, 25 quality tests. Partially satisfies #78 | |
| 2 | 92 | PR #158. First rigorous autocompact benchmarking methodology — canary fact mechanism, harness script, 3 task complexity levels, CI workflow with matrix strategy, 26 quality tests. Methodology ships before data |
Previous Release (v1.25.0)
| Priority | # | Item | Description |
|---|---|---|---|
| 1 | 89 | PR #154. .claude-plugin/plugin.json manifest, hooks support, marketplace.json. CLI reads from root. 25 plugin tests. Absorbs #66 + #87 | |
| 2 | 90 | 6 channels: npm, plugin, curl, Homebrew, gh extension, GitHub Releases. awesome-claude-skills PR #588 submitted | |
| 3 | -- | Workflow working. March 30 detected v2.1.85→v2.1.87. No fix needed |
Unprioritized — archived rows
| # | Item | Description |
|---|---|---|
| 51 | Per-repo config is the right model. Global defaults would override project-specific SDLC settings — the opposite of what users need. npx agentic-sdlc-wizard init is already one command | |
| 72 | Completed in v1.23.0 | |
| 73 | Completed in v1.23.0 | |
| 54 | Deleted — effort levels cover this | |
| 56 | Merged into #72 (v1.23.0) | |
| 57 | Completed in v1.23.0 | |
| 58 | Studied claw-code (168K stars, Rust CC alternative), OmO (48K stars, 11-agent orchestration), OmX (16K stars, $ralph/$team). 16 candidate patterns identified (GreenContract, planning gate, planner/executor separation, bounded persistence loop, recovery recipes, etc). Codex certified 8/10 round 3. All candidates require Prove It Gate before adoption. Research doc: RESEARCH_58_CLAW_OMO_OMX.md | |
| 59 | Completed in v1.23.0. Spawned #66-71 | |
| 60 | Researched 2026-04-06. Forge (102 stars) has zero first-party benchmarks — all claims are unsourced synthesis. Vocabulary routing: independently tested (arxiv 2505.17037), does NOT significantly improve performance, diminishing returns past optimal specificity. 19-requirement accuracy drop: REAL (arxiv 2505.13360, GPT-4o drops to 85% at 19 combined requirements) but applies to competing format constraints on single outputs, not system-level behavioral guidelines like SKILL.md. "10 Principles" article series: doesn't exist. No action needed, but keep 19-requirement finding in mind for future SKILL.md pruning | |
| 61 | No evidence users need this. Theoretical nice-to-have with no real demand signal. Prove It Gate: can't prove value without a user asking for it | |
| 62 | Absorbed into #89 (Plugin Format) + #90 (Distribution Channels). Plugin format shipped, 6 distribution channels live, awesome-claude-skills PR #588 submitted. Bidirectional contribution = opening issues/PRs on Anthropic repos, not a roadmap item | |
| 63 | Per-PR review + release review checklist already covers it. No evidence of gap. The roadmap item itself says "may not be needed" — that's the answer | |
| 65 | Completed in v1.23.0 | |
| 67 | Agent Teams API changed in v2.1.178: TeamCreate/TeamDelete removed, teams are now implicit. Main session enforcement covers subagent workflows. Dynamic Workflows (#425) is the public multi-agent orchestration path. No demand signal for team-specific hooks. | |
| 68 | if Conditionals | PR #151. PreToolUse tdd-pretool-check.sh wired with CC v2.1.85 if: field in .claude/settings.json to filter Write/Edit/MultiEdit by path glob. Note: file-path if matching was buggy until v2.1.176 fix — minimum CC version for reliable path filtering. |
| 69 | Completed in v1.23.0 | |
| 70 | --bare Docs | Completed in v1.23.0 |
| 64 | Completed in v1.23.0 | |
| 74 | Audited all 8 articles. Wizard already reflects 80%+ of Stefan's philosophies (Testing Diamond, anti-mocking, test-code-as-first-class, flaky-tests-are-bugs, CI shepherd). Can't audit yourself against yourself and find gaps — the wizard IS the Watercooler Index codified | |
| 75 | "Every mistake becomes a rule" already works and is proven by catches data (54 catches, 5 P0s). Formalizing SBAR/5 Whys adds process overhead without evidence of value. Prove It Gate: can't prove SBAR would have caught something our existing system missed | |
| 76 | Researched 2026-04-06. NO-GO. Promptfoo handles LLM-as-Judge rubrics and multi-trial but has zero statistical analysis (no CI calculation, no CUSUM, no score history). Would only replace evaluate.sh, not stats.sh — half the pipeline stays custom. Also: OpenAI acquired Promptfoo March 2026, Anthropic provider support becoming second-class. Key finding: our custom pipeline already independently implements Promptfoo's best patterns (deterministic-first scoring, binary YES/NO instead of 1-10, calibration examples per criterion). No action needed | |
| 77 | Researched 2026-04-06. NO-GO for our prompts. Pattern is valid (Stanford 17-33% hallucination, binary checks reduce variance) but our PR reviewer already produces specific file:line findings with real P0/P1 bugs. Constraining to binary checks risks reducing recall on novel issues (e.g., ` | |
| 81 | Absorbed into #72 (Cross-Model Review Standardization, v1.23.0). Researched multi-agent patterns (advocate/adversary/judicial, 5-agent parallel, sequential pipeline). Conclusion: single well-prompted pass with adversarial framing + verification checklist beats multi-agent decomposition at our complexity level. Multi-agent adds orchestration cost without proportional quality gain for code review. Legal domain (contracts, compliance) is where multi-agent shines — different problem structure | |
| 84 | Already implemented implicitly — this repo IS the HDLC. We dogfood the wizard on itself, have hook version checks (instructions-loaded-check.sh), template parity tests (test-cli.sh), review staleness checks, workflow audits (test-self-update.sh), and CI that tests CI. The Codex sandbox crash was caught and documented. Formalizing into a named framework adds no value beyond what we already do | |
| 85 | Base capability already shipped (weekly-update.yml + session-start nudge mirroring api-review-needed pattern, see Earlier Cycle Items line item). Phase-2 scope (parse changelog into GitHub issues + auto-post + session nudge) conflicts with the post-#231 zero-cron / maintainer-run philosophy — weekly-update.yml was deliberately gutted to $0 in Phases 1–4 (#231 v1.50.0–v1.55.0). 1.7 months stale with no demand signal. Per .reviews/roadmap-prio-codex.md kill list. | |
| 86 | Root cause was NOT the scenario design — it was a format mismatch. claude-code-action@v1 outputs SDK format {type: "assistant", message: {role, content}} but check_tdd_red expected raw API format {role, content}. The jq query never found tool_use blocks in real CI. Fix: added .message unwrapping in jq. 2 SDK-format regression tests. One-line fix in deterministic-checks.sh | |
| 88 | Shipped in v1.33.0 via #182. CLI template sets CLAUDE_AUTOCOMPACT_PCT_OVERRIDE=30 for 1M default. Wizard doc and /setup skill document the 75% fallback for 200K. 6 autocompact tests verify the gate | |
| 90 | Duplicate entry. Already shipped: 6 channels (npm, plugin, install.sh, Homebrew tap, gh extension, GitHub Releases). Optional follow-ups (Scoop, AUR) deferred — covered by install.sh curl path. | |
| 194 | Expanded and promoted to Next Up #9 on 2026-04-22 with privacy-first / any-backend framing (local Ollama/LM Studio/vLLM, enterprise Azure OpenAI + internal gateways, hosted OSS). Original scope retained there plus backend-matrix proof phase | |
| 224 | sdlc-prompt-check.sh fires exactly once per user prompt | Surfaced by Codex review of auto-PR #210 (2026-04-23). CC 2.1.118 shipped a fix for prompt hooks double-firing when an agent-hook verifier subagent itself made tool calls. Our hook fires on UserPromptSubmit but we have no test asserting "once per prompt" — only existence/content/size checks in tests/test-hooks.sh. Bug would manifest as duplicate SDLC BASELINE injections in agentic/subagent sessions; context bloat + potential confusion. Zero-API test: spawn CC in a subagent-triggering scenario, count UserPromptSubmit invocations of the hook (via stderr marker or hook-side counter file), assert exactly 1 per user turn. Ref Codex finding PR210-01 |
| 225 | cleanupPeriodDays minimum for ~/.claude/tasks/ retention | Surfaced by Codex review of auto-PR #210 (2026-04-23). CC 2.1.117 changed cleanupPeriodDays to also cover ~/.claude/tasks/. Our SDLC skill uses TodoWrite as step 1, persisted under that dir. An aggressive retention policy (e.g. 7 days) could prune in-progress SDLC checklists from paused long-running features. Wizard currently says Tasks "persist across sessions" in CLAUDE_CODE_SDLC_WIZARD.md:303 but has no guidance on retention. Action: add a "Known CC gotchas" note recommending cleanupPeriodDays >= 30 when TodoWrite/task-based workflows are used, OR pin the setting to an explicit safe default in cli/templates/settings.json. Zero-API docs change, ~30 min. Ref Codex finding PR210-02 |
| 226 | Surfaced by Codex review of auto-PR #213 (2026-04-23). The "Community Patterns" PR advertises 95% CI with 5 trials and reports STABLE/IMPROVED/REGRESSION. But the workflow captures ONE simulation output and passes it to tests/e2e/run-tier2-evaluation.sh which loops TRIALS times over the SAME OUTPUT_FILE (weekly-update.yml:1008/1088, run-tier2-evaluation.sh:29/46/50). The result: 10.0 ± 0.0, 95% CI [10.0, 10.0] — statistically impossible variance from 5 truly independent Claude invocations. Current "STABLE" verdicts are not evidence of anything. Fix: rerun Claude once per trial in the workflow BEFORE scoring, so the N scores come from N distinct simulations. OR remove the statistical wording from the PR body until the loop is honest. This is P1 because community-pattern PRs otherwise look plausible but are not trustworthy — ship state changes without real regression signal. Ref Codex finding PR213-01 | |
| 227 | score-history.txt churn into community PRs | Closed-by-virtue-of-#231-Phase-3c. The PR-noise source was weekly-update.yml's scan-community job calling cusum.sh --add, which #231 Phase 3c (v1.53.0) deleted along with the entire job. Test 227 in test-workflow-triggers.sh:2445 is a regression check that the legacy call doesn't resurrect. Cleanup: deleted the empty tests/e2e/score-history.txt from the repo + added it to .gitignore (test-cusum.sh recreates it as a runtime artifact for the legacy --add mode); removed the stale .txt mention in plans/AUTO_SELF_UPDATE.md. cusum.sh keeps both --add (legacy plain-text) and --add-json (canonical JSONL) modes for backward compat. |
| 228 | Picked option (a): per-criterion judge transport now uses claude --print --output-format json --max-turns 1 --model claude-opus-4-7 --tools "" --setting-sources user --mcp-config '{"mcpServers":{}}' --strict-mcp-config from a clean tmpdir cwd, gated behind EVAL_USE_CLI=1. local-shepherd.sh exports it by default and drops the ANTHROPIC_API_KEY hard-fail. CI default (curl path) unchanged for paths without an authed CLI. Same model (claude-opus-4-7, explicitly pinned), same prompts, same JSON parsing, retry-once preserved. Security: MCP isolation matters — the criterion prompt embeds untrusted simulation output, and --tools "" alone only blocks built-in tools (user MCP servers like mcp__playwright__* are still reachable absent --strict-mcp-config). The local-Max shepherd is now honestly zero-API — sim, evaluator, and orchestration all on Max. Statistical parity proof (paired N=15) deferred to ROADMAP #212(i) Prove-It Gate; this PR is engineering, not a parity claim. Codex round 1 caught 2 P1s (MCP not isolated, model not pinned), both fixed in round 2. 15 new tests in tests/test-evaluate-cli-mode.sh (all green). Test contract update in test-local-shepherd.sh: replaced obsolete aborts_on_missing_api_key with positive runs_without_api_key. | |
| 230 | tests/e2e/local-shepherd.sh --compare-baseline flag ships. Runs same scenario on main (via git worktree add --detach) AND current branch, posts delta as check-run + PR comment. Atomic dual-row history append (Codex P1 round 1: deferred until candidate succeeds — no orphan baseline rows). BASELINE_TMPRUN nested under TMPRUN for trap-managed cleanup. 9 new quality tests (22/22 in test-local-shepherd.sh). Codex round 2 CERTIFIED 9/10. Unblocks #231 Phase 2. | |
| 234 | Fill out GitHub About + topics + homepage | ✅ DONE 2026-04-24. gh repo edit applied description ("SDLC enforcement for Claude Code — hooks, skills, and wizard setup in one command. TDD, planning, self-review, CI shepherd."), homepage (https://www.npmjs.com/package/agentic-sdlc-wizard), and 8 topics (ai-agent, anthropic, claude-ai, claude-code, code-quality, developer-tools, sdlc, tdd). User call-out 2026-04-24: empty About section screenshot. Future: revisit topics when OpenCode port ships (add opencode), when we add an LLM-agnostic angle. |
| 233 | Shipped: (c) reviewer-at-flagship invariant in skills/sdlc/SKILL.md:233 — if user pins model: "sonnet[1m]", cross-model review still runs at Opus 4.7 max / Codex gpt-5.5 xhigh; (d) tradeoff guidance in CLAUDE_CODE_SDLC_WIZARD.md:1027-1066 "Mixed-Mode Tier" section — when to use, why reviewer stays flagship, autocompact caveat, manual model pin instructions, Prove-It Gate as a future acceptance criterion. Deferred (no demand signal): (a) repo_complexity heuristic auto-detection, (b) setup wizard auto-write of model: pin to .claude/settings.json, Prove-It pair-test on 3+ simple repos. Rationale for deferring 2026-04-26: (1) primary maintainer is a Max-subscription user who always runs Opus 4.7 max — zero internal adoption pressure; (2) pinning model: in settings.json disables Max auto-mode (per existing wizard guidance at line 1011) — wrong default for Max users; (3) repo-complexity heuristic is fragile (LOC + fixture depth + stakes detection has high false-positive risk on borderline repos); (4) Prove-It Gate requires 3+ simple repos × paired API runs ≈ real Anthropic spend to validate something the maintainer won't use; (5) manual opt-in already works for API-cost users who want it — docs + SKILL.md teach model: "sonnet[1m]" pin. Kill rationale 2026-05-24 (per .reviews/roadmap-prio-codex.md): 1 month stale since 2026-04-26 deferral, no API-cost user has asked, maintainer remains a Max user with zero internal pressure, and auto-writing model: is a footgun for Max auto-mode users (disables per-turn auto-selection). Docs already cover manual opt-in for the rare API-cost user. If a real demand signal lands, re-open under the Demand-Signal-First gate. Research evidence (2026-04-26): Opus 4.7 ≈ 1.67× Sonnet 4.6 on /MTok (\5/$25 vs $3/$15); ~2.25× effective cost factoring Opus tokenizer 35% inflation. SWE-bench Verified gap = 8 pt (Sonnet 79.6%, Opus 87.6%); ~30% gap on hard agentic Intelligence Index. The 8-pt gap means simple-repo gating is non-trivial — without it, Sonnet-as-coder degrades on complex agentic work. | |
| 232 | /update-wizard should mimic claude update UX (one-shot CLI + skill sync) | Shipped pre-2026-04-29: skills/update/SKILL.md Step 1.5 detects installed CLI via npm ls -g or npx cache (with semver-aware version compare in a Node cmp() helper, since sort -V mishandles prereleases), compares to registry.npmjs.org/agentic-sdlc-wizard/latest, surfaces the gap with three upgrade options (A: refresh CLI cache only via npx -y agentic-sdlc-wizard@latest --version; B: one-shot CLI + project re-init via ... init --force; C: skip and keep stale). check-only precedence honored (report-only, no prompt). Graceful fallback when CLI undetectable. 8 quality tests in tests/test-update-skill-cli-version.sh cover detection paths, semver compare, check-only precedence, undetectable fallback, and CHANGELOG documentation. ROADMAP-update was the only remaining piece; closed 2026-04-29. |
| 231 | Weekly + monthly workflow migration — 4-phase kill-or-port plan | Full audit 2026-04-24 (see audit table below). 7 jobs across 2 files burn $25-55/week + $11-23/month with ZERO merged artifacts in 30d except one community-patterns commit. Phased execution so each phase is independently revertable. Per-job audit: • check-updates (weekly, $0.30/run) — 0 merged PRs; migrate: release detection is cheap GH API, ranking is 1× local Claude call• version-test (weekly, $8-20/run) — 0 merged PRs; biggest burn; unblocked by ROADMAP #230• scan-community (weekly, $2-5/run) — 1 merged PR (community-patterns/2026-04-23); last-community-scan.txt stale since 2026-01-01• community-e2e-test (weekly, $8-15/run) — 0 merged PRs; duplicates version-test pattern• prove-it-test (weekly, $6-12/run) — 0 merged PRs; fires on rare has_overlap condition; local runner already exists (tests/test-prove-it.sh, tests/e2e/lib/prove-it.sh)• deep-research (monthly, $3-8/run) — 0 merged PRs; "perplexity-as-CI" antipattern• e2e-test (monthly, $8-15/run) — never gated a merged artifact (blocked on deep-research producing something actionable)Phase 1 (~45 min, revised after Codex review 2026-04-24): ✅ DONE 2026-04-24. Deleted monthly-research.yml (519 lines, 2 API blocks, 0 merged artifacts in 30d). Stubbed all 17 test_monthly_* assertions in tests/test-workflow-triggers.sh with "n/a per #231 Phase 1" pattern (165/165 tests still green). Updated live docs: CI_CD.md (workflow table + overview + secrets table), ARCHITECTURE.md (file tree), plans/AUTO_SELF_UPDATE.md (5 sections marked historical). Historical audit tables left intact. No /monthly-research skill shipped — research happens inline in a Claude Code session.Phase 2 (~4-5 hrs): ✅ DONE v1.50.0 (2026-04-27, PR #275). local-shepherd.sh --compare-baseline --strip-paths '[paths]' ships the same-commit prove-it pattern. Deleted 251-line prove-it-test job from weekly-update.yml ($6-12/run, 0 merged artifacts in 30d). Updated tests/test-prove-it.sh to test the local runner (20/20 green). Codex round 2 CERTIFIED 9/10 — round 1 found 3 P1s (early-failure tmpdir leak, --strip-paths= empty-form silent fall-through, mislabeled main/PR in strip mode), all fixed with regression tests. 12 new tests in test-local-shepherd.sh (34/34 total). Phase 3 ready.Phase 3 (~2 hrs total, after Phase 2): Port remaining weekly jobs. (a) check-updates → split: keep the gh api releases detection in a tiny cron job (no API cost), move the Claude-ranker into a local tests/e2e/analyze-cc-release.sh invoked from the session-start nudge hook when a new version is detected.(b) scan-community → port to tests/e2e/scan-community.sh using the existing .github/prompts/analyze-community.md prompt file. Maintainer runs weekly on Max.(c) community-e2e-test → rewire to use local-shepherd.sh --compare-baseline.Phase 3a: ✅ DONE v1.51.0 (2026-04-28, PR #276). Deleted 319-line version-test job. Manual replacement: npm i -g @anthropic-ai/claude-code@<v> && tests/e2e/local-shepherd.sh <PR> --compare-baseline. Codex round 2 CERTIFIED 9/10.Phase 3b: ✅ DONE v1.52.0 (2026-04-29, PR #278). Deleted 231-line community-e2e-test job + dead has_external_findings/scan_payload outputs from scan-community. Manual replacement: maintainer reads scan-community digest, applies findings, runs tests/e2e/local-shepherd.sh <PR> --compare-baseline. Codex round 2 CERTIFIED 9/10 — round 1 found 1 P1 (live "external findings trigger E2E" prompt language stale) + 1 P2 (dead outputs), both fixed.Phase 3c: ✅ DONE v1.53.0 (2026-04-29, PR #280). Deleted 252-line scan-community job + cleaned up orphaned state-file/permission/test references. weekly-update.yml now has only check-updates (1 of 4 original jobs remain). Manual replacement: claude --print --allowedTools "WebFetch,Read,Bash" "$(cat .github/prompts/analyze-community.md)". Bonus hygiene fix: tests/test-cusum.sh save/restore trap (was silently wiping tests/e2e/score-history.jsonl on every run via hardcoded path). Codex round 5 CERTIFIED 10/10.Phase 3d: ✅ DONE v1.54.0 (2026-04-29, PR #282). Deleted ~150-line in-CI Claude-ranker chain (Build prompt + Analyze with claude-code-action@v1 + Extract + Parse) from check-updates job. Replaced with 12-line placeholder writing /tmp/analysis.json with relevance=UNKNOWN; auto-update PR body links the manual on-Max command using analyze-release.md. weekly-update.yml is now ZERO-API-spend. Codex round 2 CERTIFIED 10/10 — round 1 found Test 54 grep too narrow (didn't catch - uses: form) + 2 P2 doc/count nits, all fixed. Test 54 rewritten to use Python YAML step parsing; mutation-verified.Phase 3 final: 5 jobs/workflows + 1 in-job chain deleted across phases 1/2/3a/3b/3c/3d. Cron burn $25-55/week → $0/week (just $0.30 GH-API release detection). Phase 4: ✅ DONE v1.55.0 (2026-04-29, PR #284). Shrunk weekly-update.yml 289 → 161 lines (-44%). Deleted post-Phase-3d dead code: env.VERSION_SCENARIO (no-op since 3a), has_overlap/overlap_paths job outputs (consumed by nothing post-Phase-2), placeholder + parse steps (wrote /tmp/analysis.json purely to read it back), Build PR body step + case-on-relevance switch (relevance is always UNKNOWN). PR body inlined directly in create-pull-request body: field. Steps consolidated 10 → 5 operational (6 YAML steps including checkout). Decision: shrink, not delete-entirely — the GH PR audit trail is the only place a maintainer sees a new release while away from a CC session. Phase 3+4 cumulative: weekly-update.yml from ~1670 lines (pre-Phase 3) to 161 lines (-90%). Codex round 2 CERTIFIED 10/10 — round 1 was 7/10 with 3 findings (P1 cron-stagger false-greens on commented lines + 2 P2 doc/marker nits) all fixed and mutation-verified. Bonus fix: test_workflow_cron_staggered rewritten to use Python YAML parsing (was grep-based, false-greened on commented cron lines) + new test_workflow_cron_collision_detected regression test. ROADMAP #231 closes here.Risks & mitigations: • Risk: deleting the monthly workflow orphans its research update/* branch naming convention. Mitigation (resolved in Phase 1): branch-name refs only lived inside the deleted workflow; repo grep returns zero hits post-delete.• Risk: tests/test-workflow-triggers.sh asserts these workflows exist. Mitigation: stub those tests per ROADMAP #212 pattern (already done for e2e jobs).• Risk: stateless cron kills the weekly rhythm (things fall through cracks). Mitigation: session-start nudge hook + ROADMAP #220 token-spike detection fill the "reminder" role. • Risk: losing community-pattern insight signal. Mitigation: scan-community already produced only 1 merged PR in 30d — low baseline signal; local manual run preserves the entire pipeline.Verification per phase: • Phase 1: bash tests/test-workflow-triggers.sh green, python3 -c "import yaml; [yaml.safe_load(open(f)) for f in glob.glob('.github/workflows/*.yml')]" passes.• Phase 2: local shepherd paired runs show baseline vs candidate scoring on ≥1 scenario end-to-end. • Phase 3: each ported script produces equivalent output (issue body, PR comment, score-history append) to what the CI job did, confirmed by running once on Max. • Phase 4: gh workflow list shows only ci.yml + pr-review.yml + release.yml + any kept cron detection. No orphan workflows.User call-out 2026-04-24: "well monthly research could just trigger a local ci shepherd thing? or merge weekly and monthly? hmm" → audit concluded: GH Actions can't trigger on-maintainer-Mac, so the pattern is "cheap CI detection → notify via issue/PR → maintainer runs local shepherd." Merged monthly is low-value enough to just delete outright. |
| 229 | DONE Precompact hook stale-handoff auto-expire (PR #227) | Real UX bug found 2026-04-23 live-fire. The precompact-seam-check hook blocked a manual /compact at session end because .reviews/handoff.json was stuck at status: PENDING_RECHECK from the review (review_id: precompact-seam-001) that SHIPPED the hook itself. That original review was written BEFORE the self-heal pr_number feature (ROADMAP #209), so there's no PR to auto-query — the handoff is stuck forever until manually flipped. Fix shipped: hooks/precompact-seam-check.sh now treats PENDING_* + no pr_number + mtime older than SDLC_HANDOFF_STALE_DAYS (default 14) as implicit CERTIFIED with a one-line WARN (not HOLD). PR-linked handoffs still use the #209 self-heal path regardless of mtime. Env var lets power users tune the threshold. Tests: 4 added to tests/test-hooks.sh (stale-unblocks, fresh-still-blocks, stale-with-pr_number-prefers-self-heal, threshold-override). All 19 precompact tests green. User call-out: "you told me to compact then stopped me? ugh bad". |
| 218 | CC 2.1.118 introduced type: "mcp_tool" — hooks can now directly invoke MCP tools instead of bash scripts. Our 5 current hooks are all bash (_find-sdlc-root.sh, sdlc-prompt-check.sh, tdd-pretool-check.sh, instructions-loaded-check.sh, model-effort-check.sh, precompact-seam-check.sh). Audit whether any would benefit: candidate is a hook that wants to read/write structured state (e.g. a "score-history reader" hook that queries persisted JSON via an MCP tool instead of cat+jq). Prove-It Gate applies: don't rewrite a working bash hook as an MCP call unless quality improves. Default answer is leave them — bash hooks are portable to Codex/OpenCode; MCP hooks may not be | |
| 219 | Config-side verified 2026-04-30 against CC 2.1.118 (npm latest 2.1.123): both cli/templates/settings.json and .claude/settings.json have no "model" key (jq confirms has("model") == false). #198's recommendation to omit the pin is unchanged — the new persistence semantics are orthogonal (session-picked model now remembered, doesn't require a project pin). Codex grouping-review (2026-04-30) confirmed tests/test-cli.sh:1155 already asserts no default model pin, so no new test needed. Optional manual UX check for the maintainer when they upgrade past 2.1.123: in a temp repo with no project model, pick /model opus[1m], exit, restart, confirm the startup header keeps the user-selected model. If the persistence behavior reverses (project pins re-asserted instead of session selection persisting), reopen this row. | |
| 223 | DONE GPT-5.5 adopted in review tier (PR #230) | Shipped 2026-04-24: Codex config at ~/.codex/config.toml already defaults to model = "gpt-5.5" with model_reasoning_effort = "xhigh". All cross-model reviews in this session ran on GPT-5.5. Wizard docs updated (CLAUDE_CODE_SDLC_WIZARD.md:3468 + README.md quickstart section) to say "GPT-5.5 preferred, GPT-5.4 fallback" — Codex CLI auto-picks best available. Formal calibration (replay old handoffs through 5.5, compare P1 counts vs 5.4 on disk) deferred — low priority since we're already using 5.5 in production with strong qualitative results (session shipped 3 PRs via 5.5 review loop). If a future session sees suspicious 5.5 behavior, revisit calibration then. User call-out 2026-04-24: "im confused why does that need API and replay harness — [I already use 5.5]." |
| 195 | sdlc-prompt-check.sh now reads the UserPromptSubmit payload, scans for first-person LOW/FAILED/CONFUSED phrases (i'm stuck, i'm confused, i tried twice, i can't figure, i'm not sure why, my confidence is low, confidence: low, it's still failing, it keeps failing, it failed again, failed twice, plus contraction variants), and logs a timestamped signal to $SDLC_WIZARD_CACHE_DIR/effort-signals.log. When ≥2 signals land inside a 30-minute window, the hook emits a loud !! EFFORT BUMP REQUIRED !! block with the exact /effort xhigh command. Safety: writes wrapped in `{ ... } 2>/dev/null | |
| 196 | instructions-loaded-check.sh now caches npm latest for 24h in $SDLC_WIZARD_CACHE_DIR (default ~/.cache/sdlc-wizard/) and prints a loud multi-line !! WARNING: SDLC Harness is N minor versions behind !! when the gap is ≥3 minor (delta computed from the <!-- SDLC Harness Version --> metadata). 1–2 minor keeps the existing mild one-liner so we don't over-warn. 3 new hook tests: loud-when-≥3, mild-when-2, uses-daily-cache. Negative control proved the loud branch is actually gated | |
| 93 | Workflows already migrated to Node 24-native action versions: actions/checkout@v5, actions/setup-node@v5, actions/upload-artifact@v6. No oven-sh/setup-bun usage in repo. Verified by tests/test-node24-compliance.sh (13 tests, all green). Shipped pre-deadline (deadline was June 2, 2026) | |
| 183 | advisorModel (v1.81.0) | RESEARCH COMPLETE, API beta graduated. The API-only beta (advisor-tool-2026-03-01) graduated to native CC support as advisorModel in settings.json (CC v2.1.170+). Wizard v1.81.0 integrates natively: Setup A gets advisorModel: "fable", Setup B gets advisorModel: "claude-opus-4-6". Light tier research validated — Opus advisor compensates for Sonnet's weaker reasoning. Opus-advising-Opus confirmed waste. A/B workflow deferred — native advisor makes manual subagent spawning obsolete. |
| 95 | .reviews/research-95-nous.md | Verdict: NO-GO. Different layer of the stack — Nous Research builds open-weights LLMs (Hermes), agent frameworks (Hermes Agent), RL environments (atropos), and distributed-training infra (Psyche). No surface overlap with SDLC enforcement: they build the engines, we enforce the pipeline that uses an engine. Hermes Agent is the same layer as Claude Code / Codex CLI / OpenCode — a target for the wizard, not a competitor. The OpenCode port (#9) is the right vehicle if anyone ever wants the wizard to run against a self-hosted Nous endpoint. Pattern continues with #76 (Promptfoo NO-GO), #77 (constrain-to-playbook NO-GO), #235 (Thoughtworks AI Evals NO-GO): external-product audits keep validating our niche; revisit only when an external source surfaces a specific technique we don't have, not when it surfaces a different product. Original question: evaluate nousresearch.com — what are they building, how does it compare to SDLC wizard's approach? |
| 96 | Degradation infra shipped earlier. Phase 1 DONE v1.57.0 (PR #290): de-coached benchmark prompt. Phase 2 DONE v1.58.0 (PR #293): ground-truth gate. Phase 3 PR 1 DONE v1.60.0: wizard-installation lift-proof harness. Phase 3 PR 2 DONE v1.61.0: calibration scenario suite — tests/e2e/scenarios/calibration-careful-read.md is the first in a calibration-* family designed to reward self-review (parsePrice with 5 edge-case formats; rushed agent silently corrupts '\$1,000.00' to 1, a thousand-fold pricing bug). 6 new tests in tests/test-calibration-scenarios.sh (scenario format validator). End-to-end calibration verification (does low-effort agent actually score lower?) deferred to ROADMAP #212(i) Prove-It Gate paired runs. The #96 series is now complete: prompt de-coached → tests required → wizard contribution measurable → calibration signals embedded in scenario suite. | |
| 97 | .reviews/research-97-anthropic-policy.md | Verdict: NO-GO with one validating parallel. (a) Responsible Scaling Policy: Anthropic's own internal model-dev risk thresholds — no surface for third-party tooling to align. (b) Transparency Hub: model-card disclosures + baseline refusal rates — tangential, already covered by security-guidance plugin recommendation in #101 setup flow. (e) Research page: April 2026 "Automated Alignment Researchers" paper is conceptually parallel to our cross-model review pattern (LLM reviewing LLM via "scalable oversight"); our implementation predates the paper and already mitigates its noted weaknesses (reward hacking, limited generalization) via vendor-diverse adversarial framing + mission/verification-checklist handoff format. (c) Constitution + (d) Economic Futures skipped as clearly off-topic by inspection. 6/6 external audits NO-GO (continues #76, #77, #95, #99, #235). Optional follow-up: cite the AAR paper in the wizard's Cross-Model Review section as third-party precedent when a copy-edit pass happens. |
| 98 | PR #191. Added .github/ISSUE_TEMPLATE/ (bug_report, feature_request, question) with GitHub-valid frontmatter + config.yml (blank issues disabled, contact links to Discussions + in-session /feedback). Added .github/PULL_REQUEST_TEMPLATE.md with Prove-It Gate checklist + test plan section. Enabled GitHub Discussions via gh api ... -f has_discussions=true. Added README "Feedback" section with live links to all three paths. tests/test-community-paths.sh — 14 tests, mutation-verified (deleting config.yml or stripping frontmatter about: trips CI loudly). Codex xhigh 2-round code review: 5/10 → 10/10 CERTIFIED | |
| 99 | .reviews/research-99-autogpt.md | Verdict: NO-GO. AutoGPT is now an agent platform/framework (184k stars, actively maintained, pivoted from autonomous-task agent to "build, deploy, manage continuous agents"). Three blockers: (a) layer mismatch — AutoGPT is itself an agent host like Claude Code / Codex / OpenCode, not a target for SDLC enforcement; (b) no hook primitive — AutoGPT's "blocks" system is for workflow composition, not pre-tool-call enforcement, so the wizard's TDD/seam/prompt hooks have no place to live; (c) audience mismatch — AutoGPT users are agent builders running continuous services, not SWEs in interactive coding sessions. If a real demand signal surfaces, the right layering is "AutoGPT agent invokes Claude Code as a sub-tool" — that inherits the wizard for free without an AutoGPT port. Pattern continues with #76 + #77 + #235 + #95: 5/5 external-product audits NO-GO. |
| 102 | Removed hardcoded counts from README (workflows, criteria), CI_CD.md (CLI files), CODE_REVIEW_EXCEPTIONS (workflows). Fixed stale skill count in COMPETITIVE_AUDIT.md (2→4). test-doc-consistency.sh validates workflow/CLI-file/skill/scenario counts against filesystem, checks init.js ↔ disk parity. 10 tests | |
| 175 | Piebald-AI/claude-code-system-prompts is continuously diffed (updated within minutes of each CC release, 157 versions tracked since v2.0.14, last commit 2026-04-17 tracking v2.1.114). CHANGELOG.md records token-deltas per release (v2.1.113 +26 "Bash cd-prefix ban", v2.1.111 +21018 "Opus 4.7 + 1M context"). Native coverage overlap: plan mode IS natively enforced (agent-prompt-plan-mode-enhanced.md: "STRICTLY PROHIBITED from creating/modifying files"). Wizard should defer on plan mode. Non-overlapping (additive): TDD RED-before-GREEN, TodoWrite-before-work, HIGH/MEDIUM/LOW confidence, cross-model review gates — none enforced natively. Wizard's enforcement is genuinely additive. Adoptable patterns (low risk to integrate into skills + hook output): caps-lock negative-capability ("STRICTLY PROHIBITED"), role-exclusivity ("Your role is EXCLUSIVELY to…"), whitelisted operation enumeration ("Use Bash ONLY for read-only operations"). Feeds #96: weekly workflow can git clone the Piebald repo + diff per-release; large negative token deltas = candidate degradation signal (Boris Cherny "simplest approach" injection pattern) | |
| 203 | PR: 4 new size-cap tests in tests/test-hooks.sh. tdd-pretool (<500 bytes, observed 219), model-effort (<500, observed 211), instructions-loaded worst-case stale+loud-nudge+all-subchecks (<3000, observed 557), sdlc-prompt worst-case bump-firing+baseline (<1500, observed ~1220). Negative control: injecting 2KB of echo bloat into each hook trips 3-4 tests. Full suite 96/96. Addresses CC issue #50799 surface-area: any regression that grows a hook's stdout (runaway echo loop, bloated nudge copy, duplicated warnings) now fails tests rather than billing consumers | |
| 204 | Closed as absorbed. The cache-miss pattern (cache_read collapses → cache_creation spikes) surfaces directly in costly_tokens (the metric hooks/token-spike-check.sh already tracks). v1.63.0 added the explicit prove-it: tests/test-token-spike.sh:test_cache_miss_pattern_triggers_spike_warning builds a 20-row baseline of cache-hit-heavy sessions, appends one cache-miss session (cache_creation up, cache_read down), asserts the spike warning fires. Negative control test_high_cache_read_no_warning proves the detector keys on cost-bearing fields, not raw count. SDLC skill + wizard doc gained "Cache-Cost Surprises" sections covering triggers, detection via token-spike-check.sh, and practices to avoid mid-session context edits. Prove-It Gate satisfied. | |
| 206 | Native CC command added in v2.1.101 (2026-04-10). Generates local HTML report at ~/.claude/usage-data/report.html + per-session facet JSONs at ~/.claude/usage-data/facets/<session>.json. Data exposed is qualitative/behavioral only: underlying_goal, outcome, user_satisfaction_counts, friction_counts, brief_summary. No cache_read_input_tokens, no cache-hit ratio, no model-version tracking, no per-turn token breakdown. Does NOT feed #96 or #204 — cache/degradation signals live in session JSONL (~/.claude/projects/*/session.jsonl, usage.cache_read_input_tokens per turn). Known bug: sampling defect, generates facets for only 3–5 sessions while aggregates come from all. Action: add one-liner to setup-wizard skill + CLAUDE_CODE_SDLC_WIZARD.md: "Run /insights monthly to surface friction patterns." Full writeup: .reviews/research-206-insights.md | |
| 207 | Scanner side (tests/e2e/scan-community.sh) shipped earlier; this release adds the missing fetcher (tests/e2e/fetch-community.sh) so the maintainer doesn't have to manually paste transcripts. Pulls public threads from Reddit (r/ClaudeCode + r/ClaudeAI) via JSON API + HN Algolia search ("claude code" stories). Pipe to scan-community.sh: ./tests/e2e/fetch-community.sh --reddit ClaudeCode,ClaudeAI --hn | ./tests/e2e/scan-community.sh -. Live mode hits public no-auth endpoints — zero Anthropic API spend, fits post-#231 ethos (no cron, maintainer runs locally on Max when interested). Discord skipped (bot/OAuth complexity). GH Discussions deferred (GraphQL-only, marginal value over Reddit+HN). 14 quality tests (11 happy/error-path + 3 round-1 P0/P2 regressions), all offline via 5 JSON fixtures. Codex round 2 CERTIFIED 9/10 — round 1 was 6/10 with P0 code injection in parse_or_die (Codex reproduced execution; fixed via JSON_PATH env var) + P2 silent exit on missing flag values (fixed with validation + 2 regression tests). Original spec mentioned a "scheduled workflow" — superseded by #231's anti-cron ethos: detection-only manual run is the right shape. | |
| 208 | hooks/precompact-seam-check.sh gates manual /compact (matcher: "manual" — auto-compact NOT gated because blocking it could push past 100% context). Blocks with HOLD + exit 2 when .reviews/handoff.json status is PENDING_REVIEW / PENDING_RECHECK, or when git rebase (rebase-merge OR rebase-apply), merge, or cherry-pick is in flight. Seam taxonomy documented in CLAUDE_CODE_SDLC_WIZARD.md "Compact at Seams, Not Thresholds" section: commit boundary, Codex CERTIFIED, PR merge, ROADMAP item DONE. TodoWrite state not checked (CC doesn't persist task state to a readable file from hooks). Requires CC v2.1.105+ (PreCompact event). 10 quality tests (hook exists, 2 silent-paths, 6 block-paths including rebase-merge + rebase-apply + merge + cherry-pick + PENDING_REVIEW + PENDING_RECHECK, size-cap). Negative control: injecting 50 lines of stderr bloat into a copy of the hook causes the size-cap test to fail (≥ 1000 chars) — exact size depends on bloat line length (~1200 for one-word-per-line, ~4800 for nine-words-per-line). Both forms verify the cap gates runaway echo regressions. CLI distributes the hook; plugin hooks.json registers it. | |
| 209 | Self-heal logic shipped earlier in hooks/precompact-seam-check.sh:33-89: when handoff has pr_number and gh reports the PR MERGED, hook treats handoff as implicit CERTIFIED and unblocks /compact silently. 5 quality tests (tests/test-hooks.sh:392-504) cover MERGED-unblocks-silently, OPEN-still-blocks, missing-pr_number-still-blocks, gh-missing-falls-back-to-block, gh-errors-falls-back-to-block. Final closure 2026-04-26: documented pr_number as the self-heal opt-in in all 3 handoff template schemas (skills/sdlc/SKILL.md Step 1; CLAUDE_CODE_SDLC_WIZARD.md Round 1 + cross-model section). Without docs, consumers couldn't discover the opt-in and the self-heal path was effectively dead code. New regression test test_handoff_template_documents_pr_number enforces template/doc parity going forward. Hit live in this repo 2026-04-19 (PR #205) and 2026-04-26 (PR #253) — original handoffs lacked pr_number, fell through to #229 stale-expire fallback (14d default). Together #209 (PR-linked self-heal) and #229 (mtime auto-expire) close the bug from both directions. | |
| 210 | Defensive test_no_oven_sh_setup_bun shipped in PR #217 (commit 045c938) along with test_no_oven_sh_setup_bun_negative_control per Codex P2 (the negative-control fixture writes a fake workflow with the banned action and asserts the grep catches it). Verified 2026-05-04 audit: zero workflows in .github/workflows/ use oven-sh/setup-bun; all 15 Node 24 compliance tests green. The repo's full action surface is already Node 24-compatible (actions/checkout@v5, setup-node@v5, upload-artifact@v6, peter-evans/create-pull-request@v8, marocchino/sticky-pull-request-comment@v3, claude-code-action@v1). Hard deadline 2026-06-02 (GitHub forces Node 24) is comfortably met with no further work needed. | |
| 211 | Live code path was fixed in v1.36.0 (Next Up entry); v1.62.0 closes the remaining historical-data gap by backfilling 5 corrupted rows in score-history.jsonl (lines 22-25, 30 — add-ui-component + ui-styling-change scenarios) from max_score:10 to max_score:11. Codex grouping-review (2026-04-30) caught that the backfill wasn't done despite the live code being clean. | |
| 213 | Closed: same logic as #214 — SDLC mandates xhigh/max effort floor, so adaptive-thinking degradation doesn't affect compliant users. Shipping DISABLE_ADAPTIVE_THINKING=1 by default would mask the real issue (users at high or below). Keep it documented as an opt-in for power users who want to experiment. CLAUDE_AUTOCOMPACT_PCT_OVERRIDE=30 stays opt-in too (it's already in the template merge path when user sets it). User call-out: xhigh floor > defaulting a band-aid env var. | |
| 216 | Original plan was agentic-sdlc-wizard (URL parity with npm package). Revised 2026-04-23 after Codex cross-model review on the naming decision. Final call: repo = claude-sdlc-wizard (matches sibling codex-sdlc-wizard + planned opencode-sdlc-wizard), npm package = agentic-sdlc-wizard (unchanged, brand-neutral, safer re: Anthropic trademark, avoids disruptive npm rename). GH auto-redirects old URLs for git + web. Scope done in v1.36.1: repo rename via UI, package.json repository.url, sweep-replace across docs/tests/configs (README, CLAUDE_CODE_SDLC_WIZARD.md, CONTRIBUTING.md, ROADMAP.md, test-self-update.sh raw URLs, CI workflow refs). | |
| 214 | Closed: SDLC mandates xhigh floor / max default on Opus 4.7 (model-effort-check.sh, CLAUDE_CODE_SDLC_WIZARD.md:243). Adaptive thinking only degrades below xhigh, and we've ruled that out. The A/B would just prove what our effort-floor already assumes. Also closes #213 (don't default-ship DISABLE_ADAPTIVE_THINKING=1 — users opting into sub-xhigh effort are opting into degradation; not our problem). Saves ~$12 + spares API burn during canary-dead season. User call-out 2026-04-24: "with all the opus fixes i dont think we need to do adaptive thinking test anymore." | |
| 235 | .reviews/research-235-ai-evals.md | Verdict: NO-GO. Audit confirms the article is methodology-only (no tools/frameworks named); every layer Thoughtworks describes (pre-deployment validation, post-deployment monitoring, quality gates, continuous oversight, performance consistency, output accuracy, error-mode catching, "dynamic nature of AI") already has a working analog in our pipeline (Tier 1/2 evaluator + score-history + CUSUM + token-spike-check + adversarial cross-model review + SDP scoring). The only candidate gap is bias/alignment evaluation, which is explicitly out of scope (the wizard is SDLC enforcement, not LLM ethics — that belongs to model providers and domain-specific tools). Pattern continues with #76 (Promptfoo NO-GO) and #77 (constrain-to-playbook NO-GO): external-methodology audits keep validating our pipeline is mature; revisit only when an external source points to a specific technique we don't have. Original question: does Thoughtworks' AI Evals framework (thoughtworks.com/en-us/insights/decoder/a/ai-evals) add anything we don't already have in our CI/CD? User call-out 2026-05-01: "should we be using AI Evals like Thoughtworks defines? if so add to roadmap or research." |
| 347 | Goal-mode checkpoint workflow (Codex $gdlc equivalent) — CORRECTED 2026-05-24: native /goal exists, scope shrunk to ~30-line skill wrapper | GH issue #347 asks whether Claude Code has a native primitive for long-running goal-bound work (Codex $sdlc + $gdlc pattern with persistent constraints, checkpoint cadence, explicit stop boundaries). 2026-05-23 research was WRONG — claude-code-guide subagent said no native primitive, but CC v2.1.139 shipped native /goal (confirmed via raw changelog curl; docs at code.claude.com/docs/en/goal.md; follow-up fixes v2.1.140 hook-disabled hang + v2.1.143 subagent race). Corrected research at .reviews/347-goal-mode-research-CORRECTED.md. What /goal does: session-scoped, evaluator-driven (Haiku default judges transcript after each turn yes/no), survives --resume but not /clear, no disk writes, no native turn/time cap. UX = /goal <condition> + /goal (status) + /goal clear. Old verdict OBE. Corrected scope = ~30-line skill wrapper in existing /sdlc (NOT a GOAL.md template — /goal writes nothing to disk). Per .reviews/347-goal-mode-research-CORRECTED.md, the wizard should add: (a) pre-flight checklist — workspace trusted, hooks not disabled at any settings layer, CC ≥ v2.1.143; (b) condition-writing guidance mirroring SDLC reporting standards (measurable end state + check + constraints + hard turn/time bound since /goal has no native cap); (c) compose-with-hooks note — UserPromptSubmit/SessionStart/PreCompact fire normally inside the goal loop, so sdlc-prompt-check.sh + precompact-seam-check.sh keep gating each turn; (d) resume caveat — --resume resets turn/time counters; (e) anti-pattern callout — don't use /goal for "doneness" the evaluator can't see in the transcript (evaluator can't call tools). Status: DONE 2026-05-24 (PR #351, v1.76.0 — /goal wrapper section added to /sdlc skill + /update skill changelog entry; followed by PR #355 v1.77.0 adding HIGH-95% confidence gate + DLC-binding requirement + condition-as-contract guidance). Original 5-step plan + GOAL.md template scaffolding OBE. Meta-lesson captured in CORRECTED doc: require explicit citation of an authoritative source (docs index, raw changelog grep) before accepting a "feature does not exist" claim — negative claims are easier to fake than positive ones. Process gap also caught: auto-update PR workflow stopped flagging features at CC v2.1.118 (2026-04-23, last PR #210) because #231 Phase 3d (v1.54.0) gutted the in-CI LLM-ranker to $0. Manual replacement (claude --print --allowedTools "WebFetch,Read,Bash" "$(cat .github/prompts/analyze-release.md)") was supposed to run weekly on Max — it didn't, which is how /goal slipped past unnoticed for ~5 weeks. See #350 for the cadence fix. |