βοΈ Workflow Audit Template
May 3, 2026 Β· View on GitHub
π Template Instructions: Copy to
analysis/daily/{date}/{article-type}-run{N}/intelligence/workflow-audit.md. The AI agent produces this at the end of the run as a structured self-audit of how the 10-step protocol was executed. See methodologies/per-artifact-methodologies.md Β§workflow-audit.
π― Purpose: Transparent record of workflow execution β which phases ran, which MCP tools were called, which rules of ai-driven-analysis-guide.md were satisfied, and where the run fell short. A downstream reviewer (and the next same-type run) can use this to understand the run's internal quality.
π Document Metadata
articleType: [REQUIRED]
runId: [REQUIRED]
date: [REQUIRED: YYYY-MM-DD]
analysisPhase: workflow-audit
confidenceLevel: [REQUIRED: HIGH / MEDIUM / LOW]
rulesAudited: [REQUIRED: integer β 11 Core Principles, see Β§3 below]
complianceRate: [REQUIRED: 0β100]
1οΈβ£ Workflow Execution Summary
%%{init: {"theme":"dark","themeVariables":{"primaryColor":"#1565C0","primaryTextColor":"#ffffff","primaryBorderColor":"#0A3F7F","lineColor":"#90CAF9","secondaryColor":"#2E7D32","secondaryTextColor":"#ffffff","tertiaryColor":"#FF9800","tertiaryTextColor":"#000000","mainBkg":"#1565C0","secondBkg":"#2E7D32","tertiaryBkg":"#FF9800","noteBkgColor":"#FFC107","noteTextColor":"#000000","errorBkgColor":"#D32F2F","fontFamily":"Inter, Helvetica, Arial, sans-serif"}}}%%
flowchart LR
P0["Phase 0<br/>Health gate"] --> P1["Phase 1<br/>Data collection"]
P1 --> P2["Phase 2<br/>Editorial context"]
P2 --> P3["Phase 3<br/>Analysis"]
P3 --> P4["Phase 4<br/>Significance gate"]
P4 --> P5["Phase 5<br/>Validation"]
P5 --> P6["Phase 6<br/>PR creation"]
style P0 fill:#1565C0,color:#ffffff
style P1 fill:#1565C0,color:#ffffff
style P2 fill:#0288D1,color:#ffffff
style P3 fill:#7B1FA2,color:#ffffff
style P4 fill:#FF9800,color:#000000
style P5 fill:#2E7D32,color:#ffffff
style P6 fill:#2E7D32,color:#ffffff
| Phase | Status | Notes |
|---|---|---|
| Phase 0 β Health gate | [β
/ β οΈ / β] | [REQUIRED: what gate returned] |
| Phase 1 β Data collection | [β
/ β οΈ / β] | [REQUIRED: feeds attempted, feeds succeeded] |
| Phase 2 β Editorial context | [β
/ β οΈ / β] | [REQUIRED] |
| Phase 3 β Analysis | [β
/ β οΈ / β] | [REQUIRED: artifacts produced] |
| Phase 4 β Significance gate | [β
/ β οΈ / β] | [REQUIRED: composite score vs. threshold] |
| Phase 5 β Validation | [β
/ β οΈ / β] | [REQUIRED: validator exit code] |
| Phase 6 β PR creation | [β
/ β οΈ / β / β³] | [REQUIRED] |
2οΈβ£ MCP Tool Call Log
| # | Tool | Args (truncated) | Result | Records | Latency |
|---|---|---|---|---|---|
| 1 | [REQUIRED] | [REQUIRED] | [β
/β οΈ/β] | [#] | [ms] |
| 2 | [REQUIRED] | [REQUIRED] | [...] | [#] | [ms] |
| 3 | [REQUIRED] | [REQUIRED] | [...] | [#] | [ms] |
Total MCP calls: [REQUIRED] Β· Successful: [#] Β· Degraded: [#] Β· Failed: [#]
3οΈβ£ Core Principles Compliance (11-principle protocol)
See ai-driven-analysis-guide.md Β§Core Principles.
| # | Principle | Status | Evidence in run |
|---|---|---|---|
| 1 | Folder isolation | [β
/β] | [REQUIRED: run dir path] |
| 2 | AI does analysis, scripts do HTML | [β
/β] | [REQUIRED] |
| 3 | Methodologies before analysis | [β
/β] | [REQUIRED: METHODOLOGIES_READ log line] |
| 4 | Multi-framework depth | [β
/β οΈ/β] | [REQUIRED: frameworks applied] |
| 5 | Always commit analysis | [β
/β] | [REQUIRED] |
| 6 | Article type everywhere | [β
/β] | [REQUIRED] |
| 7 | Two passes, full time budget | [β
/β οΈ/β] | [REQUIRED: Pass 1 min + Pass 2 min] |
| 8 | AI-authored headlines + descriptions | [β
/β/ N/A] | [REQUIRED] |
| 9 | Complete data + historical baseline | [β
/β οΈ/β] | [REQUIRED] |
| 10 | Read-before-article + footer + ratio + floors | [β
/β οΈ/β] | [REQUIRED: validator output] |
| 11 | OSINT / INTOP tradecraft discipline (WEP bands, Admiralty grades, β₯10 SATs in methodology-reflection) | [β
/β οΈ/β] | [REQUIRED: link to methodology-reflection.md Β§12 SATs applied + Admiralty-graded source list] |
Compliance rate: [# principles β
] / 11 = [%]
4οΈβ£ Artifact Production
| Folder | Planned | Produced | Short (< floor) | Notes |
|---|---|---|---|---|
intelligence/ | [#] | [#] | [#] | [REQUIRED] |
classification/ | [#] | [#] | [#] | [REQUIRED] |
risk-scoring/ | [#] | [#] | [#] | [REQUIRED] |
threat-assessment/ | [#] | [#] | [#] | [REQUIRED] |
documents/ | [#] | [#] | [#] | [REQUIRED] |
5οΈβ£ Time Budget
| Step | Target | Actual | Delta |
|---|---|---|---|
| Step 2 β Read methodologies | 4β6 min | [REQUIRED] | [Β± min] |
| Step 3 β Collect EP data | 5β10 min | [REQUIRED] | [Β± min] |
| Steps 4β7 β Analysis Pass 1 | β₯12 min | [REQUIRED] | [Β± min] |
| Step 9 β Pass 2 | β₯8 min | [REQUIRED] | [Β± min] |
| Step 10 β Validate + commit | 2β5 min | [REQUIRED] | [Β± min] |
| Total run time | β₯20 min active | [REQUIRED] | [Β± min] |
6οΈβ£ Issues & Deviations
For every deviation from the standard protocol:
Issue 1 β [REQUIRED: short name]
- Symptom:
[REQUIRED] - Root cause:
[REQUIRED] - Workaround in this run:
[REQUIRED] - Next-run recommendation:
[REQUIRED]
(repeat per issue; no issues β state "No deviations from protocol observed.")
7οΈβ£ Recommendations for the Next Same-Type Run
[REQUIRED: concrete, actionable β e.g. "raise historical-baseline depth to β₯200 lines by fetching 3 additional prior-run comparables"][REQUIRED][REQUIRED]
8οΈβ£ EP MCP Tool Inputs (auditor's evidence basis)
| EP MCP tool | Used for which audit principle | Notes |
|---|---|---|
get_voting_records | P3 Evidence β RCV citations in artifacts | Auditor cross-checks every RCV ID claimed. |
analyze_coalition_dynamics | P4 Coalition arithmetic veracity | Two-window verification. |
get_adopted_texts | P3 Evidence β text-citation veracity | Confirms quoted recitals exist. |
get_committee_documents | P5 Source diversity | β₯1 committee-doc per major artifact. |
track_legislation / get_procedures | P6 Legislative-pipeline accuracy | Procedure-status drift between artifacts. |
get_speeches | P3 Evidence β quote attribution | Confirms speakers + verbatim. |
get_meeting_decisions | P7 Procedural veracity | Bureau / CoP rulings cited correctly. |
correlate_intelligence | P11 Cross-tool correlation | Confirms intelligence alerts feed into artifacts. |
(Auditor uses these tools directly β does not rely on prior-stage agent outputs.)
9οΈβ£ Worked Pass-1 β Pass-2 Example (60-min breaking-news run audit)
β Pass-1 (thin, 17 words):
"Run completed in 58 minutes. All artifacts present. Quality acceptable. No major issues observed during execution."
β Pass-2 (compliant, 95 words):
Run 184 (breaking, 2026-04-18) executed in 58:42 (target 60:00). Stage budget actuals: Data 11:30 / Analysis 21:50 / Gate 3:20 / Article 17:40 / PR 4:22. Two principle deviations: P3 Evidence β
voting-patterns.mdcited RCV 2026/0089(COD) but noget_voting_recordstool call recorded in tool-call log (LOW confidence flag missing β Partial); P9 Reflection βmethodology-reflection.mddocumented only 8 of required 10 SATs (Partial). All other 9 principles Pass. Stage-C gate caught both deviations and triggered Pass-2 rewrites of the affected artifacts. Net verdict: CONDITIONAL PASS β recommendations Β§7 cover both deviations.
π 11-Core-Principles Compliance Scorecard
| # | Principle | Compliance level | Worked observation |
|---|---|---|---|
| P1 | BLUF discipline | β Pass | All intelligence-tier artifacts open with β€2-sentence BLUF. |
| P2 | WEP / Admiralty tradecraft | β Pass | All forward judgements carry WEP band + horizon. |
| P3 | Evidence sourcing (cite-or-don't-claim) | β οΈ Partial | 1 RCV cited without get_voting_records tool-call evidence. |
| P4 | Coalition arithmetic discipline | β Pass | Seat shares + cohesion deltas all from analyze_coalition_dynamics. |
| P5 | Source diversity | β Pass | 9 distinct EP MCP tools used across 25 artifacts. |
| P6 | Legislative-pipeline accuracy | β Pass | Procedure-status verified against track_legislation. |
| P7 | Procedural veracity | β Pass | Rule numbers cited match published Rules of Procedure. |
| P8 | Read-Before-Write (article cites artifacts) | β Pass | All article Β§Β§ cite per-section artifacts per 04-article-generation.md Β§7.1. |
| P9 | Methodology reflection (β₯10 SATs) | β οΈ Partial | 8 of 10 SATs documented. |
| P10 | Per-artifact depth floors | β Pass | All artifacts meet reference-quality-thresholds.json. |
| P11 | Time-budget discipline (β₯45 min active work on 60-min budget) | β Pass | 47 min active per timestamps. |
Composite: 9 Pass / 2 Partial / 0 Fail = CONDITIONAL PASS, requires Pass-2 fixes on P3 and P9 before PR.
1οΈβ£1οΈβ£ Worked 60-Minute Breaking-News Time-Budget Breakdown
| Phase | Target | Actual (Run 184) | Delta | Notes |
|---|---|---|---|---|
| Stage A β Data collection (MCP feeds) | 12:00 | 11:30 | -0:30 | Within tolerance. |
| Stage B Pass 1 β Analysis artifacts (initial draft) | 14:00 | 13:20 | -0:40 | OK. |
| Stage B Pass 2 β Analysis rewrite + depth | 09:00 | 08:30 | -0:30 | Below 9 min β auditor flag: rushed. |
| Stage C β Completeness gate | 03:00 | 03:20 | +0:20 | Includes the two Partial-principle re-runs. |
| Stage D Pass 1 β Article draft | 11:00 | 10:50 | -0:10 | OK. |
| Stage D Pass 2 β Article rewrite | 07:00 | 06:50 | -0:10 | OK. |
| PR creation + final checks | 04:00 | 04:22 | +0:22 | OK. |
| Total | 60:00 | 58:42 | -1:18 | Active work 47:00 (β₯45-min floor). |
1οΈβ£2οΈβ£ Anti-patterns β REJECT on Pass-2 Review
| # | Banned pattern | Why it fails |
|---|---|---|
| 1 | "All principles Pass" with no per-principle worked observation | Audit must show its working; reviewer cannot verify. |
| 2 | Time-budget table with target column only (no actual) | Drift cannot be measured. |
| 3 | "No deviations from protocol" claim while >1 artifact below depth floor | Internally inconsistent. |
| 4 | Recommendation "tighten Pass-2 review" without specifying artifact + principle | Non-actionable. |
| 5 | Audit assigning Pass to P9 with <10 SATs documented | Hard contract; SAT count is checked. |
| 6 | Citing tool-calls from get_voting_records while the EP roll-call publication delay >4 weeks (claim impossible) | Auditor must verify upstream feed availability. |
1οΈβ£3οΈβ£ Cross-References β Controlling Methodology
../methodologies/ai-driven-analysis-guide.mdβ Β§ Rules 1β22, the 11 Core Principles being audited.../methodologies/per-artifact-methodologies.md#workflow-auditβ construction rules.../methodologies/osint-tradecraft-standards.mdβ Β§ SAT enumeration (β₯10 required for P9 Pass)../methodology-reflection.mdβ sister artifact; auditor cross-reads.scripts/validate-analysis-completeness.jsβ depth-floor check feeds P10 verdict.scripts/lint-prompts.jsβ banned-pattern check for prompt drift.
1οΈβ£4οΈβ£ Stage-C Completeness Signals
scripts/validate-analysis-completeness.js checks for this artifact:
| Check | Threshold | Source |
|---|---|---|
| Line floor | β₯100 lines (default) | reference-quality-thresholds.json |
| Required H2 substrings | "Compliance", "Time-Budget", "Recommendations" | structural contract |
| Mermaid block | optional (Gantt of stage timings if drift >5 %) | visual contract |
| Tradecraft markers | Per-principle Pass/Partial/Fail explicit; SAT count documented | ai-driven-analysis-guide.md |
| Source diversity | β₯5 EP MCP tools cross-checked | per-artifact rule |
| Time-budget arithmetic | Targets sum to 60/120 min; actuals + deltas explicit | template logic |
Document Control: /analysis/daily/{date}/{type}-run{N}/intelligence/workflow-audit.md Β· Template v1.2 Β· Depth floor: 100 lines.