Dev Scenario Runner

The scenario-dev GitHub Actions workflow runs one selected scenario against an isolated Duckgres stack in mw-dev. It uses the same tests/mw-dev harness as e2e-mw-dev for cluster access, isolated namespace deployment, pod identity, diagnostics, and teardown.

Scheduled Runs

The scheduled trigger runs full-suite, which covers the frozen dataset metadata, perf, and dbt workloads with one shared warehouse lifecycle. Manual runs also default to full-suite.

Scenario jobs override the shared harness's default worker request to 2 CPU and 8Gi memory for the frozen pgwire perf workload. This adds process headroom for repeated full-dataset aggregates without increasing DuckDB's thread count. The shared e2e workflow keeps the harness defaults documented in tests/mw-dev/README.md.

To omit dbt, manually run the workflow with scenario set to fast-suite. The input directly names a YAML file under tests/mw-dev/scenario/scenarios/, without the .yaml suffix. This also makes targeted scenarios such as provision_rejection, provision_smoke, posthog_frozen_metadata, posthog_frozen_perf, and posthog_frozen_dbt individually runnable. Invalid or missing scenario names fail before a payload Job is created.

Required Repository Configuration

The workflow follows the e2e dev harness access pattern: GitHub OIDC assumes a dev-scoped AWS role, Tailscale reaches the private cluster API, and payload jobs execute inside Kubernetes.

Configure these repository secrets:

• AWS_ECR_PUBLISH_IAM_ROLE
• MW_DEV_ACCOUNT_ID

Configure these repository variables:

• TS_WIF_CLIENT_ID_MW_DEV
• TS_WIF_AUDIENCE_MW_DEV
• MW_DEV_SCENARIO_PERF_SECRET_ID

MW_DEV_SCENARIO_PERF_SECRET_ID names the AWS Secrets Manager JSON secret used only by historical perf publishing. The secret must contain host, port, database, username, and password. The workflow pipes it directly from AWS CLI into the publisher process so the password is not placed in a shell argument, environment variable, or log.

The workflow hardcodes the stable mw-dev cluster name/context and builds the control-plane pod identity role ARN from MW_DEV_ACCOUNT_ID, matching e2e-mw-dev. The scenario payload uses the isolated control-plane ClusterIP service and the per-run internal secret generated by tests/mw-dev/run.sh. The harness also injects DUCKGRES_SCENARIO_ORG_ID=ci-pr-<workflow-run-id>-cnpg for successful CNPG scenarios. That name matches the Crossplane composition's exact CI credential ownership boundary and the harness's cleanup convention.

For a direct local run with just scenario, set DUCKGRES_SCENARIO_ORG_ID explicitly to an authorized, disposable org id. The local runner has no default because reusing a fixed warehouse identity can collide with another run. Scenario-specific frozen-dataset variables remain required as reported by scripts/scenario_run.sh --check-env.

Failure Recovery

Every scenario sub-run provisions its own org or warehouse name and includes cleanup steps. The isolated stack also tears down the temporary namespace after the payload finishes. The harness keeps the artifact volume attached to a lightweight sidecar until the runner copies it; a missing or failed copy fails the scenario job instead of being silently ignored. Incomplete copies remain uploadable in a visible *.partial/ directory with an artifact_collection_error.txt marker. When a run fails, inspect the uploaded scenario-dev-* artifact first:

• scenario_summary.json contains the overall run verdict.
• scenario_summary.md highlights failed and skipped steps and is also published in the GitHub Actions job summary.
• step_results.csv contains per-step duration, status, and error class.
• events.jsonl contains the execution timeline.

Perf query errors mark the perf_queries DAG step failed. They do not stop independent sibling branches: for example, dbt_models still runs because it depends on setup_frozen_views, not perf_queries. Only true dependants are skipped, and always_run teardown still executes. The final workflow result is reported after all eligible steps and artifact collection finish.

After teardown, scheduled and manually dispatched runs on main publish any collected perf/summary.json and perf/query_results.csv into the persistent scenario perf result schema. Publishing runs with if: always() so measured query failures remain visible. Runs without a perf artifact skip publishing, and non-main branch runs never publish into the shared history. Each artifact has a two-minute publish timeout so one stalled database operation cannot prevent later artifacts from being attempted; the workflow step retains a ten-minute overall timeout.

If cleanup did not complete, the scenario-created org is ci-pr-<workflow-run-id>-cnpg; deprovision it manually through the relevant dev control plane. Also inspect the workflow diagnostics from tests/mw-dev/run.sh diagnostics; the namespace is duckgres-ci-pr-<workflow-run-id>.