Live qualification coverage

September 10, 2026 ยท View on GitHub

The canonical entry point is tests/live/run-profile.sh; tests/test_live.sh, scripts/smoke-test.sh, and scripts/live-cli-sweep.sh are compatibility wrappers only. The runner starts real Cortex, oracle, proxy, browser, agent, and protocol clients as required by each profile. Direct database fixtures count as setup and never as proof of an ingest route.

The surface contract denominator includes UDP and TCP syslog, OTLP, managed file-tail, host-local agent and legacy central-pull ingestion plus MCP, REST, and CLI surfaces. Browser, artifact, and upgrade behavior are scenario capabilities layered on those transport entries; they are not separate surface kinds in the generated inventory. Unit coverage remains available through just coverage (cargo llvm-cov).

Every required surface must record exactly one first-attempt semantic outcome and evidence. Missing, duplicate, skipped, unsupported, qualified, or not-authorized outcomes fail mandatory isolated profiles. Diagnostic retries do not replace a first failure. A green run also requires cleanup state CLEAN and zero unaccounted contract entries.

Stable commands and cadence are documented in Live qualification. The complete safety, artifact, troubleshooting, and recovery contract is in the harness README.

just live-selftest
just live-docs-check
just live-smoke

This table is generated from the compiled SurfaceContract and profiles.json; do not edit counts by hand.

InventoryCount
mcp surfaces60
rest surfaces93
cli surfaces180
ingest surfaces32
artifact surfaces0
browser surfaces0
all surfaces365
runnable profiles21

Profiles: agent, artifacts, auth, compose-isolated, docker-boundary-full, docker-boundary-reduced, fleet-mutating, fleet-read-only, full, isolated, legacy-central-pull, mcp, mutation, noop, notifications, security, smoke, soak, stateful, storage, upgrade