Conformance Scoreboard

June 3, 2026 · View on GitHub

The public scoreboard shows, for each Weaver Stack repo, whether its published artifacts pass the conformance suite. Participation is opt-in and costs one file plus one published JSON document.

Note

The scoreboard is a report, not a gate. A repo that publishes nothing is shown as not-submitted, never as failing. A passing row attests that a published artifact satisfies the conformance suite at a contract version — it does not attest to the correctness or security of the implementation.

How it works

  1. Each participating repo publishes a signed TraceBundle (the Extended audit-chain envelope — see TRACE_BUNDLE.md) at a stable, well-known URL.

  2. The scoreboard.yml workflow runs on a weekly schedule (and on demand). For each repo registered in conformance/siblings.yaml it fetches the URL and runs the conformance pack against the bundle:

    python conformance/run.py --bundle <fetched-bundle.json>

  3. conformance/scoreboard.py renders scoreboard.md and a shields.io endpoint badge per repo under docs/badges/. The build is published as a workflow artifact and written to the job summary.

Participate

  1. Publish a bundle. Serve a conformant TraceBundle at a stable URL. The convention is .well-known/conformance.json on your project domain, e.g. https://your-project.dev/.well-known/conformance.json. Sign it (see SIGNING.md) so verifiers who hold your signing key can confirm provenance. Note: the public scoreboard always validates the signature envelope, but only cryptographically verifies it when your signing key (kid) is in its keyring. The default scheduled run carries only this repo's test keyring, so a sibling's row currently attests schema + invariant conformance — the row detail states whether the signature was actually verified, left unverified, or the bundle was unsigned.

  2. Register. Add an entry to conformance/siblings.yaml:

    siblings:
  - repo: your-project
    url: https://your-project.dev/.well-known/conformance.json

  3. Verify locally before opening a PR:

    python conformance/run.py --bundle path/to/your/bundle.json

Hosting the rendered scoreboard (optional)

The workflow uploads scoreboard.md + docs/badges/ as an artifact and writes the table to the run's job summary, so it works with no extra setup. To serve it as a web page, enable GitHub Pages for this repo (Settings → Pages) and point it at docs/; the badge endpoints are then consumable via https://img.shields.io/endpoint?url=<raw badge URL>. Enabling Pages is a repository-admin action and is intentionally left out of the workflow.