Evidence Index
July 24, 2026 ยท View on GitHub
Evidence is split into historical/external sanitized summaries and a future current-tree release chain. Summary receipts are review context: even a passed external GPU result does not satisfy the runnable gate without source, runtime, input, and raw-artifact binding.
| File | Scope |
|---|---|
megatron_full_sequence_baseline.json | historical 32K pass and 2M full-sequence OOM summary |
2m_execution_receipt.json | predecessor-source 2M execution and actor-observed optimizer properties |
forward_parity_32k.json | source-unbound historical CP32 forward canary |
gradient_parity_32k.json | current external 32K ratio-1 and active-clip comparison; both cases passed |
online_canary_32k.json | source-unbound external 32K execution canary |
online_canary_64k.json | source-unbound external 64K execution canary |
online_grpo_2m.json | source-unbound external exact-2M policy rollout and completed G=2 update |
manifest.json | release-asset binding to a clean source commit |
The exact-2M receipt records real policy-LoRA sampling, a real DAPO row and
rule reward, distinct completions with rewards [-1,+1], 32/32 completed
ranks, two backwards, one finalized optimizer step, 64 rank traces, finite
numerics, and cleanup. It additionally binds the Tinker multi-node sampling backend,
two exact-2M eight-sample requests, a 16-output bounded decode audit, and the
ready_for_2m_training decision. The audit records no Unicode replacement
character, one obvious degenerate output, and one target-marker echo; it is an
execution diagnostic rather than a post-training quality claim. The parity
receipt records selected log-probabilities,
losses, complete parameter-shard sets and hashes, zero/nonzero masks, global
gradient metrics, target-family aggregates, representative outliers, and the
project's explicit acceptance decision. See
docs/glm52_2m_validation.md.
Runnable evidence belongs under evidence/current/. One validation plan binds
the source tree, runtime lock, OCI digest, model manifest, backend, optimizer
contract, profiles, fixtures, and topology. Receipts must form the fixed chain
doctor -> kernel -> nccl -> actor -> 32k -> 64k -> 2m. Each receipt must bind
the exact rank and artifact manifests used to evaluate its stage; a JSON status
alone is never evidence.
The operator-facing first-stage bundle is produced in an external evidence directory (never inside the checkout):
| Stage | Raw input/output | Generic receipt |
|---|---|---|
| doctor | doctor.json, doctor-ranks.json, doctor-artifacts.json | doctor-validation.json |
| kernel | kernel-smoke.json, kernel-ranks.json, kernel-artifacts.json | kernel-validation.json |
| nccl | nccl-smoke.json, nccl-ranks.json, nccl-artifacts.json | nccl-validation.json |
Run make doctor-validation before make kernel-smoke; the kernel command
revalidates the complete doctor bundle and consumes the same run UUID,
validation-plan identity, runtime lock, image digest, CUDA device, and
precompiled TileLang cache. Both commands emit one JSON status line. Exit 0
means the complete bundle passed; exit 2 means blocked or failed and cannot be
promoted to the next stage. The required order is doctor -> kernel -> NCCL -> actor -> 32K -> 64K -> 2M.
make nccl-smoke revalidates the exact kernel-validation.json sibling bundle
and uses the same validation plan and UUID. The four target Ray worker cluster
processes themselves must run the same immutable image digest and advertise
mint2m_image_<64hex>=1 plus their Ray node:<ip> resources. The launcher
requires each target IP to have exactly one live Ray record, exactly 8 GPUs, at
least 8 CPUs, and both node and digest resources; bundles bind both, while
unrelated nodes remain untouched. NCCL actors do not use
runtime_env.image_uri.
Expose the absolute runtime_identity path with the same bytes to the driver and
every target worker cluster image, open NCCL_MASTER_PORT on
NCCL_SOCKET_IFNAME, and optionally select NCCL_IB_HCA. CLI pre/post and all 32 actor identity hashes
must agree. Allow 5-10 minutes on a healthy cluster after image pull; a blocked
attempt can use the fixed launcher timeout. Acceptance requires exact ranks,
collectives, runtime/fabric consensus, process-group destruction, and
automatic cleanup; interrupted attempts require explicit run-UUID-scoped
cleanup before retry.
No source-bound live 32-GPU NCCL promotion chain is checked into this repository
today. The external exact-2M and parity summaries are not a substitute for the
current-tree CUDA, NCCL, actor, and raw-evidence chain required for public
reproduction. 5.4 implementation does not make the release runnable; the
public status therefore remains review_only_not_runnable.
The release checker keeps the execution-source identity separate from current
receipts to avoid a self-reference. After implementation and evidence are
committed, evidence/manifest.json is generated in a binding-only child commit
as described in CONTRIBUTING.md.
The public repository is updated by the private -alpha repository's
Release to Public workflow, not by copying evidence or pushing directly to
the public remote. See docs/public_release.md.
Raw prompts, unbounded model outputs, checkpoint paths, hostnames, internal network addresses, credentials, and unbounded traces are excluded. The exact-2M receipt retains only two bounded math completions, their token hashes and old log-probabilities, and content hashes for the complete external artifact. Approved rollout fixtures remain separately hash-bound release inputs rather than sanitized claims.