Evidence Index

July 24, 2026 ยท View on GitHub

Evidence is split into historical/external sanitized summaries and a future current-tree release chain. Summary receipts are review context: even a passed external GPU result does not satisfy the runnable gate without source, runtime, input, and raw-artifact binding.

FileScope
megatron_full_sequence_baseline.jsonhistorical 32K pass and 2M full-sequence OOM summary
2m_execution_receipt.jsonpredecessor-source 2M execution and actor-observed optimizer properties
forward_parity_32k.jsonsource-unbound historical CP32 forward canary
gradient_parity_32k.jsoncurrent external 32K ratio-1 and active-clip comparison; both cases passed
online_canary_32k.jsonsource-unbound external 32K execution canary
online_canary_64k.jsonsource-unbound external 64K execution canary
online_grpo_2m.jsonsource-unbound external exact-2M policy rollout and completed G=2 update
manifest.jsonrelease-asset binding to a clean source commit

The exact-2M receipt records real policy-LoRA sampling, a real DAPO row and rule reward, distinct completions with rewards [-1,+1], 32/32 completed ranks, two backwards, one finalized optimizer step, 64 rank traces, finite numerics, and cleanup. It additionally binds the Tinker multi-node sampling backend, two exact-2M eight-sample requests, a 16-output bounded decode audit, and the ready_for_2m_training decision. The audit records no Unicode replacement character, one obvious degenerate output, and one target-marker echo; it is an execution diagnostic rather than a post-training quality claim. The parity receipt records selected log-probabilities, losses, complete parameter-shard sets and hashes, zero/nonzero masks, global gradient metrics, target-family aggregates, representative outliers, and the project's explicit acceptance decision. See docs/glm52_2m_validation.md.

Runnable evidence belongs under evidence/current/. One validation plan binds the source tree, runtime lock, OCI digest, model manifest, backend, optimizer contract, profiles, fixtures, and topology. Receipts must form the fixed chain doctor -> kernel -> nccl -> actor -> 32k -> 64k -> 2m. Each receipt must bind the exact rank and artifact manifests used to evaluate its stage; a JSON status alone is never evidence.

The operator-facing first-stage bundle is produced in an external evidence directory (never inside the checkout):

StageRaw input/outputGeneric receipt
doctordoctor.json, doctor-ranks.json, doctor-artifacts.jsondoctor-validation.json
kernelkernel-smoke.json, kernel-ranks.json, kernel-artifacts.jsonkernel-validation.json
ncclnccl-smoke.json, nccl-ranks.json, nccl-artifacts.jsonnccl-validation.json

Run make doctor-validation before make kernel-smoke; the kernel command revalidates the complete doctor bundle and consumes the same run UUID, validation-plan identity, runtime lock, image digest, CUDA device, and precompiled TileLang cache. Both commands emit one JSON status line. Exit 0 means the complete bundle passed; exit 2 means blocked or failed and cannot be promoted to the next stage. The required order is doctor -> kernel -> NCCL -> actor -> 32K -> 64K -> 2M.

make nccl-smoke revalidates the exact kernel-validation.json sibling bundle and uses the same validation plan and UUID. The four target Ray worker cluster processes themselves must run the same immutable image digest and advertise mint2m_image_<64hex>=1 plus their Ray node:<ip> resources. The launcher requires each target IP to have exactly one live Ray record, exactly 8 GPUs, at least 8 CPUs, and both node and digest resources; bundles bind both, while unrelated nodes remain untouched. NCCL actors do not use runtime_env.image_uri.

Expose the absolute runtime_identity path with the same bytes to the driver and every target worker cluster image, open NCCL_MASTER_PORT on NCCL_SOCKET_IFNAME, and optionally select NCCL_IB_HCA. CLI pre/post and all 32 actor identity hashes must agree. Allow 5-10 minutes on a healthy cluster after image pull; a blocked attempt can use the fixed launcher timeout. Acceptance requires exact ranks, collectives, runtime/fabric consensus, process-group destruction, and automatic cleanup; interrupted attempts require explicit run-UUID-scoped cleanup before retry.

No source-bound live 32-GPU NCCL promotion chain is checked into this repository today. The external exact-2M and parity summaries are not a substitute for the current-tree CUDA, NCCL, actor, and raw-evidence chain required for public reproduction. 5.4 implementation does not make the release runnable; the public status therefore remains review_only_not_runnable.

The release checker keeps the execution-source identity separate from current receipts to avoid a self-reference. After implementation and evidence are committed, evidence/manifest.json is generated in a binding-only child commit as described in CONTRIBUTING.md.

The public repository is updated by the private -alpha repository's Release to Public workflow, not by copying evidence or pushing directly to the public remote. See docs/public_release.md.

Raw prompts, unbounded model outputs, checkpoint paths, hostnames, internal network addresses, credentials, and unbounded traces are excluded. The exact-2M receipt retains only two bounded math completions, their token hashes and old log-probabilities, and content hashes for the complete external artifact. Approved rollout fixtures remain separately hash-bound release inputs rather than sanitized claims.