MinT-2M Validation Status

Status date: 2026-07-16

This table separates implementation state from validation state. A local unit test proves an API or algorithmic contract; it does not substitute for a 32-rank distributed comparison.

Gate	Implementation	Best evidence	State
Resident MLA/DSA prefix pages	present	2M CP32 capture traversed all 78 layers	execution receipt
Serial response-only GRPO	present	two backwards and one terminal optimizer invocation per worker at 2M; historical 32K/64K execution canaries	execution receipts; correctness pending
Global cross-CP DSA selection	present in current tree	32K CP32 selected-logprob max abs 9.84669e-5; rank consensus 0	forward-only passed
Backend gradient finalization	explicit finalize_model_grads call present	unit/contract coverage; no current-tree 2M rerun	implemented, distributed parity pending
Transport-neutral resident session	present	resident-adapter tests; full CPU suite is bounded evidence only	CPU bounded-witness contract passed; live rank evidence pending
Full gradient parity	diagnostic path present	best complete run: cosine 0.522370905, relative L2 0.924779	failed
Optimizer-delta parity	export/comparison surfaces present	blocked behind full gradient parity	not passed

Run readiness

Public release state is review_only_not_runnable. The repository contains a candidate repository-local Megatron rank port/backend, direct plan/session/ ownership surface, and resident replay adapter. Candidate execution is enabled (IMPLEMENTATION_COMPLETE=True), while execution/redistribution authorization, the locked runtime, approved inputs, and current-tree distributed evidence remain independent gates:

Stage	Checked-in entry point
Checkout/run prerequisite aggregation	python -m glm52 doctor
Side-effect-free 4x8 direct-Ray plan	python -m glm52 plan
Training gate	python -m glm52 train (enabled after run-scope doctor passes)
Interrupted owned-resource cleanup	python -m glm52 cleanup
Runtime lock and OCI plan	environment/runtime.lock.json + containers/cuda13/Containerfile
Ordered evidence contract	glm52/validation_contract.py

The external operator procedure is docs/runbook.md. Predecessor private-service tools remain only under legacy-* Make targets.

Direct-run evidence

After the external gates are accepted, the training deployment must point at a real absolute HF snapshot and its observed model_manifest_sha256. Its strict mint2m.training-fixture/v1 manifest binds model revision, model/policy/profile hashes, an approved distribution declaration, and an adjacent mint2m.training-payload/v1 file by path, SHA-256, and byte count. A validation deployment is separate and uses the legacy mint2m.fixture-set/v1 cases for the doctor/kernel/NCCL chain.

A passed direct run writes artifacts/rank-sessions.json and artifacts/checkpoint-manifest.json. The rank evidence records the exact bind -> initialize -> materialize -> capture-prefix -> grpo-0 -> grpo-1 -> finalize -> optimizer-step -> checkpoint sequence for all 32 ranks, rank/driver cleanup, and planned/observed Ray placement (planned_node_ip, planned_gpu_slot, observed_node_ip, observed_gpu_accelerator_id). The checkpoint manifest is re-hashed against the live regular-file tree by the receipt validator. These artifacts are acceptance evidence, not a substitute for the missing 32x H20 run.

The training receipt is explicitly scoped to 2m_execution_only_not_numerical_correctness. A passed execution receipt does not promote the 32K/64K parity or public validation gates.

What the 2M receipt proves

• A 2,097,152-token deterministic prompt was processed on 32 H20 GPUs with CP32.
• Prefix state was captured across 78 decoder layers.
• A group of two responses ran two live policy backwards.
• Gradients accumulated twice and the optimizer function was invoked once per worker.
• The run reported 32 rank summaries, 128 trace files, and no execution error.

It does not prove global attention correctness, synchronized gradients, or the numerical correctness of the optimizer update. It was produced by a predecessor source snapshot and is retained with those limitations in evidence/2m_execution_receipt.json.

Current correctness blocker

The latest bounded 32K parity attempt completed the conventional reference path but stalled in the shape-aligned MoE/MLP tail of response replay, around layers 75-77. It exported zero response gradient shards, so the full comparison could not run. The best earlier complete comparison also missed every acceptance threshold:

Metric	Required	Observed
selected log-prob max abs	<= 5e-3	0.58589
loss absolute difference	<= 5e-3	0.0410374
global gradient cosine	>= 0.999	0.522370905
global gradient relative L2	<= 1e-2	0.924779

The repository therefore must not be described as a correctness-complete 2M GRPO update. The defensible headline is: 2M execution receipt with a 32K distributed forward-parity canary; current-tree correctness pending.

Acceptance sequence

1. Complete 32K response backward on all 32 ranks and export every expected gradient shard.
2. Pass log-prob, loss, gradient, zero-mask, and optimizer-delta thresholds.
3. Repeat the parity gate at 64K.
4. Bind a fresh 2M execution to the current source manifest and inspect the same distributed invariants.

No 2M rerun should be treated as a replacement for steps 1-3.