GLM-5.2 Exact-2M Validation Update

Status date: 2026-07-24

This report records the completed external GLM-5.2 validation work and keeps it separate from public-release readiness. The exact-2M online rollout and update passed. The 32K conventional-versus-response-only gradient comparison is also passed. The source artifacts are content-hashed but are not bound to this public commit, so the checkout remains review_only_not_runnable.

Exact-2M online GRPO

The run used an actual rank-8 GLM-5.2 policy LoRA, a real DAPO math row, and dapo_math_reward.compute_score. It did not inject synthetic completions or synthetic rewards.

Field	Observed value
Prompt	2,097,152 tokens
Model context	native 1,048,576; opt-in experimental YaRN capacity 2,097,168
Hardware/topology	32 H20; training TP1/CP32/EP32/PP1; rollout TP8/PP4
Rollout engine	Tinker multi-node policy sampler; actual policy LoRA; two exact-2M requests
Model/update	78 layers, one rank-8 policy LoRA, G=2
DAPO example	row 4457, ground truth 4
Bounded search	2 of at most 8 candidate rows sampled
Completions	Answer: 2, Answer: 4
Rewards	[-1, +1], independently recomputed by the DAPO rule reward
Training transaction	2 backwards, accumulation G=2, one optimizer step
Distributed closure	all 32 ranks completed; global CP DSA and Megatron gradient finalization observed
Trace inventory	64 rank traces, 25,536 events
Numerics	no NaN, Inf, or execution error
Cleanup	actors and training service stopped; workers then head stopped; GPU/process/port recheck passed

The policy checkpoint inventory matched before and after sampling. The executed prompt, completion tokens, old log-probabilities, sampling parameters, ground truth, reward details, rank results, traces, and cleanup records are retained in the external artifact. The public receipt includes only bounded outputs and content hashes.

This is an actual 2,097,152-token online rollout followed by an optimizer update. It is not a 32K or 64K canary. The first exact-2M candidate required 1,867.58 seconds; a second candidate on the cached prefix required 30.798 seconds.

The hash-bound raw sampler JSONL contains a 64K engine probe followed by two exact-2M requests with eight samples each. Across the 16 exact-2M outputs, 10 stopped normally and 6 reached the configured 16-token limit. A fresh decode audit found zero Unicode replacement characters, one obvious numeric/escape degeneration, and one bounded target-marker echo. The remaining outputs were readable answers or length-capped reasoning fragments. This demonstrates that crossing 1M does not deterministically corrupt policy decoding, but it is not a long-context quality benchmark.

Position adaptation

The released GLM-5.2 checkpoint remains a native 1M checkpoint. Exact-2M execution used an explicit, reversible YaRN configuration:

{
  "max_position_embeddings": 2097168,
  "rope_parameters": {
    "rope_type": "yarn",
    "rope_theta": 8000000.0,
    "factor": 2.0000152587890625,
    "original_max_position_embeddings": 1048576,
    "beta_fast": 32.0,
    "beta_slow": 1.0,
    "mscale": 1.0,
    "mscale_all_dim": 1.0,
    "truncate": true
  }
}

The same payload is checked in as configs/glm52_2m_yarn.json.

glm52/glm52_long_position.py parses and validates this configuration and propagates it to the policy-sampler and Megatron configuration boundaries only when GLM52_ROPE_SCALING_JSON is set. Native checkpoint defaults are unchanged. This adaptation establishes capacity, not pretrained 2M answer quality. Low base-model quality at the new positions is a reason to train, not a blocker to the training method. Experimental denotes a reversible non-native checkpoint override, not an untested execution path: both Tinker policy sampling and Megatron training completed at exact 2M under the same payload.

Training-readiness decision

The project decision is ready_for_2m_training. Retain the validated YaRN factor-2 configuration for initial training so policy sampling and Megatron replay continue to use identical position semantics. LongRoPE-style per-dimension scaling may be evaluated later if trained checkpoints show a reproducible position-dependent collapse; changing position encoding before training would reopen backend-consistency and gradient-parity work without closing a current blocker.

Gradient parity

Both comparison paths started from the same model and LoRA state, used the same rollout fixture and fixed seed, prohibited optimizer steps, ran two backwards, and exported every trainable LoRA parameter shard before zeroing gradients. The comparison covers a nominal ratio-1 case and an active-clipping case with positive and negative advantages and one lower plus one upper clip.

Case	Log-prob max abs	Loss abs	Global cosine	Relative L2	Decision
ratio1	2.38417e-7	0	0.999933603	0.0117068	passed
active clip	2.38417e-7	0	0.999933588	0.0116293	passed

Each case exported 44,608 conventional shards and 44,608 response-only shards. Parameter sets and parameter hashes match, with no missing/extra shards and no non-finite values. Exact zero/nonzero masks were compared rather than ignored: the ratio-1 and active-clip cases recorded 435,870 and 434,559 element-level mask differences respectively. These are not missing gradients, and the complete per-parameter inventory remains in the external artifacts.

The target-family aggregate is similarly aligned:

Case/family	Cosine	Relative L2
ratio1 attention	0.999938678	0.0112643
ratio1 expert MLP	0.999892533	0.0148143
ratio1 shared MLP	0.999780050	0.0225412
ratio1 unembed	1.000000000	3.74586e-7
active attention	0.999938513	0.0111990
active expert MLP	0.999893604	0.0146673
active shared MLP	0.999804103	0.0198633
active unembed	1.000000000	3.68548e-7

The largest relative per-parameter outliers at the end of the expert stack have negligible absolute scale; representative attention outliers remain below 8.4e-5 max absolute difference. The observed 1.16-1.17% global relative-L2 results meet the project acceptance contract, so both cases pass. A 64K repeat and optimizer-delta parity were not run.

Evidence inventory

The sanitized receipts are:

• evidence/online_grpo_2m.json
• evidence/gradient_parity_32k.json

The exact-2M external manifest covers 146 files and was rebuilt and verified against every listed SHA-256. Its top-level manifest SHA-256 is 3c45736cb21a7a402f033df09487480e73d5eb8b192c89e0378b625af98e0852. The two parity receipts record the independent manifest/comparison hashes for the ratio-1 and active-clip artifacts.

These results close the practical method-level questions addressed by this validation: real exact-2M policy sampling can feed a full response-only G=2 update, and its gradients closely track conventional full-sequence gradients. They do not distribute the model, dataset, immutable runtime, authorization, or raw evidence, and do not make a fresh public clone runnable.