Model parity matrix
August 24, 2026 · View on GitHub
Status: canonical exhaustive model-family table for parallel claims.
Pinned oracle source: ${VLLM_SOURCE} at
5559679229bc961848b121ccdeaa8fa5d79bec98 (vLLM 0.26.0.dev0 + transformers
5.14.1; advanced 2026-07-26 from the prior e24d1b24 / 0.25.0 pin, see
specs/pin-advance.md).
Prior audited sync target: v0.25.0 702f4814fe54; the three explicitly marked
target-pending rows below were inventoried against it (the pin has since advanced past 0.25.0).
Inventory spike: specs/model-family-inventory.md.
This document, rather than its spike, owns the exhaustive static model-registry
inventory at the pin. The summary and the per-target claim rows intentionally
live together so an agent never has to infer ownership from a scoping document.
It groups architecture aliases only when they resolve to the exact same
upstream (category, module, class) target. Consequently each table row is
a practical unit that one agent can spike without silently dropping aliases.
Architecture-support checklist
At-a-glance view of which architectures we have actually engaged, and how far.
355 architecture rows are inventoried at the pin, plus 18 rows that the pinned
registry does not contain = 373 architecture rows. Those 18 are, by why they
are not at the pin: KimiK3ForConditionalGeneration and
MuseGlimmerForConditionalGeneration, both released after the pin;
Qwen3_5ForCausalLM and Qwen3_5MoeForCausalLM, the text-only Qwen3.5 arms
upstream added after the pin in PR vllm#50210 (#490);
Dots3NoteForCausalLM and its speculative head Dots3NoteMTPModel, added to
vLLM main after the pin by
vllm#51255 and still being
patched there (#699);
BailingMoeV3ForCausalLM, a pin-lag row registered
on vLLM main and absent only at the pin (#609); MiniMaxH3DiTModel and
LTX2VideoTransformer3DModel, both DIFFUSION architectures released after the
pin and absent from the vLLM repository entirely, reached through the
separate vllm-omni repository (#435); MiniMaxMusic3ForConditionalGeneration,
absent from the pin, from vLLM main and from vllm-omni alike, whose only
reference implementations are an open diffusers PR and SGLang-Omni (#672); the
six TTS / audio-generation architectures
registered in that same vllm-omni repository — MossTTSDelayModel,
MossTTSRealtime, Qwen3TTSForConditionalGeneration,
HiggsMultimodalQwen3ForConditionalGeneration, and IndexTTS-2.5's two halves
IndexTTS2TalkerForConditionalGeneration and IndexTTS2S2MelDecoder (#634); and two whose upstream target is
still pending because the exact config.json architecture string is registered
in neither core vLLM main nor vllm-omni —
VoxtralRealtimeForConditionalGeneration and
BailingMMNativeForConditionalGeneration (#610).
The rollup below counts 377 rows, which is those 373 plus the four that are
not architectures at all and say so in their own sections:
MODEL-FACTORY-registry (the cross-cutting registry contract), the two
MODEL-AUDIO encoder-component rows, and the one MODEL-HFDYNAMIC row (dynamic
Transformers compatibility is capability-driven and excluded from finite counts).
Of those 377, 53 are past INVENTORIED (engaged) and the remaining 324 are the
known-but-not-started long tail — the same two numbers the rollup table states,
and the way to re-derive every count here is to re-run
scripts/check-agent-record.py's parser over
this file rather than to carry any of them forward. Every mark
below is grounded in the row's lifecycle State cell plus its ledger evidence,
and this section is CI-enforced against those rows by
scripts/check-model-checklist.py: a mark
can never claim more than the row backs, no engaged architecture can be omitted,
and the rollup counts cannot drift. A change that advances (or regresses) a
model's state updates its entry AND the rollup in the SAME commit.
Legend: ✅ correctness-complete + gated (a passing SACRED token-exact gate on
record; DONE/ACTIVE/GATING/PARTIAL rows) · 🚧 active / in progress
(implemented or scoped, gate or speed still pending) · 📋 spiked (scoped in a
spec, not implemented) · 🚫 blocked (hardware- or dependency-blocked, no
runnable gate) · ⬜ inventoried (known, not started — shown only as the rollup
count, never one line each). ⬜ never marks a single row; the ~300 inventoried
rows are the rollup tail.
Rollup by lifecycle state (must equal the detailed per-state row counts):
| State | Rows |
|---|---|
| INVENTORIED | 324 |
| PARTIAL | 22 |
| ACTIVE | 10 |
| SPIKE | 9 |
| BLOCKED | 5 |
| DONE | 3 |
| READY | 3 |
| GATING | 1 |
| Total | 377 |
Engaged architectures (the 53 non-INVENTORIED rows):
| Support | Architecture | Family / example | Status | Row |
|---|---|---|---|---|
| ✅ | Qwen3ForCausalLM | Qwen3 dense (0.6B/1.7B/4B/32B) | near-tie-robust token-exact 16/16 on 0.6B+4B vs vLLM 0.25.0; NVFP4A16 (W4A16) dense quant also gated; c1 every-axis speed parity, c8 decode residual; async-serving device token-ids mirror ported (ROW-SERVE-ASYNC-DENSE-MIRROR, #31 fix into the shared dense EmbedInto) — test_qwen3_dense_async_serving RED→GREEN; sibling scope CLOSED (#323): 60e71a0e fixed the eager path; DenseDecodeGraphForward ran first and replayed against stale HOST ids, so it now declines while the mirror is live and falls back to the proven eager path. Async gate 7/7 across Qwen3-0.6B/4B + Llama/Mistral/InternLM2 | MODEL-TEXT-qwen3-qwen3-for-causal-lm |
| ✅ | Qwen3MoeForCausalLM | Qwen3-Coder-30B-A3B (MoE) | STRICT token-exact 6/6 vs vLLM 0.25.0; 11/16 speed-grid cells at/above graphed vLLM, c1/c2 residual | MODEL-TEXT-qwen3-moe-qwen3-moe-for-causal-lm |
| ✅ | Qwen3_5ForConditionalGeneration | Qwen3.6-27B (text path) | text-gen STRICT token-exact 235/235 vs vLLM 0.25.0; mm INPUT pipeline (M0/M1) landed + processor-parity gate PASS; M3-W0 landed (vision-inclusive checkpoint Qwen/Qwen3.6-27B 51.7 GiB bf16 with 333 visual.* FOUND+fits+downloaded; 27B vision config resolved — depth 27/out 5120/EMPTY deepstack; MRoPE [11,11,10]/rot 64/theta 1e7; the bf16 GDN-hybrid loader ALREADY handles it). M3-b LANDED 2026-07-25: image→text STRICT token-exact 32/32 vs vLLM 0.25.0 — Qwen3.6-27B image understanding works end-to-end (forked GDN-hybrid VL forward gated on mm input ⇒ text byte-identical; 27B/35B/Coder inertness re-passed 235/315/138). M3d LANDED 2026-07-25: video→text STRICT token-exact 32/32 vs vLLM 0.25.0 — video works end-to-end too (Qwen3_5VLGenerateGreedyVideo reuses the M3c processor/windowed-tower/video-MRoPE on the GDN-hybrid backbone). Qwen video modalities COMPLETE: image+video both work e2e (audio N/A for Qwen). VISION-FORWARD SPEED (2026-07-28, CLAIM-MM-SPEED-QWEN-IMAGE, multimodal-speed.md §16): the mm-forward tower BEATS vLLM — per-image tower forward 142.3 ms (flash AttentionDenseFlash, hd-72) vs vLLM 0.25.0 ~250 ms eager encode = 0.57×; attribution-first nsys REFUTED a bigger lever (the t=784 vision attention is serial-latency-bound, flash only 1.04× over warp), STRICT 32/32 image/video HELD + goldens md5 unchanged. Row stays PARTIAL — vision-forward speed BEATS vLLM; umbrella speed pending on batched c2+/serving. SECOND CHECKPOINT TOKEN-GATED 2026-08-15 (Qwen/Qwen3.8-27B@1d4bf0f2, bf16, #915, spec): 4/7 prompts STRICT 16/16 vs the pinned oracle 555967922, and all THREE first-divergence positions are EXACT fp32 TIES — oracle-minus-ours and top-2 gap both 0.000 mnats, our token at rank 3 / 2 / 2 in the oracle top-20, so ALL_TIES_OR_IN_BAND against kNearTieMnats = 500. Every one is the #910 tie-break signature and nothing else: vLLM's pick carries the LOWER token id (1814/11/16309) and ours the HIGHER (22960/13/27180) at a bit-identical logprob. Only the first divergence per prompt is adjudicable, so this is three numbers; a raw position count over the grid is NOT a quality score and is not recorded as one. Adjudicated twice on the pinned oracle's fp32 logprobs — a greedy re-decode and an independent TEACHER-FORCED probe that asserts the echoed prefix — because the earlier transformers bf16 CPU probe could not resolve below one bf16 ULP (every runner-up gap it printed was a multiple of 0.125) and so could not have reported anything but a tie. SPEED on the same checkpoint, vs vLLM's PRODUCTION graphed config at the pin, clocks 2184 MHz: 1 of 3 concurrency cells established. c4 is the only cell where both arms completed every request — 0.963x output throughput, 1.008x median ITL. c1 and c8 throughput WERE withheld on 2026-08-15 (superseded, below): our server failed 1/6 in all three reps and 12/11/12 of 48 where vLLM failed none in nine legs (#931), and output_throughput divides tokens by a duration still containing the dead request, so c1 read 0.677x while median TPOT in the SAME file read 1.014x in our favour. SUPERSEDED 2026-08-19 by the c1/c8 RE-MEASURE (#915, #979, .agents/benchmark-record.md BENCH-QWEN38-27B-BF16 c1/c8 RE-MEASURE): #931 landed, and with VT_SERVER_SSE_PING_S=0 our arm completed 162 of 162 requests, failed=0 on every leg — c1 4.4040 tok/s (CV 0.039%), c8 22.6402 tok/s (CV 0.205%). Our half of the withholding is discharged; NEITHER cell became a ratio and the two halves are blocked differently. At c1 vLLM also completed everything (4.2835 tok/s, CV 0.033%) and gpu_clock_state compare returned PAIRING_VERDICT=DISCARD on all three pairings — the cross-arm rule PASSED (same boot, both arms 2489 MHz median, 0.0% offset) and the WITHIN-RUN rule failed on both against the 5% ceiling (#1354: clocks cannot be pinned inside an rc lease), so the c1 ratio is OWED, not withheld for being unflattering. At c8 the vLLM denominator is NOT MEASURABLE on this box at the recorded configuration — that is the answer, not a gap, and not a claim that vLLM is defective. Read the two output-throughput absolutes with #1355: our usage.prompt_tokens reports 5,942 where vLLM reports 6,144 on identical prompts, which corrupts total-token throughput outright and biases output throughput up by more than its own CV. Cold start 53 s vs 780 s = 14.7x; host memory after warmup 42.5 vs 110.1 GiB = 2.59x, caveated because vLLM's is set by --gpu-memory-utilization 0.85 pre-reserving KV | MODEL-MM-qwen3-5-qwen3-5-for-conditional-generation |
| ✅ | Qwen3_5MoeForConditionalGeneration | Qwen3.6-35B-A3B (NVFP4 text path + PUBLISHED BF16 text path; vision implemented, NOT gated) | text-gen STRICT token-exact 315/315 vs vLLM 0.25.0 on the NVFP4 requant; the PUBLISHED bf16 repo now generates too and is oracle-gated (2026-08-15, #740 + #864): greedy 7 prompts x 3 repeats x 16 tokens @995ad96e vs the pinned oracle, 6/7 prompts STRICT 16/16, the seventh one exact logit tie (top2_gap_mnats = 0.0) our on-device argmax breaks toward the higher id where torch.argmax takes the lower (#910) — PASS under the ratified near-tie doctrine; only the FIRST divergence per prompt is adjudicable, so the raw 108/112 position count is NOT a quality score. SACRED inertness 3/3 with real counts, goldens byte-identical (27B 235/235, 35B 315/315, Coder 138/138, 688 assertions). NO throughput, latency or memory number exists for this checkpoint; mm INPUT pipeline (M0/M1) landed + processor-parity gate PASS; vision tower LOADED and the forked VL forward LANDED (#891, M2/M3), but the binding image/video token-exact gates vs the pinned oracle are OWED, so the row stays PARTIAL | MODEL-MM-qwen3-5-qwen3-5-moe-for-conditional-generation |
| 🚧 | Qwen3_5ForCausalLM | Qwen3.5 text-only dense arm (no checkpoint fits this box) | REGISTERED, NOT RUN-GATED (2026-08-12, issue #490). Ahead-of-pin forward port of upstream PR vllm#50210 (ad5d29db7): additive registration against the EXISTING dense factory + one backbone weight-namespace decision per checkpoint (model. vs model.language_model., mixed index REFUSED). Dispatch, flat-config and namespace tests green; 27B/35B/Coder inert (VL prefix is still the seam default, goldens md5 unchanged). NO token-exact oracle run and NO speed number — the run gate is OWED, and no text-only Qwen3.5 checkpoint fits GB10. Unlike the MoE arm, this loader DOES route BF16/FP8/NVFP4 per projection by tensor presence, so a flat bf16 checkpoint is expected to load | MODEL-TEXT-qwen3-5-qwen3-5-for-causal-lm |
| 🚧 | Qwen3_5MoeForCausalLM | Qwen/Qwen3.8-2.4T-A95B (2.4T MoE — DOES NOT FIT this hardware) | REGISTERED, NOT RUN-GATED (2026-08-12, issue #490). Same ahead-of-pin port against the EXISTING MoE factory: 3.8 is the token-exact Qwen3.6-35B-A3B GDN-hybrid MoE backbone at larger scale, every knob config-driven, with the BACKBONE weight names identical modulo the backbone prefix. CORRECTED 2026-08-12 — REGISTRATION IS NOT LOADABILITY. This loader reads ONLY per-expert NVFP4 routed experts; the published Qwen/Qwen3.8-2.4T-A95B and Qwen/Qwen3.6-35B-A3B indices ship 3-D STACKED experts and ZERO weight_scale/input_scale (read live 2026-08-12), and our gated 35B row reads the REQUANTIZED nvidia/Qwen3.6-35B-A3B-NVFP4 — so at that date the bf16/stacked MoE routed-expert arm was NOT implemented and such a checkpoint was REFUSED by a message naming it. SUPERSEDED 2026-08-15 — THE LOADER HALF IS CLOSED, WITH A TOKEN. #740 landed the 3-D stacked bf16 routed-expert reader and #864 the bf16 GDN tower, attention tower, shared expert and lm_head, all selected by tensor presence; both published indices now satisfy the load plan completely, and the binding token-exact greedy gate RAN on Qwen/Qwen3.6-35B-A3B bf16 @995ad96e vs the pinned oracle at 6/7 prompts STRICT 16/16 (the seventh an exact logit tie, #910), with SACRED 27B/35B/Coder inertness 3/3 and goldens byte-identical. THE RUN GATE FOR THIS CLASS IS STILL OWED, and the reason is now purely hardware: that gate ran through Qwen3_5MoeForConditionalGeneration, not through Qwen3_5MoeForCausalLM, whose only published checkpoint is 2.4T bf16 ≈ 4.8 TB / FP8 ≈ 2.4 TB against 128 GB unified. NO token and NO speed number exist for this class; its load plan resolves against the published index, which is name/shape/dtype resolution and not a token. Also owed: MTP and GGUF arms for 3.8 | MODEL-TEXT-qwen3-5-qwen3-5-moe-for-causal-lm |
| 🚧 | Qwen3VLForConditionalGeneration | Qwen3-VL-4B-Instruct (image + video vehicle) | IMAGE e2e WORKING (M2 CLOSED): STRICT image→text token-exact 32/32 vs vLLM 0.25.0 (full pipeline: C++ processor→M2a tower→merge→forked MRoPE/DeepStack greedy decode); correctness complete, speed pending. Underpinned by mm processor (M1), vision tower (M2a faithful), M2b/M2c numeric contracts (85/85). VIDEO (M3c 2026-07-25): preprocessing + full wiring LANDED + unit-gated — video-processor pixel_values_videos BIT-exact 41/41 (RED-first), video MRoPE positions BIT-exact vs vLLM, per-frame windowed tower faithful (rel-L2 0.072); video e2e NEAR-TIE-ROBUST PASS (gate form chosen BY MEASUREMENT 2026-07-25): teacher-forcing vLLM 0.25.0 on OUR exact tokens proves the sole divergence is ONE genuine bf16 near-tie at tok22 (gap 0.125 nats, our token vLLM's 2nd of 4 tied choices) with every downstream token == vLLM's teacher-forced argmax at gap 0.0 — the tower already accumulates in f32 everywhere (matches vLLM), so the residual is the irreducible bf16 envelope, NOT a fixable numeric gap. VIDEO understanding now WORKS e2e (correctness complete, speed pending); image e2e STRICT 32/32 unchanged (the deterministic strict-pass proof, no regression). VISION-FORWARD SPEED (2026-07-28, CLAIM-MM-SPEED-QWEN-IMAGE, multimodal-speed.md §16): the shared vision tower now defaults to the flash-tiled AttentionDenseFlash (byte-identical to warp; STRICT image 32/32 HELD, nsys default 4B e2e = 24 flash-kernel inst/zero warp, goldens md5 unchanged); the tower already BEATS vLLM (~250 ms eager encode) — the profile REFUTED a bigger lever. OpenAI SERVING (2026-07-28, CLAIM-MM-SERVING-W1, mm-serving.md): first CPU brick landed — the chat request now parses OpenAI multimodal content parts (image_url/input_audio/audio_url) + decodes base64/data-URI + routes to the EXISTING processor → MultiModalInputs (gate test_chat_mm 5/5, CPU no-weights). This model is the named MM-SERVE-E2E closing gate: a real image+prompt OpenAI /v1/chat/completions request → token-correct output vs the mm oracle (needs the engine mm plumbing MM-SERVE-ENGINE + DGX). Not yet wired into the OpenAI server end-to-end. SERVING W2 (2026-07-28, CLAIM-MM-SERVING-W2): MM-SERVE-ENGINE DONE — the engine now CARRIES the parsed MultiModalInputs: additive LLMEngine/AsyncLLM add_request(MultiModalInputs)+generate overloads via InputProcessor::process_inputs_mm (mm_features onto EngineCoreRequest/Request), chat-template placeholder-STRING helpers, serving_chat MultiModalChatFn seam (default unset ⇒ text byte-identical); CPU gate test_input_processor +2 + test_chat_mm +2 (full chain parse→route→engine request with the 196-slot expanded prompt + mm handles). SERVING W3 (2026-07-28, CLAIM-MM-SERVING-E2E): the MM-SERVE-E2E CPU SEAM BODY DONE — MakeQwen3VLImageChatFn (chat_mm.{h,cpp}) turns an image chat request into the placeholder-EXPANDED engine input (marker-inject → chat template → EncodeWithSpecialTokens single image_pad marker→one image_token_id → RouteImageRgb EXPAND to 196 image tokens + mm_features), wired in examples/server/main.cpp (guarded on preprocessor_config.json; text path byte-identical). Gate test_chat_mm 8/8 (RED text path = 0 image tokens) + test_openai_serving (production seam invoked + routed). The GPU e2e forward is ARCHITECTURALLY BLOCKED (not box contention): the engine model runner has no mm forward — ModelForwardInput has no vision field, runner.cpp ignores Request.mm_features, this arch is UNREGISTERED (REGISTER_VLLM_MODEL absent), and the M2c Qwen3VLGenerateGreedy is a standalone driver. Fold-into-engine recipe (add a vision-embed field → runner runs the tower via the encoder cache + merge → register the arch → dgx token-exact vs the M2c golden) in specs/mm-serving.md. ENGINE MM-FORWARD LANDED (2026-07-28, CLAIM-ENGINE-MM-FORWARD): the block is RESOLVED — ModelForwardInput gains an additive default-nullopt mm field (merged inputs_embeds + 3-D MRoPE + DeepStack), Qwen3-VL is now REGISTER_VLLM_MODEL-registered (qwen3_vl_registry.cpp, supports_multimodal=true), and the registered forward FOLDS the M2c decode into ModelRegistry::Forward via the SHARED Qwen3VLForwardStepLastLogits (Qwen3VLGenerateGreedyViaRegistry drives every step through ModelRegistry::Forward). Text inertness (RED line, shared-path change): test_runner 16/16 + test_scheduler 36/36 + test_model_registry 24/24 + test_chat_mm 8/8 + test_openai_serving 41/41 all green (nullopt-for-text ⇒ byte-identical). Engine mm-forward token-exact gate test_qwen3vl_registry_e2e (image→text THROUGH ModelRegistry::Forward == M2c golden STRICT, dgx.casa GB10). Residual: the in-runner scheduler-fed tower run + the full server /v1/chat/completions GPU e2e. | MODEL-MM-qwen3-vl-qwen3-vlfor-conditional-generation |
| 🚧 | VoxtralForConditionalGeneration | Voxtral-Mini-3B-2507 (AUDIO→text, audio-track A3) | FIRST e2e AUDIO UNDERSTANDING: audio→text vs vLLM 0.25.0 — A2 Whisper-large-v3 encoder at Voxtral config (128 mel/1280/32L/head_dim 64) + AudioLanguageAdapter projector + masked-scatter merge into the LANDED Mistral/Llama decoder (untied lm_head, mistral-consolidated q/k rope-permute) → forked greedy. Gate form BY MEASUREMENT: vLLM greedy K=5 deterministic ⇒ STRICT is the bar; STRICT prefix 33/48 exact vs vLLM greedy, then the ratified near-tie-robust gate PASSES (worst teacher-forced gap 0.0 nats, sole branch = a 4-way bf16 tie at -2.069 nats) — bit-exact infeasible (encoder uses different bf16 GEMM/attn kernels). Decoder proven token-exact (ref-audio→48/48). Additive driver/loader gated on audio ⇒ Mistral text byte-identical. Speed pending (decode now graph-captured via VoxtralDecodeGraph, lever #3 W1 2026-07-27: small real win 60.94 vs 61.71 ms/tok, still ~1.49× vs vLLM 40.8 ms). Decode-kernel efficiency ATTRIBUTED + VALIDATED ceiling (2026-07-27, CLAIM-MM-SPEED-DECODE-KERN, multimodal-speed.md §11): the whole ~20 ms/tok residual is the naive scalar PagedAttentionKernel decode attention (723 µs × 30 layers = 21.7 ms/step, ~120× the KV floor); the 1:1 vLLM lever (FA2 flash_attn_varlen decode) is already in-binary, gated off only because the driver's single KV block (444) isn't ÷16. block_size÷16 → decode via FA2: TPOT 59.4→38.2 ms/tok (−21.2, ~36%) = 0.94× vLLM 40.8 ms — BEATS parity, and the FA2 sequence is a VALID vLLM greedy branch (teacher-force PASS, 0 divergences, gap 0.0). But it flips the committed near-tie golden's exact-tie branch (repro 48→18) → blocked byte-exact (RED line). bf16 near-tie / golden-pinning ceiling; RECORDS-ONLY (14/14 held, golden unchanged); win reachable via block_size÷16 + golden regen. ADOPTED 2026-07-27 (USER-APPROVED, CLAIM-MM-SPEED-DECODE-KERN-ADOPT, multimodal-speed.md §12): FA2 decode SHIPS as the Voxtral default — audio DECODE now BEATS vLLM (0.97×), the LAST mm decode-speed gap CLOSED. One-line block_size÷16 routes decode via FA2 LaunchDecodeVarlenFA2Bf16 (nsys: flash_fwd_splitkv 1410 @ 18.5 µs, ZERO PagedAttentionKernel). test_voxtral_e2e converted to the ratified near-tie DISTRIBUTIONAL gate (binding = teacher-force PASS, KERNEL-INDEPENDENT; strict prefix exact to the first bf16 tie — FA2 takes the other side of the pos-18 2-way EXACT tie ⇒ prefix 18, >=18; determinism anchor to the FA2 seq). voxtral_neartie.json md5 3d199c2d…→937b9ad3…; STRICT golden voxtral_golden.json 8ab87b7e… UNCHANGED. Gate PASS 16/16; teacher-force vLLM 0.25.0 on the FA2 seq = 0 divergent, worst gap 0.0000 nats, PASS. Capture-safe (graph S=1 + 46 replays valid; compute-sanitizer 0 errors on the graphed-FA2-decode path; 3 runs byte-identical) ⇒ ships as the DEFAULT graph path. A/B (same-binary, 6 reps rep0 dropped, steady-state): scalar 60.50 → FA2 39.50 ms/tok (−21.0, ~35%, NON-OVERLAPPING) = 0.97× vLLM 40.8 ms — BEATS. Audio DECODE now correctness- AND speed-DONE; row stays PARTIAL/ACTIVE — umbrella DONE bar has audio TTFT (32-layer Whisper encoder, UNMEASURED vs vLLM 43 ms) + c2+ batched / audio_url serving ingestion still open (same as image/video). ENCODER TTFT MEASURED + warp-attention brick 2026-07-27 (CLAIM-MM-SPEED-AUDIO-ENC, multimodal-speed.md §13): the Whisper encoder ran the naive kAttention (O(t²) per-key block-__syncthreads) — routed the encoder self-attention (hd-64, non-causal) to the warp-scoped vt::AttentionDenseFast (§7's vision-tower fix; kAttention untouched ⇒ text byte-identical). Encoder forward 8870→1890 ms (4.7×, NON-OVERLAPPING); test_voxtral_e2e 16/16 with ZERO token flips (naive arm also 16/16 same tokens; goldens md5 UNCHANGED), proof-of-run nsys AttentionWarpKernel 32 inst / zero naive. NOT at parity: ~1.89 s vs vLLM's 43 ms (~44×) — the warp kernel is STILL 31.8 ms/layer (O(t²), memory-bound on redundant K/V reads); closing needs a flash-TILED non-causal hd-64 attention (LARGE) + resident one-time encoder weights (MEDIUM, byte-exact). ENCODER FLASH-TILED KERNEL LANDED 2026-07-28 (CLAIM-MM-SPEED-AUDIO-ENC-KERNEL, multimodal-speed.md §14): §13 lever #1 done — new vt::AttentionDenseFlash (OpId::kAttentionDenseFlash, cuda_ops.cu) shared-memory-TILES K/V across a block of kFlashBr=16 query-warps (FA2 K/V tiling, structure-ported from vendored flash_fwd_kernel.h:52), per-warp math copied verbatim from AttentionWarpKernel ⇒ BIT-IDENTICAL; encoder routed to it by default (VT_WHISPER_ENC_WARP=1/EAGER=1 A/B); kAttention/kAttentionDenseFast untouched ⇒ text/vision byte-identical. 16/16 default-flash, flash/warp/eager token md5-IDENTICAL (89923566…) ⇒ ZERO flips, goldens md5 UNCHANGED, nsys proof AttentionDenseFlashKernel 32 inst / zero warp+naive, RED confirmed, compute-sanitizer 0, 3 runs identical. A/B same-binary: attention 35.11→19.29 ms/layer (1.82×), encoder forward ~1834→~1375 ms (1.33×). NOT at parity: ~1.37 s vs vLLM ~43 ms (~32×, was ~44×) — the scalar warp-per-query recurrence is now serial-latency-bound (L2 already served the redundant reads ⇒ 1.8× not 16×); gap-closer = tensor-core MMA hd-64 non-causal FA2 (LARGE) + resident encoder weights (MEDIUM). ENC lever #2 LANDED 2026-07-28 (CLAIM-MM-SPEED-AUDIO-ENC-RESIDENT, multimodal-speed.md §15): device-resident one-time encoder weights (mirror the Qwen d_dev residency seam; each of the 487 encoder weight tensors f32→bf16-converted + H2D-uploaded ONCE into a mutable shared_ptr<void> handle, reused across forwards). BYTE-EXACT (moves data, not math): test_voxtral_e2e 16/16, goldens md5 UNCHANGED (8ab87b7e…/937b9ad3…), compute-sanitizer 0 errors. Proof-of-run/RED (VT_WHISPER_ENC_REMARSHAL): nsys HtoD resident 740 ops/9.4 GB vs re-marshal 1714 ops/11.9 GB (−974 ops, −2.5 GB). A/B same-binary (flock, 6 reps rep0 dropped): encoder forward ~1377→~729 ms (−648 ms, 1.89×) — removes the confirmed ~0.75 s per-call host marshalling. Trajectory 1834→1375→729 ms; vs vLLM ~43 ms now ~17× (was ~32×). NOT at parity — residual 729 ms is GPU-compute-bound (scalar warp attention); LARGE gap-closer remains tensor-core MMA hd-64 FA2. Audio TTFT/encoder stays speed-pending | MODEL-MM-voxtral-voxtral-for-conditional-generation ENCODER TTFT lever #1 MEASURED 2026-08-12 (CLAIM-MM-SPEED-AUDIO-ENC-FA2, S17, #432): the vendored FA-2 forward on tensor cores (new dense hd-64 non-split instantiation + vt::AttentionDenseFa2) takes the encoder attention 19,278 -> 166.5 us/layer (115.8x) and the encoder forward 731.7 -> 133.0 ms (5.50x). Restated against the PIN (555967922, TTFT median 46.02 ms, production/graphed, re-measured 2026-08-12): the ENCODER FORWARD goes from 15.90x to 2.89x of vLLM's whole TTFT. That is NOT a TTFT ratio - our projector, merge and prefill are unmeasured and absent from the numerator (review finding F3); the earlier ~17.1x -> ~3.11x used the carried-forward 0.25.0 denominator of 42.8 ms, so the published figure was CONSERVATIVE. It LANDS OPT-IN (VT_WHISPER_ENC_FA2=1), NOT default: the oracle teacher-force PASSES the ratified band (worst 0.125 nats, over-band 0) but with 3 divergent positions where the shipping kernel has 0 at gap 0.0. WHY is a HYPOTHESIS: the stated cause (bf16 P before the PV MMA) was REFUTED by mutation M4 - the same rounding forced into the scalar kernel left the tokens unchanged. Default path + goldens UNCHANGED, test_voxtral_e2e 16/16; the op now has unit tests (tests/vt/test_ops_attention_dense_fa2.cpp). ADOPTION = developer decision; audio TTFT stays speed-pending. |
| 🚧 | ParakeetForCTC (encoder + CTC head) | Parakeet / FastConformer ASR: an audio COMPONENT, not a registry arch (vLLM wraps transformers' ParakeetEncoder inside nano_nemotron_vl.py) | P4 LANDED 2026-08-07 on CPU: encoder + CTC greedy + log-mel front end, gated stage-by-stage against a dumped HuggingFace ParakeetForCTC oracle (rel-L2 <= 3.5e-7) with greedy and collapsed token ids EXACT, plus an independent in-test reference and 20/20 mutants killed. Real transcript verified 2026-08-07 on nvidia/parakeet-ctc-0.6b AND -ctc-1.1b; no CUDA provider, no GPU suite, no aarch64, no speed number. ONE-SURFACE ROW 1 (2026-08-07): NOW a registered arch — ParakeetForCTC self-registers (BEYOND-PIN: pinned vLLM has Parakeet only as the NemotronH audio component, registry.py:511-513) with the SupportsTranscription-only ModelInfo; text paths refuse by task; served via vllm_transcribe (ABI v11) + /v1/audio/transcriptions; example = thin vllm.h client, transcript byte-identical to pre-fold | MODEL-AUDIO-PARAKEET-ENCODER |
| 🚧 | ParakeetForRNNT / ParakeetForTDT (transducer heads) | Parakeet RNN-T and TDT ASR: the decode half of the same family, likewise an audio COMPONENT and not a registry arch (vLLM has no transducer call site at all) | P6 LANDED 2026-08-07 on CPU: LSTM prediction network, joint network, TDT duration head and the greedy transducer loop, gated against dumped HuggingFace ParakeetForRNNT / ParakeetForTDT oracles from transformers main with the emitted sequence and per-step durations EXACT, plus an independent in-test LSTM reference. Real transcripts verified on nvidia/parakeet-rnnt-0.6b and -tdt-0.6b-v3, token ids EXACT vs HF generate() end to end. Corrects the P4 record, which called the transducer unmirrored off the locally installed transformers 5.3.0. No CUDA, no GPU suite, no aarch64, no speed number; .nemo-only checkpoints out of reach. ONE-SURFACE ROW 1 (2026-08-07): ParakeetForRNNT/ParakeetForTDT registered (BEYOND-PIN, transcription-only refuse-by-task) and served through the same vllm_transcribe / /v1/audio/transcriptions / thin-client surface as the CTC head | MODEL-AUDIO-PARAKEET-TRANSDUCER |
| ✅ | OPTForCausalLM | OPT-125m | STRICT token-exact 6/6 vs vLLM 0.25.0; speed pending | MODEL-TEXT-opt-optfor-causal-lm |
| ✅ | DeepseekV2ForCausalLM | DeepSeek-V2-Lite (MLA) | SACRED gate 8/8 token-exact vs vLLM 0.25.0; speed short (attributed, W9) | MODEL-TEXT-deepseek-v2-deepseek-v2-for-causal-lm |
| 🚧 | LlamaModel (embedding conversion) | The first POOLING arch (ARCH-ONE-SURFACE ROW 6): upstream _EMBEDDING_MODELS maps LlamaModel onto the Llama backbone via as_embedding_model (registry.py:230 + adapters.py:230) | LIVE 2026-08-08 on row/EMBEDDINGS-ONE-SURFACE (PR #137): is_pooling_model=true, text paths refuse by task; vllm_embed (ABI v15) + task-conditional /v1/embeddings; fold gate 4/4-231 on the committed synthetic fixture (engine path == direct registry path + f64 LAST+normalize ref). Residual: real embedding checkpoint + LLM(task="embed") oracle cosine; the other 7 memberships unregistered | MODEL-EMBED-llama-llama-for-causal-lm |
| ✅ | LlamaForCausalLM | Llama-3.2-1B dense (+ Yi + InternLM3ForCausalLM aliases) | STRICT token-exact 16/16 vs vLLM 0.25.0; speed pending. Llama-alias checkpoints gated 2026-07-26: Yi (01-ai/Yi-Coder-1.5B-Chat, arch=LlamaForCausalLM, zero delta) 16/16; InternLM3 (internlm3-8b-instruct, one alias line, plain-Llama+dynamic rope) 16/16 — CLOSES the recent-dense TEXT tier | MODEL-TEXT-llama-llama-for-causal-lm |
| ✅ | MistralForCausalLM | Mistral-7B-v0.3 dense | full paged-engine SACRED gate 16/16 vs vLLM 0.25.0; speed pending | MODEL-TEXT-mistral-mistral-for-causal-lm |
| ✅ | Qwen3_5MTP | Qwen3.5 MTP draft (spec-decode) | DONE 2026-07-26: k=1 MTP spec-decode e2e on the 27B GDN hybrid — three-way token-exact at c1 (our-ON == vLLM --speculative-config mtp == our-OFF, acceptance 16/16), c1 above vLLM every-axis + c2-c8 on-par-or-above, mixed-batch concurrency bit-exact, server/CLI/C-ABI --speculative-config; spec-OFF byte-identical | MODEL-SPEC-qwen3-5-mtp-qwen3-5-mtp |
| ✅ | Qwen3_5MoeMTP | Qwen3.5-MoE MTP draft (spec-decode) | DONE 2026-07-26 (M-mtp-2): k=1 MoE-MTP spec-decode e2e on the 35B — three-way token-exact at c1 (our-ON == vLLM 0.25.0 --speculative-config mtp == our-OFF == vLLM spec-OFF, 16/16 vs greedy_ids anchor, acceptance 16/16 both sides), c1 spec-ON 1.19x TPOT / +16.3% output-tput vs spec-OFF (acceptance 0.908); spec-OFF byte-identical (test-only diff) | MODEL-SPEC-qwen3-5-mtp-qwen3-5-moe-mtp |
| ✅ | DFlashDraftModel | Qwen3 DFlash draft (spec-decode) | DONE 2026-07-27 (CLAIM-DFLASH-D14): block-diffusion drafting is correctness-complete (ratified near-tie e2e 27/27, acceptance 19/39/29/25) AND at/above vLLM throughput — FINAL 3-rep A/B our-ON 29.32 tok/s ≥ vLLM-ON 29.24 (non-overlapping bands, 1.003×). D14 closed the D13 ~2% residual: nsys attributed it to the from-scratch draft attention (1.8%/step); ported to a warp-scoped online-softmax kernel (3.1× faster). SACRED 235/235 + MTP 9/9 inert. Built D0-D14 on the advanced pin 555967922/vLLM 0.26.0.dev0 (vllm#40898 resolved under VLLM_USE_V2_MODEL_RUNNER=1); the per-attempt D0-D12 progression (drafter model + first non-causal in-block attention KERNEL-ATTN-DFLASH-BLOCK, context-KV precompute, engine-loop integration, persistent paged draft-KV, capture-safe paged attention) climbed 0.69x to 0.917x then D14 closed the last ~2%. Full forensics in the parity ledger + dflash-spec-decode.md. | MODEL-SPEC-qwen3-dflash-dflash-qwen3-for-causal-lm |
| ✅ | model factory / self-registration | cross-cutting registry (not an arch) | DONE (record repaired 2026-08-05): REGISTER_VLLM_MODEL static-Registrar landed 2026-07-19 (CLAIM-MODEL-SELFREG-1, model_registry.h:167-189), fixed kRegistrations array retired; the deferred dgx no-regression debt was paid by the 2026-07-23 seam campaign (seven SACRED gates byte-identical on dgx through ModelRegistry::Forward) and every SACRED gate since; 28 TUs self-register today — every post-landing model bring-up (Qwen3 dense/MoE, Coder, Gemma-4, DS4, Laguna, Kimi-Linear) exercised the 1-TU+1-line additivity claim. Deep qwen3_5.cpp layers/ relocation stays a named cosmetic follow-on, not gated here | MODEL-FACTORY-registry |
| 📋 | ChatGLMForCausalLM | ChatGLM | scoped in the GLM/DSA spike, not implemented | MODEL-TEXT-chatglm-chat-glmfor-causal-lm |
| 📋 | DeepseekForCausalLM | DeepSeek (plain MHA, not MLA) | scoped; this is a plain-MHA row, distinct from the MLA campaign | MODEL-TEXT-deepseek-v2-deepseek-for-causal-lm |
| ✅ | DeepseekV4ForCausalLM | DeepSeek-V4 | DECODE GLUE-FUSION LEVERS 2+3 LANDED + MEASURED (2026-08-03, CLAIM-DSV4-ROPE-FLOAT + CLAIM-DSV4-MHC-LEAN, GB10 sm_121a, base 6576814b): two byte-exact decode-glue folds on top of the MHC-pre fold. (3) the fused per-head/kv/inverse-o norm+RoPE kernel (NormRopeRows) folded FP64→FP32 (VT_V4_ROPE_FLOAT default-ON) — 4.58→0.46 ms/step (~10×), decode +6.1%; (2) the MHC-pre finish occupancy widen (256→1024) + sqrsum-fold (VT_V4_MHC_LEAN default-ON=1024) — +0.7%, MHC residual launch-serialization-bound (86 sequential single-block launches/step). BYTE-EXACT: decode ids token-IDENTICAL =1/=0 for BOTH levers on ds4flash IQ2XXS via the resident-decode path (--gpu --kv-cache, device kernels genuinely exercised); test_cuda_deepseek_v4 Brick-7 + Brick-B near-tie 20/20·67073 on GB10. Net decode 14.02→14.96 tok/s (+6.7%) → 90.7% of ds4 ~16.5 (ds4 bar corrected from the unreproduced 17.13 anchor to the fair same-session ~16.5; the later MHC-SINK4 lever reached ~15.87 tok/s = ~96%). SACRED test_qwen36_paged_engine 315/315 UNMOVED on the merged build. MHC-PRE FP64→FP32 FOLD LANDED + MEASURED (2026-08-03, CLAIM-DSV4-MHC-FOLD): the float MHC-pre mirroring ds4's hc4_split_one+hc_split_weighted_sum_norm_fused_kernel (VT_V4_MHC_FUSED, default-ON; GB10 throttles FP64) is BIT-EXACT (decode ids token-identical =1/=0; test_cuda_deepseek_v4 Brick-B near-tie gate PASS on GB10) and MEASURED 85.4%→87.5% of ds4 ~16.5 (recomputed onto the corrected ds4 bar). GGUF UD-IQ2_M CPU BRING-UP (2026-08-03): multi-shard gguf-split stitching (CLAIM-GGUF-SPLIT-SHARDS, test_gguf 33/33) + IQ2_S/MXFP4 keep-quant vec_dot (CLAIM-DSV4-UDIQ2M-QUANT, CPU 17/17+19/19+37/37) load the 3-shard unsloth/DeepSeek-V4-Flash UD-IQ2_M COMPRESSED; the IQ2_S device DotSuperblock<kIQ2_S> is now CUDA-BUILT + LINKED on GB10 (sm_121a, -Werror; MXFP4 device dot kept [[maybe_unused]] until a Q8_0-activation GEMM lands). A real UD-IQ2_M GPU load is owed. — W1/W2 CPU scaffolding landed (registry stub + config parse + loader map VERIFIED vs the real NVFP4 header; clean -Werror; forward is a W3-W8 VT_CHECK stub). HW-FIT REVERSAL: the NVFP4 checkpoint is 156.7 GiB, does NOT fit ONE GB10 (119 GiB) ⇒ W1 single-GB10 oracle run is MEMORY-INFEASIBLE. W3 primitives landed (2026-07-28): DSA Lightning-Indexer sparse top-k selection + 512-wide MLA attention-sink softmax + grouped output-LoRA ported as host references + unit-gated (deepseek_v4_dsa.{h,cpp}, test_deepseek_v4_dsa 13/13·38); SGLang v0.5.15 registers V4 (viable 2nd reference). W4 primitives landed (2026-07-29): the DSA COMPRESSOR (softmax-weighted window pool + save-time APE add + RMSNorm) + the fp8_ds_mla KV-cache state layout (448 fp8 UE8M0 + 64 bf16 rope, 576B stride) ported as host references + unit-gated (deepseek_v4_compressor.{h,cpp}, test_deepseek_v4_compressor 12/12·164; new kernel row KERNEL-ATTN-DSA-COMPRESSOR). W5 primitives landed (2026-07-29): the Manifold/Markov Hyper-Connections (MHC) topology — the 20-iteration Sinkhorn-normalized doubly-stochastic mixing matrix + the mHC pre/post mixes with the folded attn/ffn RMSNorms + the hc_head collapse — ported as host references + unit-gated (deepseek_v4_mhc.{h,cpp}, test_deepseek_v4_mhc 14/14·125; new kernel row KERNEL-MHC-SINKHORN); the W0 "no eager reference" premise is CORRECTED (vLLM ships mhc/torch.py; four upstream impls agree on the Sinkhorn). W6 primitives landed (2026-07-29): the sqrtsoftplus + hash-routed MoE deltas — the sqrt(softplus(·)) router score, the noaux_tc bias-for-selection top-k with weights gathered from the UNBIASED scores, the tid2eid hash route that bypasses top-k, and the asymmetric clamped SwiGLU expert activation — ported as host references + unit-gated (deepseek_v4_moe.{h,cpp}, test_deepseek_v4_moe 12/12·716, RED-first proven all 3 levers; new kernel row KERNEL-MOE-SQRTSOFTPLUS-HASH); the shared grouped-GEMM machinery is REUSED not re-ported. W7 forward-assembly landed (2026-07-29): the VT_CHECK(false) stub is replaced by a real DeepseekV4Model::Forward (DeepseekV4ForwardHost) composing the four host primitives into an end-to-end logits producer at a tiny synthetic config, STRUCTURALLY gated (test_deepseek_v4_forward 6/6·26 — interleave runs, [T,hc,H] stream, hash-by-tid routing, DSA select+compress, finiteness; RED-first 3 levers) — DERIVED + BUILD-VERIFIED, NOT a real-checkpoint token gate. W7-device landed + DGX-gated (2026-07-29): the CUDA kernels for the four new families (MHC/DSA/compressor+fp8_ds_mla/sqrtsoftplus-hash MoE), registered through the OpProvider seam and dispatched by a real DeepseekV4Model::ForwardDevice, RUNTIME-VERIFIED on GB10 vs the host-ref oracle (test_cuda_deepseek_v4 11/11·153, memcheck 0 errors, RED-first proven; new kernel row KERNEL-DSV4-W7-DEVICE); the 512-wide MLA attn + expert grouped-GEMM REUSE the existing NVFP4/FP8 kernels. W8 single-Spark keep-quant enabler + name-map landed (2026-07-29, CLAIM-DEEPSEEK-V4-W8): the ~2-3-bit routed-expert encodings now keep-quant — vec_dot for IQ2_XXS/IQ3_XXS/Q2_K (1:1 ggml ports; test_ops_quant_dot 19/130444, RED-first) so the 158 B experts stay COMPRESSED (~91 GiB vs the ~316 GiB bf16 OOM); the blk.N.*→V4 GGUF name-map has EXACT 1328/1328 coverage (check-dsv4-gguf-namemap.py rc=0, real manifest via HTTP-range). HONEST: the real UD-IQ2_XXS uses IQ2_XXS gate/up + IQ3_XXS down (not Q2_K); keep-quant is CPU-tier (no CUDA vec_dot for ANY k-quant). W2b GGUF→tower materialization landed (2026-07-29, CLAIM-DEEPSEEK-V4-W2B, base 341dfbb9): LoadDeepseekV4FromGguf (+ DeepseekV4ParamsFromGguf) wires the blk.N.* name-map + the landed keep-quant blocks into the DeepseekV4 weight towers — routes EVERY GGUF tensor via the name-map into its tower slot, KEEPS the MW/SEW weights (MLA linears + router gate + 256 routed/shared experts + lm_head) COMPRESSED as block dtype (OwnGgufQuantBlocks, the ~91 GiB-vs-316 GiB OOM enabler) and dequants the small MHC/DSA/norm/embed/tid2eid tensors, accounts for every tensor (throws on unmapped/leftover), lifts the registry GGUF reject (kGguf→LoadDeepseekV4FromGguf), and dequants the tiny CPU composition tower so a loaded model FORWARDs. Structural gate test_deepseek_v4_gguf_load 5/5·149 on a tiny synthetic deepseek4 GGUF (real blk.N.* names + Q8_0 keep-quant): accounting 126/126 (== file tensor set, none unmapped/leftover), keep-quant residency (down-experts stay Q8_0, bytes < dequant-f32; expand-policy RED shows bf16), load→DeepseekV4ForwardHost finite+deterministic, RED-first (missing/leftover throw). DERIVED+BUILD-VERIFIED at tiny shape (loader-only, SACRED-inert: forward + W3-W6 primitives + shared MLA/MoE empty-diff; prior V4 tests 6/26 + 4/40 unchanged). The real 91 GB UD-IQ2_XXS RUN did NOT execute (NOT faked) — it is the W8-final operational residual (download + DGX keep-quant load + ForwardDevice greedy gen + self-consistency/coherence gate + benchmark). W8-final entrypoint wiring LANDED + GATED (2026-07-29, CLAIM-DEEPSEEK-V4-W8, base 376e186b): the top-level GGUF dispatch now routes a deepseek4 file to the DeepSeek-V4 loader — DeepseekV4HfConfigFromGguf maps general.architecture=deepseek4 → the registered DeepseekV4ForCausalLM (republishing the GGUF geometry into config.raw for the registry parse hook) and LoadedEngine::FromModelDir dispatches to it via HfConfigFromGgufDispatch (was qwen-only, would have thrown "unexpected architecture 'deepseek4'"); gate test_deepseek_v4_gguf_load 6/6·168 (+1 case: config maps + ModelRegistry::Resolve→V4 factory), CPU -Werror-clean full-library build, qwen GGUF path byte-neutral. The real single-Spark RUN did NOT execute and is BLOCKED by a CODE residual — not download/box (HONEST, provable from source; NOT attempted because it would OOM-reboot the box): the forward (ForwardComposeImpl) composes off the FULLY-DEQUANTIZED f32 weights.host tower, and LoadDeepseekV4FromGguf builds that host tower UNCONDITIONALLY (every routed-expert tensor via HostVec→DqRowF32→f32) — for the real 43-layer/256-expert model that is ~24 GiB per layer (3×256×2048×4096×4 B) ≈ ~1.0 TiB total f32, which OOM-reboots the 119 GiB unified pool at ~layer 5. The keep-quant weights.gguf tower (~91 GiB, the W2b memory enabler) IS built but is NEVER read by the forward — the forward-over-keep-quant-blocks path is unimplemented. So a real run additionally needs the forward rewired to consume the keep-quant blocks (via the CPU CIQ kMatmulBTQuant GEMM) + the host-dequant gated off — was the named residual W2c. W2c LANDED (2026-07-29, CLAIM-DEEPSEEK-V4-W2C, NOT pushed): the memory infeasibility is FIXED. LoadDeepseekV4FromGguf no longer f32-expands the big MLA/MoE/lm_head weights (only the small norms/embed/MHC/DSA/hash tensors dequant to the host tower); a new DeepseekV4ForwardGguf runs the SAME composition with the 512-wide MLA linears + 256 routed/shared expert GEMMs + lm_head consuming the COMPRESSED weights.gguf blocks in place via vt::MatmulBT→kMatmulBTQuant. DeepseekV4Model::Forward gates on has_gguf_weights (safetensors/NVFP4 + tiny-synthetic host path byte-identical). Gate test_deepseek_v4_gguf_load 7/7·185 (CPU Release -Werror-clean): keep-quant forward RUNS (finite, deterministic) + keep-quant(Q8_0)==dequant(bf16) RelL2 0.0116 (< 0.05 near-tie) + RED-first (no-sink miswire diverges 0.122; a load that rebuilds the f32 tower fails the loader's VT_CHECK + the host<gguf-bytes assertion). MEMORY-BOUND asserted: host 23,980 B vs keep-quant 141,676 B at tiny shape; PROJECTED full-scale — the 256 routed experts alone are ~1032 GiB f32 (OOM-reboots the 119 GiB pool) vs the UD-IQ2_XXS keep-quant ~91 GiB + small host < 3 GiB = memory-FEASIBLE on ONE GB10. Honest 3-state: the tiny keep-quant forward = DERIVED+BUILD-VERIFIED; the real 91 GB run stays the operational W8-run (download + DGX), now memory-feasible. SACRED-inert PROVEN: only deepseek_v4.{h,cpp}/deepseek_v4_weights.cpp + the test changed; W3-W6 primitive tests unchanged (dsa 38, mhc 125, moe 716, compressor 164, forward 26, scaffold 40). NO tokens generated (not faked); DGX left exactly as found (worker untouched, no download). W8-run apples-to-apples oracle SELECTED + ds4-file LOADABILITY PROVEN (2026-07-29, CLAIM-DEEPSEEK-V4-W8-RUN, base edf68c91, NOT pushed): the cross-engine oracle is antirez/ds4 (DwarfStar, make cuda-spark on GB10); the shared apples-to-apples vehicle is ds4's q2-imatrix GGUF (antirez/deepseek-v4-gguf, single 80.7 GB file — IQ2_XXS gate/up + Q2_K down + Q8_0 attn/shared/out + F16 embed, all keep-quant types we support). Verified via HF HTTP-range (NO 80 GB download) that our blk.N.* name-map covers ds4's file EXACTLY (1328/1328, 0 unmapped, 0 leftover — byte-identical tensor-name set to unsloth). Found + FIXED 3 real-file loader gaps that would throw/zero before a token: (1) compress_ratios length 44 (43 layers + trailing MTP) vs our strict ==block_count → relaxed to >= + prefix-truncate; (2) clamp read only the scalar swiglu_clamp but ds4 ships the per-layer array swiglu_clamp_exp (all 10.0) — a 0 limit ZEROES every expert (ClampedSwiGLU min(gate,0)·clamp(up,0,0)) → array fallback added; (3) hash ffn_gate_tid2eid is ggml I32 (type 26), unhandled in DequantGgufRowToF32 → I32 case added. Gate test_deepseek_v4_gguf_load 8/8·288 (CPU Release -Werror-clean) with a new ds4-flavor case reproducing all 3 quirks (load + keep-quant forward finite + deterministic); RED-first PROVEN (disabling the I32 case throws 'unsupported ggml type 26'). The GB10 RUN did NOT execute (NOT faked): blocked on the GPU flock held by a concurrent agent + two long builds (ds4 cuda-spark + our aarch64 CUDA) + the 80 GB download + a greedy driver (the engine's incremental paged decode is incompatible with V4's stateless keep-quant recompute — a manual greedy loop over DeepseekV4ForwardGguf is required; Tokenizer::FromGguf also rejects ds4's pre='joyai-llm', so the token cross-check must inject ds4's token ids). W8-run EXECUTED on the DGX GB10 (2026-07-29, CLAIM-DEEPSEEK-V4-W8-RUN, base 858b0b15, NOT pushed) — LOAD succeeds on the real 158 B model, forward HARD-FAILS at the real geometry; ds4 reference captured: the real 80.7 GB ds4 q2-imatrix file was downloaded + integrity-verified (86,720,111,488 B == HTTP Content-Length), ds4 was built (make cuda-spark) and our CPU engine + a NEW greedy driver (examples/deepseek_v4_gen, stateless full-recompute loop over DeepseekV4ForwardGguf) were built, and BOTH ran on the SAME file. ds4 (GB10 GPU) loads 80.76 GiB in ~20-29 s and greedily generates coherent on-topic text ("We need to answer: 'The capital of France is'. This is a straightforward") at prefill 358 tok/s, decode 16.5 tok/s (ctx=1024, ds4-bench). OUR engine LOADS the keep-quant tower on the real model (43 layers, 256 experts, vocab 129280, all 1328 tensors accounted, has_gguf_weights=1, ~78 s) at PEAK RESIDENT 116.2 GiB — FITS under the 119 GiB pool but ~32 GiB ABOVE the projected ~84 GiB because OwnGgufQuantBlocks is called WITHOUT mmap_src so it COPIES ~81 GiB into owned vectors while the mmap pages stay resident (fix: mmap-VIEW like qwen3_5_gguf_weights.cpp). The FORWARD then HARD-FAILS: deepseek-v4 keep-quant GEMM: weight shape mismatch: want [N=512,K=4096] got [1024,4096] (deepseek_v4.cpp:234) at layer 2 (the first DSA compressor layer) — the W3-W7 forward, gated only at tiny synthetic shape, conflates head_dim=512 with the DSA compressor's distinct 1024-dim output (real attn_compressor_gate/attn_compressor_kv are [1024,4096]). Layers 0-1 pass; layer 2 crashes. NO coherent tokens generated. Remaining brick: rework the MLA/DSA forward to the real DeepSeek-V4 geometry (compressor 1024-dim; audit indexer + grouped output-LoRA at o_groups=8/nh=64/q_lora_rank=1024) against a reference, plus the mmap-view load fix. Download + keep-quant load + tokenizer + greedy driver + ds4 oracle are DONE; the forward geometry is the one blocker. W8-run (2) — geometry FIXED, forward now RUNS the real 158 B model end-to-end; generation still INCOHERENT (2026-07-29, CLAIM-DEEPSEEK-V4-W8-RUN, base fba56f9b, NOT pushed): the layer-2 hard-fail is fixed — the DSA compressor projects to 2*head_dim (ds4 coff=2), not head_dim, so the real keep-quant run uses DENSE MLA (EXACT for seq ≤ index_topk=512, i.e. every short single-Spark step; the sparse compressor/indexer at real geometry is a named residual), guarded by a NEW real-ish gate (comp_width=2*head_dim, test_deepseek_v4_gguf_load 9/9·388, RED-first). Also landed: mmap-VIEW keep-quant load (PEAK RESIDENT 116.2 → 86.1 GiB, byte-identical, load 78 s → 6 s warm) + per-layer YaRN RoPE (compressed layers base 160000 + freq_scale 1/16, ds4 rope_tail_ext_inplace) + skip the lossy single-block fp8 KV roundtrip. The forward now runs ALL 43 layers on the real model at ~1.4-1.7 s/tok / peak 86 GiB, but greedy output is a degenerate 2-cycle loop (201 7249 465 7249…) — INCOHERENT. A different prompt gives different ids (818 7249 16 42498 20850), so the body is input-responsive; the failure is a numerical/scaling fidelity error, not structural. Remaining brick: localize it by instrumented per-layer comparison vs ds4 (suspects: MoE routing/expert-scale/shared-expert, MHC Sinkhorn stream mixing, keep-quant IQ2_XXS/Q2_K numerics at real scale). Ruled out: geometry, attention scale (ds4 1/sqrt(512) matches) + structure, fp8 roundtrip, memory path, rope. ds4 reference (same file, GB10 GPU): coherent, prefill 358 tok/s, decode 16.5 tok/s; ours CPU-tier (~0.6 tok/s decode — the #195 CUDA keep-quant GEMM is on main but this stateless-recompute forward runs the CPU queue; GPU-expert wiring is a named residual). W8-run.9 COHERENT GENERATION (2026-07-29, CLAIM-DEEPSEEK-V4-COHERENCE, NOT pushed) — the real single-Spark run WORKS. Root-caused via per-layer diff vs the antirez/ds4 oracle to a MISSING per-head query RMS-norm (ds4 head_rms_norm_inplace after wq_b): at L00 the attention input was bit-exact (rel-L2 0.0000) and the KV correct (0.018) but q was rel-L2 0.9646 → the q projection. Fix DeepseekV4QHeadRmsNormInplace (+ spec-anchored doctest, RED-first) collapsed L00 q to 0.0013 and the FULL 43-layer curve to the keep-quant floor (L33 0.53→0.003, L34 6.02→0.003). Plus the MISSING inverse-RoPE on the attention output (ds4 rope(heads,inverse=true), pos>0). Greedy generation now: "The capital of France is Paris.<｜end▁of▁sentence｜>" (correct + EOS), self-consistent, test_deepseek_v4_gguf_load 10/10·403. CPU-tier ~3.3 s/tok / peak 85.8 GiB. Named residuals: GPU-expert dispatch (CPU-tier), DSA-sparse ctx>512 (dense-fallback, exact for short gen), ds4-oracle (vLLM has no V4 GGUF). Row ADVANCES SPIKE → ACTIVE. DECODE now DEVICE-RESIDENT by DEFAULT on GB10 (2026-07-30, CLAIM-DEEPSEEK-V4-DEVICE-DECODE, base 73d4799c, NOT pushed): the full device-resident decode campaign (bricks A→D + the Q8_0 CUDA keep-quant GEMM + the top-3 glue-kernel tune) landed; the T=1 decode step now runs 100% on the GB10 (device MLA attention + resident routing + grouped keep-quant MoE GEMM incl. Q8_0, no per-op sync) and is the SHIPPED DEFAULT (VT_V4_RESIDENT_DECODE=0 rolls back to the host forward; the optional CUDA graph VT_V4_DECODE_GRAPH=1 ≈ eager once GPU-bound). Decode 7.96 tok/s (24-tok) / 8.01 (short) / ~1.8× the host path at a grown 256-tok KV context — ~48% of ds4's 16.5 tok/s on the SAME 80.7 GB file; the step is GEMM-bound (glue tuned 44%→21%). Validated broadly (4 prompts × 256 tok, resident vs host): token-identical on factual, characterized COHERENT NEAR-TIE on open-ended (deterministic, bounded-noise near-tie flips into coherent text — the ratified gate), 0 errors. Gates: test_cuda_deepseek_v4 18/18·34176, test_deepseek_v4_gguf_load 12/12·531. The last mile to ds4's 16.5 is GEMM/quant microarch (fp8 KV, tuned MMQ), a named residual. | MODEL-TEXT-deepseek-v4-deepseek-v4-for-causal-lm |
| 🚧 | DeepSeekV4MTPModel | DeepSeek-V4-Flash MTP draft (spec-decode) | W1 2026-07-30 (CLAIM-DEEPSEEK-V4-MTP): native nextn draft head WIRED — tiny-config draft forward + compute_logits (1:1 nvidia/mtp.py:128-258: enorm/hnorm, separate e_proj/h_proj, MHC mtp_block, own hc_head collapse) reusing the DS4 host composition, DeepseekV4GgufHasMtp absence guard, + the LOSSLESS gate (DS4 draft + DS4 target verified by the shared RejectionSampler == pure target greedy → MTP-on == MTP-off; test_deepseek_v4_mtp 5/5). Real-model gate WEIGHT-BLOCKED: both shipped GGUFs dropped the nextn tail. Residuals: DS4-native decode-loop propose/verify + engine spec-config registration. | MODEL-SPEC-deepseek-v4-deep-seek-v4-mtp |
| 📋 | GlmForCausalLM | GLM | scoped in the GLM/DSA spike, not implemented | MODEL-TEXT-glm-glm-for-causal-lm |
| 🚧 | Glm4ForCausalLM | GLM-4-9B-0414 (dense, sandwich norms + partial interleaved rope) | SACRED gate 16/16 vs vLLM 0.25.0 (STRICT token-exact 13/16 + near-tie-band 3/16, max gap 0 nats, 0 forward-divergent; vLLM K=5 self-deterministic → STRICT bar); speed pending | MODEL-TEXT-glm4-glm4-for-causal-lm |
| 📋 | Glm4MoeForCausalLM | GLM-4 MoE | scoped in the GLM/DSA spike, not implemented | MODEL-TEXT-glm4-moe-glm4-moe-for-causal-lm |
| ✅ | Glm4MoeLiteForCausalLM | GLM-4.7-Flash (31.2B MLA + GLM MoE) | SACRED gate 8/8 vs vLLM 0.25.0 (STRICT token-exact 1/8 + near-tie-band 7/8, 69/128 tokens strictly exact, max teacher-forced gap 0.0 nats, 0 forward-divergent; vLLM K=5 self-deterministic → STRICT bar); FIRST e2e coverage of the q_lora query branch AND the noaux_tc sigmoid router (closes the MLA campaign's two gaps, C2); speed pending | MODEL-TEXT-glm4-moe-lite-glm4-moe-lite-for-causal-lm |
| 🚧 | KimiLinearForCausalLM | Kimi-Linear-48B-A3B | PAGED-INCREMENTAL DECODE LANDS the 5× speed win (2026-08-07, §19, row/KIMI-PAGED-INCREMENTAL #113): the §18 real lever (e) BUILT — KimiDecodeCache + ForwardPrefillIncremental/ForwardDecodeStepIncremental (kimi_linear_device.cpp): prefill-once (KDA recurrent+conv state carried via vt::KdaGatedDeltaRule state in/out + vt::CausalConv1dFwd tap-carry; NoPE-MLA latent-KV cached) + recurrent decode-step, MIRRORING vLLM kimi_gdn_linear_attn._forward (prefill=chunk_kda_with_fused_gate output_final_state / decode=fused_recurrent_kda initial_state, vllm-src a4e3cb4; divergences: host state vs paged slot cache, materialized-MHA MLA vs paged-FA2 — named residuals). CPU byte-exact state-carry gate test_kimi_linear_forward 15/15·875 (NEW case l: carried decode == fresh full-recompute byte-identical + greedy-identical). Full 48.9B GB10 (single-load/config, flock, drop_caches, min-avail 18-21 GiB, no reboot, §12 golden md5 bfa5bdbf): recompute 122/128 @ 4.23 tok/s (reproduces #111); incremental+recurrence 120/128 @ 16.63; incremental+chunk-prefill 122/128 @ 18.87/19.03 tok/s (2 runs) — token-IDENTICAL to recompute (Gate A PASS, p7 got byte-exact) = 4.5× over recompute, 0.90× of vLLM ~21 (the 5× decode gap 0.20×→0.90×). Gate B STRICT NOT reached (122/128): chunk-prefill in the RIGHT vehicle reproduces recompute EXACTLY, does NOT close p7 — REFUTES the #111 "p7 in the right vehicle → STRICT" hypothesis; p7 intrinsic (§13/§14 f32-vs-bf16 near-tie at a comma). Decode decomposition (nsys, ours, 99 steps, same-tool): ~90% is the SAME cuBLAS internal::gemvx::kernel vLLM calls (batch-1 GEMV-parity), KdaScanKernel 2.3%, MoE glue 2.3%, CastBf16 3%; chunk kernels 20 inst = prefill only (prefill=chunk/decode=recurrent IN VIVO). Killing O(n²) ALONE reaches parity-class; no lever load-bearing beyond it — residual = ~15% host-orchestration idle + 3% CastBf16 (a bf16 residual stream = the ONE lever ALSO closing the p7-STRICT near-tie) + paged-FA2 MLA decode. vLLM-live-nsys@0.82 NOT run (box-safety: 95-98 GiB reservation + nsys below the 15 GiB LIFE-CRITICAL floor). --incremental opt-in; VT_KIMI_DEVICE_KDA/_CHUNK STAY OFF (122/128 ≠ STRICT). Row STAYS 🚧. chunk_kda PREFILL PHASE-2 MEASURED — op CORRECT, chunk-EVERY-STEP REGRESSES 122→102 (2026-08-07, §18, row/KIMI-CHUNK-KDA-P2 #111): the chunk_kda prefill family regenerated + vendored for ALL 6 arches (reproducible — only new kda_*+MANIFEST; GDN cubins byte-identical; drift GREEN) + wired through the new op vt::KdaChunkPrefill (the 6-cubin _chunk_kda_fwd_with_cumulative_g; cuda_gdn.cu.o -Werror clean; RED-first unit test_ops_kda_chunk_prefill 2/2·4 on GB10 — chunk-vs-recurrence mean_abs 4.68e-5, wrong-gate 72×; GDN untouched 66/66·4242). Full 48.9B GB10 gate (flock, min-avail 21 GiB, no reboot): control device-KDA reproduces 122/128, 4.24 tok/s EXACTLY; +chunk-prefill (VT_KIMI_DEVICE_KDA_CHUNK) REGRESSES to 102/128, 4.08 tok/s (p3 16→3, p6 16→11). Root cause: the island's O(n²) recompute applies chunk EVERY decode step over the growing sequence — NOT vLLM's prefill=chunk/decode=recurrent split — so it coin-flips near-ties the recurrence-every-step (control) doesn't (the recurrence matches vLLM's DECODE; chunk only matches its PREFILL). vLLM speed arm (§12 recipe, util 0.82, triton MoE, eager, single-seq; min-avail 15 GiB, no reboot): ~21 tok/s median 16-token aggregate (25.3 cold-discarded; TTFT not isolable in 0.25.0) vs ours 4.24 (recurrence) / 4.08 (chunk) STEADY decode → ours/vLLM ≈ 0.20 (vLLM ~5× faster on decode — the O(n²)-recompute vs paged-incremental distance, = the coupled STRICT+speed lever). VT_KIMI_DEVICE_KDA_CHUNK STAYS OFF (a regression isn't a flip); device-KDA (122, OFF) still best. The op + regen are the validated prefill half of the named real lever (e) paged-incremental decode (chunk-prefill ONCE + recurrent-decode over PERSISTENT state — kills the O(n²); the STRICT + speed lever, coupled). Row STAYS 🚧. DEVICE-KDA GB10 122/128 + 4.24 tok/s (§15, #104); device NoPE-MLA lever MEASURED-NEGATIVE (2026-08-07, §16, row/KIMI-STRICT-CLOSE #107): the per-channel-decay device recurrence vt::KdaGatedDeltaRule moves 106→122/128 (p0-p6 16/16; sole p7 pos-6 comma near-tie) AND 1.35→4.24 tok/s (3.1×) — vLLM's ACTUAL f32-on-bf16 arithmetic, beats §14's host-precision 120. The §15 residual (d) was attempted in device-COMPUTE form: VT_KIMI_DEVICE_MLA routes the 7 NoPE-MLA layers' softmax core through vt::Attention (pad-V: value zero-padded qk_nope+qk_rope=192 vs v=128, out[:,:,:v] byte-exact). CPU RED-first gate GREEN (test_kimi_linear_forward 14/14·825, pad-V==f64 ref rtol 3e-3; perturbation fails 108). Full 48.9B GB10 gate (single-load, flock, min-avail 21 GiB, no reboot): control device-KDA reproduces 122/128, 4.24 tok/s EXACTLY; +device-MLA REGRESSES to 109/128 AND 3.89 tok/s — vt::Attention's f32 online-softmax is the right math but a DIFFERENT reduction order than vLLM's FA2, so it coin-flips near-ties (breaks p3 16→3 into §14's repeat) and the per-(t,h) build slows the O(n²) recompute. VT_KIMI_DEVICE_MLA STAYS OFF, kept as a documented-MEASURED-NEGATIVE A/B knob (§14 ISLAND_F32ACC precedent). MLA dims VERIFIED from the real config (nah=32, qk_nope=128, qk_rope=64, v=128, kv_lora=512, q_lora=None; 7 full-attn/20 KDA). Both device knobs default OFF (122 ≠ STRICT, K=3-deterministic golden). STRICT residual, sharpened: needs vLLM's ACTUAL kernels — (c) chunk_kda prefill family (Triton-AOT regen for sm_121a) + (d) paged FA2 mla::ForwardMlaAttentionBlock (NOT the vt::Attention approximation) + (e) paged-incremental decode (needs a decode/paged-attn op, query_len≠key_len; kills the O(n²)) — each a substantial multi-kernel brick (§16). Row STAYS 🚧. FULL-MODEL GB10 e2e RUNS — NEAR-TIE 106/128 (2026-08-06, row/MODEL-KIMI-LINEAR-BF16): the bf16-resident path CLEARS the f32-loader block — the full 48.9B model now runs e2e on one GB10. dgx CUDA build (-Werror clean, 14 GDN AOT symbols nm-linked, test_kimi_linear_forward 13/13·656 in the CUDA binary); kimi-linear-gen --gpu greedy-decodes the §12 8-prompt battery x16 vs greedy_ids.npy. MEMORY: load 117.6s, host RSS PEAK 1.7 GiB (stage-then-ReleaseHost), device peak 98.5 GiB, min-avail 21.6 GiB (above the 15 GiB floor, matches the ~25 GiB pool-math headroom), NO OOM/reboot. TOKEN gate NEAR-TIE 106/128 (82.8%) — prompts 0,1,3,4,5,6 are 16/16 token-exact, p2/p7 diverge at punctuation/word near-ties; 96 consecutive exact tokens across 6 prompts prove the WIRING (a wiring bug can't). Root cause (honest): the f32 residual stream + host-f64 islands are MORE precise than vLLM's bf16 device kernels, so they flip the argmax where vLLM's deterministic bf16 top-1 has a small margin. STRICT path = the named W7-speed residuals (device GDN/MLA islands -> bf16 stream matching vLLM's rounding). 1.59 tok/s (recompute+island rate). VT_KIMI_DEVICE_COMPUTE STAYS OFF (parity-enablers: near-tie != token-exact). Row STAYS 🚧. bf16-RESIDENT loader/forward IMPLEMENTED + CPU-gated (2026-08-06, row/MODEL-KIMI-LINEAR-BF16): the §13 design is coded — LoadKimiLinearResidentBf16Weights/StageKimiResidentBf16/BuildKimiResidentFromHost (kimi_linear_weights.cpp; LoadBf16Direct -> OwnedTensor, per-tensor stage-to-d_dev + ReleaseHost, tiny vectors host f32), KimiLinearResidentWeights (kimi_linear.h), bf16 device forward DeviceForwardBodyBf16 + Gemm Bf16 cast-act at ~20 GEMM sites with the two host-fallback islands EXTRACTED+shared (kimi_linear_device.cpp), ForwardDevice resident-path dispatch (kimi_linear.cpp), and the kimi-linear-gen e2e harness. CPU 13/13·656 (12/12·614 f32 path UNTOUCHED + NEW tiny-config bf16-vs-f32 gate). PENDING: dgx CUDA build + full-model GB10 e2e vs the STRICT golden. Row STAYS 🚧. bf16-RESIDENT brick POOL-MATH+DESIGN (2026-08-06, row/MODEL-KIMI-LINEAR-BF16): pool math CLOSES (91.5 GiB bf16 device-resident + ~2.4 GiB act/norms/ctx ≈ 94 GiB, ~25 GiB headroom); design grounded §13 (Laguna GemmBf16 cast-act + OwnedTensor::d_dev, LoadBf16Direct, f32 MaterializeHost kept for the unit gate). Impl (loader/forward rewrite + gate + e2e) pending. Row STAYS 🚧. §8 GOLDEN CAPTURED — STRICT (2026-08-06, row/MODEL-KIMI-LINEAR-E2E): the §8 SACRED oracle golden is captured on GB10 (0.25.0-stage, util 0.82, moe=triton, min 15 GiB avail, NO reboot), 8/8 prompts DETERMINISTIC over K=3 → STRICT gate, committed at tests/parity/goldens/kimi_linear_greedy/. Full our-engine e2e BLOCKED on OUR f32 loader (materializes ~183 GiB > 119 pool), the bf16-residency residual; row STAYS 🚧. W7 GPU-VERIFY (2026-08-06, branch row/MODEL-KIMI-LINEAR-GPU): the device compute runs 12/12·614 GREEN on GB10 sm_121a CUDA build, BOTH arms (VT_KIMI_DEVICE_COMPUTE=1 + host-ref); prod stack (CUTLASS-NVFP4 GEMM + FA2 ENABLED + Triton-AOT GDN, 14 cubins nm-verified); f32 device==W2 ref, no divergence, no DeepSeek-class trap. Oracle gateability re-confirmed (0.25.0-stage registers KimiLinearForCausalLM). e2e §8 SACRED golden STILL disk-blocked (91.5 GiB checkpoint absent, dgx root 100% full, 34G free). Row STAYS 🚧. W7 DBuf-resident device COMPUTE landed, CPU-gated (CLAIM-KIMI-LINEAR-W7): the real device compute (ForwardDeviceCompute, kimi_linear_device.cpp) composes the whole 27-layer KDA/NoPE-MLA + 256-expert-MoE hybrid over pooled f32 DBufs through the SHARED vt:: ops (embed/FusedChain add+RMSNorm/MatmulBT projections/CausalConv1dFwd convs/L2Norm/RmsNormGated/MoeRouterTopK sigmoid-noaux_tc/MoeSiluMul/MoeCombine/lm_head), returning DEVICE-RESIDENT logits; 2 documented HOST-FALLBACK islands (the KDA per-k-channel gated-delta recurrence + its exp/softplus decay gate — vt::GdnDecode carries only a per-HEAD scalar decay; the NoPE-MLA softmax core — the paged mla::ForwardMlaAttentionBlock device path is born-on-runner) are the W7-speed residuals. CPU-gated vs the W2 host reference (the CPU backend runs the SAME vt:: dispatch): test_kimi_linear_forward 12/12·614 (per-op KDA/NoPE-MLA/MoE/dense device==ref within f32-accumulation tolerance; the whole ForwardDeviceCompute == ref logits + greedy-token-identical + device-resident). Runner opt-in via VT_KIMI_DEVICE_COMPUTE=1 (default OFF keeps the CPU-verified W6 host-ref compose). GPU numerics (bf16 activations, GDN Triton-AOT cubins, paged het-KV, grouped-MoE slabs) + the e2e SACRED golden stay a NAMED pending (box down) — row STAYS 🚧. ON TOP OF W6 DEVICE forward SEAM (CLAIM-KIMI-LINEAR-W6): the born-on-the-runner ForwardDevice (the DEFAULT gather_logits runner path) no longer refuses — it composes the [rows,vocab] logits via the CPU reference and hands them back DEVICE-RESIDENT (a pooled DBuf, wrapped like deepseek_v2 WrapDeviceLogits; on_device()==true on CPU+CUDA) so the on-GPU sampler consumes them with NO host download. Kimi-Linear now ROUTES device-resident (check-runner-routing-consistency reclassifies it, refuse-skipped stubs 2→1, NO allowlist; check-fusion-consistency green); test_kimi_linear_forward 7/7·300 (adds the ForwardDevice==host-ref device-resident gate). The DBuf-resident device COMPUTE (KDA via the GDN family, NoPE-MLA via mla::ForwardMlaAttentionBlock, DeepSeek-V2 grouped-MoE over the paged het-KV; full plan in kimi_linear.cpp) is the GPU-verify-pending W7 residual. ON TOP OF W2-W6 CPU REFERENCE forward (CLAIM-KIMI-LINEAR-W2): the real host KimiLinearModel::Forward composes the whole 27-layer hybrid from the landed primitives (KDA layer via vllm::kimi_kda refs + the gated-delta recurrence; NoPE-MLA materialized-MHA ref; sigmoid noaux_tc MoE + shared expert; dense SwiGLU); loader now materializes host float weights; test_kimi_linear_forward 6/6·246 (per-op gates + finite whole forward + greedy decode). ON TOP OF W1 scaffolding (registry + ParseKimiLinearParams 20 KDA + 7 NoPE-MLA + index-verified name-map + het-KV spec). e2e-gateable (FITS one GB10, 0.77× pool). RESIDUAL = the DEVICE born-on-runner forward (KDA kernel/absorbed-MLA/grouped-MoE slabs) + the W0/W7 e2e SACRED golden. ROW 7 RUNNER FOLD (2026-08-07, §21, #122): Kimi decode now runs THROUGH ModelRegistry::Forward on the runner's paged state (B1 config synthesis, B2 KDA-paged via KdaChunkPrefill/KdaGatedDeltaRule over gdn_state, B3 paged NoPE-MLA with mla::ForwardMlaAttentionBlock default-ON, B4 vllm_complete_tokens ABI v13 + thin-client example). GB10: engine==CLI 128/128 byte-identical, vs golden 122/128 (the intrinsic near-tie profile), SACRED post-fold green (35B 315/315, 27B 235/235); server stream 19.0 tok/s wall vs vLLM ~21 (~0.90×) = the open speed residual. Row stays ACTIVE on the speed thread | MODEL-TEXT-kimi-linear-kimi-linear-for-causal-lm |
| 📋 | MuseGlimmerForConditionalGeneration | Muse Glimmer 30B (Meta, agentic multimodal; iRoPE + gated attn + perception encoder + DFlash) | W0-W7 ON row/MODEL-MUSE-GLIMMER (PR #279, NOT merged): 52-layer text forward, 50-layer perception encoder, mm wiring, ATEM reasoning + tool parsers — the forward no longer refuses. The evidence behind that is NARROW and says so: text agrees with a torch transcription of #51655 and with HF's own muse_glimmer on REAL 30B tensors at reduced depth 4/52 only (5 prefill argmax positions, no generated tokens); full depth never ran, multi-step decode is untested, the perception encoder has NO reference check, nothing has run through the server, and the ATEM parsers' channel scoping does not work at server defaults (the seam has no adjust_request dispatch site and skip_special_tokens defaults true, so the framing is stripped first). The lifecycle token stays SPIKE/📋 deliberately: advancing it owes docs/STATUS.md and .agents/NOW.md rows, and both are byte-exactly at their shrink-only ratchet/budget, so the advance belongs to the landing commit that can pay for them. Released 2026-08-08, BEYOND-PIN: no Muse code at 555967922 nor on vLLM main; the only upstream impl is the OPEN PR #51655 head 075d645af (approved, 3/20 CI red), ported from on explicit developer direction as a tracked exception (porting-inventory §9 deviation 16). No oracle ⇒ no speed axis is claimable; correctness gates against the HF reference. Text tower fully traced (Gemma-style sandwich norms w/ baked +1 offset + split eps, Llama-4-style iRoPE NoPE/full vs RoPE/sliding, weightless pre-RoPE QK-norm, ~3.87 query pre-scale w/ dual config schema, Qwen3.5-style attn output gate reading the LAYER INPUT); 50-layer perception encoder (linear patchify, bilinear pos-emb interp, width-first 2D RoPE, block-windowed attn, pixel-shuffle merge) reuses the Qwen3-VL vision seam; DFlash is recognition + is_neox_style threading on the existing speculator, not a new drafter | MODEL-MM-muse-glimmer-muse-glimmer-for-conditional-generation |
| 📋 | KimiK3ForConditionalGeneration | Kimi K3 (2.8T MoE + MoonViT-V2, DERIVE-AND-SHIP) | W2/W5 CPU scaffolding landed (registry stub + nested text/vision/quant config descent + text-backbone structural name-map + REFUSE-by-name forward + MXFP4-refuse loader; clean CPU build, scaffold gate 6/6). text backbone IS KimiLinearForCausalLM (KDA+MLA+MoE hybrid, HEAVY reuse); does NOT fit GB10 (~1.56 TB MXFP4, ~12×) and NOT in the pinned oracle ⇒ no on-box golden — DERIVED, proxy-gated on Kimi-Linear-48B; forward + MXFP4 + KDA delta + MoonViT-V2 not implemented (NOT-YET-BUILDABLE) | MODEL-MM-kimi-k3-kimi-k3-for-conditional-generation |
| 📋 | Dots3NoteForCausalLM | dots3-note (280B-A16B multimodal MoE: DeepSeek-V3.2 text tower + sliding-window MLA, MoE ViT, dots audio encoder) | W2 LANDED (the whole weight map): the complete released shard index read for real — 38006/38006 accounted, 35381 language / 2195 vision / 430 audio, every bucket asserted BY NUMBER, zero unaccounted, and the two tower files carried as NAMED W6/W7 deferral records rather than integer counters. Headers only: 4770592 bytes over 133 safetensors headers, no tensor byte. It settled three things the W1 slice could not — the backbone has exactly FOUR distinct layer shapes so the 1/2 repeat holds, the full/sliding split derived from the shipped indexer tensors matches config.layer_types exactly, and the checkpoint carries 62 F32 tensors in TWO families (the 45 language e_score_correction_bias W1 predicted plus 17 vision router_bias, spec R5's shape). And it FOUND one: the index declares indexer_rope_layout: "leading" / converted_from: "tail", which no upstream code reads (#1846); W3 owes the slice. W1 LANDED (config + registry): the arch RESOLVES, parses the REAL released config.json, and accounts 1614/1614 tensors on a committed slice of the released shard index; load, GGUF and the forward each REFUSE BY NAME. All six §4 config traps are gated RED-first (evidence table in spec §4.1), which matters more here than on any other row because spec §6.4 records that NO oracle runs on hardware we own, so no token gate exists downstream. W1 also RESOLVED §1.4 from the checkpoint: exactly ONE nextn layer (model.layers.46.*, 18 tensors) with the SLIDING geometry and a DENSE MLP, and no shared_head.head.weight (has_own_lm_head=False); and CORRECTED spec §4 item 6 (#1804) — BOTH MLA ropes are GPT-J (deepseek_v2.py:1093-1098 for the full class), the geometries differ on the THETA (5e4 vs 8e7), not the layout. Memory format recorded: mlp.gate.e_score_correction_bias ships F32 in an otherwise BF16 tower. Row stays SPIKE deliberately — making an arch resolve is not porting a model, and the §8.1 heading restructure ACTIVE requires belongs to the brick where the forward stops refusing. W0 scope, unchanged below. Upstream subclasses DeepSeek (Dots3NoteModel(DeepseekV32Model), Dots3NoteMoE(DeepseekV2MoE)), so our gated MLA + DSA indexer + noaux_tc router + Qwen3-VL vision + Voxtral audio carry most of it; net-new is windowed MLA over 33 of 46 layers with a second latent geometry (576 vs 1088 rows), the padded/heterogeneous MLA KV spec, the headwise attention gate, the MoE ViT and the dots audio stem. BEYOND-PIN (vLLM main only, #51255, still being patched) and oracle-memory-infeasible: ~576 GB bf16 / ~290 GB fp8 against a 119-122 GiB ceiling on every host we own, so no performance axis is claimable and no token gate can bind until §6.4 of the spec is answered | MODEL-MM-dots3-note-dots3-note-for-causal-lm |
| 🚧 | MiniMaxH3DiTModel | MiniMax-H3 (33.1B omni-modal video+audio DiT, DERIVE-AND-SHIP) | W1/W2 landed: packed layout (fl2va + ref2va, fp64 position grid BIT-EXACT), latent packing, euler-ancestral eta0 scheduler, and the full DiT forward all parity-gated against the UPSTREAM vLLM-Omni modules executed at reduced dimensions (max abs diff 1.6e-7, 10/10 cases / 2539 assertions). NOT autoregressive (no KV cache, no sampler, no logits) and e2e HW-BLOCKED (~354 GB checkpoint, ~133 GB/rank on 4x B300 vs 119 GiB unified); bf16 production stream + request planning + the ComfyUI-GGUF arm also landed (535 REAL tensors resolve onto our contract, geometry from shapes alone). HW verdict CORRECTED: quantized arms FIT (~41 GB in 119 GiB), so e2e + speed are reachable; encoder/VAEs/audio VAE DONE (4.2e-9 vs the checkpoint's remote code); NVFP4 layout GATED as identical to ours (speed path is loader wiring); BOTH VAE DECODERS done (audio 4.2e-9, video ViT3D 8.9e-8); video tiling + 3D-CNN encoder (conditioning only) pending; encoder TEXT tower done (1.2e-7); serving /v1/videos DONE and the DEVICE-RESIDENT forward (W2b, f32) LANDED + GPU-VERIFIED on Thor sm_110 at video 1.49e-7 / audio 8.94e-8; bf16 stream + fusion folds + the FP4 path (needs sm_121a) + a real-checkpoint run pending. 2026-08-05: the AUDIO-VAE ENCODER is ported (DAC analysis stack + pre_block AttnProjection + mean_proj, gated stage by stage vs the checkpoint's own remote code at 2.98e-8 / 1.64e-7 / 1.86e-8) with its own checkpoint loader gated on the real 1087-tensor manifest — so ref2va AUDIO and VIDEO+AUDIO references are now WIRED (audio rows move by 0.51 / 0.71; a different waveform still moves them by 7.1e-4). Both VAEs are now complete in both directions. bf16 13-SHARD RELEASE INDEXES 2026-08-07 (row/H3-BF16-SHARDED-DIT): MiniMaxH3ShardedCheckpoint resolves the ORIGINAL 66.3 GB release through its own model.safetensors.index.json (a tensor named in the index but missing from its shard throws BY NAME), EnumerateMiniMaxH3ShardedTensors feeds the shared shapes-only geometry parser, and LoadMiniMaxH3DitFromShards is the host-f32 reference loader. Gated CPU-only at 72/72/54497 (post-rebase): every tensor resolves to the shard the index named AND to the bytes written there, the derived geometry equals the single-file path field for field, and a SPARSE 13-shard release with the REAL 535 tensors at REAL shapes (66.3 GB declared, 144 KB on disk) derives the SHIPPED geometry (50/5376/56/128/14336/24/32/1x2x2/5120). STREAMS 2026-08-07 (row/H3-BF16-SHARDED-STREAM): StreamMiniMaxH3ShardedToDeviceBf16 uploads it one tensor at a time — a BF16 tensor bound for a bf16 device slot goes straight from the mmap with ZERO host buffer, so peak host is bounded by ONE tensor (observed host_peak=8192, direct=37 converted=9); bit-exact vs the non-streamed StageMiniMaxH3DitWeights reference over all 46 views with identical logits, rope.inv_freq host-resident, 73/73/55203. Spec §8.14. bf16 TEXT ENCODER + THE CONDITIONING NUMBER 2026-08-07 (row/H3-ENC-BF16-COND-DIFF): the 14-shard 63 GB bf16 Qwen3-VL-32B encoder streams to device too (StreamMiniMaxH3EncoderShardsToDevice, q/k/v and gate/up fused ON DEVICE), --encoder-only runs the tower alone (peak ~96 -> ~49 GiB by not loading the DiT first), and the widening is gated BIT-IDENTICAL vs an f32-staged tower so the A/B cannot be confounded. MEASURED on Thor over 233 tokens: Q4_K_M vs bf16 conditioning is cos 0.99745 mean / 0.909 min, rel RMS 6.85% excluding the attention sink, median rotation 3.5 deg — same energy as a ONE-WORD prompt edit but DIFFUSE (232/233 tokens rotate vs 172/233). Whether the RENDER changes is NOT established. 75/75/55609. Spec §8.15. This UNBLOCKS the quantization-quality question; no bf16-vs-quant render or speed number is claimed. Spec §8.13. W-FP4a LANDED (CPU) 2026-08-06 (row/H3-FP4-SPEED): the device DiT forward now routes the NVFP4 projections through the shared Marlin W4A16 dispatcher (fp4 kept packed; no new quant code), fp4-vs-bf16 wiring gate GREEN (62/62·30039). W-FP4a GB10 leg LANDED 2026-08-06 (row/H3-FP4-GPU-E2E, PR #64): on sm_121a the Marlin W4A16 path RAN for all 11 projections (dense_gemms==11 default — VT_MARLIN_DENSE is default-ON → vLLM's own DENSE Marlin GEMM, not the grouped route; marlin_gemms==11 under VT_MARLIN_DENSE=0; fallback_gemms==0), fp4-vs-bf16 BYTE-EXACT (max|diff|=0), and the fp4 arm is a MEMORY win not a diffusion-forward speed win (per-forward bf16/fp4 3.47× @seq64 → 0.79–0.83× @seq4224–7040; ~16 vs ~66 GB device). Real-checkpoint fp4-resident t2va e2e RUNS (real 18.75 GB NVFP4 DiT + VAEs + GGUF Qwen3-VL-32B encoder → valid mp4/wav; DiT s/step 5.45/20.0/209 s @512/768/REF-209f) but frames are a non-scene patch-grid at 12/20/50 steps → OPEN render bug (device VAE/denoise). vLLM-Omni has no quantized H3 arm (BF16-only) so any comparison is HW/loader-forced-indirect — spec §8. ONE-SURFACE ROW 2 LANDED 2026-08-08 (row/H3-VIDEO-ABI): the whole assembly pipeline is library-owned (vllm::multimodal::MiniMaxH3VideoEngine, minimax_h3_video.cpp) behind the ABI v12 vllm_video_* entry points; /v1/videos routes through the SAME seam; minimax_h3_gen+minimax_h3_mux are thin vllm.h clients, frames+WAV byte-identical to the pre-fold binary on the committed fold fixture (test_minimax_h3_video_fold 3-arm gate + the v12 test_capi section); GB10 real-video re-verify via the v12 ABI = named residual. ROW 2 DEVICE-SEAM FOLLOW-UP (#135; replaces #134): ABI 0/1 maps once to vt::DeviceType; shared code dispatches through GetBackend(device_type), restoring DSR 34→32 without a baseline/allowlist change; CPU compile/fold test pending in CI due shared-disk pressure PRUNED CHECKPOINTS LOAD AND RUN 2026-08-10 (row/H3-PRUNED, #241, spec §8.21): the community pruned variants (unsloth/MiniMax-H3-GGUF Q2_K..Q8_0, lilcheaty/MiniMax-H3-NVFP4 *_pruned_nvfp4) are not lossily pruned but ComfyUI's AdaLN timestep-CURVE refactor (comfy/ldm/minimax/model.py:419-432,610-615): an adaln_t_table [1025, 8] lerp replaces the sinusoidal+MLP time embedder, no SiLU before the AdaLN linear, and its in_features drop 2688 -> 8, collapsing adaln_proj from 13.04B of 33.12B to 0.04B and the DiT to 20.11B — so a pruned Q8_0 (21.4 GB) costs what our unpruned Q4_K_M (19.9 GB) costs. Gated on the REAL 532-tensor pruned GGUF manifest (532 = 535 - 4 time_embedder.* + 1 table, header-only), the clamped-lerp golden, and a CONSTRUCTED-curve test proving the pruned forward is numerically identical to the unpruned one; 79/79/57299. | MODEL-DIFFUSION-minimax-h3-mini-max-h3-dit |
| 🚧 | MiniMaxMusic3ForConditionalGeneration | MiniMax-Music3 (text-to-music: ~8.6B Qwen3 global LLM + 0.646B 8-codebook RVQ depth decoder + 2.4B fp32 flow-matching DiT + 0.054B DAC Flow-VAE) | W0 DONE + W1 LANDED (#672, spec). Diffusers oracle gateable with committed per-stage goldens (#708); modular loader in the tree, 1413/1413 assertions against the real checkpoint (#714). Geometry MEASURED from every component config.json plus safetensors headers by range request. First row to exercise the fallback-oracle rule (#647): absent from vLLM AND vLLM-Omni, so its primary oracle is the OPEN diffusers PR #14456 head c6da9936 with SGLang-Omni 748a0b43 as the e2e cross-check. W2/W3 LANDED: the AR half computes and is gated. W4/W5 LANDED: the ACOUSTIC half — flow-matching DiT, FlowMatchEulerDiscreteScheduler with invert_sigmas, the CFG mix, the denoise loop's window bookkeeping and the DAC Flow-VAE vocoder over the shared vocoder1d primitives — gated at reduced dimensions (27 cases / 265 assertions, no checkpoint) AND at full scale against the committed capture (scheduler BIT-EXACT 22 016/22 016; waveform 88 064 samples, 0 outside a tolerance calibrated against torch reproducing its own goldens). W6 LANDED: registered as a SpeechRegistry family (minimax-music3, detection INSPECTS modular_model_index.json), SpeechGenParams extended ADDITIVELY with lyrics/description plus three generation controls (IndexTTS-2.5 byte-identical, 0 lines changed), the new vllm_speech_* C-ABI surface at v20, and POST /v1/audio/speech on the example server as a thin ABI client. The denoise + decode composition reproduces the capture's waveform.npy — 88 064 values, 0 outside tolerance, max abs-diff 4.52e-06 — and the WAV payload is BIT-EXACT against the quantization of that golden. W7 LANDED: ONE ARM IMPLEMENTED AND VALUE-GATED, the rest refused. The RVQ depth decoder loads from GGUF Q4_K (audio-cpp/MiniMax-Music3-GGUF@c36aaeed, rvq_depth_decoder_q4_k.gguf, 405 752 480 B, sha256 4c5d41b2...c70cbdd0 — pinned because an unpinned quantized checkpoint is not reproducible): 47 tensors as 36 Q4_K + 9 BF16 + 2 F16, dequantized to bf16 through the SHARED gguf_dequant.h seam, lineage keyed on audiocpp.model_spec.family because general.architecture reads audiocpp/mm3/qwen3/wan for this ONE model and wan collides with real Wan video GGUFs. Gated 6 cases / 319 assertions against the W3 golden's 716 800 values at a bound DERIVED rather than chosen: worst per-tensor relative L2 0.0742 (bound 0.10), full-scale mean abs-diff 0.0324 / max abs-diff 0.3125 / 2.84% bit-identical. THE GATE-DESIGN FINDING, which generalizes past this row: an upper-bound-only tolerance CANNOT distinguish a real quantized arm from a silent dequant fallback, because the fallback is CLOSER to the golden — bf16 weights through the identical forward give mean abs-diff 0.00182, 17.8x nearer than the genuine Q4_K arm, so every plausible upper bound PASSES the failure. What catches it is a LOWER bound (5e-3, sitting between the two), gated by a positive control that actually runs the fallback and asserts the gate rejects it. Two further INDEPENDENT legs prove the quantized path was taken rather than inferring it: the RESIDENT ggml type of every tensor (loader bookkeeping — localizes a fault to a name, but a loader that lied about its tallies would pass it, which is why mutation QM2 exists) and the Q4_K LATTICE (a property of the DATA needing no loader cooperation — 0 of 524 288 32-element windows exceed 16 distinct values, against a bf16 control from the same file at 127 of 128). Second finding: the 2 F16 islands do NOT round-trip exactly (rel 3.23e-08 / 2.36e-08) because the quantizer re-encoded them BF16->F16 and F16's exponent range is NARROWER, flushing weights below ~6e-08 to zero — so the gate splits the islands, BF16 bit-exact and F16 bounded; asserting "unquantized means exact" for all 11 REDS a correct reader. 18 mutations across both layers all fire, one of them (QM4, the lineage guard) only after a COVERAGE HOLE it exposed was closed with synthetic GGUF fixtures. The 4 OTHER components' GGUF arms, the mm3 and ComfyUI lineages, and every non-GGUF format are REFUSED BY NAME and owed. Quantized MiniMax-Music3 checkpoints DO exist — a HuggingFace survey on 2026-08-14 found 14 community repositories in 5 formats (GGUF of all five components, int8/w4a8, MLX, plus a proprietary Cortiq arm), recorded query-by-query with result counts in spec §9.1. minimax_music3_quant.{h,cpp} is a SEPARATE translation unit per porting-a-model.md that diagnoses 8 formats at the THREE places a quantized checkpoint announces itself (tree .gguf, manifest sidecars plus the dtype-only fp8/int8 cases, and quantization_config/MLX quantization) and refuses each BY NAME with the evidence, the missing piece, the supported arm, the phase and the issue. 29 cases / 125 assertions without a checkpoint plus 6 cases / 319 assertions against the pinned artifact, RED first at 8/8 probe checks, 18 of 18 mutations fire; every tolerance is DERIVED from a printed measurement rather than chosen, and no speed number is claimed. GGUF headers MEASURED by range request (56 MiB, no weight bytes): "the GGUF arm" is THREE MUTUALLY INCOMPATIBLE LINEAGES and general.architecture cannot separate them (audiocpp/mm3/qwen3/wan, and wan collides with real Wan video GGUFs), the ComfyUI lineage ships the DiT + condition encoder ONLY so it can never generate audio, and comfy.gguf.orig_shape.* is absent from all ten files. The 8.6B W2's remainder LANDED (this PR): the 8.6B Qwen3ForCausalLM forward runs. Qwen3DenseModel::ForwardEmbeds gives the landed dense path the inputs_embeds entry its multimodal siblings already had (qwen3_vl.h:145,159, gemma4.h:210-218, muse_glimmer.h:369-380), additively and proved BIT-IDENTICAL to the token-id Forward in logits AND in the paged KV it writes. Teacher-forced on the capture's own codes, 25 steps reproduce frame_hiddens[:, :4096] - 102 400 values, 9337 bit-identical, mean abs-diff 1.763e-02 - INSIDE a matched control (upstream's own model under sdpa_kernel(MATH): 12 036 bit-identical, mean abs-diff 1.475e-02), with the per-step error FLAT rather than compounding. The oracle's OWN sampled semantic codes rank 2.48 on average under our guided logits where chance is 8191.5, and the gate carries its own NEGATIVE CONTROL: a one-step-shifted alignment reads mean abs-diff 0.803, 46x outside. The e2e case at POST /v1/audio/speech is OBSERVED TO PASS (2026-08-15, #852): 200 audio/wav, 12332 bytes, 2 AR frames -> 6 latent frames -> 3072 samples/channel of 44100 Hz stereo, 6144 int16 samples all non-zero, 0 clipped, 2818 of 3072 positions differing between L and R; 21 assertions in 7:54, and 5 cases / 535 assertions for the whole file in 31:14. It had never been seen because the request body posted audio_duration_s, the FIELD name, where ParseSpeechRequest reads audio_duration - so the key was silently dropped and the family's 60 s default substituted, making every run a ~750x job (1500 AR frames not 2, 8 denoise windows not 1, 5167 vocoder latents not 6) that could never have passed its own payload assertion. The near-miss key is now REFUSED (#925). The earlier diagnosis was wrong and the way it was wrong generalizes: open LM shard fds do NOT locate the program counter, because LoadBf16Direct BORROWS the mapping and a borrowed OwnedTensor keeps it alive for the whole request, while the depth decoder's tensors are COPIED so its fd closes on return. A symbol-resolved profile puts the LM weight load at 180 s (1 thread, state D, ~92 MB/s off the NAS) in BOTH binaries, and the cost in LinearNoBias 42-57% / Threadpool::Barrier 25% / vocoder1d::ConvTranspose1d 88.5%; the suspected 200000 x 4096 lm_head transpose is 1.388 s. The GGUF k-quant ARM for the remaining components is still owed; spec §5's token-exact gate is WITHDRAWN because upstream's AR stage has no greedy path PARITY SWEEP + MUSIC-ONLY SERVER LANDED (#672, #953): upstream's whole user-facing surface was enumerated field by field (SGLang-Omni 748a0b43 models/minimax_music3/, diffusers c6da9936 modular_pipelines/minimax_music3/) and recorded with file:line in spec §10.1. --model is now OPTIONAL when --speech-model is given, so vllm-server --speech-model <dir> serves /v1/audio/speech and registers NOTHING else — upstream's own shape (sgl-omni serve --model MiniMaxAI/MiniMax-Music3, config.py:29-63: three stages, no chat LLM), and the third instance of the task-conditional dispatch already in server_main.cpp for pooling and transcription-only checkpoints (vLLM api_server.py:255-265). ADDITIVE and proved: the ONLY case whose verdict changes is BOTH flags absent, which was an error and remains one with a message naming both options; the route table is gated in BOTH directions over a REAL SOCKET. New examples/minimax_music3_gen — the music family was the one generative family in this tree with NO example — a THIN C-ABI client (vllm_speech_engine_load + vllm_synthesize, include/vllm.h only). Five keys upstream refuses BY NAME were SILENT here and are now refused (#953): temperature/top_p/top_k/repetition_penalty (request_builders.py:14-19,109-114; the AR stage has ONE sampler, a fixed top-50 draw) and max_new_tokens (request_builders.py:56-68; upstream's length spelling in 25 Hz FRAMES, not seconds) — the #925 class exactly. The e2e gate no longer reports a SKIP WEARING A PASS: it read 5 cases / 5 passed with assertions: 0 when the checkpoint was absent, the same shape that fooled this project on test_qwen3_paged_engine. Split into a CHECKPOINT-FREE half that runs unconditionally in CI (request contract, both ceilings, the speech-only route table over a real socket with a stub synthesizer) and the env-gated checkpoint half, whose HTTP case now runs over a REAL SOCKET against the music-only server shape rather than calling handle_audio_speech; a coverage-report case prints which arms ran every run. No-env arm 9 cases / 37 assertions where it was 5 / 0. The WEIGHTS are documented per porting-a-model.md §2.1: docs/USAGE.md carries component-by-component tables for the diffusers arm at MiniMaxAI/MiniMax-Music3@fbdf52fbaaca799592917417eb05f1899f1255ec (28.5 GB resident, 28 517 617 303 B MEASURED, out of a 57.4 GB repository, and why they differ), the native .pth arm we refuse and SGLang-Omni serves, the one implemented GGUF Q4_K artifact with its sha256, and the 14 THIRD-PARTY quantized repositories in 5 formats each marked refused. The revision is VERIFIED not copied: condition_encoder/diffusion_pytorch_model.safetensors on disk hashes to 83179c5e…a202c2a4d, that revision's own LFS record. OWED and named rather than discovered: the non-wav response formats (no encoder vendored; note upstream DOWNMIXES TO MONO for every non-wav format, client/audio.py:328-334), request batching and /v1/audio/speech/batch (SGLang runs continuous batching at 16 with TWO engine rows per request for the CFG twin, engine_builder.py:74-77), and the 32 kHz delivery resample. Streaming is a PERMANENT refusal, not a debt: neither upstream arm has it (supports_streaming_vocoder=False). One place we are AHEAD of both arms: guidance_scale is a real per-request control here, where diffusers freezes it at 1.7 into the guider component (denoise.py:180) and SGLang exposes it only as a serve-time knob. DEVICE ARM LANDED (#672, this PR) — PARTIAL, and the split is the point. SpeechModelParams grew device (0 = CPU, 1 = the accelerator this build resolves; the VideoModelParams::device polarity, NOT vllm_model_params.device's 0=auto, because every Music3 gate was taken on the CPU arm and auto would move every zero-filled caller off it), resolved through a NEW SHARED multimodal::SpeechEngineDeviceType rather than a third copy of a mapping the tree already carried twice (minimax_h3_video.cpp:255, ltx2_video.cpp:706). The engine's vt::Queue was a COMPILE-TIME CONSTANT (minimax_music3_speech.cpp:492, whose own comment said "a device arm is a queue, not a fork"); it is now that queue, built once in the constructor BEFORE any weight is read (the unified-memory recipe). Music3LmSession allocates its 36-layer paged KV on the queue's device, because dense_attn::KvSlice labels the pointer with d.q.device and a host std::vector handed to a CUDA forward is a host pointer wearing a device tensor's label. C ABI v21: vllm_speech_model_params.device + vllm_speech_engine_device, which reports what was GRANTED not what was requested; --speech-device on the server, --device on minimax-music3-gen. WHAT MOVES is the 8.6B Qwen3ForCausalLM half and nothing else: it already rides the shared Qwen3DenseModel::ForwardEmbeds five text registrations use, so no model was forked. WHAT DOES NOT, named rather than implied: the 0.646B RVQ depth decoder and the 2.4B fp32 DiT are host std::vector<float> scalar loops under -ffp-contract=off (kept reproducible against torch by W2-W5) and are OWED; the DAC Flow-VAE vocoder needs ConvTranspose1d, and vt HAS NO SUCH OP — vt::Conv2d and vt::DepthwiseConv1d exist but are registered for the CPU only and neither expresses a transposed convolution — so it is recorded as owed rather than hand-rolled outside the seam. The CPU arm is BIT-IDENTICAL: it takes the same host code with the same std::vector KV, and every Music3 gate is unchanged. | MODEL-MUSIC-minimax-music3-mini-max-music3-for-conditional-generation |
| 📋 | LTX2VideoTransformer3DModel | LTX-2.5 (21.00B joint video+audio flow-matching DiT, Lightricks) | L0 spec committed 2026-08-11 (#435, spec). Geometry MEASURED from the FP8 checkpoint's own safetensors header by HTTP range request (6124 tensors, 881,048-byte header, no payload downloaded): 21.00B params — 48 blocks @ 386.7M = 18.560B, audio embeddings connector 2.016B, global 0.427B. The filename says 22b and the Diffusers card says ~19B; the MEASURED count is what this row uses. Video stream 4096 (32 heads x 128), audio stream 2048 (32 heads x 64), in/out channels 128 both. Per block SIX attentions — attn1 (video self), attn2 (video<->text, cross 4096), audio_attn1, audio_attn2 (cross 2048), and the two CROSS-MODAL audio_to_video_attn / video_to_audio_attn — which is the structural break from MiniMax-H3: H3 packs every modality into ONE sequence with per-row token tags, LTX runs TWO streams coupled by explicit cross-attention. Per-head gated attention on every attention (to_gate_logits = Linear(query_dim, heads, bias=True), attention.py:513-514, applied AFTER the attention output at :577) — H3 has no analogue and getting it wrong renders plausibly-wrong rather than erroring. FFN is gelu-approximate 4096->16384->4096 with NO bias while audio_ff 2048->8192->2048 HAS bias, which independently confirms ff_bias=false / audio_ff_bias=true from model_configurator.py:78-80 — checkpoint and source agreeing, not either alone. RETRACTED 2026-08-12, was billed as a FREE WIN. The spec claimed 2.5 sets use_prompt_adaln_single=false so the cross-attention K/V are timestep-free and cacheable. The shipped checkpoint DISPROVES it: it carries 12 prompt_adaln_single/audio_prompt_adaln_single tensors including a timestep_embedder.linear_1 [4096, 256] (256 = the sinusoidal timestep width), and model.py:223-227 builds that module ONLY when the flag is TRUE. transformer.py:441 was quoted as proof of no timestep term, but :442-443 add one whenever prompt_timestep is not None, and the comment above them says exactly that. NO SHIPPED DEFECT: ltx2_dit.cpp:672 refuses the cache by name when the flag is on, so the feature is correct-and-inapplicable rather than silently wrong, and stays gated bit-identical and prompt-bound for any checkpoint that does set it false. ORACLE: vLLM-Omni does NOT support 2.5 — resolve_ltx_pipeline_recipe keys on 2 / 2.3 only and RAISES otherwise (ltx2_recipes.py:162-166), with upstream vllm-omni#6066 filed 2026-08-11 — but its DiffusersAdapterPipeline is generic (DiffusionPipeline.from_pretrained, pipeline_diffusers_adapter.py:116), so vLLM-Omni CAN execute 2.5 via --load-format diffusers. Binding oracle = that adapter; immediate cross-check = Lightricks ltx-pipelines. HW: FITS ONE GB10 at ~29 GB (NVFP4 DiT 18.72 + NVFP4 Gemma-4 TE 7.40 + VAEs 1.83 + upscaler 1.00) vs H3's ~41 GB. OWED UP FRONT, not to be discovered later: the speed axis lands PENDING because the adapter is a black box (supports_step_execution=False, supports_request_batch=False) and therefore NOT vLLM's production configuration, which AGENTS.md requires as the denominator; DiffVAE (NADiffusionDecoder, neighborhood attention) is REFUSED BY NAME until its own row rather than silently downgraded to the Conv VAE; and no render-quality claim is made from structural e2e. | MODEL-DIFFUSION-ltx-2-5-ltx2-video-transformer-3d-model |
| ✅ | LagunaForCausalLM | Poolside Laguna-S-2.1 (118B/8B MoE) | LONG-CTX DECODE LEVERS LANDED + MEASURED (2026-08-03, CLAIM-LAGUNA-LONGCTX-LEVERS): window-bounded SWA reads (VT_LAGUNA_SWA_WINDOW, default-ON, BYTE-EXACT) bound the four DecodeAttnGqa* kernels' read to the ~512 sliding window (vLLM laguna.py:412) — GB10 A/B token-IDENTICAL =1 vs =0 at 520-token context (truncation active), MEASURED −0.30 ms/step at ~2k (~0 at ≤512, grows linearly). bf16 paged KV (VT_LAGUNA_KV_BF16, default-OFF opt-in) a distributional near-tie left UNRATIFIED. See BENCHMARKS CLAIM-LAGUNA-LONGCTX-LEVERS. — NVFP4 W4A4 ARM RAN on GB10 (N4, 2026-08-01, CLAIM-LAGUNA-NVFP4-N4): the additive safetensors NVFP4 arm (N1a/N1b/N2/N3 — Nvfp4Weight expert fields + LoadLagunaForCausalLMWeights + LqGemmNvfp4Fp4 per-expert TRUE-W4A4 + LagunaFfnBlock fp4 branch + laguna_gen dir-autodetect; CPU-gated test_laguna_nvfp4_loader 3/3·61, GGUF path byte-identical) generates COHERENTLY on the real 67 GiB poolside/Laguna-S-2.1-NVFP4. vs the vLLM MARLIN golden (vLLM's exact prompt ids injected): FIRST 2 TOKENS MATCH exactly, then near-tie divergence (our TRUE-W4A4 fp4-activations vs the MARLIN golden's W4A16 bf16-activations — different precision, EXPECTED; shares golden vocab). SPEED (N5, trace-driven, 2026-08-01): 0.16 → ~4.5 tok/s (~28× THIS SESSION), now ~4× from vLLM 18.8. Lever #2 (nsys found the bf16 tower running host MatmulNK on the CUDA queue): route it to the GPU (LqGemm bf16 → CastBf16 + MatmulBT, weight stays bf16) → 6.34 → 0.39 s/tok (16×). Lever #1 (nsys found the emulation expert GEMM at 92%, GPU 87% busy): the engine's native sm120a fp4 tensor-core MMA (MatmulNvfp4Fp4Native) reads the SAME linear scales — it was gated OFF behind VT_NVFP4_FP4_NATIVE; default it ON in the driver → 0.39 → ~0.20-0.24 s/tok (~2×). Both coherent + near-tie (byte-identical ids to emulation; first token matches golden). Two GB10 memory fixes landed to run (shard-release + context-before-load). OPEN #234 (remaining ~4×): grouped W4A4 MoE (top_k×3 launches → 3), ResidentNvfp4, decode CUDA-graph + on-GPU sampling (the host-orchestration tail). Spec .agents/specs/laguna-nvfp4-arm-2026-07-31.md §N4/§N5. The GGUF-Q4_K track (below) is the separate keep-quant vehicle. Prior FASTER DECODE (W9, 2026-07-31, CLAIM-LAGUNA-W9-GROUPED): the 30 un-grouped per-expert keep-quant GEMV launches/step (top_k × {gate,up,down} LqGemmRowSlice) fold onto the SHARED vt::MatmulBTQuantGrouped op — per token, Pk experts' gate/up/down each collapse to ONE grouped launch over the already-stacked [E*N,H] tower (no loader change). Same-binary A/B on real UD-Q4_K_XL (GB10, --gpu, drop_caches cold, 24 tok): grouped (VT_LAGUNA_GROUPED_MOE=1, default) == per-expert (=0) BYTE-IDENTICAL (md5 754728c6, both == W6 golden) + decode 0.18 → 0.13 s/tok (1.38×). Routes through the shared vt op (fold policy). Cumulative with W8: decode 0.66 → 0.13 s/tok (5.1×; 1.5 → 7.7 tok/s; 18× → 3.6× vs llama.cpp 27.8). Next lever: device-resident decode (#1). See spec §W9. Prior FASTER DECODE (W8, 2026-07-31, CLAIM-LAGUNA-W8-EMBED): LagunaEmbed no longer converts the whole 1.23 GB embed table to f32 every token (it gathered T rows out of the whole [Vsz,H] table via ReadF32 — ~311M host element-converts/token, the DOMINANT decode cost the W7 profile under-filed as "#5"); now gathers only the T needed rows directly (BIT-IDENTICAL — same per-element conversion, same rows). GATED on the real 3-shard UD-Q4_K_XL GGUF (GB10, --gpu, W6 cached, drop_caches cold, 24 tok): TOKEN-IDENTICAL to the W5/W6 golden (22345 83 350 785 …, coherent " Paris.") + decode 0.66 → 0.17 s/tok = 3.9× (1.5 → 5.9 tok/s; 18× → 4.7× vs llama.cpp 27.8). See .agents/specs/laguna-s21-w7-speed-2026-07-31.md §W8. Next: grouped-expert GEMM (=A3) then device-resident decode. Prior DECODE-SPEED ATTRIBUTED (W7 profile-only, 2026-07-31, CLAIM-LAGUNA-W7-SPEED): nsys of the W6 decode (real UD-Q4_K_XL GGUF, GB10) attributes the 0.66 s/tok (~1.5 tok/s vs llama.cpp 27.8 on identical bytes, ~15-18x) to HOST-ORCHESTRATION, not kernel compute — GPU active only 32.7% of the step, 67.3% host/idle; 22,115 cudaStreamSynchronize (~2,764/step, zero GPU overlap) from the ~1,795 per-GEMM DrainQueue in LagunaForwardGgufCached + scalar host glue; 39.4% of GPU time is QuantizeQ8K activation-quant (per-GEMM), weight GEMVs un-grouped at ~22% of the 240 GB/s peak (llama.cpp ~76%); no H2D/D2H (unified memory). Ranked levers (all in-tree from ds4): device-resident decode 1.5->~5-7 tok/s, grouped-expert GEMM (MatmulBTQuantGrouped) +1.5-2x + dedupes the activation-quant, decode CUDA-graph, tuned MMVQ; + free host cleanups (LagunaEmbed copies the whole 1.23 GB embed table/token, per-token RoPE-cache rebuild). Honest reachable ~13-20 tok/s, 27.8 a stretch. NO code changed. See .agents/specs/laguna-s21-w7-speed-2026-07-31.md. Prior RUNNABLE + FAST DECODE (W6, 2026-07-31): a per-layer K/V cache + single-token incremental decode replaces W5's O(n²) STATELESS recompute — TOKEN-IDENTICAL (byte-equal ids, md5 754728c6… match, == the W5 golden) and 5.05× faster per token: decode 3.33 → 0.66 s/tok on the real UD-Q4_K_XL GGUF (GB10, --gpu, keep-quant), same " Paris.…" text. LagunaKvCache (mirrors DeepseekV4KvCache, MLA-latent → GQA multi-head K/V; caches post-QK-RMSNorm/post-RoPE K + raw V at f32 — bit-exact since RoPE/QK-norm are position-only and attention is causal), MIXED attention per-layer: 12 GLOBAL layers grow unbounded + 36 SLIDING-WINDOW-512 layers EVICT rows beyond the 512 window (gemma2/3 is_sliding); LagunaForwardGgufCached + shared LagunaAttention/LagunaFfnBlock helpers used by BOTH forwards (identical float ops; recompute ids unchanged after refactor), examples/laguna_gen --stateless A/B flag. No cache bug (bit-exact first run). Next speed = grouped-expert GEMM + device-resident decode (both in-tree from ds4). See .agents/specs/laguna-s21-w6-2026-07-31.md. Prior RUNNABLE (W5, 2026-07-31): our engine greedy-generates COHERENT text on the REAL 3-shard UD-Q4_K_XL GGUF (GB10 keep-quant) — "The capital of France is" → " Paris. …", first token "Paris." matches the llama.cpp-Poolside reference. Multi-shard GGUF reader + keep-quant tower (LoadLagunaFromGgufShards) + LagunaForwardGguf (ds4 keep-quant Gemm/GemmRowSlice) + examples/laguna_gen; load 20.6s, peak 71 GiB, 3.27 s/tok stateless recompute (speed=W6). Prior W3: W3 REAL forward + 3 new ops landed (laguna_ops.cpp: per-head softplus attn out-gate + ungrouped sigmoid-noaux router + dual per-layer RoPE cos/sin builders; LagunaModel::Forward now a REAL runnable host-reference composition — variable-Q-head GQA + dual RoPE + sliding-window mask + softplus gate + dense L0 / ungrouped-MoE L1..47 + untied lm_head — replacing the W1/W2 VT_CHECK(false) stub; CPU -Werror full-library build clean; test_laguna_scaffold 8/8·166 incl. softplus math, router selection+tie-break RED-first, dual-RoPE cos/sin bit-match, variable-Q-head shapes, forward composition on synthetic weights; test_model_registry 24/24). W1 oracle DECISION: vLLM native laguna.py in pin ⇒ config constructs; dual-oracle = vLLM-NVFP4/-FP8 (fits GB10, BF16 235 GiB does NOT) + llama.cpp-Q4_K token-exact. DEFERRED to W4 (needs 73 GB checkpoint): GGUF keep-quant tower materialization + device/paged production forward + strict dual-oracle greedy gate. ~85-90% reuse (ds4-MoE + gemma-sliding + olmo3-dual-rope + landed Q4_K keep-quant); NEW = the 3 landed host ops + name-map + variable-Q-head device runner. W4 (2026-07-31, CLAIM-LAGUNA-W4, in progress): the UD-Q4_K_XL GGUF (73.4 GiB, 3 shards) FETCHED to dgx + its metadata/tensor-map READ AUTHORITATIVELY (814 tensors, arch laguna, expert_gating_func=2 sigmoid, leading_dense_block_count=1, expert_weights_scale=2.5). Three CPU-verified FIDELITY corrections the W1-W3 scaffold got wrong, each grounded in the real GGUF + llama.cpp: (1) per-head QK-RMSNorm (attn_q_norm/attn_k_norm F32[128]) added to params+forward — the scope MISSED it (surfaces only in the tensor map); (2) dual-RoPE mscale now uses llama.cpp's yarn_attn_factor·(1+0.1·ln(factor)) off the GGUF-authoritative factor=32/yarn_attn_factor=1.0 (256K-ctx build, NOT HF's factor-128/1.4852 1M-ctx scalar) — resolves the numerics-delicate residual; (3) separate ffn_gate_exps/ffn_up_exps (Q4_K) + ffn_down_exps (Q5_K) + Q8_0 shared/attn (the scaffold assumed merged gate_up). GGUF keep-quant tower materialization (Mw/Sew mirror of ds4) + keep-quant ForwardGguf (vt::MatmulBT/GemmRowSlice) + the real-model greedy run vs the llama.cpp-laguna same-quant oracle remain the W5 close (73 GB single-GB10, host-orchestrated) | MODEL-TEXT-laguna-laguna-for-causal-lm |
| 🚫 | DeepseekV3ForCausalLM / DeepseekV32ForCausalLM | DeepSeek-V3 / V3.2 | HW-blocked (671B, ~642 GiB fp8 vs 119 GiB unified memory); V3.2 additionally DEP-blocked (DSA indexer) | MODEL-TEXT-deepseek-v2-deepseek-v3-for-causal-lm |
| 🚫 | GlmMoeDsaForCausalLM | GLM-5 (DSA) | HW-blocked (1404 GiB bf16) and DEP-blocked (GLM-5.x is DeepSeek-V3.2 verbatim) | MODEL-TEXT-deepseek-v2-glm-moe-dsa-for-causal-lm |
| 🚫 | MiniMaxM2ForCausalLM | MiniMax-M2 | HW-blocked (~230B / ~428 GiB bf16, ~4x over unified memory) | MODEL-TEXT-minimax-m2-mini-max-m2-for-causal-lm |
| ✅ | GemmaForCausalLM | Gemma 1 (gemma-2b) | STRICT token-exact SACRED gate 48/48 greedy vs vLLM 0.25.0 (K=5 ALL-DETERMINISTIC → STRICT; BOS-verified; ungated unsloth/gemma-2b mirror). The original Gemma: two fused add+RMSNorm/layer, head_dim^-0.5 scale, GeGLU + sqrt(hidden) embed-scale, tied lm_head, no soft-cap/QK-norm/sliding; reuses the W1 GeGLU/embed-scale primitives; speed pending | MODEL-TEXT-gemma-gemma-for-causal-lm |
| ✅ | Gemma2ForCausalLM | Gemma 2 (gemma-2-2b-it) | near-tie-band SACRED gate 48/48 vs vLLM 0.25.0 (44/48 strict on vLLM's greedy + 4/48 at gap 0.0000 nats in vLLM's OWN teacher-forced logits — pure argmax-tiebreak ties, 0 forward-divergent; vLLM K=5 self-deterministic; BOS-verified; ungated unsloth/gemma-2-2b-it). PROVES the soft-cap primitives: attn_logit_softcapping 50 threaded through PagedAttentionArgs into paged attention (verified applied by a cap-on≠cap-off same-binary A/B + unit + CPU differs-test) + final logit soft-cap 30 (monotone); the inverse of Gemma-3 (BOTH soft-caps, NO QK-norm, single rope); speed pending | MODEL-TEXT-gemma2-gemma2-for-causal-lm |
| ✅ | Gemma3ForCausalLM | Gemma 3 (gemma-3-1b-it) | FIRST Gemma family; tokenizer-free SACRED MODEL gate STRICT token-exact 48/48 greedy tokens vs vLLM 0.25.0 (K=5 per-prompt ALL-DETERMINISTIC → STRICT bar; BOS-verified); GeGLU (kGeluAndMul) + dual per-layer rope theta + Gemma-RMSNorm sandwich norms + sqrt(hidden) embed-scale + qpas scaling; speed pending | MODEL-TEXT-gemma3-gemma3-for-causal-lm |
| 🚫 | Gemma4ForCausalLM | Gemma 4 (newest registered; PLE+YOCO+MoE) | BLOCKED as a STANDALONE bare-text vehicle, but the ORACLE-RUNS sub-blocker is REFUTED (W0 2026-07-28, CLAIM-GEMMA4-W0): vLLM 0.25.0 loads+runs+generates the Gemma-4 PLE/YOCO/Gemma-4-MoE backbone e2e INSIDE the unsloth/gemma-4-E4B-it mm wrapper (STRICT greedy golden), so the earlier "0.25.0 oracle-support UNVERIFIED (likely absent)" worry is disproven. Residual block: still NO standalone bare-text Gemma4ForCausalLM checkpoint (all public ckpts mm-wrapped; text extractable via text_config), and the backbone needs a PLE/YOCO/Gemma-4-MoE/k_eq_v/double-wide-MLP campaign no other row uses. G1 (CLAIM-GEMMA4-G1, 2026-07-28) IMPLEMENTED the E4B text backbone (PLE/YOCO/plain-RMSNorm/proportional-rope/GeGLU/per-layer-scalar; MoE/k_eq_v/double-MLP off for E4B) as additive files loaded via the mm-wrapped unsloth/gemma-4-E4B-it text_config — compile-clean + loader header-verified, but e2e-gate BLOCKED on the runner's uniform-KV-head-dim topology (heterogeneous 256/512, runner.cpp:600-646). Row STAYS BLOCKED as a bare-text vehicle (still no standalone Gemma4ForCausalLM checkpoint). W6 honesty test_gemma4_honesty 6/6 unchanged | MODEL-TEXT-gemma4-gemma4-for-causal-lm |
| ✅ | Olmo2ForCausalLM / Olmo3ForCausalLM | OLMo-2 / OLMo-3 dense (pure post-norm + full-width QK-norm) | FIRST OLMo family; SACRED gate 16/16 vs vLLM 0.25.0 on allenai/OLMo-2-0425-1B (STRICT token-exact 13/16 + near-tie-band 3/16, max teacher-forced gap 0.094 nats, 0 forward-divergent; vLLM K=5 self-deterministic → STRICT bar; NO-BOS-verified real ByteLevel-tokenizer gate). ZERO new compute kernel: pure post-norm norm_after = standalone vt::RmsNorm on each sublayer output + plain vt::Add, NO pre-norm (the GLM-4 output-norm op, dropped pre-norms); full-width QK-norm = two standalone vt::RmsNorm over [T,q_size]/[T,kv_size] before NeoX rope (NOT the per-head fused recipe). Olmo-3 (Olmo3ForCausalLM) W5 is IMPLEMENTED on this same row (config-gated dual rope: plain sliding theta 500000 + YaRN full-attn, per-layer layer_types sliding window; our engine loads+runs allenai/OLMo-3-1025-7B) but is oracle-BLOCKED — NO SACRED bar: RUN-VERIFIED 2026-07-26 (W0), vLLM 0.25.0 CANNOT build the model (olmo2.py:143 config.rope_parameters["rope_theta"] → KeyError, full traceback captured on dgx) because transformers 5.13.1's Olmo3Config.rope_parameters is a NESTED per-layer-type dict ({sliding_attention:{…}, full_attention:{…}}), not the flat rope_theta dict 0.25.0 expects. Config-CONSTRUCTION succeeds (AutoConfig builds Olmo3Config); model-RUN fails — the earlier "CONSTRUCTS ⇒ unblocked" note conflated the two. Gate awaits a vLLM/transformers oracle advance. speed pending | MODEL-TEXT-olmo2-olmo2-for-causal-lm |
| ✅ | Phi3ForCausalLM | Phi-3 / Phi-4 dense (Phi3ForCausalLM) | Correctness-complete + gated (RATIFIED near-tie ROOT-divergence methodology). Impl: include/vllm/model_executor/models/phi3.h + src/vllm/model_executor/models/{phi3,phi3_weights,phi3_registry}.cpp. Pre-norm Llama forward + PRE-FUSED qkv/gate_up loader + precomputed LongRoPE cache (partial rotary 96/128) fed to RopeFromCache by REAL positions. Gate test_phi3_paged_engine (both checkpoints vLLM K=5 ALL-DETERMINISTIC): Phi-4-mini-instruct 16/16 by root divergence (7 strict + 9 near-tie; every ROOT/first divergence <=0.5 nats, 0 forward-divergent; p12's only >0.5 positions tok6/tok8=1.0 are CASCADE downstream of an exact-tie root tok4 gap 0.0000 where vLLM's own teacher-forced argmax IS our token). RATIFIED BIGGER-DENSE STRICT anchor microsoft/phi-4 (14B) 16/16 with STRICT token-exact 14/16 + 2 exact-tie (gap 0.0000, our token IS vLLM's argmax) -- the deterministic proof the forward is genuinely correct (not near-tie-lucky). RED-first: disabling the LongRoPE mscale produces 11 ROOT-fails (0.75-3.0 nats) the gate CATCHES (5/16). LongRoPE cos/sin cache BIT-IDENTICAL to vLLM's (.to(bf16), mscale 1.1902380714238083). memcheck 0 errors. Speed pending. | MODEL-TEXT-phi3-phi3-for-causal-lm |
| ✅ | GraniteForCausalLM | Granite-3 dense (GraniteForCausalLM) | IMPLEMENTED + GATED (batch3, new files): Llama + 4 default-1 scalar multipliers (embedding 12.0 / residual 0.22 / attention 0.015625=1/64 not 1/sqrt(dh) / logits 8.0). SACRED gate on ibm-granite/granite-3.3-2b-instruct = 16/16 (vLLM K=5 ALL-DETERMINISTIC → STRICT; 15/16 token-exact + 1 near-tie 0.062 nats, 0 forward-divergent). ZERO-NEW-KERNEL (reuses shared dense glue + vt::MulScalar/vt::Add); speed pending | MODEL-TEXT-granite-granite-for-causal-lm |
| ✅ | StableLmForCausalLM | StableLM dense (StableLmForCausalLM) | IMPLEMENTED + GATED (new files only): nn.LayerNorm (weight+bias, NON-fused residual — REUSE vt::LayerNorm) + partial NeoX RoPE (rotary_dim 16 of head_dim 64, partial_rotary_factor 0.25 — REUSE vt::RopeFromCache plain cache) + merged qkv BIAS (use_qkv_bias=True — REUSE vt::Add broadcast) + SiLU-SwiGLU. ZERO NEW KERNEL. SACRED gate stabilityai/stablelm-2-1_6b 16/16 vs vLLM 0.25.0 (per-prompt K=5 ALL-DETERMINISTIC ⇒ STRICT bar: 14/16 token-exact + 2/16 near-tie-band, max teacher-forced gap 0.438 nats, 0 forward-divergent; RED: disabling qkv bias → prompt0 tok0 wrong). memcheck 0 errors. Arcade100k cl100k-family split regex (literal CR/LF variant) accepted via a canonicalization in the shared tokenizer (inert: pretokenizer 97926/97926 + tokenizer-parity 1175/1175). speed pending | MODEL-TEXT-stablelm-stablelm-for-causal-lm |
| ✅ | MiniCPMForCausalLM | MiniCPM dense (openbmb/MiniCPM-2B-sft-bf16) | SACRED gate PASSES 16/16 vs vLLM 0.25.0 (K=5 ALL-DETERMINISTIC → STRICT bar; 10/16 token-exact + 6/16 near-tie band, max teacher-forced gap 0.0 nats, 0 forward-divergent). ZERO-NEW-KERNEL: Llama dense forward + THREE scalars threaded (scale_emb 12 after embed via vt::MulScalar; scale_depth/sqrt(40)=0.2214 scaled residual add per sublayer via vt::MulScalar+vt::Add; hidden/scale_width 9.0 = hidden_size/dim_model_base before lm_head), tied lm_head — reuses the shared dense glue exactly like Granite. RED: dropping scale_depth → 256/256 positions divergent, max gap 29.375 nats (gate CATCHES). W0 .bin-only vehicle resolved by trusted torch .bin→safetensors (same weights, both oracle+engine); gated via oracle prompt ids (TokensPrompt path — MiniCPM SP normalizer not ported). memcheck 0 errors. Speed pending | MODEL-TEXT-minicpm-mini-cpmfor-causal-lm |
| ✅ | InternLM2ForCausalLM | InternLM2 dense (internlm2-chat-1_8b) | SACRED 16/16 vs vLLM 0.25.0 (K=5 deterministic → STRICT; 12/16 token-exact + 4/16 near-tie all gap 0.0 nats, 0 forward-divergent). ZERO new kernel: reuses the landed Llama/Qwen3-dense forward VERBATIM; the ONLY delta is a loader-side fused-wqkv INTERLEAVED de-split (RED-proven load-bearing). Gated via oracle prompt token-ids (InternLM2's non-standard fast BPE not ported — orthogonal). Speed pending | MODEL-TEXT-internlm2-intern-lm2-for-causal-lm |
| 🚫 | CohereForCausalLM | Command-R / Cohere (CohereForCausalLM) | IMPLEMENTED (new files only, ZERO-NEW-KERNEL) but SACRED gate BLOCKED — no runnable vehicle. Deltas ported + grounded in commandr.py: weight-only mean-centred Cohere LayerNorm (REUSE vt::LayerNorm NULL bias) + GPT-J full-width RoPE (REUSE vt::RopeFromCache is_neox_style=false) + PARALLEL residual (ONE norm feeds attn AND mlp, h=residual+attn+mlp) + logit_scale scalar (REUSE vt::MulScalar) + tied embeddings + SiLU-SwiGLU. Compiles + links + self-registers (CPU -Werror clean, host TUs). W0 oracle RUN-VERIFIED (hf-internal-testing/tiny-random-CohereForCausalLM builds+runs on vLLM 0.25.0, greedy golden captured; arch confirmed CohereForCausalLM NOT Cohere2ForCausalLM) but the SACRED gate cannot be produced: every REAL small CohereForCausalLM (aya-expanse-8b / aya-23-8B / c4ai-command-r-v01) is HF-GATED (dgx has no token), the only ungated vehicles are tiny-RANDOM (head_dim 8/2, outside our validated FA2/paged-attn), and dgx is disk-full (26 GiB free, no build tree). Gate awaits an ungated real checkpoint / HF token + dgx disk. | MODEL-TEXT-commandr-cohere-for-causal-lm |
| ✅ | PhiForCausalLM | Phi-1 / Phi-2 (PhiForCausalLM, microsoft/phi-2) | SACRED gate PASSES 16/16 vs vLLM 0.25.0 (9 STRICT token-exact + 7 near-tie band, max gap 0.25 nats < 0.5, 0 forward-divergent); ZERO-NEW-KERNEL (gelu_new reuses the landed vt::GeluTanh, not a new op) — parallel-residual nn.LayerNorm(+bias) + biased qkv/dense/fc1/fc2 + partial NeoX rope + non-gated NewGELU MLP + untied biased lm_head; F16→BF16 dtype-aware loader (local). DISTINCT from Phi3ForCausalLM. Speed pending | MODEL-TEXT-phi-phi-for-causal-lm |
| ✅ | MiniCPM3ForCausalLM | MiniCPM3 dense+MLA (openbmb/MiniCPM3-4B) | SACRED gate PASSES 16/16 vs vLLM 0.25.0 (K=5 ALL-DETERMINISTIC → STRICT bar; 13/16 token-exact + 3/16 near-tie band, max teacher-forced gap 0.0 nats, 0 forward-divergent). ZERO-NEW-KERNEL: the landed MiniCPM 3 scalars (scale_emb 12, scale_depth/sqrt(62), hidden/scale_width=2560/256=10) with attention swapped GQA→MLA, REUSING the landed DeepSeek-V2 MLA block (mla::ForwardMlaAttentionBlock + load-time kv_b_proj absorption). Shared MLA seam: MlaBlockDims::is_neox_style (default false → DeepSeek byte-identical; MiniCPM3 true) + LongRoPE cos/sin cache (phi3_long_rope, short cache, mscale 1.0). FA-2 MLA prefill extended to zero-pad qk_head_dim 96→128 (reuses the compiled hdim128 kernel; DeepSeek d=192/GLM d=256 unchanged). RED: wrong rope style (is_neox false) → first-token divergence (gate CATCHES). DeepSeek-V2-Lite non-regression re-gated 8/8. W0 .bin-only vehicle resolved by trusted torch .bin→safetensors; gated via oracle prompt ids (TokensPrompt path). memcheck 0. Speed pending (eager; no decode CUDA-graph yet) | MODEL-TEXT-minicpm3-mini-cpm3-for-causal-lm |
| 🚧 | Gemma4ForConditionalGeneration | Gemma-4 multimodal (text STRICT; image mm e2e near-tie GREEN via the registered forward; vision+audio towers gated-in-isolation; audio e2e + STRICT-18/18 + speed pending) | TEXT PATH STRICT 32/32 TOKEN-EXACT vs vLLM 0.25.0 (G1b LANDED 2026-07-28, CLAIM-GEMMA4-G1B). unsloth/gemma-4-E4B-it loads through our engine + greedily emits the EXACT 32 golden token ids (tests/parity/goldens/gemma4_e4b_text/gen_manifest.json); gate tests/parity/test_gemma4_paged_engine.cpp (dgx CUDA, flock). The G1-named blocker is CLEARED: the runner now allocates a PER-LAYER KV head_dim (KVCacheConfig::per_layer_attn_specs, runner.cpp initialize_kv_cache) so Gemma-4's heterogeneous 256 (sliding) / 512 (global) heads each get a correctly-strided cache — BYTE-NEUTRAL for every uniform-KV model (empty ⇒ old path; full CPU runner/KV suite green + OLMo-2 SACRED GPU re-gate 16/16 UNCHANGED). Three additive loader gaps found+fixed on the way to the first-ever forward: nested per-layer rope_parameters (config loader), the Gemma Replace(" "→"▁") metaspace normalizer (tokenizer), and reading Gemma-4 scalars from raw["text_config"] not the full config (the real 256/512 head_dim source). Primitives grounded 1:1 in gemma4.py: PLAIN RMSNorm, PLE, YOCO KV-sharing, proportional partial-RoPE, GeGLU, per-layer scalar, final soft-cap 30, tied lm_head. G2 IMAGE ORACLE + PORT MAP LANDED 2026-07-28 (CLAIM-GEMMA4-G2, honest partial): the unsloth/gemma-4-E4B-it IMAGE→text greedy golden is captured STRICT (K=5 deterministic — same bar as Qwen3-VL image; 18 tokens → coherent gradient-image description, 256 soft tokens, tests/parity/goldens/gemma4_e4b_image/) with the four staged vision refs (proc/patch-embed/encoder/pooled/projected) for M2a-style unit-gating, and the SigLIP/NaFlex tower is fully port-mapped (spec §G2). ★ CORRECTION: the tower is a custom NaFlex SigLIP2 WITH multidim vision-RoPE + q/k/v-norm + Gemma2 sandwich norms + learned-2D pos-embed + avg-pool-by-position pooler — the earlier §0.1 "no vision-RoPE, simpler than Qwen3-VL" claim is REFUTED. G2-impl LANDED 2026-07-28 (CLAIM-GEMMA4-G2-IMPL): the C++ NaFlex SigLIP2 vision TOWER is implemented (additive gemma4_vision.{h,cpp}) and PASSES all four per-stage gates vs the transformers-eager refs (dgx CUDA flock): patch-embed relL2 2.15e-3, encoder 3.14e-2, pooled 1.36e-2, projected 1.85e-2 (220/220, compute-sanitizer 0). ★ Port-map correction: E4B use_clipped_linears=True with FINITE trained QAT clamps on the 7 encoder linears (implemented). Multidim vision-RoPE via 2× vt::RopeFromCache sharing one cache; attn scale 1.0; NO new vt op (reused MatmulBT/RmsNorm/RopeFromCache/AttentionDenseFlash/GeluAndMul). Text test_gemma4_paged_engine STRICT 32/32 UNCHANGED on the linking binary. Image e2e NOT yet gated — C++ NaFlex image processor + engine mm-plumbing (merge/hasher/encoder-cache/decode-fork) are the named residual. G3 LANDED 2026-07-28 (CLAIM-GEMMA4-G3): the C++ USM-Conformer AUDIO tower + audio projector are implemented (additive gemma4_audio.{h,cpp}) and PASS all 7 per-stage gates f32-exact vs the transformers-eager refs (host f32, 1256/1256): subsample 5.4e-7, posemb 8.9e-8, block0 4.2e-7, block_mid 3.8e-7, block_last 4.4e-6, output_proj 5.9e-6, projected 6.3e-6. Chunked-local attn (Transformer-XL rel-shift + softcap 50 + per_dim_scale softplus), GLU + depthwise causal conv (k5), half-step FFNs, FINITE QAT clamps; RED-first (wrong 13-key window → 2.8e-2, fixed 12-key → 1e-6); NO new vt op (host f32). Golden tests/parity/goldens/gemma4_e4b_audio/. Audio e2e (feature extractor + engine merge) is the named residual. MM-IMAGE-E2E LANDED 2026-07-29 (CLAIM-GEMMA4-MM-E2E): the SigLIP2 vision tower is FOLDED into the ENGINE registered forward (supports_multimodal=true + the mm branch in gemma4_registry.cpp → Gemma4Model::ForwardMm; projected soft tokens masked-scattered into the <image> rows via ModelForwardInput.mm, PLE image-rows→0). Image→text through ModelRegistry::Forward (driver Gemma4GenerateGreedyViaRegistry, gemma4_mm.cpp) is dgx-GREEN: 16/18 content tokens BIT-EXACT vs the STRICT gemma4_e4b_image golden (the full sentence), the single divergence being the terminal "."↔"," at a bf16 NEAR-TIE (margin ~0.10-0.12 logit), INVARIANT to vision-input precision (live C++ bf16 tower AND committed f32 ref_projected diverge identically ⇒ backbone bf16-accumulation, not the fold). Text SACRED 32/32 UNCHANGED (inertness). Gate tests/vllm/multimodal/test_gemma4_registry_e2e.cpp. Row ACTIVE: text STRICT + image mm e2e (near-tie) gated + all three towers gated-in-isolation; residuals = STRICT-18/18 (bit-match vLLM prefill bf16), audio e2e (mel A1 + merge), speed | MODEL-MM-gemma4-mm-gemma4-for-conditional-generation |
| 📋 | Gemma4UnifiedForConditionalGeneration | Gemma-4 unified multimodal (encoder-free) | Scoped in the Gemma-4 mm + audio track — GATEABLE (W0 2026-07-28, CLAIM-GEMMA4-W0): the sibling Gemma4ForConditionalGeneration W0 RUN-VERIFIED on vLLM 0.25.0 proves the shared registered mm path loads+runs+generates; the ungated unsloth/gemma-4-12b-it (23.92 GB) IS this encoder-free variant (Gemma4UnifiedVisionEmbedder, no SigLIP/audio AutoModel tower) and fits GB10 (no HF token needed). Oracle block RETIRED; IMPLEMENTATION-blocked only, staged behind the Gemma-4 backbone. Row stays SPIKE, not implemented | MODEL-MM-gemma4-unified-gemma4-unified-for-conditional-generation |
Row contract
MODEL-*IDs are deterministic: category prefix plus normalized upstream module and class. Never renumber them. If upstream retires a target, move its closed row to.agents/completed/and reserve the ID.Spikerecords coverage evidence;✅means an implementation spec exists with upstream code/dependency anchors, weight and quantization mapping, deviation record, gates, A/B plan, and a complete Tests to port inventory.StatefollowsINVENTORIED -> SPIKE -> READY -> ACTIVE -> GATING -> DONE. Partial modality support staysPARTIAL. Shared layers are dependencies, not model support.Ownerisunassignedor oneagent/path @ branch. Claim the row before implementation. One owner per ID; agents may share a dependency spike but not independently implement the same row.Evidenceisnoneuntil real local model code and tests exist. Exact code and test anchors are mandatory for partial or complete support.- Dependency cells are source-scanned hypotheses. The target spike must inspect
the whole upstream execution chain and correct the cell before becoming
READY.
Static summary and invariants
| Block | Category | Architecture memberships | Implementation targets | Registry anchor |
|---|---|---|---|---|
MODEL-TEXT | Text generation | 130 | 114 | vllm/model_executor/models/registry.py:71-208 |
MODEL-EMBED | Embedding | 40 | 28 | vllm/model_executor/models/registry.py:210-262 |
MODEL-LATE | Late interaction | 11 | 9 | vllm/model_executor/models/registry.py:264-278 |
MODEL-REWARD | Reward | 3 | 3 | vllm/model_executor/models/registry.py:280-284 |
MODEL-TOKCLS | Token classification | 4 | 4 | vllm/model_executor/models/registry.py:286-300 |
MODEL-SEQCLS | Sequence classification | 10 | 9 | vllm/model_executor/models/registry.py:302-329 |
MODEL-MM | Multimodal | 115 | 107 | vllm/model_executor/models/registry.py:331-583 @ 702f481 |
MODEL-SPEC | Speculative decoding models | 46 | 38 | vllm/model_executor/models/registry.py:586-638 @ 702f481 |
MODEL-HFALIAS | Static Transformers aliases | 4 | 2 | vllm/model_executor/models/registry.py:635-645 |
MODEL-HFBACKEND | Generic Transformers backend | 10 | 10 | vllm/model_executor/models/registry.py:647-680 |
The tables assert 373 category memberships, 356 unique static architecture
IDs, 324 category/target rows, 310 unique (module, class) targets,
261 implementation modules, and 17 architecture IDs present in more than
one task category. The embedding dictionary comprehension at registry.py:225-230
is expanded exactly as upstream does. Later dictionaries overwrite duplicate IDs
with identical targets when _VLLM_MODELS is assembled at registry.py:682-693.
Generation check: parse the ten registry dictionaries with Python ast, expand
the embedding comprehension from text targets whose implementation class is
LlamaForCausalLM, merge in _VLLM_MODELS declaration order, group each
category by (module, class), and assert all six invariants above. Dynamic
Transformers compatibility is capability-driven and excluded from finite counts.
Cross-cutting factory spike
| ID | Item | Upstream | Our code + tests/evidence | Spike/spec | State | Owner |
|---|---|---|---|---|---|---|
MODEL-FACTORY-registry | Type-erased architecture-to-factory contract, explicit unsupported-architecture rejection (including subset-supported-list formatting), capability metadata, task-aware loader/runner construction. Self-registration reshape LANDED (extensibility item 5, CLAIM-MODEL-SELFREG-1): the fixed constexpr std::array kRegistrations is replaced by a REGISTER_VLLM_MODEL(...) static-Registrar idiom (copies RegisterOp/RegisterBackend/RegisterPlatform); each architecture registers itself from its own TU into the shared ModelFactory. Registration order across TUs is unspecified under C++ static init, so the registry imposes a stable canonical sort by architecture name on first query (deviation §9 note 8) — resolution (first config-arch match) is order-independent. GATING: CPU build/test suite green; both gate-model greedy + no-regression campaign deferred while CLAIM-SERVE-GATE-1 owns dgx | registry.py:998-1082,1166-1190,1244-1296,682-693,1396-1404 | generic registry include/vllm/model_executor/models/model_registry.h:41-189 (self-reg seam RegisterModel/ModelRegistrar/REGISTER_VLLM_MODEL :167-189) + src/vllm/model_executor/models/model_registry.cpp:30-320; per-arch entry-point TUs src/vllm/model_executor/models/qwen3_5_dense.cpp (REGISTER :141) + qwen3_5_moe.cpp (REGISTER :132) over shared qwen3_5_common.{h,cpp} (ParseConfig/MakeKVCache/HostLogits/ModelInfo/BorrowedWeightsTag); live loader src/vllm/entrypoints/model_loader.cpp:184-223; runner indirection src/vllm/v1/worker/gpu/runner.cpp:252-303,546-561; ported tests tests/vllm/models/test_model_registry.cpp (138 assertions + one tracked skip; adds self_registration case) and live reject tests/vllm/test_model_loader_gguf.cpp:54-72; ledger .agents/parity-ledger.md:296 + 2026-07-19 self-registration row (DGX-confirmed 27B 235/235 + 35B 315/315 token-exact @ 669679a) | ✅ model-factory-registry.md | GATING | - |
MODEL-TEXT - Text generation
| ID | Item / architecture aliases | Upstream | Task / modality | Dependencies | Spike/spec | State | Our code + tests/evidence | Owner |
|---|---|---|---|---|---|---|---|---|
MODEL-TEXT-afmoe-afmoe-for-causal-lm | AfmoeForCausalLM | registry.py:73; vllm/model_executor/models/afmoe.py::AfmoeForCausalLM | causal generation / text | model loader/forward; paged attention/KV; FusedMoE/grouped GEMM; sliding-window attention | ☐ required | INVENTORIED | none | unassigned |
MODEL-TEXT-apertus-apertus-for-causal-lm | ApertusForCausalLM | registry.py:74; vllm/model_executor/models/apertus.py::ApertusForCausalLM | causal generation / text | model loader/forward; paged attention/KV; sliding-window attention | ☐ required | INVENTORIED | none | unassigned |
MODEL-TEXT-arcee-arcee-for-causal-lm | ArceeForCausalLM | registry.py:75; vllm/model_executor/models/arcee.py::ArceeForCausalLM | causal generation / text | model loader/forward | ☐ required | INVENTORIED | none | unassigned |
MODEL-TEXT-arctic-arctic-for-causal-lm | ArcticForCausalLM | registry.py:76; vllm/model_executor/models/arctic.py::ArcticForCausalLM | causal generation / text | model loader/forward; paged attention/KV; FusedMoE/grouped GEMM | ☐ required | INVENTORIED | none | unassigned |
MODEL-TEXT-axk1-axk1-for-causal-lm | AXK1ForCausalLM | registry.py:77; vllm/model_executor/models/AXK1.py::AXK1ForCausalLM | causal generation / text | model loader/forward; paged attention/KV; FusedMoE/grouped GEMM; MLA/latent KV | ☐ required | INVENTORIED | none | unassigned |
MODEL-TEXT-bailing-moe-bailing-moe-for-causal-lm | BailingMoeForCausalLM | registry.py:78; vllm/model_executor/models/bailing_moe.py::BailingMoeForCausalLM | causal generation / text | model loader/forward; paged attention/KV; FusedMoE/grouped GEMM | ☐ required | INVENTORIED | none | unassigned |
MODEL-TEXT-bailing-moe-bailing-moe-v2-for-causal-lm | BailingMoeV2ForCausalLM | registry.py:79; vllm/model_executor/models/bailing_moe.py::BailingMoeV2ForCausalLM | causal generation / text | model loader/forward; paged attention/KV; FusedMoE/grouped GEMM | ☐ required | INVENTORIED | none | unassigned |
MODEL-TEXT-bailing-moe-linear-bailing-moe-v25-for-causal-lm | BailingMoeV2_5ForCausalLM | registry.py:80; vllm/model_executor/models/bailing_moe_linear.py::BailingMoeV25ForCausalLM | causal generation / text | model loader/forward; FusedMoE/grouped GEMM; Mamba/SSM state; GDN/linear-attention state; MLA/latent KV | ☐ required | INVENTORIED | none | unassigned |
MODEL-TEXT-bailing-moe-v3-bailing-moe-v3-for-causal-lm | BailingMoeV3ForCausalLM | BEYOND-PIN (pin-lag) — NOT registered in 555967922; present on vLLM main at registry.py:82 (_TEXT_GENERATION_MODELS) → module vllm/model_executor/models/bailing_moe_v3.py, class BailingMoeV3ForCausalLM (1495 lines, 9 classes), with its own MTP draft registered at main registry.py:654 → module bailing_moe_v3_mtp.py, class BailingMoeV3MTPModel. Module and class are given as separate fields rather than in the compact colon-pair form: that form marks a target inside the PINNED registry inventory, and this row is beyond the pin — exactly how the MuseGlimmerForConditionalGeneration, KimiK3ForConditionalGeneration and MiniMaxH3DiTModel rows already render theirs. Succeeds BailingMoeV2_5ForCausalLM but is a NEW MODULE, not a class rename in the old one: V2.5 at the pin (bailing_moe_linear.py, 821 lines) mixes MLA full attention with Bailing linear attention (vllm/model_executor/layers/mamba/linear/bailing_linear_attn, imported bailing_moe_linear.py:32), whereas V3 keeps MLA but swaps the linear arm for Kimi Delta Attention (vllm/third_party/flash_linear_attention/ops/kda — chunk_kda, fused_kda_gate, fused_recurrent_kda, imported bailing_moe_v3.py:74-79) with a per-projection causal conv1d (:650-671), adds grouped-topk MoE through config.n_group (:1140), a clamped SwigluStepAndMul where a per-layer swiglu_limit is configured (:387, :1107-1135), an fp8 quant-config hook _configure_ling_fp8_quant_config (:215, called :1354), and REFUSES a checkpoint without no_kda_lora=True (:603-604). Recipe inclusionAI/Ling-3.0-flash declares architectures: ["BailingMoeV3ForCausalLM"]. Tracked by #609 | causal generation / text | model loader/forward; FusedMoE/grouped GEMM; KDA/linear-attention state; causal conv1d; MLA/latent KV; fp8 | ☐ required | INVENTORIED | none — cannot advance past this state until the pin advances to a revision that registers it | unassigned |
MODEL-TEXT-bloom-bloom-for-causal-lm | BloomForCausalLM | registry.py:81; vllm/model_executor/models/bloom.py::BloomForCausalLM | causal generation / text | model loader/forward; paged attention/KV | ☐ required | INVENTORIED | none | unassigned |
MODEL-TEXT-chatglm-chat-glmfor-causal-lm | ChatGLMModel, ChatGLMForConditionalGeneration | registry.py:82-83; vllm/model_executor/models/chatglm.py::ChatGLMForCausalLM | causal generation / text | model loader/forward; paged attention/KV; legacy ChatGLM2/3 lineage — hand-written attention with multi_query_group_num (chatglm.py:60-63) and a HARDCODED partial rotary factor 0.5 (:103, applied :131), on an out-of-tree config requiring trust_remote_code (:34). Shares only the NEW partial-rotary primitive with the rest of GLM. Smallest genuine checkpoint zai-org/chatglm3-6b 6.2B / 23.3 GiB fp16 — FITS GB10. Lowest priority in the campaign (spike W9), deferrable without blocking anything | glm-dsa-latest-deepseek spike | SPIKE | none | CLAIM-GLM-DSA-LATEST-DEEPSEEK |
MODEL-TEXT-commandr-cohere-for-causal-lm | CohereForCausalLM, Cohere2ForCausalLM | registry.py:84-85; vllm/model_executor/models/commandr.py::CohereForCausalLM | causal generation / text | impl include/vllm/model_executor/models/commandr.h + src/vllm/model_executor/models/commandr{,_weights,_registry}.cpp (weight-only Cohere LayerNorm grounded commandr.py:76-87,65-73; GPT-J full-width RoPE is_neox_style=false commandr.py:174-179; PARALLEL residual commandr.py:257-273; logit_scale commandr.py:376; tied embeds commandr.py:372; ZERO new kernel, one REGISTER line); test tests/parity/test_commandr_paged_engine.cpp (SACRED gate, dgx-only, SKIPs — no runnable vehicle). CPU -Werror build compiles+links+registers. use_qk_norm/sliding rejected at parse (Cohere2 arch). | recent-dense-batch | BLOCKED | W0 oracle RUN-VERIFIED (tiny-random builds+runs, golden captured, CohereForCausalLM≠Cohere2); gate BLOCKED — real ckpts HF-gated (no dgx token) + only ungated are tiny-random head_dim 8/2 (outside validated attn) + dgx disk-full | CLAIM-SWEEP-RECENT-DENSE |
MODEL-TEXT-cohere2-moe-cohere2-moe-for-causal-lm | Cohere2MoeForCausalLM | registry.py:86; vllm/model_executor/models/cohere2_moe.py::Cohere2MoeForCausalLM | causal generation / text | model loader/forward; paged attention/KV; FusedMoE/grouped GEMM; sliding-window attention | ☐ required | INVENTORIED | none | unassigned |
MODEL-TEXT-llama-llama-for-causal-lm | CwmForCausalLM, InternLM3ForCausalLM, IQuestCoderForCausalLM, LlamaForCausalLM, LLaMAForCausalLM, TeleChat3ForCausalLM | registry.py:87,134-135,143,146,205; vllm/model_executor/models/llama.py::LlamaForCausalLM | causal generation / text | model loader/forward; paged attention/KV; llama3 rope-scaling; qk-norm-optional dense attn | sweep-llama-3.2 spike | PARTIAL (W0-W4 landed; correctness DONE — near-tie-robust token-exact 16/16 vs a MEASURED-deterministic vLLM 0.25.0 oracle; SPEED bar explicitly PENDING, no throughput measured or claimed) | The breadth-sweep CROSS-FAMILY DENSE additivity proof (Tier-2 rank 5, .agents/specs/breadth-sweep-plan.md §B.3 — the roadmap's explicit "Llama-first" increment). Every prior model was a Qwen variant or OPT; Llama is the mainstream dense arch. Llama IS the Qwen3-dense forward with exactly TWO deltas, both handled ADDITIVELY: (1) NO per-head q/k RMSNorm (Qwen3 has qk-norm, Llama does not) — the shared dense_attn_block.h::AttnBlock skips the norm step when q_norm/k_norm are empty (has_qk_norm), byte-preserving for Qwen3; (2) llama3 RoPE frequency scaling (rope_theta=500000 + rope_scaling type "llama3" — a piecewise low/high wavelength rescale of the base inv_freq, Llama3RotaryEmbedding._compute_inv_freq llama3_rope.py:33-54) — the one genuinely new primitive, added as 4 default-0 vt::RopeArgs fields + a Llama3ScaleFreq helper applied in the CUDA + CPU RopeNeox/RopeCosSinCache kernels (no-op when disabled ⇒ every existing model byte-identical; formula VERIFIED to 2e-7 rel vs vLLM's inv_freq). Everything else (merged QKV, GQA 32/8, SwiGLU MLP, standard non-gemma RMSNorm, tied lm_head, the full-attention paged forward) is REUSED VERBATIM (LlamaModel == Qwen3DenseModel). W0+W1 (2026-07-23): registry TU src/vllm/model_executor/models/llama_registry.cpp (REGISTER_VLLM_MODEL(llama_dense, "LlamaForCausalLM") + full-attention-only MakeLlamaForCausalLMKVCache + ParseLlamaForCausalLMConfig; is_dense_model=true) + header include/vllm/model_executor/models/llama.h; the reusable-piece generalizations MakeRopeArgs(cfg) + qk-norm-optional AttnBlock in dense_attn_block.h and the llama3 rope kernels in cuda_ops.cu/cpu_ops.cpp. HfConfig/RopeParameters ALREADY parse+validate the llama3 rope_scaling dictionary (hf_config.cpp:230-250,:370-377) — ZERO hf_config.{h,cpp} edit. W2 (2026-07-23): loader src/vllm/model_executor/models/llama_weights.cpp (LoadLlamaForCausalLMWeights — the Qwen3 name map MINUS q_norm/k_norm, merged qkv/gate_up raw-NK, lm_head.weight SKIPPED under tie_word_embeddings). Test test_llama_load.cpp — all tensors mapped/shaped, q_norm/k_norm EMPTY, tied lm_head empty, no leftover, llama3 config fields asserted (792 assertions). W3 (2026-07-23): forward reuses LlamaModel::Forward; head_dim 64 runs the generic LaunchPrefillFlash/LaunchDecode CUDA-core path (epl from runtime d) with ZERO kernel edit. Forward doctest test_llama_forward.cpp (CPU synthetic: qk-norm-ABSENT finite+deterministic; llama3-scaling ACTUALLY applied — scaled vs plain-rope logits DIFFER; fusion ADOPT byte-exact) + a CUDA prefill-argmax diagnostic (510 assertions). W4 correctness gate (SACRED, 2026-07-23): gate SELECTED BY MEASUREMENT per [[near-tie-distributional-gate]] — qwen3-oracle-capture.py --per-prompt --runs 6 found vLLM 0.25.0 greedy DETERMINISTIC on all 16 prompts, 0 multi-valued (prompt,pos) cells ⇒ the bar is STRICT token-exact. tests/parity/test_llama_paged_engine.cpp + goldens tests/parity/goldens/llama_greedy_1b/ ⇒ 16/16 prompts PASS (12/16 STRICT token-exact + 4/16 near-tie band, max gap 0.0000 nats, 0 forward-divergent) — at ALL 13 divergent positions vLLM's OWN teacher-forced argmax GIVEN OUR PREFIX IS our token (gap 0.0000), i.e. our forward reproduces vLLM's logits bit-identically and vLLM only contradicts its own incremental greedy. ONE tokenizer bug was found + fixed by this gate, correctness-fatal: Llama's post_processor is a Sequence wrapping a ByteLevel + a TemplateProcessing, and ExtractBosEos (tokenizer.cpp) only handled a TOP-LEVEL TemplateProcessing ⇒ the BOS token (begin_of_text, id 128000) was never prepended ⇒ every prompt was one token short (silently 1/16). The CUDA prefill-argmax diagnostic proved the FORWARD correct (4/4 argmax match vLLM's greedy tok0 GIVEN vLLM's exact prompt tokens), isolating the fault to the tokenizer; the fix (parse the Sequence member) is byte-preserving for every existing model (Qwen/OPT/DeepSeek all use a ByteLevel or top-level-TemplateProcessing post_processor — verified). Checkpoint: unsloth/Llama-3.2-1B (ungated mirror of gated meta-llama/Llama-3.2-1B — same weights; 401 on the gated repo's weight download even with metadata access). ADDITIVITY VERDICT: the dense MODEL seams held 100% — 3 new Llama files (llama.h/llama_weights.cpp/llama_registry.cpp, reusing the shared dense forward), ZERO edit to the runner, scheduler, platforms, attention registry, hf_config.{h,cpp}, or any qwen3*/opt* model file; the shared touches are the guarded llama3 rope primitive (byte-preserving), the qk-norm-optional AttnBlock (byte-preserving), and the tokenizer Sequence post_processor parse (byte-preserving) — the leak was one layer down in the tokenizer, exactly as OPT's BOS leak was. REGRESSIONS (each STANDALONE, UNCHANGED): 27B test_qwen27_paged_engine 235/235 · 35B test_qwen36_paged_engine 315/315 · Qwen3-Coder 6/6 · Qwen3-dense 16/16 · OPT 6/6 · DeepSeek-V2 8/8; CUDA -Werror 0 warnings; memcheck 0 on the Llama engine path; DSR 67 unchanged. SPEED bar explicitly PENDING (head_dim 64 falls to the generic paged path; the FA2-toggle decode path is the Llama-3.2-3B/head_dim-128 W-next) — row stays ACTIVE, not DONE. ALIASES GATED (2026-07-26, sweep-yi-internlm3, CLOSES the recent-dense TEXT tier): two Llama-alias checkpoints brought up as near-zero-work reuse of this forward. Yi — modern 01-ai/Yi-* declare architectures:["LlamaForCausalLM"] (Yi adopted the Llama arch) ⇒ ZERO code delta, no YiForCausalLM alias (vLLM 0.25.0 registers none); test_yi_paged_engine (01-ai/Yi-Coder-1.5B-Chat, ungated) SACRED 16/16 vs vLLM 0.25.0 (W0 K=5 deterministic ⇒ STRICT; 13 strict + 3 near-tie, max gap 0.125 nats, 0 divergent). InternLM3 — registry.py:134 maps InternLM3ForCausalLM→llama (a PLAIN Llama alias with dynamic-NTK rope factor 6.0, NOT InternLM2+sliding-window — the spike note corrected); ONE additive alias line REGISTER_VLLM_MODEL(internlm3_llama,"InternLM3ForCausalLM",kLlamaFactory,kLlamaInfo) in llama_registry.cpp; test_internlm3_paged_engine (internlm/internlm3-8b-instruct) SACRED 16/16 vs vLLM 0.25.0 (W0 K=5 deterministic ⇒ STRICT; 14 strict + 2 near-tie, max gap 0.0 nats, 0 divergent). Both W0 RUN-VERIFIED on the pinned oracle (build+run coherent, the OLMo-3 lesson). Additive-only ⇒ Llama-3.2-1B re-gated 16/16 UNCHANGED. Gated ID-based via TokensPrompt (scripts/stage-tokenizer-metaspace.py stages a Metaspace tokenizer.json; forward is ID-gated). SPEED PENDING.; registration src/vllm/model_executor/models/llama_registry.cpp:148; test tests/vllm/models/test_model_registry.cpp:89 | CLAIM-MODEL-LLAMA-3.2 |
MODEL-TEXT-dbrx-dbrx-for-causal-lm | DbrxForCausalLM | registry.py:88; vllm/model_executor/models/dbrx.py::DbrxForCausalLM | causal generation / text | model loader/forward; paged attention/KV; FusedMoE/grouped GEMM | ☐ required | INVENTORIED | none | unassigned |
MODEL-TEXT-nemotron-nas-deci-lmfor-causal-lm | DeciLMForCausalLM | registry.py:89; vllm/model_executor/models/nemotron_nas.py::DeciLMForCausalLM | causal generation / text | model loader/forward | ☐ required | INVENTORIED | none | unassigned |
MODEL-TEXT-deepseek-v2-deepseek-for-causal-lm | DeepseekForCausalLM | registry.py:90; vllm/model_executor/models/deepseek_v2.py::DeepseekForCausalLM | causal generation / text | model loader/forward; paged attention/KV; FusedMoE/grouped GEMM; NOT MLA — model_type=="deepseek" selects plain MHA (deepseek_v2.py:1201-1211); corrected from the source-scan hypothesis by the campaign spike | mla-deepseek-campaign spike | SPIKE | W10 (blocked-row honesty pass, 2026-07-22) — THIS IS NOT AN MLA ROW AND MUST NOT BE READ AS ONE. model_type == "deepseek" selects plain MHA (deepseek_v2.py:1201-1211 -> :133 DeepseekAttention), so NONE of the campaign's MLA machinery (W1 spec-driven KV, W2 MLA backend priorities, W3 ConcatAndCacheMla, W4 MQA decode, W5 MHA prefill + chunked context, W6 absorption block) is what this row needs; W7's config parse REFUSES this architecture BY NAME rather than registering it, because a registration would be a false support claim. What it actually needs is our EXISTING dense attention plus the DeepSeek MoE block (grouped-topk router + ungated shared experts) that W7 built and W8 gated e2e on V2-Lite. NOT hardware-blocked: per the spike's §11 the 16B DeepSeek-MoE variant would fit GB10 (the 67B does not), so this row stays SPIKE as the cheapest remaining member of the campaign — no e2e gate has ever been run on it and none is claimed. Below: the shared-layer chronology this row INHERITS, which is context, not support. none W6 LANDED 2026-07-22 — the MLA attention BLOCK + LOAD-TIME WEIGHT ABSORPTION; the row STILL does not move, because W6 adds a LAYER, not a model. W3's cache write, W4's MQA decode and W5's MHA prefill are now composed by one layer (mla_attention.h, mla_attention.cpp): projections with BOTH q_lora_rank branches, the two RMSNorms, the decoupled is_neox_style=False RoPE with its YaRN cos/sin cache and the separate mscale^2 softmax scale, the kv_b_proj -> W_UK/W_UV split, and the prefill-MHA / decode-MQA dispatch with decode tokens packed FIRST. Absorption needed NO new attention kernel — it is a load-time transform plus two batched GEMMs (vt::BatchedMatmul <- torch.bmm at mla_attention.py:789,1034; vt::ConcatMlaNopeRope <- concat_mla_q generalized to also serve _concat_k_nope_k_pe). The absorbed-vs-unabsorbed equivalence is PROVEN numerically three ways (double-precision identity < 1e-11; ours vs the unabsorbed oracle < 2e-4 in f32; and the SAME batch through our ABSORBED MQA decode vs our UNABSORBED materialized-MHA prefill, < 3e-4 CPU f32 / < 4e-2 CUDA bf16). Evidence: test_mla_attention_block.cpp 10/10 / 2,372,644 assertions + test_ops_mla_absorb.cpp 9/9 / 1,644,807 on dgx sm_121; memcheck/racecheck/synccheck all 0; clean CUDA build 0 warn/0 err; regression set UNCHANGED (27B 235/235, 35B 315/315, Coder 138/138, Qwen3-dense 664/664, OPT 36/36). The q_lora query branch is UNIT-GATED ONLY and always will be on this box — DeepSeek-V2-Lite has q_lora_rank=null, so it is gated at DeepSeek-V3's real dimensions (7168 / 128 heads / 1536) and gets NO e2e coverage; GLM-4.7-Flash (q_lora_rank=768) is what would close it. NO model TU, NO registry entry, NO loader, NO model forward — W7. W7 (2026-07-22) DELIBERATELY DOES NOT REGISTER THIS ARCHITECTURE. The W7 DeepSeek-V2 model TU registers only DeepseekV2ForCausalLM, and ParseDeepseekV2Params THROWS BY NAME on this one — model_type == "deepseek" (or both qk dims zero) is upstream's use_mha branch (deepseek_v2.py:1201-1211 -> :133 DeepseekAttention), i.e. plain MHA and a DIFFERENT architecture that needs none of the MLA campaign's code. Registering it would have been a false support claim; the refusal is asserted in tests/vllm/models/test_deepseek_v2_load.cpp. This row remains what §0.7 said it was: our existing dense attention plus a DeepSeek MoE router, and the cheapest DeepSeek row to do — but it is NOT done. | CLAIM-MLA-DEEPSEEK |
MODEL-TEXT-deepseek-v2-deepseek-v2-for-causal-lm | DeepseekV2ForCausalLM | registry.py:91; vllm/model_executor/models/deepseek_v2.py::DeepseekV2ForCausalLM | causal generation / text | model loader/forward; paged attention/KV; FusedMoE/grouped GEMM; MLA/latent KV — W0-CONFIRMED geometry against the real deepseek-ai/DeepSeek-V2-Lite config.json: kv_lora_rank=512 + qk_rope_head_dim=64 = a 576-wide latent, num_kv_heads=1, no separate V; qk_nope=128, v_head_dim=128, q_lora_rank=null, n_group=topk_group=1/softmax/greedy (so NO e_score_correction_bias — the noaux_tc router and the fused_qkv_a_proj branch stay unit-gated only) | mla-deepseek-campaign spike (W0-W9 landed) | ACTIVE (correctness COMPLETE, speed SHORT — W9 attributed miss; W10 closed the record) | W0 OBSERVATION only (no model code yet): oracle DEBUG startup on sm_121 selects TRITON_MLA (dense MLA) + FLASH_ATTN (MLA prefill), evidence ~/scratch_mla_w1/w0_probe.log on dgx; W1 engine seam gated behaviour-preserving (tests/vllm/v1/test_kv_cache_interface.cpp MLA-spec cases, tests/vllm/v1/worker/test_runner.cpp spec-driven allocation case); W2 backend selection — use_mla=true on sm_121 RESOLVES to TRITON_MLA with FLASHINFER_MLA_SPARSE_SM120 filtered on is_sparse() (tests/vllm/v1/attention/test_attn_backend_registry.cpp, ports of test_attention_backends_selection.py/test_mla_prefill_selector.py/test_mla_prefill_registry.py); W3 two ops CPU-ref-gated — vt::ConcatAndCacheMla at the real 576-wide geometry (tests/vt/test_ops_mla_cache.cpp, port of test_cache.py::test_concat_and_cache_mla) and the grouped-topk router (tests/vt/test_ops_moe_router_grouped.cpp, ports of test_grouped_topk.py/test_routing.py). STILL NO MLA ATTENTION MATH, NO MODEL, NO FORWARD — row stays SPIKE W4 the MLA DECODE kernel — vt::MlaDecodeAttention, a structure port of the two-stage split-KV Triton pair (_fwd_grouped_kernel_stage1 triton_decode_attention.py:278-458 IS_MLA branch + _fwd_kernel_stage2 :575-639) that W0 OBSERVED EXECUTING on sm_121, plus _compute_num_kv_splits (triton_mla.py:40-47) and the forward_mqa contract (:189-260); gated at the REAL 576/512/64 block-16 mscale^2-scale geometry vs an INDEPENDENT two-pass CPU reference in tests/vt/test_ops_mla_attn.cpp (port of test_mla_decode_cpu.py::ref_mla incl. its NaN-padding out-of-bounds detector), covering ragged / multi-block / single-block / every-split-boundary / 128-head(V3) / non-BLOCK_H head counts, run-to-run BIT-reproducible, compute-sanitizer memcheck+racecheck+synccheck 0; TritonMLABackend::get_impl_cls() now returns a real TritonMLAImpl (DECODE only — prefill is W5, and forward() says so). STILL NO MLA MODEL AND NO FORWARD — row stays SPIKE W5 the MLA PREFILL path + the workspace-bounded CHUNKED-CONTEXT loop — vt::MlaPrefillAttention (<- mla/prefill/flash_attn.py:153-248, the only backend reachable on sm_121, running over the vendored FA-2 with V zero-padded 128->192 exactly as upstream's requires_v_padding path does), vt::GatherMlaCache (<- cache_kernels.cu:992-1064), vt::MergeAttnStates (<- merge_attn_states.cu:18-192, both -inf edge cases verbatim), and mla_chunked_context.h (<- mla_attention.py:1422-1451,1667-1745,2094-2199,2344-2425). Gated UNIT-ONLY (there is still no model): test_ops_mla_prefill.cpp 4/4 / 2,377,052 assertions + test_ops_mla_chunked_context.cpp 5/5 / 306,037 assertions on dgx sm_121, at the REAL QK 192 / V 128 prefill geometry, against an INDEPENDENT double-precision two-pass oracle and — for the loop — a SINGLE-SHOT whole-sequence oracle that never chunks; exact/+1/-1 chunk boundaries, zero-context and zero-key-in-chunk requests, ADVERSARIAL reverse-interleaved block tables, NaN-poisoned outputs, run-to-run bit-exact; memcheck/racecheck/synccheck all 0; regression set UNCHANGED. Still NO MLA model, NO MLA forward and NO weight absorption (W6/W7) — the row stays SPIKE. W6 LANDED 2026-07-22 — the MLA attention BLOCK + LOAD-TIME WEIGHT ABSORPTION; the row STILL does not move, because W6 adds a LAYER, not a model. W3's cache write, W4's MQA decode and W5's MHA prefill are now composed by one layer (mla_attention.h, mla_attention.cpp): projections with BOTH q_lora_rank branches, the two RMSNorms, the decoupled is_neox_style=False RoPE with its YaRN cos/sin cache and the separate mscale^2 softmax scale, the kv_b_proj -> W_UK/W_UV split, and the prefill-MHA / decode-MQA dispatch with decode tokens packed FIRST. Absorption needed NO new attention kernel — it is a load-time transform plus two batched GEMMs (vt::BatchedMatmul <- torch.bmm at mla_attention.py:789,1034; vt::ConcatMlaNopeRope <- concat_mla_q generalized to also serve _concat_k_nope_k_pe). The absorbed-vs-unabsorbed equivalence is PROVEN numerically three ways (double-precision identity < 1e-11; ours vs the unabsorbed oracle < 2e-4 in f32; and the SAME batch through our ABSORBED MQA decode vs our UNABSORBED materialized-MHA prefill, < 3e-4 CPU f32 / < 4e-2 CUDA bf16). Evidence: test_mla_attention_block.cpp 10/10 / 2,372,644 assertions + test_ops_mla_absorb.cpp 9/9 / 1,644,807 on dgx sm_121; memcheck/racecheck/synccheck all 0; clean CUDA build 0 warn/0 err; regression set UNCHANGED (27B 235/235, 35B 315/315, Coder 138/138, Qwen3-dense 664/664, OPT 36/36). The q_lora query branch is UNIT-GATED ONLY and always will be on this box — DeepSeek-V2-Lite has q_lora_rank=null, so it is gated at DeepSeek-V3's real dimensions (7168 / 128 heads / 1536) and gets NO e2e coverage; GLM-4.7-Flash (q_lora_rank=768) is what would close it. NO model TU, NO registry entry, NO loader, NO model forward — W7. W7 LANDED 2026-07-22 — the DeepSeek-V2 MODEL: registry + config parse + loader + forward. The row STILL does not move, because a loading, forwarding model is not a SUPPORTED model — the SACRED token-exact gate is W8. Four new files (deepseek_v2.h, deepseek_v2_weights.cpp, deepseek_v2.cpp, deepseek_v2_registry.cpp) plus ONE shared-code edit (a two-line additive condition in runner.cpp recognising a kMlaAttention KV group as the attention group; upstream maps MLA onto the ordinary FullAttentionManager, single_type_kv_cache_manager.py:1539). LOADER GATE: 5291/5291 checkpoint tensors accounted for, ZERO unmapped, ZERO leftover (test_deepseek_v2_load.cpp, 4/4 cases / 37,331 assertions on dgx), every shape asserted incl. the LOAD-TIME kv_b_proj -> W_UK_T [16,128,512] / W_UV [16,512,128] absorption split and the shared-expert MLP; V2-Lite takes the DIRECT q_proj branch (q_lora_rank: null) with the fused branch asserted EMPTY. FORWARD GATE: the real checkpoint prefill of The capital of France is -> argmax Paris (top-5 Paris/ the/ a/ one/ also, run-to-run bit-exact). BATCH-ORDERING GATE: BuildMlaBatchSplit reproduces split_decodes_and_prefills + prefill_tokens_with_context and THROWS on either violation of the ordering W6 measured 0.86 relative error from. SHARED EXPERTS (new for this family, ungated unlike Qwen3.6's sigmoid-gated one) gated two ways: zeroed-routed MoE layer BIT-IDENTICAL to a dense layer with the same MLP, and shared-off CHANGES the logits. CUDA path EXERCISED at the real MLA head geometry (grouped bf16 MoE GEMM + MLA kernels): bit-exact on device, 0.0061 worst relative error vs the CPU reference path; test_deepseek_v2_forward.cpp 11/11 cases; memcheck/racecheck/synccheck all 0; clean CUDA build 0 warn/0 err; regression set UNCHANGED. Only DeepseekV2ForCausalLM is registered — DeepseekForCausalLM (plain MHA), V3 (fp8/671B) and V3.2 (DSA indexer) are REFUSED BY NAME in the config parse rather than falsely claimed; W10 owns their rows. NO token-exact gate, NO goldens, NO benchmark — W8/W9. W8 LANDED 2026-07-22 — THE SACRED CORRECTNESS GATE PASSES 8/8, so the row moves SPIKE -> ACTIVE with SPEED EXPLICITLY PENDING (never DONE: DONE requires vLLM-speed parity on every axis, which is W9 and has no number at all yet). test_deepseek_v2_paged_engine.cpp drives an 8-prompt battery through the FULL paged LLMEngine (LoadedEngine::FromModelDir -> scheduler -> decode-first reorder -> MLA cache write + MLA decode/prefill + DeepSeek MoE -> sampler) and compares greedy decode to the pinned vLLM 0.25.0 oracle: 8/8 prompts PASS — STRICT token-exact 5/8, near-tie band 3/8, 92/128 tokens strictly exact, max teacher-forced gap 0.25 nats, 0 forward-divergent (1/1 case / 223 assertions). The bar was ARRIVED AT, not chosen: vLLM is DETERMINISTIC here at batch=1 (W0's K=5 8/8, re-confirmed by W8's own capture at T=16 with 0 multi-valued cells), so the STRICT form was applied FIRST and came out 5/8; the ratified TEACHER-FORCING diagnostic (scripts/deepseek-v2-neartie-gap.py) then showed 36 divergent positions, 35 at gap EXACTLY 0.0000 nats (vLLM's OWN argmax given OUR prefix IS our token), exactly ONE root flip with any gap at all (prompt[3] tok 9, 0.2500 nats — inside the ratified 0.5-nat band) and ZERO tokens outside vLLM's top-20, with the per-position nats COMMITTED (tests/parity/goldens/deepseek_v2_greedy/neartie_gap_mnats.npy, anchored by our_ids.npy). THE SCHEDULER/RUNNER WIRING NEEDED NO NEW CODE, and W8 PROVES that rather than asserting it: runner.cpp:671 already reorders with decode_threshold = 1 — exactly MLA's reorder_batch_threshold (mla_attention.py:1420) — and its decode -> short_extend -> long_extend -> pure_prefill ordering satisfies BOTH MLA invariants; new diagnostic MlaBatchSplitStats counters measure what the engine really produced: phase 2 admits the battery CONCURRENTLY with staggered admission and observes steps=30, MIXED=7, max_num_reqs=8, max_num_decodes=8 with BuildMlaBatchSplit never throwing, and phase 3 drives a prefix-cache hit producing a real WITH-CONTEXT prefill (with_context_prefill_steps=1); phase 0 asserts the engine allocated a genuine MLA cache (fa_page_size_bytes = 36864 = block 32 x 576 x 2B, no factor 2). THE REAL BLOCKER WAS THE TOKENIZER, NOT THE MODEL: the first run REFUSED to load (unsupported normalizer Sequence), and behind it sat a whole NEW pre-tokenizer family — DeepSeek's is a HF Sequence PIPELINE of SEVEN stages (five Split(Isolated) over ENUMERATED codepoint ranges, then Digits(individual_digits=true), then ByteLevel(use_regex=false)), landed as SplitPattern::kDeepSeek and MEASURED token-for-token against the REAL HF tokenizers library over a stage-stress corpus (test_tokenizer_parity_deepseek.cpp, 6/6 cases / 2461 assertions). The TOKENIZATION goldens earned their keep by REFUTING a fix that was already written: tokenizer_config.json declares add_bos_token: true, but vLLM's resolved tokenizer (TokenizersBackend) adds NO BOS, our loader already matched, and the fix would have BROKEN a passing gate — reverted, with the measured behaviour pinned by a guard case in test_bpe.cpp. Regression set UNCHANGED (27B 235/235, 35B 315/315, Coder 6/6, Qwen3-dense 16/16, OPT 6/6, plus every tokenizer test); clean CUDA rebuild 0 warn/0 err; memcheck/racecheck/synccheck 0. Batch invariance is REPORTED (6/8) and deliberately NOT a bar, because the ORACLE itself changed on 3/8 of this battery under batched generation (W0). Both coverage gaps stand unchanged: q_lora_rank=null and n_group=topk_group=1/softmax/greedy, so the fused_qkv_a_proj query branch and the whole noaux_tc router remain unit-gated only on this box. NO speed number — W9. W9 LANDED 2026-07-22 — THE FIRST SPEED NUMBER, and it is an ATTRIBUTED MISS, so the row STAYS ACTIVE (correctness COMPLETE, speed SHORT) and does NOT move to DONE. Binding grid in docs/BENCHMARKS.md § "Binding DeepSeek-V2-Lite (MLA) every-axis grid". THE DENOMINATOR IS SETTLED WITH EVIDENCE, NOT ASSUMED: vLLM AUTO-selects the FlashInfer CUTLASS unquantized MoE backend for this model and it CANNOT RUN ON GB10 — five hard reboots now (three at W8; two at W9 with the Qwen3-Coder mitigations deliberately applied, the second on a PRISTINE freshly-rebooted box with a 0 GiB page cache and 116 GiB free at gmu=0.40/max-model-len 2048/max-num-batched-tokens 1024/max-num-seqs 4), both W9 deaths at the IDENTICAL phase (immediately after torch.compile, in the memory-profiling dummy run where the CUTLASS MoE workspace is first allocated); sudo is password-gated on this box so drop_caches was never actually available and W9 #2 substitutes for it exactly. So --moe-backend triton IS vLLM's best STABLE GRAPHED production configuration here and is the legitimate bar — stated openly because the substitution does NOT flatter us: we lose to it. Both sides otherwise resolve identically (TRITON_MLA decode + FLASH_ATTN MLA prefill) and vLLM runs GRAPHED (FULL_AND_PIECEWISE, capture sizes [1,2,4,8,16]), never --enforce-eager. THE LEVER WAS FOUND BY nsys, NOT BY THE PLAN: §10's W9 cell planned MLA fusion recipes; the trace (--cuda-graph-trace=node, both sides, same workload) refuted that priority before a line was written — MlaDecodeStage1 was 44.7% of ALL GPU time at 837 us/instance (~22.6 ms of a ~47.7 ms TPOT) and about two orders of magnitude off its own memory-bound floor, because _compute_num_kv_splits (triton_mla.py:40-47) derives the split count from max_seq_len alone and returned 2, MLA has ONE kv head so the head dimension collapses to a single tile, and at batch 1 the grid was 2 CTAs on the whole GPU. ComputeNumKvSplitsFilled applies upstream's OWN occupancy target (maximum = sm_count * _SPLIT_OCCUPANCY_MULTIPLIER, which the length-only heuristic never reaches on short sequences) — a RECORDED DEVIATION, numerics-visible via the stage-2 reduction order and therefore GATED not assumed, rollback VT_MLA_SPLIT_FILL=0: 837 us -> 45.8 us, 18.3x, 44.7% -> 3.9% of GPU time, worth +69.5%/+53.3%/+32.0%/+19.5% end-to-end at c1/c2/c4/c8. DeepseekV2DecodeGraph (the MLA member of the Qwen3_5DecodeGraph/Qwen3_5DenseDecodeGraph/Qwen3MoeDecodeGraph family; ForwardBody split into EmbedInto + a capturable ForwardLayers; rollback VT_DEEPSEEK_CUDAGRAPH=0) is worth only ~+2% — an order of magnitude less than its Qwen3-Coder analogue, and the profile says why: this model's decode is GPU-BOUND, ~1.2 ms/step of host tax rather than ~5. A REAL LATENT CUDA-GRAPH DEFECT WAS FOUND AND FIXED, not worked around: BuildMlaStep uploaded the decode block_table/seq_lens from FUNCTION-LOCAL TEMPORARIES — correct eagerly, a use-after-free under capture (the copy becomes a graph node that bakes the source address), producing a graph that was right on the replay issued immediately after capture and then diverged ~19 logits and finally faulted; compute-sanitizer SERIALIZED enough to hide the fault and reported 0 memory errors with wrong tokens. Fixed by UploadRange (uploads a range of the CALLER's persistent vector — legitimate because the decode half is a batch PREFIX and the prefill half a SUFFIX by the scheduler's own reorder contract), verified max abs(graph - eager) = 0 by a VT_DEEPSEEK_GRAPH_VERIFY=1 diagnostic kept in the tree. A second hazard — the split-KV workspace GROWING inside a capture once the fill made its size batch-dependent, which turns cudaMallocAsync into a graph-owned allocation node — is closed two ways: EnsureMidScratch now carries the house capture guard (mirroring cuda_dropin.cu:124 and cuda_flash_attn_fa2.cu:717-720) so growth-under-capture is a LOUD error, and the workspace is sized at the CONSTANT bound sm_count * 2 so the allocation depends only on (batch, num_heads). CORRECTNESS IS INTACT WITH EVERY OPTIMIZATION DEFAULT-ON: the SACRED gate is 8/8 with CHARACTER-FOR-CHARACTER the W8 result (STRICT 5/8, near-tie 3/8, 92/128 strictly exact, max gap 0.25 nats, 0 forward-divergent, 0 multi-valued oracle cells) and IDENTICAL MlaBatchSplitStats (phase 1 steps=128/decode_only=120; phase 2 MIXED=7/max_num_reqs=8; phase 3 with_context_prefill_steps=1) — which required the graph driver to record the split on the REPLAY path, since BuildMlaStep does not run there; capture set exercised S={1,2,4,8}; test_ops_mla_attn 11/11 / 2,303,193 assertions. THE GRID (medians of 3, 1024-in/128-out, fresh vllm serve per concurrency, prefix-cache hit VERIFIED 0.0% at c1/c2/c4 and 1.1% at c8 — reported, not hidden): output tok/s ours-over-vLLM 0.87 / 0.95 / 0.86 / 0.88 (was 0.50 at c1), request req/s identical ratios, median TTFT 1.06 / 1.14 / 0.96 / 0.88 (we WIN at c4 and c8), median TPOT 1.11 / 0.97 / 1.16 / 1.17, median ITL 1.11 / 1.00 / 1.16 / 1.13. PEAK MEMORY is the one axis we win decisively: 31.38 GB peak RSS vs vLLM's 68.5 GiB reservation (ratio 0.46) — stated with its caveat, since vLLM PRE-RESERVES a fixed fraction while we allocate the KV the configured concurrency needs, so it is a real operating-footprint difference but NOT evidence of a cheaper per-token KV (the MLA page geometry is identical on both sides). RESIDUAL, ATTRIBUTED TO KERNELS: (1) the batch-1 dense projections fall to cuBLAS gemvx for 31.8% of our GPU time where vLLM splits the same work gemvx 12.7% + tensor-core nvjet_sm121_tst_mma_* 6.6% — the LARGEST single difference and the NEXT LEVER, a dispatch change not a new kernel; (2) MoeGroupedGemmBf16NaiveSplitK is now our top kernel at 40.5%, ~1.2x its own bandwidth floor; (3) vLLM runs Inductor-codegenned FUSED glue we lack (triton_red_fused_add_fused_add_rms_norm_1, triton_red_fused_fused_add_rms_norm_moe_forward_shared_0, triton_poi_fused_mul_silu_slice_0, triton_poi_fused_2), ~7-8% of its profile; (4) our MLA decode is still ~2.3x vLLM's per call (45.8+22.6 us vs _fwd_grouped_kernel_stage1 29.6 us). NOT DONE, said plainly: the planned MLA fusion catalog entries were NOT built (the trace ranked them far below the occupancy fix); vt::MergeAttnStates stays scalar (chunked-prefill-only, absent from the decode profile); vt::ConcatMlaNopeRope stays scalar (0.1% of GPU time); the fused A-GEMM is folded into lever (1) rather than done blind; cuda_arch_tactics stays deferred (still exactly one MLA decode kernel). Regression set UNCHANGED (27B 235/235, 35B 315/315, Coder 138/138, Qwen3-dense 664/664, OPT 36/36, run SERIALLY); clean full CUDA rebuild 0 warn / 0 err; memcheck/racecheck/synccheck 0. Both §5.1 coverage gaps stand unchanged. The row reaches DONE when the c1..c8 grid passes on EVERY axis. W10 (blocked-row honesty pass, 2026-07-22) — the campaign's W-plan is COMPLETE; this row is the ONLY member that is e2e-gateable on GB10 and it stays ACTIVE, not DONE. DONE requires BOTH token-exact correctness AND vLLM-speed parity on every axis; correctness is COMPLETE (SACRED gate 8/8: STRICT 5/8, near-tie 3/8, 92/128 tokens strictly exact, max teacher-forced gap 0.25 nats, 0 forward-divergent, vLLM deterministic at batch=1) and speed is SHORT (ours/vLLM tok/s 0.87 / 0.95 / 0.86 / 0.88 at c1/c2/c4/c8; TTFT 1.06 / 1.14 / 0.96 / 0.88; TPOT 1.11 / 0.97 / 1.16 / 1.17; peak memory 0.46). NEXT LEVER, NAMED and measured, not guessed: at batch 1 our dense projections fall to cuBLAS gemvx for 31.8% of our GPU time where vLLM splits the same work between gemvx (12.7%) and tensor-core nvjet_sm121_tst_mma_* (6.6%) — a DISPATCH change, not a new kernel. TWO PERMANENT COVERAGE GAPS on this hardware, stated in the row rather than in the spec only: (1) the noaux_tc grouped router has NO e2e coverage — V2-Lite is n_group=topk_group=1 with softmax/greedy and carries no e_score_correction_bias at all (deepseek_v2.py:315-320), so sigmoid scoring, the biased-select/unbiased-weight asymmetry and the two-level group mask are UNIT-GATED ONLY (test_ops_moe_router_grouped.cpp, at V3's real 256/8/4 dimensions); (2) the q_lora query branch has NO e2e coverage — V2-Lite is q_lora_rank=null, so fused_qkv_a_proj / q_a_layernorm / q_b_proj and the packed_modules_mapping un-fusing (deepseek_v2.py:1812-1820) are built and UNIT-GATED ONLY at V3 dims (test_mla_attention_block.cpp). Neither gap closes by porting harder: no checkpoint that exercises them fits this box except one — Glm4MoeLiteForCausalLM / GLM-4.7-Flash (31.2B, 58.2 GiB bf16) FITS GB10 and would close BOTH (q_lora_rank=768 AND noaux_tc with correction bias), which makes it the NAMED NEXT VEHICLE for this campaign's coverage (row MODEL-TEXT-glm4-moe-lite-glm4-moe-lite-for-causal-lm, glm-dsa spike). DEVIATIONS LEDGER (every deviation this campaign recorded, discoverable HERE and not only in the spec): (a) W5 works around an UPSTREAM FA-2 DEFECT — its empty-K early exit (flash_fwd_kernel.h:1030-1043) ignores unpadded_lse and always writes at the padded offset, so we drive the kernel with unpadded_lse=false into a [b,h,max_seqlen_q] scratch and convert, also normalizing FA-2's +INFINITY empty-row LSE to -inf; (b) vt::MergeAttnStates is SCALAR where upstream's is 128-bit packed (same arithmetic; chunked-prefill-only, absent from the decode profile); (c) vt::ConcatMlaNopeRope is SCALAR/width-generic where upstream's concat_mla_q is vectorized and templated on NOPE_DIM=512 (a concat is a pure copy, so bytes are identical; 0.1% of GPU time, so vectorizing cannot pay); (d) the A-projections are issued per weight ROW-SLICE rather than as one fused GEMM, because vt::RmsNorm requires contiguous inputs and relaxing the hottest op in every model for no MLA-specific gain is the wrong trade — checkpoint PACKING is unchanged; (e) cuda_arch_tactics registration is DEFERRED (there is still exactly one MLA decode kernel; an empty selector would be ceremony); (f) DCP/context-parallel (_context_parallel_compute_prefill_context, reorg_kvcache) and the fp8-prefill arms are NOT ported — single-GPU only, and fp8 prefill needs device-capability family 100 (mla_attention.py:1382-1385), unreachable on sm_121; (g) the split-KV occupancy fill is a RECORDED DEVIATION from upstream's max_seq_len-only _compute_num_kv_splits (triton_mla.py:40-47) — it applies upstream's OWN sm_count * _SPLIT_OCCUPANCY_MULTIPLIER bound, is numerics-visible, was GATED not assumed, and rolls back with VT_MLA_SPLIT_FILL=0; (h) a PRE-EXISTING, tree-wide, NOT-MLA hazard found on the way: the process-wide DevicePool singleton (device_pool.h) keys only on a byte size class, so a test binary running a CPU forward and a CUDA forward hands the second backend the first backend's recycled pointers (observed as a SIGSEGV) — worked around test-locally with per-arm ActivePoolScope, recorded rather than papered over. Denominator honesty, restated because it bears on every number above: vLLM AUTO-selects the FlashInfer CUTLASS unquantized MoE backend for this model and that backend cannot run on GB10 — five hard reboots, one of them on a pristine freshly-rebooted box with a 0 GiB page cache — so --moe-backend triton is vLLM's best STABLE graphed configuration here and is the legitimate bar; the substitution does not flatter us, because we LOSE to it. | CLAIM-MLA-DEEPSEEK |
MODEL-TEXT-deepseek-v2-deepseek-v3-for-causal-lm | DeepseekV3ForCausalLM, DeepseekV32ForCausalLM | registry.py:92-93; vllm/model_executor/models/deepseek_v2.py::DeepseekV3ForCausalLM | causal generation / text | model loader/forward; paged attention/KV; FusedMoE/grouped GEMM; MLA/latent KV — same 576-wide latent as V2-Lite (W0-confirmed on the V2-Lite vehicle); V3's own deltas (q_lora_rank=1536, sigmoid/noaux_tc, 256+1 experts, n_group=8/topk_group=4) remain HW-BLOCKED e2e (~1250 GiB bf16 vs 119 GiB unified) and gateable only by config/loader-slice/unit parity | mla-deepseek-campaign spike (W0-W7 landed) | BLOCKED | W10 (blocked-row honesty pass, 2026-07-22) — HW-BLOCKED e2e on GB10, and V3.2 is additionally DEP-BLOCKED. This row will not sit in a state that implies a gate we cannot run. HW: every shipped V3 / V3.2 checkpoint is 671B — ~642 GiB even in its native fp8 (~1250 GiB bf16) against 119 GiB of unified memory and ~184 GiB of free disk on dgx. No amount of software work changes that; this is not "pending optimization". DEP (V3.2 / DSA only): for a SPARSE model vLLM's XOR filter eliminates TRITON_MLA entirely, leaving FLASHINFER_MLA_SPARSE_SM120 as the SOLE candidate on sm_121 (cuda.py:129-133), and that path is NON-FUNCTIONAL on the flashinfer available here: the sm12x dispatch goes to XQA (flashinfer/mla/_core.py:1169-1172), which is DENSE-ONLY and discards sparse_mla_top_k (_core.py:1483), while vLLM's capability probe has_flashinfer_sparse_mla_sm120() (vllm/utils/flashinfer.py:216-231) only checks three importable symbols and never the function actually called; upstream's own single test (tests/v1/attention/test_flashinfer_sparse_mla_sm120_api.py:37) MONKEYPATCHES that probe to True and asserts nothing numerical. This is a DEPENDENCY gap to watch, not work we can close by porting harder. Also recorded so nobody ports the wrong file: vllm/models/deepseek_v32/ is UNREGISTERED DEAD CODE upstream — registry.py:92-93 routes BOTH DeepseekV3ForCausalLM and DeepseekV32ForCausalLM to deepseek_v2.py::DeepseekV3ForCausalLM. Kimi-K2/K2.5's TEXT backbone lands in this same row by config composition (configs/kimi_k25.py:10 imports DeepseekV3Config; kimi_k25.py:360-364 re-enters the registry) and is HW-BLOCKED harder still at ~1.03T params / ~2000 GiB bf16. WHAT CAN STILL BE GATED here, and what cannot: CAN — config/registry resolution against the real config.json (a few KB, no weights); the weight-map/loader assertion on a single-layer SLICE from the safetensors index; and UNIT parity at V3's REAL dimensions, of which the noaux_tc router half is ALREADY GATED (see the V2 row). CANNOT — any end-to-end token-exact or speed gate, now or after any amount of software work. W7's config parse REFUSES both architectures BY NAME rather than registering them. W3 grouped-topk router UNIT parity at V3's REAL dimensions (256 experts, n_group=8, topk_group=4, top_k=8, sigmoid, routed_scaling_factor=2.5, WITH e_score_correction_bias) vs an independent transcription of grouped_topk_router.py:106-161 — tests/vt/test_ops_moe_router_grouped.cpp. This is the ONLY correctness evidence the noaux_tc router can have here: the V2-Lite e2e vehicle has n_group=topk_group=1 and no bias, so it exercises none of it (§5.1). No V3 model code; e2e still HW-BLOCKED W4 the MLA DECODE kernel — vt::MlaDecodeAttention, a structure port of the two-stage split-KV Triton pair (_fwd_grouped_kernel_stage1 triton_decode_attention.py:278-458 IS_MLA branch + _fwd_kernel_stage2 :575-639) that W0 OBSERVED EXECUTING on sm_121, plus _compute_num_kv_splits (triton_mla.py:40-47) and the forward_mqa contract (:189-260); gated at the REAL 576/512/64 block-16 mscale^2-scale geometry vs an INDEPENDENT two-pass CPU reference in tests/vt/test_ops_mla_attn.cpp (port of test_mla_decode_cpu.py::ref_mla incl. its NaN-padding out-of-bounds detector), covering ragged / multi-block / single-block / every-split-boundary / 128-head(V3) / non-BLOCK_H head counts, run-to-run BIT-reproducible, compute-sanitizer memcheck+racecheck+synccheck 0; TritonMLABackend::get_impl_cls() now returns a real TritonMLAImpl (DECODE only — prefill is W5, and forward() says so). STILL NO MLA MODEL AND NO FORWARD — row stays SPIKE W5 the MLA PREFILL path + the workspace-bounded CHUNKED-CONTEXT loop — vt::MlaPrefillAttention (<- mla/prefill/flash_attn.py:153-248, the only backend reachable on sm_121, running over the vendored FA-2 with V zero-padded 128->192 exactly as upstream's requires_v_padding path does), vt::GatherMlaCache (<- cache_kernels.cu:992-1064), vt::MergeAttnStates (<- merge_attn_states.cu:18-192, both -inf edge cases verbatim), and mla_chunked_context.h (<- mla_attention.py:1422-1451,1667-1745,2094-2199,2344-2425). Gated UNIT-ONLY (there is still no model): test_ops_mla_prefill.cpp 4/4 / 2,377,052 assertions + test_ops_mla_chunked_context.cpp 5/5 / 306,037 assertions on dgx sm_121, at the REAL QK 192 / V 128 prefill geometry, against an INDEPENDENT double-precision two-pass oracle and — for the loop — a SINGLE-SHOT whole-sequence oracle that never chunks; exact/+1/-1 chunk boundaries, zero-context and zero-key-in-chunk requests, ADVERSARIAL reverse-interleaved block tables, NaN-poisoned outputs, run-to-run bit-exact; memcheck/racecheck/synccheck all 0; regression set UNCHANGED. Still NO MLA model, NO MLA forward and NO weight absorption (W6/W7) — the row stays SPIKE. W6 LANDED 2026-07-22 — the MLA attention BLOCK + LOAD-TIME WEIGHT ABSORPTION; the row STILL does not move, because W6 adds a LAYER, not a model. W3's cache write, W4's MQA decode and W5's MHA prefill are now composed by one layer (mla_attention.h, mla_attention.cpp): projections with BOTH q_lora_rank branches, the two RMSNorms, the decoupled is_neox_style=False RoPE with its YaRN cos/sin cache and the separate mscale^2 softmax scale, the kv_b_proj -> W_UK/W_UV split, and the prefill-MHA / decode-MQA dispatch with decode tokens packed FIRST. Absorption needed NO new attention kernel — it is a load-time transform plus two batched GEMMs (vt::BatchedMatmul <- torch.bmm at mla_attention.py:789,1034; vt::ConcatMlaNopeRope <- concat_mla_q generalized to also serve _concat_k_nope_k_pe). The absorbed-vs-unabsorbed equivalence is PROVEN numerically three ways (double-precision identity < 1e-11; ours vs the unabsorbed oracle < 2e-4 in f32; and the SAME batch through our ABSORBED MQA decode vs our UNABSORBED materialized-MHA prefill, < 3e-4 CPU f32 / < 4e-2 CUDA bf16). Evidence: test_mla_attention_block.cpp 10/10 / 2,372,644 assertions + test_ops_mla_absorb.cpp 9/9 / 1,644,807 on dgx sm_121; memcheck/racecheck/synccheck all 0; clean CUDA build 0 warn/0 err; regression set UNCHANGED (27B 235/235, 35B 315/315, Coder 138/138, Qwen3-dense 664/664, OPT 36/36). The q_lora query branch is UNIT-GATED ONLY and always will be on this box — DeepSeek-V2-Lite has q_lora_rank=null, so it is gated at DeepSeek-V3's real dimensions (7168 / 128 heads / 1536) and gets NO e2e coverage; GLM-4.7-Flash (q_lora_rank=768) is what would close it. NO model TU, NO registry entry, NO loader, NO model forward — W7. W7 (2026-07-22) REFUSES BOTH OF THESE BY NAME rather than registering them. The W7 DeepSeek-V2 model TU registers EXACTLY ONE architecture (DeepseekV2ForCausalLM); ParseDeepseekV2Params throws a precise message for a V3.2 checkpoint (detected by index_topk, deepseek_v2.py:1068 — the DSA indexer at :613-642 does not exist here) and for any quantized checkpoint (every shipped V3 is fp8 block-quantized, and the loader is BF16-only), so neither can be silently mis-served. The CODE PATHS V3 would need ARE implemented and unit-gated at V3's real dimensions — the q_lora_rank query branch (the loader un-fuses packed_modules_mapping["fused_qkv_a_proj"] into one merged raw-NK owner) and the sigmoid/noaux_tc grouped router — but a registry entry is a SUPPORT claim, not a code-coverage claim, and e2e stays HW-BLOCKED. W10 owns the honest disposition of this row. | CLAIM-MLA-DEEPSEEK |
MODEL-TEXT-deepseek-v4-deepseek-v4-for-causal-lm | DeepseekV4ForCausalLM | registry.py:94; vllm/models/deepseek_v4/__init__.py::DeepseekV4ForCausalLM | causal generation / text | model loader/forward; paged attention/KV; FusedMoE/grouped GEMM; sliding-window attention; NOT MLA — DeepseekV4Attention (attention.py:97) does NOT subclass MLAAttention, has no kv_lora_rank and no nope/v split. V4 is a NEW architecture, not an increment: (a) Manifold Hyper-Connections — the residual stream becomes [tokens, hc_mult=4, hidden] (nvidia/model.py:1066) with runtime 20-iteration SINKHORN normalization inside TileLang kernels that also swallow the RMSNorms (:866-939), and upstream has ZERO numerical tests and no eager reference for them; (b) CSA/HCA compressor with a stateful fp32 recurrent CompressorStateCache (compressor.py:121) paged via its own CompressorBackend (:37), per-layer compress_ratio 4 (indexed) vs 128; (c) grouped OUTPUT LoRA + per-head attention sinks + SWA 128; (d) MoE sqrtsoftplus scoring + 3 HASH-routed layers (no learned gate, tid2eid lookup on token id, nvidia/model.py:563-577) + clamped SwiGLU; (e) dual-theta RoPE with YaRN mscale disabled (common/rope.py:18-28). Tokenizer risk LOW — vllm/tokenizers/deepseek_v4.py:95 is plain PreTrainedTokenizerFast, standard HF fast BPE; only the ~700-line hand-written chat template (deepseek_v4_encoding.py) needs porting, and upstream ships golden fixtures (tests/tokenizers_/test_deepseek_v4.py:286) making it offline-gateable. HW-BLOCKED e2e: deepseek-ai/DeepSeek-V4-Flash 158.1B stored / 148.7 GiB (fp4 experts) vs GB10's 119 GiB; fits the 184 GiB disk but not memory. DEEPER SCOPE + HW-FIT CORRECTION (2026-07-28, deepseek-v4-flash.md, CLAIM-DEEPSEEK-V4-SCOPE; registry now registry.py:95): the native fp8/fp4 checkpoints (~148–167 GiB) do NOT fit, BUT nvidia/DeepSeek-V4-Flash-NVFP4 (W4-everything, group 16) is ≈83 GiB and FITS the 119 GiB pool with ~36 GiB headroom (matches our sm_12x NVFP4 fast path). Two GB10-viability facts: (i) the FlashInfer sparse DSV4 backend supports_compute_capability(major∈{10,12}) (nvidia/flashinfer_sparse.py:100) + a dedicated DeepseekV4FlashInferSM120Attention ⇒ the DSA sparse-MLA attention path IS supported on sm_121 (V4 ships its OWN sparse backend, unlike the V3.2 sm_120 XQA-dense block — W1 RUN-verifies); (ii) MegaMoE requires SM100 (nvidia/model.py:309, major==10) so GB10 uses the fallback FusedMoE grouped-GEMM path (reuses our NVFP4 MoE). Disk action: free DGX to ~90 GiB before the ~83 GiB download (~68 GiB free now). The decisive W1 oracle-RUN gate + W-plan live in the new spec. W1/W2 IMPL SCAFFOLDING LANDED (2026-07-28, CLAIM-DEEPSEEK-V4-IMPL, base df18ca91): additive registry stub + config parse + checkpoint loader name-map VERIFIED against the real nvidia/DeepSeek-V4-Flash-NVFP4 safetensors header (HTTP-range, NO download) — new TUs deepseek_v4.{h,cpp} / _weights.cpp / _registry.cpp + test_deepseek_v4_scaffold (4/4·40: arch resolves + config descends + per-layer topology), clean CPU -Werror 0-warn; the forward is an honest VT_CHECK(false, "W3-W8 pending") stub (never a silent wrong answer). HW-FIT REVERSAL (corrects the ~83 GiB spike estimate): the NVFP4 checkpoint is 156.7 GiB (index total_size 168,266,793,544 B) — only the 256 routed experts are NVFP4/W4; the 512-wide MLA + shared-expert linears are FP8 block, plus NVFP4 double-scale (weight_scale+weight_scale_2+input_scale) overhead ⇒ it does NOT fit ONE GB10's 119 GiB unified pool (consistent with the native fp4 148.7 GiB). W1 (single-GB10 oracle run) is therefore MEMORY-INFEASIBLE, not merely disk-contended — needs multi-node TP / CPU offload / a smaller quant. Row stays SPIKE (forward + strict gate = W3-W8). W3 PRIMITIVES LANDED (2026-07-28, CLAIM-DEEPSEEK-V4-W3, base 308c312a): the genuinely-NEW-vs-V2/V3 attention math ported as portable host references + unit-gated — (A) the DSA "Lightning Indexer" sparse SELECTION (weighted-MQA logit Σ_h w·ReLU(q·k) with the load-bearing per-head ReLU, triton_fp8_mqa_logits.py:129; weight fold D^-0.5·H^-0.5; causal top-k index_topk=512 + short-context all-select) and (B) the 512-wide MLA output seams V2/V3 lack (per-head attention-sink softmax + grouped output-LoRA wo_a bmm→wo_b, o_proj.py:58-73) — new TUs deepseek_v4_dsa.{h,cpp} + test_deepseek_v4_dsa 13/13·38 (hand-derived literals + double-precision references, rel-L2 < 1e-6), clean CPU -Wall -Werror -Wextra. Honest gate: hand-case + structural review (arch not constructible at a tiny shape ⇒ no dumped-oracle rel-L2). SACRED-inert (additive; mla_attention/cuda_mla_attn untouched — shared-mla extraction is a W7 follow-on). New kernel row KERNEL-ATTN-DSA-SPARSE-INDEX (SPIKE). SGLang v0.5.15 registers+implements DeepseekV4ForCausalLM (full DSA/MHC/o_lora stack) ⇒ a viable second benchmark/dump reference, same single-GB10 memory constraint (spec §W3). W4 COMPRESSOR + fp8_ds_mla KV-STATE LANDED (2026-07-29, CLAIM-DEEPSEEK-V4-W4, base 4d1be010): the second half of the DSA stack ported as portable host references + unit-gated — the DSA COMPRESSOR forward (CompressorPoolNorm = per-head-dim-column softmax(score,dim=0)·kv window pool, the load-bearing per-column nuance, then RMSNorm; CompressorSaveScoreApe = the fused save-time score += ape[pos % compress_ratio]) + the fp8_ds_mla KV-cache state read/write layout (MakeFp8DsMlaLayout/Fp8DsMlaEncodeToken/Fp8DsMlaDecodeToken = 448-wide NoPE FP8 with per-64 UE8M0 power-of-two block scales exp=ceil(log2(absmax/448)) byte exp+127, 64-wide RoPE bf16, 576B token stride, 7+1 scale region, + the dequant read) — new TUs deepseek_v4_compressor.{h,cpp} + test_deepseek_v4_compressor 12/12·164 (hand-derived literals + double-precision references + independent UE8M0 recompute; RED-first PROVEN: scale-bias perturb fails 4/135, revert restores). Ported 1:1 from vLLM common/ops/fused_compress_quant_cache.py:198-297 + save_partial_states.py:92-101 + compressor.py:307-309, cross-checked vs SGLang v0.5.15 dsv4/dequant_k_cache.py. Clean CPU Debug full-library build 0-warn (the pre-existing GCC-13 -O2 -Werror=array-bounds false positive in voxtral.cpp is why the gate uses the Debug config). SACRED-inert (additive; shared mla_attention/cuda_mla_attn empty-diff; test_deepseek_v4_dsa still 13/13·38, test_deepseek_v4_scaffold 4/4·40). New kernel row KERNEL-ATTN-DSA-COMPRESSOR (SPIKE). Residuals: MHC (W5), sqrtsoftplus/hash MoE (W6), device kernel + forward integration + strict gate (W7-W8 = multi-Spark) W5 MHC HYPER-CONNECTIONS LANDED (2026-07-29, CLAIM-DEEPSEEK-V4-W5, base d0bc0f41): the hardest V4 brick — the Manifold/Markov Hyper-Connections residual topology ported as portable host references + unit-gated: MhcSinkhorn (the 20-iteration Sinkhorn: row-softmax seed +eps then alternating col/row normalization toward a doubly-stochastic matrix), MhcPre (folded weight-free RMSNorm projection → pre/post/comb sigmoid+Sinkhorn gates → stream collapse → optional folded attn/ffn RMSNorm), MhcPost (comb-matrix mix + post-gate residual fold), HcHeadCollapse (weight-free RMSNorm → hc_head_fn → sigmoid → weighted stream sum). New TUs deepseek_v4_mhc.{h,cpp} + test_deepseek_v4_mhc 14/14·125 (hand-derived literals + from-first-principles double-precision references, rel-L2 < 1e-5..1e-4; RED-first PROVEN both levers — iteration-count perturb fails 1/9, axis-swap fails 2/12, via a dedicated SMALL-iteration-count gate since the Sinkhorn converges by 20 iters; revert restores). EAGER-REF FINDING — corrects this row's "ZERO numerical tests and no eager reference" premise: the pinned vLLM DOES ship an eager PyTorch reference (model_executor/kernels/mhc/torch.py mhc_pre_torch/mhc_post_torch + triton.py head collapse), and four upstream impls (torch.py, tilelang_kernels.py _sinkhorn_fwd, tilelang.py, SGLang mhc.py) agree byte-for-byte on the Sinkhorn — ported 1:1 AND cross-checked against an independent double-precision derivation. Honest gate: DERIVED-eager-reference + hand-case + structural review (fixed-config 167B not constructible at a tiny shape ⇒ no dumped-oracle rel-L2). OPEN QUESTION: end-to-end bf16 residual rounding between steps is a W7 device concern, left out of the f32/f64 refs. New kernel row KERNEL-MHC-SINKHORN (SPIKE). SACRED-inert (additive; no existing forward touched; prior V4 tests 12/12·164 + 13/13·38 + 4/4·40 unchanged). Residuals now: sqrtsoftplus/hash MoE (W6), device kernel + DeepseekV4Model::Forward assembly (W7), strict/near-tie gate (W8) — all multi-Spark. W6 sqrtsoftplus + HASH-routed MoE LANDED (2026-07-29, CLAIM-DEEPSEEK-V4-W6, base 5b843be5): the three genuinely-new-vs-V2/V3 MoE pieces ported as portable host references + unit-gated — SqrtSoftplus (the V4 router score sqrt(softplus(x)), distinct from V2/V3 sigmoid/softmax), SqrtSoftplusRouteTopk (score all experts → add e_score_correction_bias for SELECTION ONLY → top-k OR the tid2eid token-id→expert HASH lookup that BYPASSES top-k → GATHER weights from the UNBIASED scores → renormalize → ×routed_scaling_factor), ClampedSwiGLU (SiluAndMulWithClamp: gate clamped max-only, up clamped both-sided, gate·σ(α·gate)·(up+β)). New TUs deepseek_v4_moe.{h,cpp} + test_deepseek_v4_moe 12/12·716 (hand-derived literals — the sqrt∘softplus composition, bias-flips-selection-but-weight-stays-unbiased, hash bypass, asymmetric clamp — + from-first-principles double-precision references; RED-first PROVEN all three levers: drop sqrt fails 8/493, biased-weight-gather fails 2/181, symmetric gate-clamp fails 2/6; revert restores 12/12·716). Ported 1:1 from vLLM fused_topk_bias_router.py:75-118 + activation.py:197-201, cross-checked SGLang v0.5.15 moe/{topk.py, hash_topk.py}. REUSE not re-port: the shared grouped-GEMM / 256-expert / shared-expert / NVFP4 machinery is untouched; only scoring+hash+clamp are net-new. MegaMoE is SM100-only so GB10 mirrors the FusedMoE-fallback router. Clean CPU Debug full-library build (the voxtral -O2 false positive forces Debug; new TUs -Wall -Werror -Wextra-clean). SACRED-inert (additive; prior V4 tests 14/14·125 + 12/12·164 + 13/13·38 + 4/4·40 unchanged). New kernel row KERNEL-MOE-SQRTSOFTPLUS-HASH (SPIKE). Residuals now: device kernels (reuse the existing grouped-GEMM) + DeepseekV4Model::Forward assembly (W7), strict/near-tie gate (W8) — all multi-Spark. GGUF QUANT-PATH UNBLOCKED (2026-07-29, CLAIM-DSV4-GGUF-LOADER, gguf-iquant-dsv4.md): the single-Spark unsloth/DeepSeek-V4-Flash-GGUF UD-IQ2_XXS (~91 GB, fits ONE GB10) vehicle's ~2-bit weight types now DEQUANT — IQ2_XXS (id 16) + Q2_K (id 10) ported 1:1 from llama.cpp ggml-quants.c (QUANT-GGUF-IQ2_XXS/QUANT-GGUF-Q2_K → ACTIVE). HTTP-range-verified the GGUF header: general.architecture=deepseek4, general.file_type=19 (=IQ2_XXS, confirms the target type), split.tensors.count=1328, and the full deepseek4.* config-KV schema (matches the safetensors arch scalars). The V4-GGUF loader (registry reject lift + blk.N.* name map) stays BLOCKED on the tensor manifest (beyond the CDN range cap; 90 GB download prohibited) and the forward (W3-W8), so an actual V4-GGUF RUN is still gated. Row stays SPIKE. W7 FORWARD ASSEMBLY LANDED (2026-07-29, CLAIM-DEEPSEEK-V4-W7, base a856383c): the VT_CHECK(false, "W3-W8 pending") stub is REPLACED by a REAL DeepseekV4Model::Forward (DeepseekV4ForwardHost) that COMPOSES the four landed host primitives (W3 DSA/MLA seams, W4 compressor + fp8_ds_mla KV, W5 MHC + Sinkhorn, W6 sqrtsoftplus/hash MoE) into an end-to-end logits producer on the portable CPU path at a SMALL synthetic config — the brick that finally makes V4 structurally runnable. Interleave grounded 1:1 (nvidia/model.py:1080-1148 model.forward + :866-957 decoder.forward): embed → per layer [first-layer MHC-pre stream EXPAND [T,H]→[T,hc,H], else fused MhcPost(prev-ffn)+MhcPre(attn)] → 512-wide MLA (q/kv+norms, RoPE, DSA indexer→topk→compressor→fp8_ds_mla KV round-trip, sink softmax, grouped o-LoRA) → fused MhcPost(attn)+MhcPre(ffn) → MoE (sqrtsoftplus/hash router + shared+routed clamped-SwiGLU) → final MhcPost → hc_head collapse → norm → lm_head. Gate test_deepseek_v4_forward 6/6·26 — STRUCTURAL/composition (finite logits end-to-end + deterministic + shape [T,vocab]; MHC stream [T,hc,H] via residual_stream_elems==T*4*H, hc_mult==4; hash layers route by tid2eid layer_hash_routed=={1,1,0,0} vs gated learned top-k; DSA indexer SELECTS layers 1,3 index_topk=3 + compressor POOLS layers 1,2,3; logits_indices gather) + RED-first PROVEN 3 levers (all-gated hash, skip-final-MhcPost, no-sink each change the output). Honest 3-state: the CPU forward assembly at tiny shape = DERIVED + BUILD-VERIFIED (structural) — does NOT claim V4 "runs" a real model; documented tiny-vs-167B divergences (W=2 compressor window, full-latent MLA value, single rope_theta, 1-block quant). SACRED-inert (only deepseek_v4.{h,cpp} + new test + CMake; shared MLA/MoE + W3-W6 TUs empty-diff; prior V4 tests 4/40+13/38+12/164+14/125+12/716 unchanged); CPU Debug (voxtral -O2 false positive), new TU -Wall -Werror -Wextra-clean. NO new kernel row / no checker bump. Residuals: W7-device CUDA kernels (MHC Sinkhorn, DSA indexer/compressor, sqrtsoftplus router, clamped SwiGLU; expert GEMM REUSES NVFP4/FP8 grouped-GEMM) + ForwardDevice; W2b real-tower materialization; W8 strict/near-tie engine gate (multi-Spark, 156.7 GiB); the IQ2_XXS-GGUF vehicle also needs the GGUF blk.N.* name-map (W2, manifest-download-blocked). W7-DEVICE LANDED + DGX-GATED (2026-07-29, CLAIM-DEEPSEEK-V4-W7-DEVICE, base 33016f34): the CUDA kernels for the four NEW V4 op families (MHC Sinkhorn/pre/post/head; DSA indexer weight-fold + weighted-MQA ReLU logits + causal top-k + attention-sink softmax + grouped output-LoRA; compressor pool+norm + save-APE + fp8_ds_mla KV encode/decode; sqrtsoftplus/hash router + clamped SwiGLU), each a 1:1 device port of the landed host reference, registered through the vt OpProvider seam (kDeepseekV4{Mhc,Dsa,Compressor,Moe}) and dispatched by a real DeepseekV4Model::ForwardDevice (the ONE composition now runs on host refs OR the device kernels via a V4Backend policy). NEW TUs src/vt/cuda/cuda_deepseek_v4.cu + deepseek_v4_device.{h,cpp} + test_cuda_deepseek_v4.cpp; the 512-wide MLA attn + expert grouped-GEMM REUSE the existing NVFP4/FP8 kernels (NOT re-ported). DGX GB10 (sm_121a) UNIT GATE test_cuda_deepseek_v4 11/11 cases · 153 assertions GREEN vs the host-ref oracle at small shape (BIT-EXACT top-k/router ids, near-tie rel-L2 under 1e-4 for the fp reductions, fp8_ds_mla within e4m3 granularity), the ForwardDevice composition gate (device == host rel-L2 under 2e-3), compute-sanitizer memcheck 0 errors, RED-first proven (drop sqrt fails 3/6). CUDA + CPU -Werror clean (the pre-existing GCC-13 voxtral -Werror=array-bounds/-Wstringop-overflow false positive #155 neutralized locally with a scoped pragma). SACRED-inert (shared MLA/MoE CUDA + W3-W6 host TUs empty-diff; host oracle 6/6·26 unchanged). New kernel row KERNEL-DSV4-W7-DEVICE (SPIKE, count 42→43). Honest 3-state: kernels RUNTIME-VERIFIED at small shape on real GB10; the real-checkpoint e2e stays W8 (156.7 GiB does not fit ONE GB10) plus W2b tower materialization plus the GGUF blk.N.* name-map. Row stays SPIKE (real strict gate = W8) W2b GGUF→TOWER MATERIALIZATION LANDED (2026-07-29, CLAIM-DEEPSEEK-V4-W2B, base 341dfbb9): the last CODE brick before the real single-Spark run — LoadDeepseekV4FromGguf (+ DeepseekV4ParamsFromGguf) in deepseek_v4_weights.cpp (+ the DeepseekV4GgufWeights tower + declarations in deepseek_v4.h) wires the landed blk.N.*→V4 name-map + the landed keep-quant vec_dot into the DeepseekV4 weight towers, mirroring the Qwen3.6 GGUF path (qwen3_5_gguf_weights.cpp). Every GGUF tensor routes through GgufLoadPolicy::Route with its name-map role: MW/SEW (512-wide MLA linears wq_a/wq_b/wkv/wo_a/wo_b, router gate, shared + 256 routed experts, lm_head) KEEP their ~2-3-bit blocks COMPRESSED via OwnGgufQuantBlocks (the memory enabler — dequant-to-bf16 would need ~316 GiB and OOM-reboot the unified pool); the small V/ET/HASH tensors (norms, MHC hc_*, DSA compressor/indexer, attention sinks, embed, the tid2eid hash table, exp_probs_b bias) dequant to f32. The load ACCOUNTS for every tensor (throws on an unmapped required tensor OR a leftover file tensor the map does not cover), ALSO dequants the tiny-config CPU composition host tower so a loaded model FORWARDs, and the registry GGUF reject is LIFTED (source.kind == kGguf → LoadDeepseekV4FromGguf). Structural gate test_deepseek_v4_gguf_load 5/5·149 on a TINY SYNTHETIC deepseek4 GGUF (real blk.N.* naming + Q8_0 keep-quant, tiny dims): accounting 126/126 (== the file tensor set, none unmapped/leftover, per-layer hash/compressor/indexer topology mapped), keep-quant residency (the 256-expert down proj stays Q8_0, byte size < the dequant-f32 image; an MLA linear + lm_head stay Q8_0; a norm stays F32), load→DeepseekV4ForwardHost finite+deterministic end-to-end, RED-first (a missing required tensor OR a leftover unmapped tensor OR an expand-instead-of-keep policy each changes the outcome). Honest 3-state: tiny-synthetic load→forward = DERIVED + BUILD-VERIFIED (CPU Debug, loader-only TUs -Werror-clean); the real 91 GB UD-IQ2_XXS load + generate stays W8-FINAL (operational: download + DGX keep-quant load + ForwardDevice greedy gen + self-consistency/coherence gate + benchmark). SACRED-inert PROVEN: only deepseek_v4_weights.cpp + deepseek_v4.h + deepseek_v4_registry.cpp + the new test + CMake changed; the forward (deepseek_v4.cpp) + the W3-W6 primitive TUs + the shared MLA/MoE (mla_attention, cuda_mla_attn) empty-diff; prior V4 tests test_deepseek_v4_forward 6/26 + test_deepseek_v4_scaffold 4/40 unchanged. No new kernel-matrix row (loader wiring, not a kernel family) ⇒ no checker count bump. W8-run COHERENT GENERATION (2026-07-29, CLAIM-DEEPSEEK-V4-COHERENCE, NOT pushed): the real single-Spark run WORKS. The forward runs the real 80.7 GB ds4 q2-imatrix model end-to-end on the DGX GB10 and greedily generates "The capital of France is Paris.<｜end▁of▁sentence｜>" (correct answer + EOS). Root-caused via per-layer diff vs the antirez/ds4 oracle: (1) MISSING per-head query RMS-norm (ds4 head_rms_norm_inplace after wq_b; DeepseekV4QHeadRmsNormInplace added + doctest, RED-first) — L00 q rel-L2 0.9646→0.0013, and the full 43-layer curve COLLAPSED to the keep-quant floor (L33 0.53→0.003, L34 6.02→0.003); (2) MISSING inverse-RoPE on the attention output (ds4 rope(heads, inverse=true); identity at pos 0, load-bearing for pos>0). Gate test_deepseek_v4_gguf_load 10/10·403. Self-consistent (deterministic greedy). CPU-tier decode ~3.3 s/tok / peak 85.8 GiB (ds4 GPU: 16.5 tok/s). Named residuals: GPU-expert dispatch (CPU-tier speed), the DSA-SPARSE path for ctx>512 (dense-fallback, exact for short gen), ds4-oracle (vLLM has no V4 GGUF plugin ⇒ not vLLM-token-exact). Row ADVANCES SPIKE → ACTIVE. | deepseek-v4-flash spike | ACTIVE | DeepseekV4QHeadRmsNormInplace + inverse-RoPE; test_deepseek_v4_gguf_load 10/10·403; coherent GB10 gen | CLAIM-DEEPSEEK-V4-IMPL |
MODEL-TEXT-ernie45-ernie4-5-for-causal-lm | Ernie4_5ForCausalLM | registry.py:95; vllm/model_executor/models/ernie45.py::Ernie4_5ForCausalLM | causal generation / text | model loader/forward | ☐ required | INVENTORIED | none | unassigned |
MODEL-TEXT-ernie45-moe-ernie4-5-moe-for-causal-lm | Ernie4_5_MoeForCausalLM | registry.py:96; vllm/model_executor/models/ernie45_moe.py::Ernie4_5_MoeForCausalLM | causal generation / text | model loader/forward; paged attention/KV; FusedMoE/grouped GEMM | ☐ required | INVENTORIED | none | unassigned |
MODEL-TEXT-exaone-exaone-for-causal-lm | ExaoneForCausalLM | registry.py:97; vllm/model_executor/models/exaone.py::ExaoneForCausalLM | causal generation / text | model loader/forward; paged attention/KV | ☐ required | INVENTORIED | none | unassigned |
MODEL-TEXT-exaone4-exaone4-for-causal-lm | Exaone4ForCausalLM | registry.py:98; vllm/model_executor/models/exaone4.py::Exaone4ForCausalLM | causal generation / text | model loader/forward; paged attention/KV; sliding-window attention | ☐ required | INVENTORIED | none | unassigned |
MODEL-TEXT-exaone-moe-exaone-moe-for-causal-lm | ExaoneMoEForCausalLM | registry.py:99; vllm/model_executor/models/exaone_moe.py::ExaoneMoeForCausalLM | causal generation / text | model loader/forward; FusedMoE/grouped GEMM | ☐ required | INVENTORIED | none | unassigned |
MODEL-TEXT-fairseq2-llama-fairseq2-llama-for-causal-lm | Fairseq2LlamaForCausalLM | registry.py:100; vllm/model_executor/models/fairseq2_llama.py::Fairseq2LlamaForCausalLM | causal generation / text | model loader/forward | ☐ required | INVENTORIED | none | unassigned |
MODEL-TEXT-falcon-falcon-for-causal-lm | FalconForCausalLM, RWForCausalLM | registry.py:101,193; vllm/model_executor/models/falcon.py::FalconForCausalLM | causal generation / text | model loader/forward; paged attention/KV | ☐ required | INVENTORIED | none | unassigned |
MODEL-TEXT-mamba-mamba-for-causal-lm | FalconMambaForCausalLM, MambaForCausalLM | registry.py:102,148; vllm/model_executor/models/mamba.py::MambaForCausalLM | causal generation / text | model loader/forward; Mamba/SSM state | ☐ required | INVENTORIED | none | unassigned |
MODEL-TEXT-falcon-h1-falcon-h1-for-causal-lm | FalconH1ForCausalLM | registry.py:103; vllm/model_executor/models/falcon_h1.py::FalconH1ForCausalLM | causal generation / text | model loader/forward; paged attention/KV; Mamba/SSM state | ☐ required | INVENTORIED | none | unassigned |
MODEL-TEXT-flex-olmo-flex-olmo-for-causal-lm | FlexOlmoForCausalLM | registry.py:104; vllm/model_executor/models/flex_olmo.py::FlexOlmoForCausalLM | causal generation / text | model loader/forward; FusedMoE/grouped GEMM | ☐ required | INVENTORIED | none | unassigned |
MODEL-TEXT-gemma-gemma-for-causal-lm | GemmaForCausalLM | registry.py:105; vllm/model_executor/models/gemma.py::GemmaForCausalLM | causal generation / text | model loader/forward; paged attention/KV. Simplest Gemma: only post_attention_layernorm (fused add), no soft-cap, no QK-norm; needs the NEW GeGLU (gelu_pytorch_tanh+mul) activation + sqrt(hidden) embed-scale (we have neither); gemma-RMSNorm + SentencePiece + tied embeddings REUSED. Gate vehicle google/gemma-1.1-2b-it 2.5B — FITS. Lowest priority (W5) | sweep-gemma spike | PARTIAL | W3-W6 LANDED 2026-07-24: gemma.{h,cpp} + gemma_weights.cpp + gemma_registry.cpp; STRICT SACRED test_gemma_forward 48/48 (gemma-2b) + loader test_gemma_load 164 tensors; ledger 2026-07-24. Speed PENDING; registration src/vllm/model_executor/models/gemma_registry.cpp:112; test tests/vllm/models/test_model_registry.cpp:106 | CLAIM-SWEEP-GEMMA |
MODEL-TEXT-gemma2-gemma2-for-causal-lm | Gemma2ForCausalLM | registry.py:106; vllm/model_executor/models/gemma2.py::Gemma2ForCausalLM | causal generation / text | model loader/forward; paged attention/KV; sliding-window attention. Gemma-2 = dense + attn logit soft-cap (:202) + final logit soft-cap (:345) + query_pre_attn_scalar scaling + interleaved sliding/global ("hybrid attention" upstream test) + sandwich norms. Sliding window + sandwich norms + gemma-RMSNorm REUSED; NEW = GeGLU + final soft-cap + attn-soft-cap wiring + qpas. Gate vehicles google/gemma-2-2b-it 2.6B / gemma-2-9b-it 9.2B — FIT. Proves the soft-cap primitives (W4) | sweep-gemma spike | PARTIAL | W3-W6 LANDED 2026-07-24: gemma2.{h,cpp} + gemma2_weights.cpp + gemma2_registry.cpp + vt::SoftCap/PagedAttentionArgs.logits_soft_cap (ops.cpp); near-tie-band SACRED test_gemma2_forward 48/48 (44 strict + 4 at 0.0-nat vLLM ties) + loader test_gemma2_load 288 tensors; ledger 2026-07-24. Speed PENDING; registration src/vllm/model_executor/models/gemma2_registry.cpp:116; test tests/vllm/models/test_model_registry.cpp:105 | CLAIM-SWEEP-GEMMA |
MODEL-TEXT-gemma3-gemma3-for-causal-lm | Gemma3ForCausalLM | registry.py:107; vllm/model_executor/models/gemma3.py::Gemma3ForCausalLM | causal generation / text | model loader/forward; paged attention/KV; sliding-window attention. THE recommended gate vehicle (recent-first + clean text + oracle-certain). Gemma-3 REMOVED the attn soft-cap and added QK-norm (gemma-RMSNorm on head_dim, REUSES kAttnQkNormRopeGate) + dual per-layer rope theta (global rope_theta vs local rope_local_base_freq, :158-176) + sandwich norms. REUSED: gemma-RMSNorm, sandwich norms (glm4 b568d20), SentencePiece, sliding window (FA-2 + SlidingWindow/ChunkedLocalAttention specs), QK-norm recipe, tied embeddings. NEW: GeGLU activation + final soft-cap + qpas/embed-scale scalars + dual-rope routing. Gate vehicles google/gemma-3-1b-it 1.0B (primary) / gemma-3-270m 268M / gemma-3-4b-it 4.3B text backbone (strict) — all FIT (W2) | sweep-gemma spike | PARTIAL | W0-W2 LANDED 2026-07-24: gemma3.{h,cpp} + gemma3_weights.cpp + gemma3_registry.cpp + kGeluAndMul/kMulScalar (ops.cpp); tokenizer-free SACRED gate test_gemma3_forward STRICT 48/48 + loader test_gemma3_load 340 tensors; ledger 2026-07-24. Speed PENDING; registration src/vllm/model_executor/models/gemma3_registry.cpp:122; test tests/vllm/models/test_model_registry.cpp:101 | CLAIM-SWEEP-GEMMA |
MODEL-TEXT-rnj1-rnj1-for-causal-lm | Rnj1ForCausalLM | registry.py:108; vllm/model_executor/models/rnj1.py::Rnj1ForCausalLM | causal generation / text | model loader/forward; paged attention/KV; sliding-window attention | ☐ required | INVENTORIED | none | unassigned |
MODEL-TEXT-gemma3n-gemma3n-for-causal-lm | Gemma3nForCausalLM | registry.py:109; vllm/model_executor/models/gemma3n.py::Gemma3nForCausalLM | causal generation / text | model loader/forward; paged attention/KV; sliding-window attention | ☐ required | INVENTORIED | none | unassigned |
MODEL-TEXT-gemma4-gemma4-for-causal-lm | Gemma4ForCausalLM | registry.py:110; vllm/model_executor/models/gemma4.py::Gemma4ForCausalLM | causal generation / text | model loader/forward; paged attention/KV; FusedMoE/grouped GEMM; sliding-window attention. The NEWEST registered Gemma — leads the characterization, gate-BLOCKED as a first vehicle. Real (public google/gemma-4-12B-it 11.96B / gemma-4-31B-it 32.68B / gemma-4-26B-A4B-it 26.5B MoE) but ALL checkpoints are multimodal-wrapped (Gemma4*ForConditionalGeneration/Gemma4Unified…); the bare text row has no standalone checkpoint (extract via text_config). Needs a large NEW primitive stack no other row uses: Per-Layer Embeddings (PLE), YOCO self/cross split, KV-sharing, a parallel Gemma-4 MoE (softmax-over-all + per_expert_scale router), k_eq_v layers, weight-less V-norm, double-wide MLP, per-layer scalar; scaling=1.0. Two blocking preconditions, one now RESOLVED: (a) CLAIM-GEMMA4-W0) — vLLM 0.25.0 loads+runs+generates the Gemma-4 backbone e2e inside the unsloth/gemma-4-E4B-it mm wrapper (STRICT greedy golden), so oracle support is PROVEN; (b) STILL no standalone bare-text Gemma4ForCausalLM checkpoint (extract via text_config) + the backbone campaign. Scoped follow-on campaign; W6 honesty pass only | sweep-gemma spike | BLOCKED | W6 honesty pass LANDED 2026-07-24: test_gemma4_honesty 6/6 — NOT registered (no over-claim), HW/DEP-BLOCKED (MM-wrapped-only ckpts + PLE/YOCO/MoE campaign); ledger 2026-07-24 | CLAIM-SWEEP-GEMMA |
MODEL-TEXT-qwen3-next-qwen3-next-for-causal-lm | Qwen3NextForCausalLM | registry.py:111; vllm/model_executor/models/qwen3_next.py::Qwen3NextForCausalLM | causal generation / text | model loader/forward; paged attention/KV; FusedMoE/grouped GEMM; Mamba/SSM state; GDN/linear-attention state | ☐ required | INVENTORIED | none | unassigned |
MODEL-TEXT-glm-glm-for-causal-lm | GlmForCausalLM | registry.py:112; vllm/model_executor/models/glm.py::GlmForCausalLM | causal generation / text | model loader/forward; a 24-line subclass of LlamaForCausalLM with three post-construction deltas (glm.py:11,14,22-24) — partial_rotary_factor = 0.5, is_neox_style = False, o_proj bias dropped with skip_bias_add. Needs the NEW partial-rotary primitive (we have NONE: no partial_rotary_factor anywhere in src//include/) but NOT sandwich norms. Independent of MLA. Gate vehicle zai-org/glm-4-9b-chat-hf 9.4B / 17.5 GiB bf16 — FITS GB10. NOTE the same class is double-registered as an embedding model at registry.py:218, tracked by MODEL-EMBED-glm-glm-for-causal-lm, NOT claimed here | glm-dsa-latest-deepseek spike | SPIKE | none | CLAIM-GLM-DSA-LATEST-DEEPSEEK |
MODEL-TEXT-glm4-glm4-for-causal-lm | Glm4ForCausalLM | registry.py:113; vllm/model_executor/models/glm4.py::Glm4ForCausalLM | causal generation / text | GLM-4-9B-0414 dense LANDED (G2). Impl: include/vllm/model_executor/models/glm4.h + src/vllm/model_executor/models/{glm4,glm4_weights,glm4_registry}.cpp (one REGISTER_VLLM_MODEL, reuses the shared dense glue). The two "new primitives" reduced to EXISTING infra: partial + INTERLEAVED rope routes RopeFromCache with is_neox_style=false (cuda_ops.cu:697-698 / cpu_ops.cpp:744-746, the DeepSeek-V2 decoupled-rope path) over rotary_dim=0.5*head_dim=64, tail passed through; SANDWICH NORMS (glm4.py:206,211) are standalone vt::RmsNorm on the attn/mlp output. Biased qkv (attention_bias:true, vt::Add row-broadcast, 1-D LoadMergedBf16Vector), pre-merged gate_up_proj, no QK-norm, GQA 32/2, untied lm_head, MTP-tail skip. Loader 523 tensors, zero missing/unmapped. Tests: tests/parity/test_glm4_paged_engine.cpp (SACRED), rope unit gate at GLM dims both layouts (tests/vt/test_ops_rope_cache.cpp), registry resolution. Runs EAGER (bf16, no decode graph). Gate: 16/16 vs vLLM 0.25.0 (STRICT 13/16 + near-tie 3/16, max gap 0 nats) — speed pending | glm-dsa-latest-deepseek spike | READY | test_glm4_paged_engine 16/16 (dgx); rope unit 6692/6692; registry 22/22; registration src/vllm/model_executor/models/glm4_registry.cpp:120; test tests/vllm/models/test_model_registry.cpp:97 | - |
MODEL-TEXT-glm4-moe-glm4-moe-for-causal-lm | Glm4MoeForCausalLM | registry.py:114; vllm/model_executor/models/glm4_moe.py::Glm4MoeForCausalLM | causal generation / text | model loader/forward; paged attention/KV; FusedMoE/grouped GEMM; GLM-4.5/4.6/4.7 (glm4_moe.py:24-25) = Qwen3-MoE attention + DeepSeek-V2 router: GQA with optional QK-norm (:305-322) and partial NeoX rope (:289), NOT MLA; but a router that is a near-verbatim DeepseekV2MoE port — fp32 nn.Linear gate (:147-152), e_score_correction_bias (:153-155), sigmoid scoring (:204), grouped top-k (:200-202), routed_scaling_factor (:206-207), first_k_dense_replace (:362-379). Needs the SHARED router extension (our vt::MoeRouterTopKArgs has only top_k+renormalize). HW-BLOCKED e2e: smallest genuine checkpoint zai-org/GLM-4.5-Air 110.5B / 205.8 GiB bf16 vs GB10's ~119 GiB; the 104.8 GiB FP8 variant depends on an fp8 checkpoint-loading row we do not own. Gateable subset = config/registry resolution + weight-map on a single shard + router unit parity at 128-expert/top-8 | glm-dsa-latest-deepseek spike | SPIKE | none | CLAIM-GLM-DSA-LATEST-DEEPSEEK |
MODEL-TEXT-glm4-moe-lite-glm4-moe-lite-for-causal-lm | Glm4MoeLiteForCausalLM | registry.py:115; vllm/model_executor/models/glm4_moe_lite.py::Glm4MoeLiteForCausalLM | causal generation / text | model loader/forward; FusedMoE/grouped GEMM; MLA/latent KV; ★ THE RECOMMENDED SECOND MLA GATE VEHICLE. GLM-4.7-Flash = DeepSeek-V2 with GLM's MoE bolted in: glm4_moe_lite.py:94-95 and :98-99 are LITERAL zero-override subclasses of DeepseekV2Attention / DeepseekV2MLAAttention; decoder layer, model and load_weights (incl. the fused_qkv_a_proj merge :330-335, :544-551) are structural copies of deepseek_v2; only Glm4MoeLite = Glm4MoE (:86-87) is GLM-specific. Verified live config: q_lora_rank: 768 (non-null), topk_method: noaux_tc (so e_score_correction_bias present), kv_lora 512, qk_nope 192 / qk_rope 64 / v_head 256, 47L, 64 experts + 1 shared, top-4, NO index_topk so is_v32 == False — DENSE MLA, not DSA. zai-org/GLM-4.7-Flash 31.2B / 58.2 GiB bf16 — FITS GB10. EXTENDS the MLA campaign: closes BOTH coverage gaps that spike named as unit-gated-only on DeepSeek-V2-Lite (q_lora_rank=null; no e_score_correction_bias). Gated on CLAIM-MLA-DEEPSEEK reaching its W6 | glm-dsa-latest-deepseek spike | ACTIVE (correctness COMPLETE, speed PENDING) | G1 LANDED 2026-07-24 — SACRED gate 8/8 vs vLLM 0.25.0. GLM-4.7-Flash reuses the ENTIRE DeepSeek-V2 MLA stack over the SAME DeepseekV2Weights — the noaux_tc grouped router (already landed in the MLA campaign's W3), the MLA attention block with the q_lora branch (W6), the bf16 grouped MoE + shared expert (W7), the decode CUDA-graph (W9). Genuinely NEW work: (1) an ADDITIVE head_dim=256 dispatch in the MLA prefill launcher (cuda_flash_attn_fa2.cu LaunchMlaPrefillFA2Bf16 — GLM's qk_nope 192 + qk_rope 64 = 256; the 256 split-KV instantiation was already compiled for the 27B/35B paged prefill, so the 192 path is byte-identical); (2) a MTP-tolerant parse/loader (allow_mtp_tail on ParseDeepseekV2Params/LoadDeepseekV2ForCausalLMWeights — GLM ships num_nextn_predict_layers: 1, defaulted false so DeepSeek-V2 is byte-identical); (3) the GLM registry TU glm4_moe_lite_registry.cpp; (4) a scoring-func fix — GLM's config OMITS scoring_func and its model class hardcodes sigmoid, so noaux_tc + absent-key now defaults to sigmoid (DeepSeek-V2-Lite is greedy → softmax UNCHANGED). THE GATE (measured, not assumed): vLLM 0.25.0 is DETERMINISTIC at batch=1 (K=5, 0 multi-valued cells) → STRICT bar. Our engine: STRICT token-exact 1/8, near-tie-band 7/8, 69/128 tokens strictly exact; the teacher-forcing diagnostic shows ALL 59 divergent positions at gap EXACTLY 0.0000 nats (vLLM's OWN argmax on OUR prefix IS our token), 0 tokens outside vLLM's top-20, 0 forward-divergent — a cleaner pass than DeepSeek-V2's (0.25-nat root flip). PROOF THE PATH RAN: fa_page_size 36864 = block 32 × 576 × 2B (NO factor 2 — real MLA cache); split stats prefill_only=8/decode_only=120 (8 MLA prefills + 120 MLA decodes); coherent English ("Paris. The capital of the United Kingdom is London..."). CLOSES the MLA campaign's C2 gaps: the q_lora query branch (fused_qkv_a_proj/q_a_layernorm/q_b_proj) and the whole noaux_tc router (sigmoid + e_score_correction_bias + routed_scaling_factor 1.8 + renormalize) now have e2e coverage (were unit-gated-only on DeepSeek-V2-Lite). LOADER GATE 3/3 cases / 57117 assertions (9491 main-model tensors mapped, 212 MTP-tail skipped, 0 unmapped, 0 missing; q_lora branch + f32 e_score_correction_bias asserted). ROUTER unit gate at real GLM dims (64E/top-4/n_group1/routed_scale 1.8/noaux_tc/renorm) vs CPU ref, CUDA==CPU, memcheck 0 errors. compute-sanitizer memcheck on the FULL GLM gate (head_dim-256 MLA prefill + MoE glue + MLA decode): ERROR SUMMARY 0 errors, gate still 8/8. eager==graph bit-identical (VT_DEEPSEEK_CUDAGRAPH=0 same 8/8 + anchored tokens). Clean full CUDA -Werror 0 warn/0 err. Regressions ALL byte-identical STANDALONE under flock: 27B 235/235, 35B 315/315, Qwen3-Coder 138/138, Qwen3-dense 184/184, OPT 63/63, DeepSeek-V2 223/223 (Release AND asserts-on exit 0 — the shared-TU canary), Llama 92/92, Mistral 92/92. Speed PENDING (no number; DONE needs vLLM every-axis parity).; registration src/vllm/model_executor/models/glm4_moe_lite_registry.cpp:179; test tests/vllm/models/test_glm4_moe_lite_load.cpp:102 | CLAIM-GLM-DSA-LATEST-DEEPSEEK |
MODEL-TEXT-deepseek-v2-glm-moe-dsa-for-causal-lm | GlmMoeDsaForCausalLM | registry.py:116; vllm/model_executor/models/deepseek_v2.py::GlmMoeDsaForCausalLM | causal generation / text | model loader/forward; paged attention/KV; FusedMoE/grouped GEMM; MLA/latent KV; DSA sparse indexer. GLM-5.x is DeepSeek-V3.2 VERBATIM — deepseek_v2.py:1917-1918 is class GlmMoeDsaForCausalLM(DeepseekV2ForCausalLM): pass; the ONLY behavioural special case is fp32 router dtype forced by model_type == "glm_moe_dsa" (:120-130). Numerical deltas vs V3.2 live in the newer tree: interleaved indexer RoPE vs NeoX split-half (vllm/models/deepseek_v32/nvidia/kernels.py:300,697) and index_topk_freq=4 (nvidia/attention.py:206); live zai-org/GLM-5 config confirms indexer_rope_interleave: true, index_topk: 2048, index_n_heads: 32. DOUBLY BLOCKED: (1) DEP-BLOCKED — on sm_121 the sparse XOR filter (vllm/v1/attention/backend.py:345-350) ELIMINATES TRITON_MLA, leaving FLASHINFER_MLA_SPARSE_SM120 as the SOLE candidate, and that path is non-functional on flashinfer 0.6.12 (4 concrete failures, spike §0.2); (2) HW-BLOCKED — 753.9B / 1404.2 GiB vs 119 GiB memory and 184 GiB disk. Registry/config resolution only | glm-dsa-latest-deepseek spike | BLOCKED | W10 (blocked-row honesty pass, 2026-07-22), cross-claim edit by CLAIM-MLA-DEEPSEEK with the disposition recorded in coordination.md — HW-BLOCKED and DEP-BLOCKED. HW: zai-org/GLM-5 is 1404.2 GiB bf16 against 119 GiB of unified memory (measured 2026-07-21 by the GLM spike). DEP: GLM-5.x is DeepSeek-V3.2 VERBATIM (deepseek_v2.py:1917-1918 is class GlmMoeDsaForCausalLM(DeepseekV2ForCausalLM): pass), so it is a SPARSE/DSA model and inherits exactly the sm_121 dependency dead-end recorded on MODEL-TEXT-deepseek-v2-deepseek-v3-for-causal-lm: the sparse XOR filter removes TRITON_MLA, leaving FLASHINFER_MLA_SPARSE_SM120 alone, and that path dispatches to flashinfer's dense-only XQA backend which discards sparse_mla_top_k. Upstream-watch item, not work. What can still be gated: config resolution and the fp32-router special case (deepseek_v2.py:120-130) at unit level; nothing e2e. none | CLAIM-GLM-DSA-LATEST-DEEPSEEK |
MODEL-TEXT-gpt-oss-gpt-oss-for-causal-lm | GptOssForCausalLM | registry.py:117; vllm/model_executor/models/gpt_oss.py::GptOssForCausalLM | causal generation / text | model loader/forward; paged attention/KV; FusedMoE/grouped GEMM; sliding-window attention | ☐ required | INVENTORIED | none | unassigned |
MODEL-TEXT-gpt2-gpt2-lmhead-model | GPT2LMHeadModel | registry.py:118; vllm/model_executor/models/gpt2.py::GPT2LMHeadModel | causal generation / text | model loader/forward; paged attention/KV; cross-attention | ☐ required | INVENTORIED | none | unassigned |
MODEL-TEXT-gpt-j-gptjfor-causal-lm | GPTJForCausalLM | registry.py:119; vllm/model_executor/models/gpt_j.py::GPTJForCausalLM | causal generation / text | model loader/forward; paged attention/KV | ☐ required | INVENTORIED | none | unassigned |
MODEL-TEXT-gpt-neox-gptneo-xfor-causal-lm | GPTNeoXForCausalLM | registry.py:120; vllm/model_executor/models/gpt_neox.py::GPTNeoXForCausalLM | causal generation / text | model loader/forward; paged attention/KV | ☐ required | INVENTORIED | none | unassigned |
MODEL-TEXT-granite-granite-for-causal-lm | GraniteForCausalLM | registry.py:121; vllm/model_executor/models/granite.py::GraniteForCausalLM | causal generation / text | IMPLEMENTED (batch3): include/vllm/model_executor/models/granite.h + src/vllm/model_executor/models/{granite,granite_weights,granite_registry}.cpp (one REGISTER_VLLM_MODEL). Llama dense forward + FOUR default-1 scalar multipliers threaded (embedding_multiplier 12.0 after embed via vt::MulScalar; residual_multiplier 0.22 scaled residual add per sublayer via vt::MulScalar+vt::Add; attention_multiplier 0.015625=1/64 as the paged-attn softmax scale, NOT 1/sqrt(64); logits_scaling 8.0 as vt::MulScalar 1/8 on logits). Reuses shared dense glue (dense_attn_block.h), separate q/k/v→merged qkv loader, tied lm_head. ZERO new compute kernel. Runs EAGER (bf16). Gate: test_granite_paged_engine 16/16 vs vLLM 0.25.0 on ibm-granite/granite-3.3-2b-instruct (vLLM K=5 ALL-DETERMINISTIC → STRICT bar; 15/16 token-exact + 1 near-tie 0.062 nats @ p13tok0, 0 forward-divergent). Speed pending. | ✅ recent-dense-batch | PARTIAL | test_granite_paged_engine 16/16 (dgx, 92 assertions); loader zero-unmapped on granite-3.3-2b; registry resolves GraniteForCausalLM; registration src/vllm/model_executor/models/granite_registry.cpp:114; test tests/vllm/models/test_model_registry.cpp:113 | CLAIM-SWEEP-RECENT-DENSE (Claude Code opus-4-8) |
MODEL-TEXT-granitemoe-granite-moe-for-causal-lm | GraniteMoeForCausalLM | registry.py:122; vllm/model_executor/models/granitemoe.py::GraniteMoeForCausalLM | causal generation / text | model loader/forward; paged attention/KV; FusedMoE/grouped GEMM | ☐ required | INVENTORIED | none | unassigned |
MODEL-TEXT-granitemoehybrid-granite-moe-hybrid-for-causal-lm | GraniteMoeHybridForCausalLM | registry.py:123; vllm/model_executor/models/granitemoehybrid.py::GraniteMoeHybridForCausalLM | causal generation / text | model loader/forward; paged attention/KV; Mamba/SSM state | ☐ required | INVENTORIED | none | unassigned |
MODEL-TEXT-granitemoeshared-granite-moe-shared-for-causal-lm | GraniteMoeSharedForCausalLM | registry.py:124; vllm/model_executor/models/granitemoeshared.py::GraniteMoeSharedForCausalLM | causal generation / text | model loader/forward | ☐ required | INVENTORIED | none | unassigned |
MODEL-TEXT-gritlm-grit-lm | GritLM | registry.py:125; vllm/model_executor/models/gritlm.py::GritLM | causal generation / text | model loader/forward; sliding-window attention | ☐ required | INVENTORIED | none | unassigned |
MODEL-TEXT-hrm-text-hrm-text-for-causal-lm | HrmTextForCausalLM | registry.py:126; vllm/model_executor/models/hrm_text.py::HrmTextForCausalLM | causal generation / text | model loader/forward; paged attention/KV | ☐ required | INVENTORIED | none | unassigned |
MODEL-TEXT-hunyuan-v1-hun-yuan-mo-ev1-for-causal-lm | HunYuanMoEV1ForCausalLM | registry.py:127; vllm/model_executor/models/hunyuan_v1.py::HunYuanMoEV1ForCausalLM | causal generation / text | model loader/forward; paged attention/KV; FusedMoE/grouped GEMM; cross-attention | ☐ required | INVENTORIED | none | unassigned |
MODEL-TEXT-hunyuan-v1-hun-yuan-dense-v1-for-causal-lm | HunYuanDenseV1ForCausalLM | registry.py:128; vllm/model_executor/models/hunyuan_v1.py::HunYuanDenseV1ForCausalLM | causal generation / text | model loader/forward; paged attention/KV; FusedMoE/grouped GEMM; cross-attention | ☐ required | INVENTORIED | none | unassigned |
MODEL-TEXT-hy-v3-hyv3-for-causal-lm | HYV3ForCausalLM | registry.py:129; vllm/model_executor/models/hy_v3.py::HYV3ForCausalLM | causal generation / text | model loader/forward; paged attention/KV; FusedMoE/grouped GEMM | ☐ required | INVENTORIED | none | unassigned |
MODEL-TEXT-hyperclovax-vision-hcxvision-for-causal-lm | HCXVisionForCausalLM | registry.py:130; vllm/model_executor/models/hyperclovax_vision.py::HCXVisionForCausalLM | causal generation / text | model loader/forward | ☐ required | INVENTORIED | none | unassigned |
MODEL-TEXT-hyperclovax-vision-v2-hcxvision-v2-for-causal-lm | HCXVisionV2ForCausalLM | registry.py:131; vllm/model_executor/models/hyperclovax_vision_v2.py::HCXVisionV2ForCausalLM | causal generation / text | model loader/forward | ☐ required | INVENTORIED | none | unassigned |
MODEL-TEXT-hyperclovax-hyper-clovaxfor-causal-lm | HyperCLOVAXForCausalLM | registry.py:132; vllm/model_executor/models/hyperclovax.py::HyperCLOVAXForCausalLM | causal generation / text | model loader/forward; paged attention/KV | ☐ required | INVENTORIED | none | unassigned |
MODEL-TEXT-internlm2-intern-lm2-for-causal-lm | InternLM2ForCausalLM | registry.py:133; vllm/model_executor/models/internlm2.py::InternLM2ForCausalLM | causal generation / text | impl include/vllm/model_executor/models/internlm2.h (using InternLM2Model = Qwen3DenseModel, zero-new-kernel) + src/vllm/model_executor/models/internlm2{_weights,_registry}.cpp (fused-wqkv interleaved de-split grounded internlm2.py:158-176; one REGISTER line); test tests/parity/test_internlm2_paged_engine.cpp SACRED 16/16 vs vLLM 0.25.0 (STRICT 12 + near-tie 4, max gap 0.0 nats, 0 divergent; RED wrong-split CAUGHT prompt0 tok0 55040≠1934) | recent-dense-batch | PARTIAL | none; registration src/vllm/model_executor/models/internlm2_registry.cpp:140; test tests/vllm/models/test_model_registry.cpp:114 | CLAIM-SWEEP-RECENT-DENSE |
MODEL-TEXT-iquest-loopcoder-iquest-loop-coder-for-causal-lm | IQuestLoopCoderForCausalLM | registry.py:136; vllm/model_executor/models/iquest_loopcoder.py::IQuestLoopCoderForCausalLM | causal generation / text | model loader/forward; paged attention/KV; sliding-window attention | ☐ required | INVENTORIED | none | unassigned |
MODEL-TEXT-jais2-jais2-for-causal-lm | Jais2ForCausalLM | registry.py:137; vllm/model_executor/models/jais2.py::Jais2ForCausalLM | causal generation / text | model loader/forward; paged attention/KV; sliding-window attention | ☐ required | INVENTORIED | none | unassigned |
MODEL-TEXT-jamba-jamba-for-causal-lm | JambaForCausalLM | registry.py:138; vllm/model_executor/models/jamba.py::JambaForCausalLM | causal generation / text | model loader/forward; paged attention/KV; FusedMoE/grouped GEMM; Mamba/SSM state | ☐ required | INVENTORIED | none | unassigned |
MODEL-TEXT-kimi-linear-kimi-linear-for-causal-lm | KimiLinearForCausalLM | registry.py:139; vllm/model_executor/models/kimi_linear.py::KimiLinearForCausalLM | causal generation / text | model loader/forward; FusedMoE/grouped GEMM; GDN/linear-attention state; MLA/latent KV (NoPE variant — kimi_linear.py:213 asserts use_nope); KDA per-channel-decay gated-linear-attention kernel (NEW — adjacent to GDN, reuses its state/conv/chunked-delta machinery but NOT its decay or output kernel) | kimi-linear W0 spike; mla-deepseek-campaign | ACTIVE | bf16-RESIDENT LOADER/FORWARD — POOL MATH + DESIGN (2026-08-06, branch row/MODEL-KIMI-LINEAR-BF16, NOT pushed to main): the final phase-3 brick unblocking the full-model e2e (§12 was blocked on OUR f32 loader = 183 GiB). POOL MATH (done before building, spec §13): bf16 weights 98,245,528,576 B = 91.5 GiB staged device-resident via OwnedTensor::d_dev (cudaMalloc+one-H2D, native GPU mem, no ATS penalty), per-tensor stage-then-ReleaseHost so the LOAD peak is ~91.5 GiB device + <1 GiB host + the ReleaseSourcePages mmap window; f32 residual-stream activations at T≈36 < ~0.3 GiB; host f32 norm/scale vectors (ReadF32-on-demand) < 0.1 GiB; CUDA context ~2 GiB reserved FIRST. STEADY ≈ 94 GiB → ~25 GiB headroom, CLOSES. DESIGN grounded file:line in the winning patterns: bf16 storage via dense_loaders::LoadBf16Direct→OwnedTensor (mirror laguna_weights.cpp/gemma_weights.cpp), the f32 MaterializeHost KEPT only for the SMALL-config unit gate; resident GEMM = KimiResidentBf16W (mirror laguna.cpp:125-139) + GemmBf16 vt::CastBf16(f32-act→bf16) then vt::MatmulBT (bf16,bf16)→f32 (the combo CUDA MatmulBT SUPPORTS cuda_matmul.cu:3; the elementwise f32-act×bf16-weight it LACKS cuda_deepseek_v4.cu:1821), so the GEMM numerics are vLLM-bf16 (best token-exact vs the bf16 golden); the residual stream + the two host-fallback islands stay f32; norms via ReadF32/ResidentWeightF32; runner drops the host.materialized precondition on the resident path; GB10 load recipe (context-first + shard-release, examples/laguna_gen/main.cpp:185-237); e2e vehicle = a greedy-decode harness over the bf16 ForwardDeviceCompute vs tests/parity/goldens/kimi_linear_greedy/greedy_ids.npy (the =1 bf16 arm only — the =0 f32 host Forward cannot fit the full model). HONEST — implementation PENDING: the ~500-line loader/forward rewrite + tiny-config bf16 gate + CUDA build + the memory-critical full-model e2e are the scoped next execution (§13 Gates); the pool-math/design phase is complete and de-risks it. Row STAYS ACTIVE. §8 SACRED ORACLE GOLDEN CAPTURED — STRICT; full our-engine e2e f32-loader-blocked (2026-08-06, branch row/MODEL-KIMI-LINEAR-E2E, NOT pushed to main): disk unblocked upstream (129 GiB free), resumed off origin/main c1a7b452. Step 3 (download) DONE: moonshotai/Kimi-Linear-48B-A3B-Instruct in the HF cache, 20 shards, exact 98,245,528,576 B = 91.5 GiB bf16 / ~49.1B, config.json matches the spec. Step 4 (§8 golden) DONE, STRICT: captured on GB10 with the 0.25.0-stage oracle ALONE (both locks, drop_caches, 116 GiB free), recipe moe_backend=triton + enforce_eager + max_num_batched_tokens=512 + max_num_seqs=1, gpu_memory_utilization=0.82 (footprint ≈ util×119 ≈ 97.6 GiB; the coordinator's ≤0.60 cannot even hold 91.5 GiB weights, 0.60×119=71<91.5; 0.82 gives ~20 GiB slack), min available 15 GiB throughout, no reboot; ALL 8 prompts DETERMINISTIC over K=3 → STRICT token-exact gate (coherent: Paris/Rome/Berlin, fibonacci, "Au…aurum", Hamlet). Golden committed tests/parity/goldens/kimi_linear_greedy/ (greedy_ids.npy [8,16] + greedy_dist.npy + p{0-7}_prompt.i32); recipe scripts/kimi-linear-oracle-capture.py. Step 5 (our-engine full e2e) BLOCKED on OUR f32 loader — NOT disk, NOT oracle: kimi_linear_weights.cpp::MaterializeHost decodes every weight to std::vector<float> f32 (all 27 layers + 256 experts, ~49.1B×4 B ≈ 183 GiB > 119 GiB pool), so a full-model our-engine load would OOM-reboot; NOT attempted. This is the §10/§11 bf16-resident residual; the device compute is already GPU-verified (test_kimi_linear_forward 12/12·614) and the STRICT golden is now ready to gate against once the bf16 loader/forward lands. Row STAYS ACTIVE; VT_KIMI_DEVICE_COMPUTE default STAYS OFF (e2e gate not run). Anchor: golden dir + capture script; oracle log dgx:~/kimi-e2e/oracle_full.log. W7 GPU-VERIFY LANDED (2026-08-06, CLAIM-KIMI-LINEAR-W7 GPU-verify, branch row/MODEL-KIMI-LINEAR-GPU, NOT pushed to main): the §10 device compute now RUNS on GB10 (sm_121a). Clean origin/main CUDA build (-DVLLM_CPP_CUDA=ON -DVLLM_CPP_TRITON=ON -DVLLM_CPP_CUDA_ARCHITECTURES=121a -DVLLM_CPP_CUTLASS_DIR=$HOME/cutlass-4.5.0, nvcc 13.0.88, Release, -Werror clean) resolved the full PRODUCTION stack — CUTLASS … sm120a NVFP4 cutlass GEMM, FlashAttention-2 prefill/decode ENABLED [121a], vendored Triton AOT sm_121a (MANIFEST OK) GDN cubins; nm confirms all 14 gdn_{chunko,chunko_bf16,decode,deltah,kkt,tril,wu}_h{32,48}_default symbols linked. tests/vllm/models/test_kimi_linear_forward.cpp runs 12/12·614 GREEN on the GPU, BOTH arms (VT_KIMI_DEVICE_COMPUTE=1 device path + default host-ref) — the f32 vt:: device dispatch == the W2 host reference on real Blackwell within the CPU-gate tolerances; no numerics divergence, no DeepSeek-class trap (norms/async/keep-quant/capture). Oracle gateability RE-CONFIRMED (live ~/venvs/vllm-oracle→v0.25.0-stage registers KimiLinearForCausalLM + models/kimi_linear.py + config). e2e SACRED §8 golden STILL disk-BLOCKED: the 91.5 GiB bf16 checkpoint is ABSENT (not in ~/models nor the HF cache) and dgx root is 100% full — 34 GiB free (need ~91.5 GiB; the ~60 GiB reclaim = other agents' ACTIVE laguna/ds4/bench trees, not done unilaterally; no stale source-* grid trees). GPU-verify PROVES the f32 device WIRING (vt-op dispatch, MoE routing, device-resident logits, GDN cubin linkage) on GB10; it does NOT prove bf16 numerics vs the oracle or the e2e token gate — those + speed stay the NAMED residual. Anchor: GPU build log + test_kimi_linear_forward on dgx (both arms, /dev/shm). Row STAYS ACTIVE. W7 DBuf-RESIDENT DEVICE COMPUTE LANDED, CPU-gated (2026-08-05, CLAIM-KIMI-LINEAR-W7, NOT pushed): the W7 residual named by W6 — the real DBuf-resident device COMPUTE — is IMPLEMENTED in the additive TU src/vllm/model_executor/models/kimi_linear_device.cpp. KimiLinearModel::ForwardDeviceCompute composes the whole 27-layer KDA/NoPE-MLA + 256-expert-MoE hybrid over POOLED f32 DBufs through the SHARED vt:: device ops (mirroring deepseek_v2.cpp's device-forward structure): vt::Embedding → per-layer vt::FusedChain(kFusedAddRmsNormStd) residual add+RMSNorm → KDA/NoPE-MLA self-attn → FusedChain → dense/MoE MLP → final FusedChain → lm_head, returning DEVICE-RESIDENT [rows,vocab] f32 logits (the same pooled-DBuf WrapDeviceLogits carrier). ON DEVICE (genuine vt:: dispatch): every projection vt::MatmulBT (host weights torch [out,in]=[N,K]), the 3 KDA short convs vt::CausalConv1dFwd (silu, fresh zero conv-state, per qwen3_5.cpp:3032-3040), q/k vt::L2Norm, the KDA output vt::RmsNormGated (sigmoid), the MoE router vt::MoeRouterTopK (sigmoid noaux_tc, group 1/1, e_score_correction_bias, routed_scaling=2.446), per-expert/dense/shared SwiGLU vt::MoeSiluMul, weighted vt::MoeCombine. TWO documented HOST-FALLBACK islands (the W7-speed residuals): (1) the KDA per-k-channel gated-delta RECURRENCE + decay gate g=-exp(A_log)*softplus(f_b(f_a(x))+dt_bias) + beta=sigmoid — vt::GdnDecode/GdnPrefill carry only a per-HEAD scalar decay g[T,Hv] (ops.h) so CANNOT express KDA's per-channel g[T,H,D]; computed on host from device-resident q/k/v/g1/beta via the kimi_kda refs then uploaded; (2) the NoPE-MLA attention CORE (causal softmax) — the device path is mla::ForwardMlaAttentionBlock over the paged het-KV + load-time W_UK/W_UV absorption (born-on-runner), so this seam keeps every MLA projection + kv_a_layernorm + kv_b ON DEVICE and computes only the softmax core on host (identical materialized-MHA math, NoPE). Because the CPU backend runs the SAME vt:: dispatch (a pooled DBuf is a device buffer on CPU; weights alias the host f32 bytes exactly as ResidentWeight's CPU branch), the f32 device compute matching the W2 host f32 reference is a REAL wiring gate: tests/vllm/models/test_kimi_linear_forward.cpp 12/12·614 (was 7/7·300) — (f) KDA layer device==ref rtol 3e-3, (g) NoPE-MLA rtol 1e-4, (h) MoE block + dense MLP rtol 2e-4/1e-4 (the device router selects the SAME experts — lowest-index tie rule matches KimiMoeRoute), (i) the WHOLE ForwardDeviceCompute == ref logits rtol 5e-3 AND greedy-token-identical AND device-resident (on_device(), logits_indices gather→1 row), (j) ForwardDevice reaches the device compute under the opt-in flag. Runner routing: ForwardDevice routes to ForwardDeviceCompute under VT_KIMI_DEVICE_COMPUTE=1 (default OFF keeps the CPU-verified W6 host-ref compose as production until GPU-verified). test_kimi_linear_scaffold 9/9 + test_kimi_kda 14/14 UNCHANGED; clean CPU build, -Wall -Wextra -Werror clean; check-fusion-consistency/check-runner-routing-consistency/check-env-doc/check-agent-record rc=0. HONEST — NOT DONE: GPU numerics (bf16 activations for vLLM parity, GDN Triton-AOT cubins, paged het-KV, grouped-MoE slabs) + the e2e SACRED golden stay a NAMED pending (box down). Anchor src/vllm/model_executor/models/kimi_linear_device.cpp; test tests/vllm/models/test_kimi_linear_forward.cpp cases (f)-(j). Row STAYS ACTIVE. W6 DEVICE FORWARD SEAM LANDED (2026-08-05, CLAIM-KIMI-LINEAR-W6, NOT pushed): the born-on-the-runner KimiLinearModel::ForwardDevice (the DEFAULT gather_logits runner path) replaces the refuse-by-name stub — it composes the [rows,vocab] logits via the landed CPU reference then hands them back DEVICE-RESIDENT (a pooled DBuf, wrapped verbatim like deepseek_v2.cpp:633 WrapDeviceLogits; on_device()==true on CPU+CUDA alike) so the runner's on-GPU sampler consumes them with NO host logit download on the default path — the third MUST-route seam. check-runner-routing-consistency now reclassifies Kimi-Linear device-resident (25 routed models; refuse-skipped stubs 2→1) with NO allowlist entry, and check-fusion-consistency stays green (the wrapper hand-rolls no fusion/residual stream). Gate tests/vllm/models/test_kimi_linear_forward.cpp 7/7·300 — the new case (e) asserts ForwardDevice returns on_device() logits == the host reference byte-exact, greedy tokens identical, and the logits_indices gather returns exactly one device row in request order. Row → ACTIVE (implementation underway; e2e gate pending, like deepseek_v4). The DBuf-resident device COMPUTE — KDA routed through the GDN device family (decode = reused fused-recurrent + the KDA gate as a host-fallback: no device exp/softplus op), NoPE-MLA through mla::ForwardMlaAttentionBlock, the DeepSeek-V2 sigmoid-noaux_tc grouped-MoE, all over the paged het-KV caches (the full reuse-wiring plan, with the exact vt:: op per layer, is documented in kimi_linear.cpp) — is the GPU-verify-pending W7 residual: CPU cannot gate the GDN Triton-AOT decode cubins or the bf16 numerics against the pinned oracle, so authoring the device compute now would present as "done" without evidence. New code: kimi_linear.cpp (ForwardDevice), kimi_linear.h doc, test_kimi_linear_forward.cpp case (e). Anchor src/vllm/model_executor/models/kimi_linear.cpp ForwardDevice; test tests/vllm/models/test_kimi_linear_forward.cpp. W0 DEDICATED SPIKE LANDED (2026-08-05, CLAIM-KIMI-LINEAR-W0) — the row stays SPIKE (protocol: it is actively claimed by MLA-DEEPSEEK + KDA-KERNEL, and a claimed row is SPIKE/ACTIVE, never READY); the dedicated full W0 spike is complete and W1 implementation can start. W1 CPU SCAFFOLDING LANDED (2026-08-05, CLAIM-KIMI-LINEAR-W1, NOT pushed): additive registry TU (kimi_linear_registry.cpp, REGISTER_VLLM_MODEL(kimi_linear, "KimiLinearForCausalLM"), info text-gen + is_hybrid, supports_multimodal=false) + config/loader (kimi_linear_weights.cpp: ParseKimiLinearParams/EnumerateKimiLinearTensors/LoadKimiLinearForCausalLMWeights) + REFUSE-by-name forward (kimi_linear.cpp, VT_CHECK(false)) + heterogeneous KV spec (MakeKimiLinearKVCache: MLA latent-576 group + KDA mamba group / ) + header include/vllm/model_executor/models/kimi_linear.h. Config parse resolves the authoritative 48B schedule (20 KDA + 7 NoPE-MLA, MLA 512/128/64/128 q_lora-null, 256e/top-8/1-shared sigmoid noaux_tc 2.446, first_k_dense_replace=1) and REJECTS a positional MLA / q-LoRA branch / missing linear_attn_config / non-sigmoid router. Loader throws BY NAME on a missing or mis-shaped tensor (never silent zeros); name-map VERIFIED vs the real safetensors index — the MoE block is block_sparse_moe.* (the concrete correction of the DERIVED K3 mlp.* map). Gate tests/vllm/models/test_kimi_linear_scaffold.cpp 9/9·83 (registry-resolve + config schedule + per-layer KDA/MLA + MoE/dense split + name-map + synthetic 2-layer loader round-trip + RED missing/mis-shape + reject cases) + test_model_registry 24/24 (sorted arch-set + error message + hybrid model-property) + test_kimi_k3_scaffold 6/6 UNCHANGED; clean CPU -DVLLM_CPP_CUDA=OFF Release build; check-fusion-consistency + check-runner-routing-consistency green (the refuse-by-name forward is a skipped stub — NO allowlist). Registration src/vllm/model_executor/models/kimi_linear_registry.cpp; test tests/vllm/models/test_kimi_linear_scaffold.cpp. W2-W6 CPU REFERENCE FORWARD LANDED (2026-08-05, CLAIM-KIMI-LINEAR-W2, NOT pushed): the refuse-by-name host KimiLinearModel::Forward is replaced by a REAL per-op-gated CPU reference (kimi_linear_forward.cpp) that composes the whole 27-layer KDA/NoPE-MLA + MoE decoder from the landed primitives — KDA layer (q/k/v proj → 3 silu short convs → per-head q/k L2-norm → the gated-delta recurrence fused_recurrent.py:122-149 with the KDA per-channel decay gate → FusedRMSNormGated → o_proj; all via vllm::kimi_kda), NoPE-MLA (the UNABSORBED materialized-MHA reference, scaling=qk_head_dim**-0.5, rotary_emb=None), sigmoid noaux_tc MoE (grouped-topk num_expert_group=1 trivial group: bias-select / unbiased-weight, renormalize, routed_scaling=2.446) + always-on shared expert, dense layer-0 SwiGLU, pre-norm residual stream — and the loader now MATERIALIZES the host float weights (bf16/f32→f32). The DEVICE forward ForwardDevice (the DEFAULT gather_logits runner path) STAYS refuse-by-name — the born-on-runner device forward (KDA device kernel, absorbed-MLA decode, grouped-MoE slabs, het-KV wiring) is W6/W7. Gate tests/vllm/models/test_kimi_linear_forward.cpp 6/6·246: (a) KDA layer == a hand-composition of the kimi_kda refs + an independent recurrence; (b) NoPE-MLA layer == an independent materialized-MHA reference; (c) sigmoid noaux_tc router + block == a hand-computed top-k case; (d) the loader materializes end-to-end + the whole 2-layer forward is finite/coherent-shaped + greedy-decodes 5 tokens with the (recomputed) recurrent+latent context advancing. test_kimi_linear_scaffold 9/9·83 + test_kimi_kda 14/14 UNCHANGED; clean CPU build; check-fusion-consistency/check-runner-routing-consistency green (the DEVICE forward is a refuse-skipped stub — NO allowlist; the host reference uses no vt:: fusable device ops). Row STAYS SPIKE (the DEVICE forward refuses; the W0/W7 e2e SACRED token golden on GB10 — spec §8 recipe — plus speed are the residual). Forward src/vllm/model_executor/models/kimi_linear_forward.cpp; test tests/vllm/models/test_kimi_linear_forward.cpp. Row STAYS SPIKE (the forward refuses; the W3-W6 KDA-kernel/NoPE-MLA/MoE/het-KV assembly + the e2e SACRED gate are the residual). Full spike contract in kimi-linear.md: authoritative 48B-A3B config (27 layers = 20 KDA + 7 NoPE-MLA; 256 experts/top-8/1-shared sigmoid noaux_tc, routed_scaling_factor=2.446, first_k_dense_replace=1; num_nextn_predict_layers=0 ⇒ no MTP head in this checkpoint), reuse-vs-new map, HW-fit (FITS one GB10, 91.5 GiB / 0.77× pool — the ONE Kimi text model with a real e2e SACRED gate, unlike 2.8T K3), the W0-W7 breakdown, and the GPU golden-capture recipe. NET-NEW: the KDA device kernel (host refs already landed, see below), the NoPE-MLA branch (rotary_emb=None), the hybrid layer-schedule + het-KV wiring (GDN state + MLA latent per layer), and the loader name-map. CORRECTS the 2026-07-25 sweep: the full-attn layers are MLA, not "MHA head_dim 72". Below: the prior blocked-row honesty pass, which stands as inherited context. W10 (blocked-row honesty pass, 2026-07-22) — the MLA half of this row is UNLOCKED, the row is not. The campaign's shared MLA unlock (W1-W6) reaches Kimi-Linear because its full-attention layers ARE MLA (kimi_linear.py:310 -> KimiMLAAttention:179, self-documented "Main reference: DeepseekV2 vllm Implementation"), and the NoPE variant (:213 hard-asserts use_nope) is a SIMPLIFICATION of what we built, not an extension. What remains is a separate KERNEL CAMPAIGN, not a model row: KDA reuses our GDN state machinery (cache layout, GDNAttentionMetadata, conv update, chunked delta recurrence, WY solve) but NOT its decay or output kernel — it needs a per-channel [H,D] decay from a low-rank f_a_proj/f_b_proj bottleneck, the gated-linear-attention output kernel (kda.py:1019,1126) plain GDN does not have, and three separate q/k/v short convs. KDA kernel-delta FIRST BRICK LANDED (2026-07-28, KERNEL-KDA-DELTA, CLAIM-KDA-KERNEL): the four net-new-vs-GDN deltas (the per-channel [H,D] low-rank decay f_a/f_b, the sigmoid-gated output norm FusedRMSNormGated, the 3 q/k/v short convs + q/k L2-norm) are ported as portable host references (kimi_kda.{h,cpp}) and unit-gated (test_kimi_kda 14/14·36, clean CPU -Wall -Werror -Wextra) vs hand-derived literals + double-precision references; additive/GDN-inert (cuda_gdn.cu/gdn_attn.cpp untouched). The KDA CUDA device kernel + the Kimi-Linear-48B proxy e2e gate remain the named residuals; the row still stays SPIKE. HW-MARGINAL, not blocked: Kimi-Linear-48B-A3B is ~89.4 GiB against ~110 GiB usable, which fits but pushes dgx's root filesystem past 97%. Stays SPIKE: no code, no checkpoint, no gate. none W4 the MLA DECODE kernel — vt::MlaDecodeAttention, a structure port of the two-stage split-KV Triton pair (_fwd_grouped_kernel_stage1 triton_decode_attention.py:278-458 IS_MLA branch + _fwd_kernel_stage2 :575-639) that W0 OBSERVED EXECUTING on sm_121, plus _compute_num_kv_splits (triton_mla.py:40-47) and the forward_mqa contract (:189-260); gated at the REAL 576/512/64 block-16 mscale^2-scale geometry vs an INDEPENDENT two-pass CPU reference in tests/vt/test_ops_mla_attn.cpp (port of test_mla_decode_cpu.py::ref_mla incl. its NaN-padding out-of-bounds detector), covering ragged / multi-block / single-block / every-split-boundary / 128-head(V3) / non-BLOCK_H head counts, run-to-run BIT-reproducible, compute-sanitizer memcheck+racecheck+synccheck 0; TritonMLABackend::get_impl_cls() now returns a real TritonMLAImpl (DECODE only — prefill is W5, and forward() says so). STILL NO MLA MODEL AND NO FORWARD — row stays SPIKE W5 the MLA PREFILL path + the workspace-bounded CHUNKED-CONTEXT loop — vt::MlaPrefillAttention (<- mla/prefill/flash_attn.py:153-248, the only backend reachable on sm_121, running over the vendored FA-2 with V zero-padded 128->192 exactly as upstream's requires_v_padding path does), vt::GatherMlaCache (<- cache_kernels.cu:992-1064), vt::MergeAttnStates (<- merge_attn_states.cu:18-192, both -inf edge cases verbatim), and mla_chunked_context.h (<- mla_attention.py:1422-1451,1667-1745,2094-2199,2344-2425). Gated UNIT-ONLY (there is still no model): test_ops_mla_prefill.cpp 4/4 / 2,377,052 assertions + test_ops_mla_chunked_context.cpp 5/5 / 306,037 assertions on dgx sm_121, at the REAL QK 192 / V 128 prefill geometry, against an INDEPENDENT double-precision two-pass oracle and — for the loop — a SINGLE-SHOT whole-sequence oracle that never chunks; exact/+1/-1 chunk boundaries, zero-context and zero-key-in-chunk requests, ADVERSARIAL reverse-interleaved block tables, NaN-poisoned outputs, run-to-run bit-exact; memcheck/racecheck/synccheck all 0; regression set UNCHANGED. Still NO MLA model, NO MLA forward and NO weight absorption (W6/W7) — the row stays SPIKE. W6 LANDED 2026-07-22 — the MLA attention BLOCK + LOAD-TIME WEIGHT ABSORPTION; the row STILL does not move, because W6 adds a LAYER, not a model. W3's cache write, W4's MQA decode and W5's MHA prefill are now composed by one layer (mla_attention.h, mla_attention.cpp): projections with BOTH q_lora_rank branches, the two RMSNorms, the decoupled is_neox_style=False RoPE with its YaRN cos/sin cache and the separate mscale^2 softmax scale, the kv_b_proj -> W_UK/W_UV split, and the prefill-MHA / decode-MQA dispatch with decode tokens packed FIRST. Absorption needed NO new attention kernel — it is a load-time transform plus two batched GEMMs (vt::BatchedMatmul <- torch.bmm at mla_attention.py:789,1034; vt::ConcatMlaNopeRope <- concat_mla_q generalized to also serve _concat_k_nope_k_pe). The absorbed-vs-unabsorbed equivalence is PROVEN numerically three ways (double-precision identity < 1e-11; ours vs the unabsorbed oracle < 2e-4 in f32; and the SAME batch through our ABSORBED MQA decode vs our UNABSORBED materialized-MHA prefill, < 3e-4 CPU f32 / < 4e-2 CUDA bf16). Evidence: test_mla_attention_block.cpp 10/10 / 2,372,644 assertions + test_ops_mla_absorb.cpp 9/9 / 1,644,807 on dgx sm_121; memcheck/racecheck/synccheck all 0; clean CUDA build 0 warn/0 err; regression set UNCHANGED (27B 235/235, 35B 315/315, Coder 138/138, Qwen3-dense 664/664, OPT 36/36). The q_lora query branch is UNIT-GATED ONLY and always will be on this box — DeepSeek-V2-Lite has q_lora_rank=null, so it is gated at DeepSeek-V3's real dimensions (7168 / 128 heads / 1536) and gets NO e2e coverage; GLM-4.7-Flash (q_lora_rank=768) is what would close it. NO model TU, NO registry entry, NO loader, NO model forward — W7. — anchor tests/vt/test_ops_mla_attn.cpp:359; W0 dedicated spike specs/kimi-linear.md | CLAIM-MLA-DEEPSEEK, CLAIM-KIMI-LINEAR-W0, CLAIM-KIMI-LINEAR-W1, CLAIM-KIMI-LINEAR-W2, CLAIM-KIMI-LINEAR-W6, CLAIM-KIMI-LINEAR-W7 |
MODEL-TEXT-lfm2-lfm2-for-causal-lm | Lfm2ForCausalLM | registry.py:140; vllm/model_executor/models/lfm2.py::Lfm2ForCausalLM | causal generation / text | model loader/forward; paged attention/KV; Mamba/SSM state | ☐ required | INVENTORIED | none | unassigned |
MODEL-TEXT-lfm2-moe-lfm2-moe-for-causal-lm | Lfm2MoeForCausalLM | registry.py:141; vllm/model_executor/models/lfm2_moe.py::Lfm2MoeForCausalLM | causal generation / text | model loader/forward; paged attention/KV; FusedMoE/grouped GEMM; Mamba/SSM state | ☐ required | INVENTORIED | none | unassigned |
MODEL-TEXT-laguna-laguna-for-causal-lm | LagunaForCausalLM | registry.py:142; vllm/model_executor/models/laguna.py::LagunaForCausalLM | causal generation / text | model loader/forward; paged attention/KV; FusedMoE/grouped GEMM; sliding-window attention | W3 spec (W1/W2, W0 scope specs/laguna-s21-scope-2026-07-30.md) | ACTIVE | NVFP4 W4A4 ARM (2026-08-01, CLAIM-LAGUNA-NVFP4-N4): the additive safetensors NVFP4 arm (N1a/N1b/N2/N3, CPU-gated test_laguna_nvfp4_loader 3/3·61) RAN on GB10 — coherent, first-2-tokens match the vLLM MARLIN golden (near-tie: our TRUE-W4A4 vs MARLIN W4A16), but 0.16 tok/s = 120× slower than vLLM 18.8 because LqGemmNvfp4Fp4 uses the emulation vt::MatmulNvfp4Fp4, not the cutlass fp4 tensor-core path (MatmulNvfp4Fp4DirectD). OPEN #234 (N5): route experts to cutlass fp4 + grouped W4A4 + ResidentNvfp4. Spec .agents/specs/laguna-nvfp4-arm-2026-07-31.md. — Prior W3 REAL forward + 3 new ops landed: laguna_ops.cpp (per-head softplus attn out-gate + ungrouped sigmoid-noaux router + dual per-layer RoPE cos/sin builders), LagunaModel::Forward now a REAL runnable host-reference composition (variable-Q-head GQA + dual RoPE + sliding-window mask + softplus gate + dense L0 / ungrouped-MoE L1..47 + untied lm_head) replacing the VT_CHECK(false) stub; test_laguna_scaffold 8/8·166 (softplus math, router selection+tie-break RED-first, dual-RoPE cos/sin bit-match, variable-Q-head shapes, forward composition on synthetic weights), test_model_registry 24/24, CPU -Werror full-library build clean. DEFERRED to W4 (needs 73 GB checkpoint): GGUF keep-quant tower materialization + device/paged production forward + strict dual-oracle greedy gate (llama.cpp-Q4_K token-exact + vLLM-NVFP4 near-tie). ~85-90% reuse (ds4-MoE + gemma-sliding + olmo3-dual-rope + landed Q4_K keep-quant); NEW = the 3 landed host ops + name-map + variable-Q-head device runner. | CLAIM-LAGUNA-W3 |
MODEL-TEXT-llama4-llama4-for-causal-lm | Llama4ForCausalLM | registry.py:144; vllm/model_executor/models/llama4.py::Llama4ForCausalLM | causal generation / text | model loader/forward; paged attention/KV; FusedMoE/grouped GEMM | ☐ required | INVENTORIED | none | unassigned |
MODEL-TEXT-longcat-flash-longcat-flash-for-causal-lm | LongcatFlashForCausalLM | registry.py:147; vllm/model_executor/models/longcat_flash.py::LongcatFlashForCausalLM | causal generation / text | model loader/forward; FusedMoE/grouped GEMM | ☐ required | INVENTORIED | none | unassigned |
MODEL-TEXT-mamba2-mamba2-for-causal-lm | Mamba2ForCausalLM | registry.py:149; vllm/model_executor/models/mamba2.py::Mamba2ForCausalLM | causal generation / text | model loader/forward; Mamba/SSM state | ☐ required | INVENTORIED | none | unassigned |
MODEL-TEXT-mellum-mellum-for-causal-lm | MellumForCausalLM | registry.py:150; vllm/model_executor/models/mellum.py::MellumForCausalLM | causal generation / text | model loader/forward; paged attention/KV; sliding-window attention | ☐ required | INVENTORIED | none | unassigned |
MODEL-TEXT-minicpm-mini-cpmfor-causal-lm | MiniCPMForCausalLM | registry.py:151; vllm/model_executor/models/minicpm.py::MiniCPMForCausalLM | causal generation / text | IMPLEMENTED (new files): include/vllm/model_executor/models/minicpm.h + src/vllm/model_executor/models/{minicpm,minicpm_weights,minicpm_registry}.cpp (one REGISTER_VLLM_MODEL). Llama dense forward + THREE scalar deltas grounded in minicpm.py @ e24d1b24 (scale_emb 12 after embed :441-443 via vt::MulScalar; scale_depth/sqrt(num_layers)=1.4/sqrt(40) scaled residual add per sublayer :384-393 via vt::MulScalar+vt::Add; hidden/scale_width=hidden_size/dim_model_base=9.0 before lm_head :604,633,640). Standard 1/sqrt(head_dim) attn scale (NO custom multiplier, unlike Granite). Reuses shared dense glue (dense_attn_block.h), separate q/k/v→merged qkv loader, tied lm_head (no lm_head in checkpoint). ZERO new compute kernel. Runs EAGER (bf16). Gate: test_minicpm_paged_engine 16/16 vs vLLM 0.25.0 on openbmb/MiniCPM-2B-sft-bf16 (K=5 ALL-DETERMINISTIC → STRICT bar; 10/16 token-exact + 6/16 near-tie, max gap 0.0 nats, 0 forward-divergent; RED drop-scale_depth → 256/256 divergent max 29.375 nats). Speed pending. | ✅ recent-dense-batch | PARTIAL | test_minicpm_paged_engine 16/16 (dgx, 140 assertions); loader zero-unmapped on MiniCPM-2B; registry resolves MiniCPMForCausalLM; memcheck 0; registration src/vllm/model_executor/models/minicpm_registry.cpp:121; test tests/vllm/models/test_model_registry.cpp:116 | CLAIM-SWEEP-RECENT-DENSE (Claude Code opus-4-8) |
MODEL-TEXT-minicpm3-mini-cpm3-for-causal-lm | MiniCPM3ForCausalLM | registry.py:152; vllm/model_executor/models/minicpm3.py::MiniCPM3ForCausalLM | causal generation / text | IMPLEMENTED (new files): include/vllm/model_executor/models/minicpm3.h + src/vllm/model_executor/models/{minicpm3,minicpm3_weights,minicpm3_registry}.cpp (one REGISTER_VLLM_MODEL). The landed MiniCPM 3 scalars (minicpm.py:588-640) with attention swapped GQA→MLA, REUSING the landed DeepSeek-V2 MLA block (mla::ForwardMlaAttentionBlock, load-time kv_b_proj→W_UK/W_UV absorption) threaded with MiniCPM3 dims (grounded minicpm3.py:52-134 @ e24d1b24). THREE MLA deltas vs DeepSeek: is_neox_style=True (shared MlaBlockDims::is_neox_style, default false → DeepSeek unchanged); LongRoPE (phi3_long_rope short cache, mscale 1.0) not YaRN; q_lora always present. FA-2 MLA prefill dispatch/pad extended for qk_head_dim 96→128 (reuses compiled hdim128 kernel; DeepSeek/GLM byte-identical). ZERO new compute kernel. Runs EAGER (bf16; no decode CUDA-graph yet). Gate: test_minicpm3_paged_engine 16/16 vs vLLM 0.25.0 on openbmb/MiniCPM3-4B (K=5 ALL-DETERMINISTIC → STRICT; 13/16 token-exact + 3/16 near-tie, max gap 0.0 nats, 0 forward-divergent; RED wrong-rope → first-token divergence). DeepSeek-V2-Lite re-gated 8/8 (shared-MLA non-regression). Speed pending. | ✅ recent-dense-batch | PARTIAL | test_minicpm3_paged_engine 16/16 (dgx, 140 assertions); test_deepseek_v2_paged_engine 8/8 non-regression; registry resolves MiniCPM3ForCausalLM; memcheck 0; registration src/vllm/model_executor/models/minicpm3_registry.cpp:122; test tests/vllm/models/test_model_registry.cpp:117 | CLAIM-SWEEP-RECENT-DENSE (Claude Code opus-4-8) |
MODEL-TEXT-minimax-m2-mini-max-m2-for-causal-lm | MiniMaxM2ForCausalLM | registry.py:153; vllm/model_executor/models/minimax_m2.py::MiniMaxM2ForCausalLM | causal generation / text | model loader/forward; paged attention/KV; FusedMoE/grouped GEMM; sliding-window attention; partial RoPE; expert-bias router. NOT MLA, NOT lightning attention — plain dense GQA (minimax_m2.py:139,206-214); corrected by the campaign spike | mla-deepseek-campaign spike | BLOCKED | W10 (blocked-row honesty pass, 2026-07-22) — HW-BLOCKED e2e on GB10, and DISJOINT from the MLA track it was filed under. HW: ~230B params / ~428 GiB bf16 — roughly 4x over 119 GiB of unified memory, and it does not fit on free disk either, so no e2e token-exact or speed gate is runnable now or after any amount of software work. DISJOINT: the campaign CORRECTED this row's source-scan dependency hypothesis — MiniMaxM2ForCausalLM is plain dense GQA + MoE with partial RoPE and per-layer sliding window (minimax_m2.py:139,178,196-199,206-214), with NO MLA and no linear attention (lightning attention is retired upstream, registry.py:721-724). It therefore needs the SLIDING-WINDOW track (ROAD-V1-C5) plus partial-rotary support and the expert-bias router (its one genuine overlap with DeepSeek, already built as the W3 MoeRouterTopKArgs extension) — it must NOT be sequenced behind MLA. What can still be gated: config/registry resolution and unit parity at its dimensions; nothing e2e. Below: the MLA campaign chronology this row was carried through, retained as context only. none W6 LANDED 2026-07-22 — the MLA attention BLOCK + LOAD-TIME WEIGHT ABSORPTION; the row STILL does not move, because W6 adds a LAYER, not a model. W3's cache write, W4's MQA decode and W5's MHA prefill are now composed by one layer (mla_attention.h, mla_attention.cpp): projections with BOTH q_lora_rank branches, the two RMSNorms, the decoupled is_neox_style=False RoPE with its YaRN cos/sin cache and the separate mscale^2 softmax scale, the kv_b_proj -> W_UK/W_UV split, and the prefill-MHA / decode-MQA dispatch with decode tokens packed FIRST. Absorption needed NO new attention kernel — it is a load-time transform plus two batched GEMMs (vt::BatchedMatmul <- torch.bmm at mla_attention.py:789,1034; vt::ConcatMlaNopeRope <- concat_mla_q generalized to also serve _concat_k_nope_k_pe). The absorbed-vs-unabsorbed equivalence is PROVEN numerically three ways (double-precision identity < 1e-11; ours vs the unabsorbed oracle < 2e-4 in f32; and the SAME batch through our ABSORBED MQA decode vs our UNABSORBED materialized-MHA prefill, < 3e-4 CPU f32 / < 4e-2 CUDA bf16). Evidence: test_mla_attention_block.cpp 10/10 / 2,372,644 assertions + test_ops_mla_absorb.cpp 9/9 / 1,644,807 on dgx sm_121; memcheck/racecheck/synccheck all 0; clean CUDA build 0 warn/0 err; regression set UNCHANGED (27B 235/235, 35B 315/315, Coder 138/138, Qwen3-dense 664/664, OPT 36/36). The q_lora query branch is UNIT-GATED ONLY and always will be on this box — DeepSeek-V2-Lite has q_lora_rank=null, so it is gated at DeepSeek-V3's real dimensions (7168 / 128 heads / 1536) and gets NO e2e coverage; GLM-4.7-Flash (q_lora_rank=768) is what would close it. NO model TU, NO registry entry, NO loader, NO model forward — W7. | CLAIM-MLA-DEEPSEEK |
MODEL-TEXT-minimax-m3-mini-max-m3-sparse-for-causal-lm | MiniMaxM3SparseForCausalLM | registry.py:154-157; vllm/models/minimax_m3/__init__.py::MiniMaxM3SparseForCausalLM | causal generation / text | model loader/forward; paged attention/KV; FusedMoE/grouped GEMM; sparse-attention indexer (NOT MLA — GQA + a learned indexer, minimax_m3/nvidia/model.py:286,386,673-690; corrected by mla-deepseek-campaign §0.6) | ☐ required | INVENTORIED | none | unassigned |
MODEL-TEXT-mistral-mistral-for-causal-lm | Ministral3ForCausalLM, MistralForCausalLM | registry.py:158-159; vllm/model_executor/models/mistral.py::MistralForCausalLM | causal generation / text | model loader/forward | sweep-mistral spike | PARTIAL | Fifth-family near-additive dense bring-up (Tier-2 rank 6). vLLM's mistral.py is "Mistral adaptation of the LLaMA architecture" ⇒ Mistral = the shared Qwen3/Llama dense forward with NO new primitive: plain RoPE θ1e6 (no rope_scaling → rope_type=="default"), no per-head qk-norm, UNTIED lm_head, sliding_window: null → one full-attention KV group. New: include/vllm/model_executor/models/mistral.h (using MistralWeights = Qwen3DenseWeights; using MistralModel = Qwen3DenseModel), src/vllm/model_executor/models/mistral_registry.cpp, src/vllm/model_executor/models/mistral_weights.cpp; tests tests/vllm/models/test_mistral_load.cpp + tests/vllm/models/test_mistral_forward.cpp + tests/parity/test_mistral_paged_engine.cpp + tests/vllm/models/test_model_registry.cpp (7→8). W2 loader real-weights 1541 assertions; W3 forward CPU 510; registry 299; CUDA -Werror 0-warn; DSR 32. MODEL forward gate oracle-validated tokenizer-free: 30/30 greedy tokens match vLLM 0.25.0 (29 STRICT + 1 near-tie [p3 last: 4608 ∈ vLLM's {1782,4608}], 0 forward-divergent; vLLM greedy det 4/5 K=3). Full paged-engine SACRED gate now RUNS + PASSES (2026-07-23) — unblocked by the LOAD-SENTENCEPIECE tokenizer (Metaspace + byte-fallback SP-BPE, encode/decode byte-exact vs vLLM 0.25.0, test_tokenizer_parity_mistral 6/6·421 + SACRED cross-check 0/45): tests/parity/test_mistral_paged_engine.cpp 16/16 prompts vs the vLLM 0.25.0 per-prompt greedy oracle (vLLM ALL-DETERMINISTIC K=5) — 15/16 STRICT token-exact + 1/16 near-tie-band (prompt[5]@tok10, max gap 0.0000 nats ≪ 0.5), 0 forward-divergent; goldens tests/parity/goldens/mistral_greedy_7b/. CORRECTNESS DONE; SPEED PENDING (separate increment). Row ACTIVE, not DONE.; registration src/vllm/model_executor/models/mistral_registry.cpp:137; test tests/vllm/models/test_model_registry.cpp:93 | CLAIM-MODEL-MISTRAL |
MODEL-TEXT-mistral-large-3-mistral-large3-for-causal-lm | MistralLarge3ForCausalLM | registry.py:160; vllm/model_executor/models/mistral_large_3.py::MistralLarge3ForCausalLM | causal generation / text | model loader/forward | ☐ required | INVENTORIED | none | unassigned |
MODEL-TEXT-mixtral-mixtral-for-causal-lm | MixtralForCausalLM | registry.py:161; vllm/model_executor/models/mixtral.py::MixtralForCausalLM | causal generation / text | model loader/forward; paged attention/KV; FusedMoE/grouped GEMM | ☐ required | INVENTORIED | none | unassigned |
MODEL-TEXT-mpt-mptfor-causal-lm | MptForCausalLM, MPTForCausalLM | registry.py:163-164; vllm/model_executor/models/mpt.py::MPTForCausalLM | causal generation / text | model loader/forward; paged attention/KV | ☐ required | INVENTORIED | none | unassigned |
MODEL-TEXT-mimo-mi-mo-for-causal-lm | MiMoForCausalLM | registry.py:165; vllm/model_executor/models/mimo.py::MiMoForCausalLM | causal generation / text | model loader/forward | ☐ required | INVENTORIED | none | unassigned |
MODEL-TEXT-mimo-v2-mi-mo-v2-flash-for-causal-lm | MiMoV2FlashForCausalLM | registry.py:166; vllm/model_executor/models/mimo_v2.py::MiMoV2FlashForCausalLM | causal generation / text | model loader/forward; paged attention/KV; FusedMoE/grouped GEMM; sliding-window attention | ☐ required | INVENTORIED | none | unassigned |
MODEL-TEXT-mimo-v2-mi-mo-v2-for-causal-lm | MiMoV2ForCausalLM | registry.py:167; vllm/model_executor/models/mimo_v2.py::MiMoV2ForCausalLM | causal generation / text | model loader/forward; paged attention/KV; FusedMoE/grouped GEMM; sliding-window attention | ☐ required | INVENTORIED | none | unassigned |
MODEL-TEXT-nemotron-nemotron-for-causal-lm | NemotronForCausalLM | registry.py:168; vllm/model_executor/models/nemotron.py::NemotronForCausalLM | causal generation / text | model loader/forward; paged attention/KV | ☐ required | INVENTORIED | none | unassigned |
MODEL-TEXT-nemotron-h-nemotron-hfor-causal-lm | NemotronHForCausalLM, NemotronHPuzzleForCausalLM | registry.py:169-170; vllm/model_executor/models/nemotron_h.py::NemotronHForCausalLM | causal generation / text | model loader/forward; paged attention/KV; FusedMoE/grouped GEMM; Mamba/SSM state; sliding-window attention. Driver checkpoint nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4 (20.1 GiB, fits one GB10). The KERNEL-SSM-MAMBA block this cell carried from 2026-08-12 is FALSE, and a file glob is why it survived six days. #496 landed the host arm at 47960a009 (W1) and the CUDA arm at 43a6c5518 (W2). The kernel is cuda_mamba2_ssd.cuh — a 692-line .cuh INCLUDED at cuda_gdn.cu:48 and registered as kMamba2ChunkScan at cuda_gdn.cu:6669, not a translation unit of its own — so a src/vt/*mamba* FILE GLOB finds nothing and reads as absence. That glob is the search this cell and #1074 both rested on, and src/vt/ops.cpp::Mamba2ChunkScan plus nemotron_h.cpp:597 refute it: this model calls the op today. #496 stays open for its GENERIC MambaSpec producer, which this architecture does not need, because A1 made the runner read the model's OWN KV spec. Two of the three things this cell called nonexistent now exist. The non-gated relu² MoE landed at 4d0c399e1 — src/vt/ops.cpp::MoeRelu2, called from nemotron_h.cpp:354 — and ModelOpt MIXED_PRECISION per-module loading landed at 1bc5ef82c (modelopt_mixed_precision.h). The third, the DeepSeek-style MTP head, is genuinely OWED as W5: the loader DEFERS its 270 tensors BY NAME at nemotron_h_weights.cpp:1123 | nemotron-h-model (row spec, W1-W7); nemotron-h-abi-e2e (A1/A2/A3); nemotron-h-a2p-paged-forward; nemotron-h-a2q1-fp8-mamba; nemotron-h-a2q2-nvfp4-moe-lmhead; nemotron-h-a2q2b-realckpt-lmhead; mamba2-ssd | INVENTORIED | INVENTORIED still, because the END-TO-END TOKEN GATE HAS NOT BEEN RUN AGAINST main — this cell claims nothing beyond that. The code and its unit gates below are real, and they are what a state move will rest on once a gate result measured on main exists. Code: the hybrid host forward nemotron_h.cpp:597 (vt::Mamba2ChunkScan) and nemotron_h.cpp:354 (vt::MoeRelu2); the PAGED forward nemotron_h_device.cpp:1474 (NemotronHPagedForward, A2-P a6df72777), which puts K/V into the runner's pages at attn_meta.slot_mapping and gathers and scatters the conv and SSM rows at the metadata's state indices — the first time this architecture carries state between decode steps; the device MoE arm (A2-Q2a, 23 blocks, NVFP4 g16 Marlin) and the A2-R dense arm (embeddings, 52 norms, 6 GQA blocks); the weight loader nemotron_h_weights.cpp:1123 (18487/18487 accounted, 18217 materialized in their SHIPPED formats, 270 MTP deferred by name); and the ABI driver examples/nemotron_h_gen/main.cpp (A3 c83b96934), a thin include/vllm.h client. Tests: test_nemotron_h_paged_forward.cpp:595 (a multi-step runner decode matches the host reference token for token) and :685 (the recurrent pages carry state across steps and are INDEXED); test_nemotron_h_forward.cpp:990 (the Mamba2 mixer vs an independent sequential recurrence) and :1190 (the MoE vs an independent non-gated relu² reference); test_nemotron_h_scaffold.cpp:204 (the REAL released config.json parses); test_nemotron_h_loader.cpp:150 (the REAL checkpoint loads and the forward produces logits); test_nemotron_h_quantized_forms.cpp:297 (a shipped quantized weight is REFUSED, never silently dequantized); test_nemotron_h_moe_device.cpp:221 (A2-Q2a vs the host reference); test_ops_mamba2_ssd.cpp:470 (chunk-size invariance). GATED END TO END ON A MEASURED TREE, NOT ON main TODAY, and no number exists. The A3 gate against the committed goldens tests/parity/goldens/nemotron_35_lightning_greedy PASSES on GB10: TOKEN MATCH: 96/96 over 3 prompt(s) (full rows=3, short rows=0, mode=decode), STRICT PASS, against the pinned oracle vllm=0.23.1rc1.dev1511+g555967922 on nemotron-3.5-lightning-30b-nvfp4 revision 29f2d1746d8f41e316523194b19018707749b1b1. It is the DEVICE leg and not the host queue: the binary is libvllm 0.0.3+cuda, cfg.log records fp4-mma, cutlass-nvfp4 and cutlass-fp8 ENABLED for [121a], and the run logs Asynchronous scheduling is enabled (max_concurrent_batches=2) — the path on which ModelForwardInput::device_token_ids is non-null, and the only one on which the #1157 defect can occur. The proof is the DELTA, not the pass: the same binary and the same checkpoint with ONLY nemotron_h_device.cpp reverted to the fix's parent scores 4/24 (full rows=0, short rows=3) and bails at 8 generated tokens. Evidence /usr/local/nas_share/rc/nh1157/ — gate_fixed.out, gate_red.out, cfg.log, build.log. That pass was measured on the tree that landed as 0ea5d249f (#1221, MERGED 2026-08-18), which is now main's last touch of nemotron_h_device.cpp. The fix is therefore on main and nothing is pending a merge; but main has advanced many commits since that tree, NO run against current main exists, and this cell does not claim one. One config caveat travels with the run: --gpu-memory-utilization 0.92 did NOT size the KV pool, which fell back to 256 blocks (#83). The step still refuses batched decode by name at nemotron_h_registry.cpp:157 (input.num_reqs <= 1, A2-B owns it), and scripts/runner-routing-allowlist.txt:26 still carries nemotron_h because lm_head is NVFP4 g16 and refuses on a non-CPU queue at nemotron_h.cpp:1032, so the last step is the HOST projection. OWED: A2-Q2b the device lm_head, A2-Q1 the 46 FP8 W8A8 mamba projections (#940), A2-B batched decode, W5 the MTP head, W7 the GGUF k-quant arm, and a RE-RUN of the A3/S5 gate against main, which is now the only thing that can move this row. No throughput, latency or memory number exists for this architecture and none is claimed. The wall times in gate_fixed.out (264.4s load, 327-343s per 32-token prompt) are a CORRECTNESS run whose lm_head and 46 FP8 mamba projections still execute host-side; they are not benchmarks and nothing may read them as one. Rows: #517 (model), #810 (ABI e2e); record reconciled by #1074 | unassigned |
MODEL-TEXT-olmo-olmo-for-causal-lm | OlmoForCausalLM | registry.py:171; vllm/model_executor/models/olmo.py::OlmoForCausalLM | causal generation / text | model loader/forward; paged attention/KV | ☐ required | INVENTORIED | none | unassigned |
MODEL-TEXT-olmo2-olmo2-for-causal-lm | Olmo2ForCausalLM, Olmo3ForCausalLM | registry.py:172-173; vllm/model_executor/models/olmo2.py::Olmo2ForCausalLM | causal generation / text | OLMo-2-0425-1B dense LANDED (W0-W4). Impl: include/vllm/model_executor/models/olmo2.h + src/vllm/model_executor/models/{olmo2,olmo2_weights,olmo2_registry}.cpp (TWO REGISTER_VLLM_MODEL for Olmo2ForCausalLM+Olmo3ForCausalLM, one shared factory; reuses the shared dense glue). ZERO new compute kernel — the two "distinctives" both reduce to WIRING over landed ops: (1) PURE POST-NORM (norm_after, olmo2.py:261-277) = standalone vt::RmsNorm on each sublayer OUTPUT + plain vt::Add residual re-join, NO pre-norm (the GLM-4 standalone-output-norm op, pre-norms dropped); (2) FULL-WIDTH QK-norm (olmo2.py:113-117,160-172) = two standalone vt::RmsNorm over [T,q_size]/[T,kv_size] before NeoX rope (NOT the per-head kAttnQkNormRope). Reuse: SiLU SwiGLU (kSiluAndMul), NeoX RoPE (RopeFromCache/RopeNeox), GQA/MHA paged glue (dense_attn_block.h), merged qkv/gate_up loader. Checkpoint is F32 on-disk → loader downcasts f32→bf16 (round-to-nearest-even, vt::F32ToBF16) to match vLLM-bf16. UNTIED lm_head (tie_word_embeddings:false). Loader 179 tensors, zero missing/unmapped. Tokenizer: ONE shared-TU touch — tokenizer.cpp accepts the OLMo-2 Split behavior=Removed,invert=true encoding (equiv. to Isolated for the full-cover cl100k regex → kLlama3); guarded additive branch, no other checkpoint uses it. NO-BOS-verified real ByteLevel gate. Runs EAGER (bf16, no decode graph). Gate: 16/16 vs vLLM 0.25.0 (STRICT 13/16 + near-tie 3/16, max gap 0.094 nats, 0 forward-divergent) — speed pending. OLMo-3 W5 (batch3) IMPLEMENTED (guarded additive edits to olmo2.{h,cpp,weights}, diff-inert for OLMo-2 — re-run OLMo-2 gate 16/16 UNCHANGED): per-layer interleaved routing off config.layer_types — sliding_attention layers use plain NeoX rope (theta 500000) + finite window (masked at the FA kernel, inert for short contexts), full_attention layers use a precomputed YaRN cos/sin cache (get_rope yarn: factor 8, original 8192, mscale=yarn_get_mscale(8)=1.2079=config attention_factor, indexed by real positions); dtype-aware loader (OLMo-3 is BF16 on-disk vs OLMo-2 F32). BUT the pinned vLLM 0.25.0 oracle CANNOT run allenai/OLMo-3-1025-7B — its transformers version predates OLMo-3's nested per-layer-type rope schema: olmo2.py:143 does rope_parameters["rope_theta"] → KeyError: 'rope_theta' (top-level rope_theta not folded into standardized rope_parameters), and forcing it surfaces Unrecognized keys {'sliding_attention','full_attention'} → TypeError: unhashable type: 'dict'. So there is NO pinned-oracle SACRED bar for OLMo-3 (DEP-blocked, spec D5); our engine LOADS + RUNS it (bootstrap loaded clean). W5 SACRED gate pending an oracle that can construct OLMo-3's rope config. | ✅ sweep-olmo2 | PARTIAL | test_olmo2_paged_engine 16/16 (dgx, 92 assertions, batch3 re-run UNCHANGED); test_olmo3_paged_engine present (oracle-blocked, skips); loader 179 tensors zero-unmapped; registry resolves both arch strings; registration src/vllm/model_executor/models/olmo2_registry.cpp:126; test tests/vllm/models/test_model_registry.cpp:118 | CLAIM-SWEEP-OLMO2 (Claude Code opus-4-8) |
MODEL-TEXT-olmo-hybrid-olmo-hybrid-for-causal-lm | OlmoHybridForCausalLM | registry.py:174; vllm/model_executor/models/olmo_hybrid.py::OlmoHybridForCausalLM | causal generation / text | model loader/forward; paged attention/KV; Mamba/SSM state; GDN/linear-attention state | ☐ required | INVENTORIED | none | unassigned |
MODEL-TEXT-olmoe-olmoe-for-causal-lm | OlmoeForCausalLM | registry.py:175; vllm/model_executor/models/olmoe.py::OlmoeForCausalLM | causal generation / text | model loader/forward; paged attention/KV; FusedMoE/grouped GEMM | ☐ required | INVENTORIED | none | unassigned |
MODEL-TEXT-opt-optfor-causal-lm | OPTForCausalLM | registry.py:176; vllm/model_executor/models/opt.py::OPTForCausalLM | causal generation / text | model loader/forward; paged attention/KV; ENG-RUNNER-MODELSHAPE (composes with ZERO runner change); NEW vt::{LayerNorm,Relu,Add} | sweep-opt-125m spike | PARTIAL (W0-W4 landed; correctness DONE — STRICT token-exact 6/6 vs a MEASURED-deterministic vLLM oracle; SPEED bar explicitly PENDING, no throughput measured or claimed) | The breadth-sweep CROSS-FAMILY additivity canary (rank 4, .agents/specs/breadth-sweep-plan.md §B.3): the first ported family that is NOT a Qwen variant — no RoPE (LEARNED absolute positions with the fairseq OFFSET OF 2, OPTLearnedPositionalEmbedding opt.py:59-68), BIASED q/k/v/out/fc1/fc2 projections (config.enable_bias), LayerNorm-with-bias instead of RMSNorm, a plain ReLU fc1/fc2 MLP instead of SwiGLU, the do_layer_norm_before pre/post-LN switch, and tied embeddings by default. W0+W1 (2026-07-21): registry TU src/vllm/model_executor/models/opt_registry.cpp (REGISTER_VLLM_MODEL(opt, "OPTForCausalLM") + full-attention-only MakeOPTKVCache + ParseOPTConfig/GetOPTConfigExtras, which read OPT's eight family fields — ffn_dim (NOT intermediate_size), word_embed_proj_dim, do_layer_norm_before, enable_bias, layer_norm_elementwise_affine, _remove_final_layer_norm, tie_word_embeddings, activation_function — out of HfConfig::raw so the shared typed HfConfig POD is NOT widened; is_dense_model=true) + header include/vllm/model_executor/models/opt.h. Reusable-piece additions: LoadMergedBf16Vector (dense_weight_loaders.h, merged rank-1 bias vectors) and THREE NEW vt:: ops — LayerNorm (torch nn.LayerNorm: biased 1/N variance, f32 accumulation, weight AND bias, one rounding on store), Relu (get_act_fn("relu")), and Add (elementwise residual joins + rank-1 row-broadcast nn.Linear bias) — in two NEW self-registering TUs src/vt/cuda/cuda_layernorm.cu + src/vt/cpu/cpu_layernorm.cpp, with only append-only OpId/typedef/wrapper additions to include/vt/ops.h + src/vt/ops.cpp. Test tests/vt/test_ops_layernorm.cpp (5 cases / 9230 assertions, CPU + CUDA, vs the torch reference). W2 (2026-07-21): loader src/vllm/model_executor/models/opt_weights.cpp (LoadOPTForCausalLMWeights — merged qkv weight [3H,H] raw-NK + merged qkv BIAS [3H] in the same [q,k,v] order, out_proj/fc1/fc2 + biases, both LayerNorms with weight AND bias, the [max_pos+2, H] learned position table asserted, lm_head.weight SKIPPED under tie_word_embeddings mirroring opt.py:390-392). Test tests/vllm/models/test_opt_load.cpp — all 197 tensors mapped/shaped, NO leftover, 958 assertions. W3 (2026-07-21): forward src/vllm/model_executor/models/opt.cpp (OPTModel::Forward/ForwardDevice: embed_tokens + embed_positions[pos+2] -> 12 layers [LayerNorm -> merged biased qkv -> vt::QkvSplit -> ReshapeAndCache -> paged causal MHA -> biased out_proj -> residual add -> LayerNorm -> fc1+bias -> ReLU -> fc2+bias -> residual add] -> decoder final LayerNorm -> TIED lm_head), reusing the dense_attn_block.h DEVICE GLUE (Dev/DBuf/shared DevicePool/ResidentWeight/KvSlice) but deliberately NOT dense_attn::AttnBlock (which hard-codes q/k RMSNorm + NeoX RoPE + VT_CHECK(qkv_bias.Empty())). head_dim 64 runs the generic LaunchPaged path with ZERO kernel edit. W4 correctness gate (SACRED, 2026-07-21): gate SELECTED BY MEASUREMENT per [[near-tie-distributional-gate]] — scripts/opt-oracle-capture.py --runs 5 found vLLM 0.25.0 greedy DETERMINISTIC on all 6 prompts, 0 multi-valued (prompt,pos) cells ⇒ the bar is STRICT token-exact, with NO near-tie band. tests/vllm/models/test_opt_paged_engine.cpp + goldens tests/parity/goldens/opt_greedy/ ⇒ 6/6 prompts / 96/96 tokens token-exact, 36/36 assertions, prompt tokenization also exact. TWO tokenizer gaps were found and fixed by this gate, both correctness-fatal: (a) the ORIGINAL GPT-2 byte-level split (ByteLevel{use_regex:true} with no explicit Split component) was an explicit Fail -> NEW SplitPattern::kGpt2 + its own scanner (4 of 6 alternatives differ from Qwen/Llama-3), goldens HF-oracle-generated (test_pretokenizer 18 cases / 97926 assertions); (b) the TemplateProcessing post-processor BOS was PARSED but NEVER APPLIED (Encode documented "caller policy"; no Qwen tokenizer declares one) -> NEW Tokenizer::EncodeWithSpecialTokens (HF add_special_tokens=True) called from input_processor.cpp, with template_bos_/template_eos_ kept DISTINCT from the model's bos/eos ids so the GGUF path (which declares ids but adds nothing) is unaffected. Without (b) the gate scored 0/6 with fluent output — caught only because vLLM's own prompt_token_ids are committed as goldens. CHECKPOINT REALITY CORRECTION: §B.1 listed opt-125m as "present"; the HF cache held only a config.json (no weights), and the downloaded snapshot is a torch-pickle pytorch_model.bin (fp16) with no tokenizer.json — materialized to bf16 safetensors + a fast tokenizer by the committed scripts/opt-materialize-checkpoint.py (spec D2). DTYPE: both arms run BF16 (--dtype bfloat16, a first-class vLLM mode) because kF16 is unimplemented outside cuda_gdn.cu (spec D1/L6). ADDITIVITY VERDICT (the primary deliverable): the MODEL seams held 100% — 4 new model files, ZERO edit to the runner, scheduler, platforms, attention registry, hf_config.{h,cpp}, any CUDA kernel, or any existing model; 11 shared files touched, of which 9 are append-only extension-point/build-list additions, 1 is test-surface counts, and exactly ONE (input_processor.cpp) changes a line of shared behavior — provably a no-op for every existing model. The leaks are one layer down: the vt:: op table and the tokenizer had both been silently specialized to Qwen. GATES (dgx ~/work/vllm.cpp-opt125m, git archive transfer NOT rsync, goldens md5-verified identical before AND after, production flags CUTLASS sm120a + Marlin + FA2 sm_121a + Triton AOT, whole series under one flock /tmp/gpu): clean full CUDA rebuild -Werror 0 warnings / 0 errors; on the FINAL binary test_ops_layernorm 5 cases / 29966 assertions (CUDA leg live), test_opt_load 958 assertions, test_opt_paged_engine 6/6 prompts / 96/96 tokens / 36 assertions; REGRESSIONS ALL UNCHANGED — 27B test_qwen27_paged_engine 235/235, 35B test_qwen36_paged_engine 315/315, Qwen3-Coder test_qwen3coder_paged_engine 6/6 (138 assertions), Qwen3-dense test_qwen3_paged_engine 664 assertions, test_model_registry 231, test_pretokenizer 97926, test_model_loader_gguf 3/3. memcheck, stated precisely: compute-sanitizer --tool memcheck --leak-check=full on the OPT engine path gives 0 invalid reads / 0 invalid writes / 0 invalid __global__ accesses with the gate still 6/6 under the sanitizer; the 66 --leak-check=full reports (34.4 MB) are NOT an OPT leak — 63 of 66 trace to dense_attn::DBuf, i.e. the shared DevicePool, whose Put never calls Backend::Free when uncapped (cap 0 on GB10, device_pool.h:65-77), so pooled blocks are retained for the process lifetime BY DESIGN. Same-binary baseline: pre-existing Qwen3-0.6B reports 125 such allocations (98 from DBuf, 35.9 MB) and full Qwen3-dense 184 (39.7 MB), both also with 0 invalid accesses — OPT retains FEWER pool blocks than the dense model preceding it. CPU: -Werror 0-warn, full CTest 136/138 with the two HTTP-server misses PASSING isolated (2/2, the documented -j port-contention flake) METAL (2026-07-22, BACKEND-METAL-MLX work row M3a): OPT-125m IS THE FIRST MODEL TO RUN ON APPLE GPU, and it runs at the SAME STRICT BAR — 6/6 prompts token-exact, 96/96 tokens, against the SAME committed dgx-captured vLLM 0.25.0 goldens, i.e. a device-INDEPENDENT bar CUDA already met rather than one re-derived on Metal. Chosen because the study measured it the cheapest correct non-CUDA model: all four OPT TUs contain ZERO CUDA references and it needed exactly five new MSL kernels (kEmbedding, kQkvSplit, kReshapeAndCache, kPagedAttention, kGreedyArgmax) on top of the W0 set plus the landed MSL GEMM. THE METAL PATH IS PROVEN TO HAVE EXECUTED, not inferred: test_opt_paged_engine.cpp asserts runner().device().type == kMETAL and, for ALL NINE ops OPT dispatches, selections > 0 AND declines == 0 (kPagedAttention selections = 1152 over the 96 generated tokens) — last_selected alone would NOT be proof, since a provider can decline inside its kernel and forward down. The same source now gates BOTH accelerators: SelectQueue asks the Platform seam instead of hardcoding kCUDA. TWO REAL PORTABILITY BUGS were found only because a model finally ran on a non-NVIDIA device — runner.cpp:516 gated KV-cache device residency on is_cuda(), so the cache fell into a host vector and vt::ReshapeAndCache was handed a HOST pointer (fixed to !is_cpu(), the same shape as the earlier dense_attn_block.h fix); and model_loader.cpp SelectQueue hardcoded kCUDA, which is the single line that had kept every non-NVIDIA accelerator on the CPU reference no matter how complete it was. NO METAL SPEED NUMBER IS CLAIMED OR OWED — the M4 could not be quieted (root LaunchDaemon + aerial wallpaper), so any timing would be void; correctness and speed are separate bars and only correctness is met here.; registration src/vllm/model_executor/models/opt_registry.cpp:184; test tests/vllm/models/test_model_registry.cpp:80 | CLAIM-MODEL-OPT-125M |
MODEL-TEXT-orion-orion-for-causal-lm | OrionForCausalLM | registry.py:177; vllm/model_executor/models/orion.py::OrionForCausalLM | causal generation / text | model loader/forward; paged attention/KV | ☐ required | INVENTORIED | none | unassigned |
MODEL-TEXT-ouro-ouro-for-causal-lm | OuroForCausalLM | registry.py:178; vllm/model_executor/models/ouro.py::OuroForCausalLM | causal generation / text | model loader/forward; paged attention/KV; sliding-window attention | ☐ required | INVENTORIED | none | unassigned |
MODEL-TEXT-openpangu-pangu-embedded-for-causal-lm | PanguEmbeddedForCausalLM | registry.py:179; vllm/model_executor/models/openpangu.py::PanguEmbeddedForCausalLM | causal generation / text | model loader/forward; paged attention/KV; FusedMoE/grouped GEMM; MLA/latent KV; sliding-window attention | ☐ required | INVENTORIED | none | unassigned |
MODEL-TEXT-openpangu-pangu-pro-mo-ev2-for-causal-lm | PanguProMoEV2ForCausalLM | registry.py:180; vllm/model_executor/models/openpangu.py::PanguProMoEV2ForCausalLM | causal generation / text | model loader/forward; paged attention/KV; FusedMoE/grouped GEMM; MLA/latent KV; sliding-window attention | ☐ required | INVENTORIED | none | unassigned |
MODEL-TEXT-openpangu-pangu-ultra-mo-efor-causal-lm | PanguUltraMoEForCausalLM | registry.py:181; vllm/model_executor/models/openpangu.py::PanguUltraMoEForCausalLM | causal generation / text | model loader/forward; paged attention/KV; FusedMoE/grouped GEMM; MLA/latent KV; sliding-window attention | ☐ required | INVENTORIED | none | unassigned |
MODEL-TEXT-param2moe-param2-mo-efor-causal-lm | Param2MoEForCausalLM | registry.py:182; vllm/model_executor/models/param2moe.py::Param2MoEForCausalLM | causal generation / text | model loader/forward; paged attention/KV; FusedMoE/grouped GEMM | ☐ required | INVENTORIED | none | unassigned |
MODEL-TEXT-persimmon-persimmon-for-causal-lm | PersimmonForCausalLM | registry.py:183; vllm/model_executor/models/persimmon.py::PersimmonForCausalLM | causal generation / text | model loader/forward; paged attention/KV | ☐ required | INVENTORIED | none | unassigned |
MODEL-TEXT-phi-phi-for-causal-lm | PhiForCausalLM | registry.py:184; vllm/model_executor/models/phi.py::PhiForCausalLM | causal generation / text | Correctness-complete + gated. The OLDER Microsoft Phi arch (microsoft/phi-2), DISTINCT from Phi3ForCausalLM. Impl: include/vllm/model_executor/models/phi.h + src/vllm/model_executor/models/{phi,phi_weights,phi_registry}.cpp. ZERO-NEW-KERNEL: GPT-J parallel-residual (ONE nn.LayerNorm+bias feeds BOTH attn+mlp, summed together — reuses the Command-R wiring), biased q/k/v/dense (OPT BiasedProj), partial NeoX rope (rotary_dim 32/head_dim 80, RopeFromCache), NON-gated NewGELU MLP (fc1->gelu_new->fc2 — gelu_new == the landed vt::GeluTanh, NO new op), untied lm_head+bias. F16→BF16 dtype-aware loader (LOCAL to phi_weights.cpp, mirrors OLMo-2's F32→BF16; shared header untouched). Gate test_phi_paged_engine (tests/parity/test_phi_paged_engine.cpp), golden tests/parity/goldens/phi2_greedy_2_7b/. vLLM 0.25.0 per-prompt K=5 ALL-DETERMINISTIC ⇒ STRICT well-posed; 16/16 PASS by the ratified near-tie ROOT-divergence gate: 9 STRICT token-exact + 7 near-tie band, max teacher-forced gap 0.25 nats < 0.5, 0 forward-divergent (every downstream token == vLLM's teacher-forced argmax at gap 0.0). RED-first (3 load-bearing deltas): full-instead-of-partial rotary, dropped qkv bias, and sequential-instead-of-parallel residual each blow past 0.5 nats — gate CATCHES. memcheck 0 errors. Speed pending. | 🚧 recent-dense-batch | PARTIAL | test_phi_paged_engine PASS: microsoft/phi-2 16/16 (92 assertions), dgx; 3 RED variants gate-caught; loader zero-unmapped; registration src/vllm/model_executor/models/phi_registry.cpp:146; test tests/vllm/models/test_model_registry.cpp:120 | CLAIM-SWEEP-RECENT-DENSE (Claude Code opus-4-8) |
MODEL-TEXT-phi3-phi3-for-causal-lm | Phi3ForCausalLM | registry.py:185; vllm/model_executor/models/phi3.py::Phi3ForCausalLM | causal generation / text | Correctness-complete + gated (RATIFIED near-tie ROOT-divergence methodology; gate now a CLEAN PASS). Impl: include/vllm/model_executor/models/phi3.h + src/vllm/model_executor/models/{phi3,phi3_weights,phi3_registry}.cpp. Pre-norm Llama forward + PRE-FUSED qkv/gate_up loader + precomputed LongRoPE cache (partial rotary 96/128) fed to RopeFromCache by REAL positions. Gate test_phi3_paged_engine (tests/parity/test_phi3_paged_engine.cpp), goldens tests/parity/goldens/{phi4_mini_greedy,phi4_14b_greedy}/. Both checkpoints vLLM 0.25.0 K=5 ALL-DETERMINISTIC. Phi-4-mini-instruct 16/16 by the ratified ROOT-divergence gate (mirrors olmo2/qwen3-dense: teacher-force vLLM on OUR sequence, PASS iff every ROOT/first divergence's gap <=0.5 nats, cascade positions downstream of a root NOT counted): 7 strict + 9 near-tie, 0 forward-divergent, max on-path gap 0.5 @ p6. RCA re-verified on current main: every root divergence <=0.5; p12's only >0.5 positions (tok6/tok8=1.0) are CASCADE downstream of an exact-tie root (tok4 gap 0.0000, vLLM's own teacher-forced argmax IS our token -- vLLM's incremental decode contradicts its own argmax at a bf16 tie). LongRoPE cos/sin cache BIT-IDENTICAL to vLLM's (.to(bf16), mscale 1.1902380714238083). RATIFIED BIGGER-DENSE STRICT anchor microsoft/phi-4 (14B, same forward) 16/16 with STRICT token-exact 14/16 + 2 exact-tie (gap 0.0000) -- the deterministic proof the forward is genuinely correct. RED-first: disabling the LongRoPE mscale (1.19->1.0) produces 11 ROOT-fails (0.75-3.0 nats) the gate CATCHES (FATAL 5/16), proving the gate is not vacuously lenient. Harness made EOS-aware (14B emits EOS on 3 factual prompts). memcheck 0 errors. Speed pending. | 🚧 recent-dense-batch | PARTIAL | test_phi3_paged_engine PASS: Phi-4-mini 16/16 (108 assertions) + phi-4-14B 16/16 STRICT 14/16 (134 assertions), dgx; RED-first broken-mscale gate FATAL 5/16; loader zero-unmapped on both; registration src/vllm/model_executor/models/phi3_registry.cpp:114; test tests/vllm/models/test_model_registry.cpp:121 | CLAIM-SWEEP-RECENT-DENSE (Claude Code opus-4-8) |
MODEL-TEXT-phimoe-phi-mo-efor-causal-lm | PhiMoEForCausalLM | registry.py:186; vllm/model_executor/models/phimoe.py::PhiMoEForCausalLM | causal generation / text | model loader/forward; paged attention/KV; FusedMoE/grouped GEMM; sliding-window attention | ☐ required | INVENTORIED | none | unassigned |
MODEL-TEXT-plamo2-plamo2-for-causal-lm | Plamo2ForCausalLM | registry.py:187; vllm/model_executor/models/plamo2.py::Plamo2ForCausalLM | causal generation / text | model loader/forward; paged attention/KV; Mamba/SSM state | ☐ required | INVENTORIED | none | unassigned |
MODEL-TEXT-plamo3-plamo3-for-causal-lm | Plamo3ForCausalLM | registry.py:188; vllm/model_executor/models/plamo3.py::Plamo3ForCausalLM | causal generation / text | model loader/forward; paged attention/KV; sliding-window attention | ☐ required | INVENTORIED | none | unassigned |
MODEL-TEXT-qwen2-qwen2-for-causal-lm | Qwen2ForCausalLM | registry.py:189; vllm/model_executor/models/qwen2.py::Qwen2ForCausalLM | causal generation / text | model loader/forward; paged attention/KV | ☐ required | INVENTORIED | none | unassigned |
MODEL-TEXT-qwen2-moe-qwen2-moe-for-causal-lm | Qwen2MoeForCausalLM | registry.py:190; vllm/model_executor/models/qwen2_moe.py::Qwen2MoeForCausalLM | causal generation / text | model loader/forward; paged attention/KV; FusedMoE/grouped GEMM | ☐ required | INVENTORIED | none | unassigned |
MODEL-TEXT-qwen3-qwen3-for-causal-lm | Qwen3ForCausalLM | registry.py:191; vllm/model_executor/models/qwen3.py::Qwen3ForCausalLM | causal generation / text | W0+W1 (2026-07-20): registry TU src/vllm/model_executor/models/qwen3_dense.cpp (REGISTER_VLLM_MODEL(qwen3, "Qwen3ForCausalLM") + full-attention-only MakeQwen3ForCausalLMKVCache + stub factory) + header include/vllm/model_executor/models/qwen3.h; runner generalization src/vllm/v1/worker/gpu/runner.cpp:458-470,651-680 (full-attention-only KV, model-agnostic has_mamba_group/gdn_group_id_>=0 guards). W2 (2026-07-20): weight loader src/vllm/model_executor/models/qwen3_weights.cpp (LoadQwen3ForCausalLMWeights) + Qwen3DenseWeights struct in include/vllm/model_executor/models/qwen3.h (merged qkv/gate_up raw-NK per vLLM packed_modules_mapping, per-head q/k norm, tied lm_head aliasing embed_tokens — checkpoint lm_head.weight skipped mirroring vLLM skip_prefixes); shared BF16 helpers extracted to include/vllm/model_executor/models/dense_weight_loaders.h (27B qwen3_5_dense_weights.cpp load byte-identical). W3 (2026-07-20): dense forward src/vllm/model_executor/models/qwen3.cpp (Qwen3DenseModel::Forward/ForwardDevice) composed from public vt:: ops + fusion catalog; per layer std add+RMSNorm → merged qkv (bf16 MatmulBT) → per-head q/k RMSNorm+RoPE (bf16, no gate) → paged attn → o_proj → post norm → SwiGLU MLP → final norm → tied lm_head; wired into the qwen3_dense.cpp factory. 2 NEW fusion recipes include/vt/recipes.h (kFusedAddRmsNormStd non-gemma add-RMSNorm used at 3 norm sites; kAttnQkNormRope non-gated qk-norm-rope composite) each byte-exact composite-tested in tests/vt/test_ops_fused_chain.cpp. Two GENUINE latent bugs the first pure-dense bf16 model FORCED OUT: (a) tokenizer SplitPattern::kQwen2Classic (src/vllm/tokenizer/pretokenizer.{h,cpp}, tokenizer.cpp) — the classic Qwen2/Qwen3 pre-tokenizer regex was hard-rejected; (b) src/vt/cuda/cuda_paged_attn.cu WMMA prefill gated to head_dim 256 only (the tensor-core ladder mistokenizes at other head_dims — validated only for the d=256 gate models). Tests tests/vllm/models/test_qwen3_forward.cpp (CPU synthetic + dgx prefill-argmax==12095 CPU+CUDA + ADOPT==fallback byte-exact), tests/parity/test_qwen3_paged_engine.cpp (SACRED gate) + oracle capture scripts/qwen3-oracle-capture.py. W4 (2026-07-20, GATING — NOT token-exact yet): Qwen3-0.6B greedy vs vLLM 0.25.0 oracle = 11/16 prompts token-exact (all 16 first-tokens exact; 5 fail at late-decode SEMANTIC NEAR-TIES, e.g. "question"↔"answer"). ISOLATED as per-op bf16-rounding drift vs vLLM's kernels, NOT a structural bug: CPU-exact-f32 also 10/16 with overlapping divergences; RMSNorm rounding verified matching vLLM csrc. 16/16-EXACT IS ILL-POSED (2026-07-20 near-tie razor): the ISOLATE-then-BIT-MATCH pass showed vLLM's OWN bf16 greedy is non-deterministic on these near-ties — vLLM ↔ its committed golden = 16·15·16 (N=3) and flips 7/16 prompts over N=10 (p6 4/10, p15 3/10; golden is the MINORITY token there); vLLM = FLASH_ATTN/FA2 for head_dim 128. All 5 of our divergences sit on gaps ≤0.125 nats (p5@5 is an EXACT 0.0 tie), and we emit vLLM's MAJORITY token on p6/p15. vt::GreedyArgmax already mirrors torch.argmax lowest-index tiebreak (cuda_sample.cu:94). ⇒ NOT a kernel bug and there is no fixed golden to bit-match to; a flash-exact d=128 rewrite cannot make a non-deterministic golden deterministic. REGRESSION preserved by construction (no code change): 27B test_qwen27_paged_engine 235/235 + 35B test_qwen36_paged_engine 315/315 UNCHANGED. Correct closure = a near-tie-robust distributional gate (accept if our output ∈ vLLM's K-run output set) — gate-DESIGN decision reserved for the user (spike honesty escape hatch). W4 CORRECTNESS COMPLETE 2026-07-20 (tests/parity/test_qwen3_paged_engine.cpp rewritten to the near-tie-robust gate; scripts/qwen3-neartie-gap.py + --runs/--per-prompt in scripts/qwen3-oracle-capture.py; goldens greedy_dist/our_ids/neartie_gap_mnats for both dirs). KEY CORRECTION: the "vLLM greedy is non-deterministic" premise was a BATCHING artifact. Per-prompt (batch=1 — the gate's single-request regime), vLLM 0.25.0 greedy is DETERMINISTIC: Qwen3-0.6B 0 multi-member cells over K=10, Qwen3-4B 0 over K=5 (batched, all 16 prompts in one call, it flips — 0.6B 56, 4B 43 multi-member cells). So the strict token-exact bar IS well-posed. Near-tie robustness comes from TEACHER-FORCING vLLM on OUR exact prefix (prompt_logprobs): at all-but-2 positions vLLM's OWN argmax given our prefix IS our token with gap 0.0000 (bit-identical logprobs), proving our forward matches vLLM's prefill logits; the residual flips are bf16 near-ties (0.6B ≤0.125 nats ×2; 4B ≤0.25 nats) where vLLM's OWN one-shot prefill argmax disagrees with its incremental decode (e.g. 4B p13 tok1 prefill→13=ours, decode→11) — vLLM contradicts itself, so NO forward bug and no single 16/16 decode target. Gate PASS = our token within 0.5 nats of vLLM's teacher-forced argmax (strict where equal). Qwen3-0.6B 16/16 PASS (strict token-exact 12/16 + near-tie-band 4/16, max gap 0.125 nats). BIGGER-MODEL COMPLETE-CORRECTNESS PROOF Qwen3-4B (BF16, 36 layers, GQA 32/8, hidden 2560 — a DIFFERENT config than 0.6B, loads+runs on the SAME forward code): 16/16 PASS (strict 10/16 + near-tie 6/16, max gap 0.25 nats). GATES: dgx CUDA -Werror 0-warn (test TU host-C++); 27B 235/235 + 35B 315/315 token-exact UNCHANGED (no engine source touched → preserved by construction, re-confirmed); compute-sanitizer memcheck 0 errors on the 0.6B gate path; CPU syntax RC=0 (gate skips off-dgx). Correctness code + evidence anchors: forward src/vllm/model_executor/models/qwen3.cpp + loader src/vllm/model_executor/models/qwen3_weights.cpp; gate tests/parity/test_qwen3_paged_engine.cpp + goldens tests/parity/goldens/qwen3_greedy_{0_6b,4b}/; ledger 2026-07-20 W4-correctness-complete row. Correctness-complete; SPEED benchmark pending (next task, precondition met) → row stays ACTIVE until speed parity per the DONE=correctness+speed policy. STRICT-DECODE RAZOR investigated 2026-07-20 (worktree agent-a7e66dd02ed286cb2): attempt to tighten the near-tie band to STRICT 16/16 by routing d128 decode through the vendored FA2 group-swap split-KV kernel was MEASURED WORSE (0.6B 12→11/16, 4B 10→9/16) and REVERTED — source-confirmed that vLLM runs FA2 varlen (flash_attn_varlen_func, fa_version 2 on sm_121), NOT the group-swap flash_attn_with_kvcache path; the real bit-match target is an FA2 VARLEN d128 decode, a scoped sub-campaign in qwen3-decode-strict-bitmatch spec. FA2 VARLEN d128 RESOLVED 2026-07-20 (worktree agent-a31b006ff4766f03b): vendored the EXACT vLLM varlen decode — d128 bf16 split-KV instantiations flash_fwd_split_hdim128_bf16_{,causal_}sm80.cu + LaunchDecodeVarlenFA2Bf16 (src/vt/cuda/cuda_flash_attn_fa2.cu: plain varlen cu_seqlens_q=query_start_loc, h=hq, is_causal=true, NO group swap, num_splits=exact heuristic) routed by VT_FA2_DECODE_QWEN3 (default OFF, scoped d128 so the d256 arms are untouched); op-parity byte-exact vs f32 ref (tests/vt/test_ops_paged_attn.cpp *varlen d128*, incl. num_splits>1 split-combine) + memcheck 0. The kernel BIT-MATCHES vLLM's decode attention OUTPUT (teacher-forced logit gap 0.0000 nats at all-but-near-tie positions) but STRICT 16/16 is NOT reached: FA2-varlen 0.6B 11/16 & 4B 9/16 — ONE WORSE than the CUDA-core fallback (12/16 & 10/16). Escape hatch CONFIRMED: strict is bf16-tie-bounded (residual flips are ≤0.375-nat ties where vLLM's own prefill argmax disagrees with its incremental decode). Shipped OPT-IN; near-tie-robust gate stays the closure; engine gate NOT tightened. 27B 235/235 + 35B 315/315 + default Qwen3 gate 16/16 UNCHANGED, -Werror 0-warn NVFP4A16 / W4A16 QUANT-SCHEME EXTENSION 2026-07-21 (row QUANT-NVFP4-CT-W4A16, spike sweep-qwen3-32b-nvfp4a16, worktree agent-a141e107f210eb6c6, base aa65ce7): the SAME dense forward now also loads and natively computes compressed-tensors NVFP4A16 (W4A16) on RedHatAI/Qwen3-32B-NVFP4A16 (64L dense, hidden 5120, GQA 64/8, intermediate 25600, UNTIED BF16 lm_head — a 64-layer size-scale check for the dense path on top of 0.6B/4B). Checkpoint premise VERIFIED BEFORE implementation (unlike OPT the plan's B.1 claim was CORRECT: 5 shards / 20.6 GB / 1603 tensors). Loader: per-Linear .weight_packed probe + fp4 fields on Qwen3DenseAttnWeights/Qwen3DenseMlpWeights (include/vllm/model_executor/models/qwen3.h), append-only helpers in dense_weight_loaders.h; forward: two dispatch branches (qwen3.cpp::MlpBlock, dense_attn_block.h::AttnBlock) into the NEW include/vllm/model_executor/models/dense_nvfp4_gemm.h. dense_device_glue.h is a PURE VERBATIM RELOCATION of Dev/DBuf/MakeTensor/Reshape/DevicePoolPolicy out of dense_attn_block.h (same namespace ⇒ the BF16 Qwen3-dense/Coder/OPT forwards stay BYTE-IDENTICAL). CORRECTNESS: strict scored 4/6 (67/96 tokens), the teacher-forcing isolation then SETTLED it, and the gate closes 6/6 under the ratified near-tie-robust bar WITH the evidence committed (tests/vllm/models/test_qwen3_32b_nvfp4a16_paged_engine.cpp). Gate selection MEASURED in two stages: vLLM's own greedy is DETERMINISTIC (K=5, 0 multi-valued cells) so STRICT was run FIRST and was NOT loosened on that evidence; then the ratified prompt_logprobs TEACHER-FORCING isolation (new scripts/qwen3-32b-nvfp4a16-neartie-gap.py + a VT_DUMP_IDS=1 bootstrap) measured all 29 divergent positions at <= 0.0625 nats with 28/29 EXACTLY 0.0 — our token IS vLLM's own argmax given OUR prefix. THIS ROW OWNS THE FINDING. The prompt[5] root flip is an EXACT bf16 TIE (" moon"/" Moon" have bit-identical vLLM logprobs -0.727154) at which vLLM's teacher-forced argmax is OUR token while vLLM's incremental greedy chose the other — vLLM contradicts ITSELF, exactly the prefill-argmax-vs-incremental-decode disagreement this row characterized at 0.6B/4B. The prompt[2] root flip gaps 0.0625 nats, against 0.125 nats of vLLM's OWN jitter at that same position (measured by batch-composition sweep: 0.1875 alone → 0.0625 batched) and below the 0.25-nat gap already ratified on the UNQUANTIZED 4B; the other 27 divergences are downstream cascade. Because the PREFILL argmax is exact on all six prompts, the prefix at both root flips is BIT-IDENTICAL to vLLM's, making the gap a direct single-position logit comparison rather than a prefix-drift artefact. ⇒ the residual is THIS row's pre-existing dense bf16 near-tie drift run 64 layers deep — the DEEPEST dense model we have run — and NOT a W4A16 defect; the quant path is exonerated four independent ways (bit-exact CPU proof max abs delta-logit = 0, fallback_gemms=0 vs 18432 Marlin GEMMs, invariance across both quantized GEMMs — VT_NVFP4_MARLIN=0 scores 3/6 with the SAME divergences — and the <=0.0625-nat gaps). Final: 6/6 prompts PASS (strict 4/6 + near-tie band 2/6, max gap 0.062 nats vs the 0.5-nat bar shared with the 0.6B/4B/Coder gates, 0 forward-divergent, 142/142 assertions incl. 96 hard anchor REQUIREs), path proven exercised (marlin_gemms=18432, fused_gate_up=6144, fallback_gemms=0), all six vLLM tokenizations match. CONSISTENT SCALING EVIDENCE for this row's drift: 0.6B 60 divergent positions ALL at gap 0.0; 4B max 0.25 nats; 32B/64L max 0.0625 nats — deeper does NOT mean worse, and every one is a tie vLLM's own logits cannot separate. BF16 behavior UNCHANGED by construction (the probe selects the fp4 arm only when .weight_packed exists): Qwen3-dense/Coder/OPT gates unaffected; 27B 235/235 + 35B 315/315 UNCHANGED; CUDA -Werror 0-warn. | first-additive-model spike SPEED — d128 FA2 PREFILL IMPLEMENTED + FA2 DECODE DEFAULT FLIPPED 2026-07-20 (Qwen3-4B BF16 vs vLLM 0.25.0 production/graphed, in1024/out128 ignore-eos closed-loop) — big gap-close but STILL below vLLM, stays ACTIVE: the dominant prefill lever is now a d128 FlashAttention-2 varlen prefill — generalized the vendored FA2 launcher LaunchPrefillFA2Bf16 (src/vt/cuda/cuda_flash_attn_fa2.cu) to head_dim 128 (reuses the already-vendored flash_fwd_split_hdim128_bf16_{,causal_}sm80.cu split-KV instantiations, num_splits=1 Split=false = exactly vLLM's flash_attn_varlen_func), routed by a new d128-scoped fa2_prefill_qwen3 gate + Fa2PrefillQwen3Enabled (VT_FA2_PREFILL_QWEN3 default-ON, src/vt/cuda/cuda_paged_attn.cu); AND flipped the FA2 varlen d128 DECODE default ON (Fa2DecodeQwen3Enabled now default-ON). Op-parity byte-exact vs the f32 ref at both Qwen3 ratios (tests/vt/test_ops_paged_attn.cpp *head_dim 128*, host-metadata bit-identical + toggle-off fallback), memcheck 0. Near-tie gate RE-PASSES 16/16 on 0.6B AND 4B with FA2 prefill+decode ON (goldens refreshed via scripts/qwen3-neartie-gap.py; teacher-forced max gap 0.125 nats, 0 forward-divergent — FA2 prefill matches vLLM's prefill logits). RESULT: total tput 0.90× (c1) / 0.62× (c8) (up from 0.80×/0.48×), c1 decode at parity (TPOT 1.04×, ITL P99 0.98× WIN); prefill-kernel A/B (same binary) = +7%/+41% total, −55%/−48% TTFT, −22% c8 TPOT. STILL FAIL: TTFT median 5.85×/10.2× + total <1× because the full prefill STEP (not the attention kernel — now vLLM's FA2 family) is ~6× vLLM = non-attention glue (GEMM/MLP fusion) + host-side launch overhead (un-graphed prefill); plus c8 decode batch efficiency (TPOT 1.38×). RESIDUAL LEVERS: (1) portable prefill-step fusion + graphed prefill (dominant); (2) c8 split-KV decode occupancy. Repro + table: ledger 2026-07-20 FA2-prefill SPEED row + docs/BENCHMARKS.md; goldens tests/parity/goldens/qwen3_greedy_{0_6b,4b}/{our_ids,neartie_gap_mnats}.npy refreshed for FA2-ON. 27B 235/235 + 35B 315/315 + -Werror 0-warn UNCHANGED. TTFT LEVERS 2026-07-20 (base 812a57a, worktree agent-aea71a6a5024b17d4) — measure-first overturned profile #81's two premises: (L1) extracted the pooled DevicePool VERBATIM to shared include/vllm/model_executor/models/device_pool.h (gate-model qwen3_5.cpp + dense qwen3.cpp both use it) — byte-identical (27B 235/235, 35B 315/315, 0.6B/4B 16/16 UNCHANGED) but a clean same-binary A/B DISPROVED the "~44% GPU-idle from cudaMalloc/cudaFree" premise: PERF-NEUTRAL (async scheduler overlaps the host-side alloc syncs); kept as byte-safe hygiene. (L2) RoPE cos/sin cache — route the bf16 preamble to RopeFromCache off the per-step cache (identity row index; bf16 cache == vLLM RotaryEmbedding) instead of RopeNeox's per-element fp64 pow/cos/sin (~36.5% prefill GPU-busy); MEASURED the dominant dense-TTFT lever (same-binary A/B, in1024/out128, idle box, 2 reps, vs graphed vLLM bench serve): c1 median TTFT 209→135 ms = 0.87× vLLM (median parity); c8 median 316→144 ms = 0.38× vLLM; TPOT/tput flat (prefill-only). Shipped DEFAULT-ON VT_QWEN3_ROPE_CACHE (2026-07-20; =0 opts back to RopeNeox) after the opt-in blocker was GROUNDED + DISPROVEN. The claimed "1-ULP RopeFromCache shift lands on the engine's ~1-ULP FA2-split-KV near-tie NONDETERMINISM → flaky gate (RopeNeox 4/4, RopeFromCache flips)" was NOT reproducible: the paged engine is byte-DETERMINISTIC run-to-run (RoPE-off gate 4/4 + RoPE-on dumps md5-identical 3/3 + flipped-default gate 16/16 identical K=5), and for the short gate contexts num_splits==1 so flash_fwd_splitkv_combine_kernel (flash_fwd_kernel.h:1126, launched only when num_splits>1) never runs; the combine is a fixed-order register reduction (no atomics) anyway. The 1-ULP FMA shift is real but deterministic — it moves 2 genuine bf16 near-tie tokens, handled by regenerating the near-tie goldens on the CANONICAL $HOME/cutlass-4.5.0 build (the flashinfer-bundled cutlass copy tips the 27B tok6 razor to 271→234/235; cutlass-4.5.0 gives 198→235/235; Qwen3 near-ties are likewise cutlass-sensitive). Gate re-passes 16/16 both (0.6B strict 10+nt 6 max 0.0 nats; 4B strict 11+nt 5 max 0.25), deterministic K=3. RoPE flip isolated from 27B/35B (RopeCacheEnabled/Qwen3DenseModel referenced only in qwen3.cpp/qwen3_dense.cpp/qwen3.h): 27B 235/235 + 35B 315/315 UNCHANGED on the canonical build. SPEED (RoPE-ON default vs vLLM 0.25.0 production): c1 total 0.97× / TPOT 1.00× / ITL P99 0.96× WIN / TTFT 2.27×; c8 total 0.82× / TPOT 1.17× / TTFT 1.90× (RoPE A/B: c1 TTFT −34%, c8 TTFT −43% / total +10%). Still ACTIVE — residuals = prefill-STEP non-attention glue (TTFT) + c8 split-KV decode occupancy. Anchors: src/vllm/model_executor/models/qwen3.cpp (RopeCacheEnabled), include/vllm/model_executor/models/device_pool.h, ledger 2026-07-20 TTFT-levers row, docs/BENCHMARKS.md. | ACTIVE (correctness COMPLETE — 0.6B + 4B near-tie gates PASS 16/16; SPEED re-bound 2026-07-21 same-session matching-recipe vs vLLM 0.25.0 production: c1 EFFECTIVE EVERY-AXIS PARITY — tput 0.98× / TTFT 0.90× WIN / TPOT 1.01× / P99 ITL 0.996× WIN; c8 residual = decode — tput 0.93× / TTFT 0.38× WIN / TPOT 1.10× / P99 ITL 1.12×. The prior "TTFT 2.27×/5.85×" + "c8 ITL 4.3×" were BAD-DENOMINATOR/num-prompts artifacts (fresh vLLM c1 TTFT ~152 ms, ours WINS TTFT both concurrencies); TTFT residual RESOLVED. c8 decode is 93% GPU-busy/compute-bound (~72% small-M=8 cutlass_80_wmma projections). qkv-merge (new GQA QkvSplit op + merged single-QKV GEMM, mirrors vLLM QKVParallelLinear) IMPLEMENTED + MEASURED NEUTRAL (doesn't cut decode FLOPs) ⇒ ships DEFAULT-OFF VT_QWEN3_QKV_MERGE. cutlass verify RESOLVED: 27B 235/235 on the flashinfer-cutlass build (prior 234/235 = build artifact). Named residual = c8 decode-GEMM efficiency, a decode-fusion sub-campaign; not yet done) === METAL COVERAGE 2026-07-23 (work row M3b, CLAIM-BACKEND-METAL-M3B-1 + RCA/oracle CLAIM-BACKEND-METAL-M3B-RCA) — Qwen3-dense is the SECOND model on a non-CUDA backend; forward ORACLE-CONFIRMED CORRECT, NOT strict-token-exact. === Qwen3-0.6B generates END TO END on the Apple M4 GPU. HONEST STATE — supersedes an earlier "16/16 strict, 4 near-ties at gap-0" claim that was UNSUBSTANTIATED (branch test had no teeth; see RCA + oracle ledger rows). The Metal forward is a DIFFERENT but equally correct bf16 decoder that resolves the model's genuine near-ties the other way (p0 tok5 = 15344 " Italy" vs 9625 " France"; also p5 tok10, p10 tok10, p11 tok1). ORACLE-BACKED near-tie confirmation (the decisive measurement, scripts/qwen3-neartie-gap.py teacher-forcing vLLM 0.25.0 on the METAL prefix, batch=1, gpu_mem_util=0.40): all 60 Metal-vs-CUDA divergent positions are within 0.5 nats of vLLM's own argmax given the Metal prefix — MAX GAP 0.125 nats, none outside top-20; p0 tok5 gap 0.0000 (vLLM's teacher-forced argmax on the identical prefix IS Italy 15344, contradicting its CUDA-capture France pick — the literal near-tie signature). Passes test_qwen3_paged_engine on Metal 16/16 against Metal's OWN oracle golden (our_ids_metal.npy + neartie_gap_mnats_metal.npy): 10 strict token-exact vs vLLM greedy + 6 near-tie-band, 0 forward-divergent, IDENTICAL gate logic to CUDA (hard anchor REQUIRE + ≤0.5-nat band, no cross-device latitude), teeth PROVEN (perturbed anchor → drift FAIL; perturbed gap→0.6 nats → band FAIL; restore → PASS). Metal execution PROVEN (device==kMETAL; all 9 ops selections>0 ∧ declines==0, kRopeFromCache/kPagedAttention 7168 each). Needed 3 new MSL RoPE kernels: kRopeFromCache (bit-exact — the DEFAULT VT_QWEN3_ROPE_CACHE-ON apply) + kRopeCosSinCache + kRopeNeox → Metal 18/75 ops. Per-op stays the NMSE ≤5e-4-vs-CPU proof (RCA, all 28 layers). First ours-vs-MLX benchmark produced (Qwen3-1.7B, INDICATIVE/BLOCKED-ON-SUDO): ours a knowingly-unoptimised FLOOR ~6–11× slower decode / ~7–10× TTFT / ~2× memory vs MLX. STRICT token-exactness on 0.6B is ILL-POSED (near-tie model); a strict Metal gate needs a bigger deterministic dense model (Qwen3-4B, not on the M4) — DEFERRED. CUDA GATE IS GREEN — the earlier "dgx CUDA RED / stale golden" note is DISPROVEN. The France/Italy flip is BUILD-SENSITIVE: the PRODUCTION dgx build (FA2+Marlin+Triton+CUTLASS) resolves p0 tok5 → France 9625 and passes 16/16 (0.6B: 10 strict + 6 near-tie, max gap 0; 4B: 16/16, 11 strict + 5 near-tie, max 0.25 nats) — France anchor held, SUCCESS; only a portable-kernel-only CUDA build resolves it → Italy 15344 (matching Metal). The golden is NOT stale — it is the production build's resolution of a genuine numerical near-tie. Regressions GREEN (OPT 6/6, Qwen3-Coder 6/6; 27B/35B/DeepSeek binaries byte-identical — on dgx-CUDA the only compiled change is this one test .cpp, all Metal TUs VLLM_CPP_METAL AUTO→OFF). Goldens md5 2965ef5772b556d3f3f86fedf4221b2f UNCHANGED (the 2 Metal files are additive). | dep MODEL-FACTORY-registry (item 5); registration src/vllm/model_executor/models/qwen3_dense.cpp:145; test tests/vllm/models/test_model_registry.cpp:76 | CLAIM-MODEL-QWEN3-DENSE |
MODEL-TEXT-qwen3-moe-qwen3-moe-for-causal-lm | Qwen3MoeForCausalLM | registry.py:192; vllm/model_executor/models/qwen3_moe.py::Qwen3MoeForCausalLM | causal generation / text | W0+W1 (2026-07-21): registry TU src/vllm/model_executor/models/qwen3_moe_registry.cpp (REGISTER_VLLM_MODEL(qwen3_moe, "Qwen3MoeForCausalLM") + full-attention-only MakeQwen3MoeKVCache + ParseQwen3MoeConfig MoE-field validation + is_dense_model=false; load throws "W2", forward throws "W3") + header include/vllm/model_executor/models/qwen3_moe.h (Qwen3MoeWeights reusing Qwen3DenseAttnWeights + MoeBlockWeights; Qwen3MoeModel::Forward/ForwardDevice decls). W1 behaviour-preserving refactors: (#1) dense self-attention block + device glue EXTRACTED verbatim from qwen3.cpp to include/vllm/model_executor/models/dense_attn_block.h (namespace vllm::dense_attn; qwen3.cpp using namespace dense_attn — Qwen3-dense byte-identical); (#2) bf16 MoeBlock EXPOSED cross-TU via RunMoeBlock (include/vllm/model_executor/models/qwen3_5_moe_block.h, defined in qwen3_5.cpp outside anon-ns, MoeBlockOutput owning carrier — 35B untouched); (#3) no-shared-expert GUARD in qwen3_5.cpp MoeBlock (shared_expert_intermediate_size==0 → skip SharedExpert, nullptr to vt::MoeCombine; inert for the shared-expert-having 35B). Tests tests/vllm/models/test_model_registry.cpp (registry resolves Qwen3MoeForCausalLM + full-attn-only KV spec + is_dense_model==false; count 3→4 + arch-list msgs). W2+W3 (2026-07-21): BF16 loader src/vllm/model_executor/models/qwen3_moe_weights.cpp (LoadQwen3MoeForCausalLMWeights — merged qkv/o + per-head q/k norm via dense_weight_loaders.h; NEW bf16 per-expert loader LoadQwen3MoeBlock = router gate + 128 experts × gate/up/down transposed to Matmul-B, NO shared expert; UNTIED lm_head loaded separately, NOT aliased) + forward src/vllm/model_executor/models/qwen3_moe.cpp (Qwen3MoeModel::Forward/ForwardDevice = the dense ForwardBody with the per-layer MLP replaced by the MoE block: embed→N layers [std add+RMSNorm → reused dense AttnBlock → std add+RMSNorm → RunMoeBlock router softmax+top8+renorm + bf16 ExpertMlp + combine, no shared]→final norm→untied lm_head vt::Matmul) wired into the registry factory (W2/W3 throw stubs replaced). Tests tests/vllm/models/test_qwen3_moe_load.cpp (all 18867 tensors mapped/shaped, NO leftover, 131746 assertions) + tests/vllm/models/test_qwen3_moe_forward.cpp (CPU synthetic finite+deterministic, fusion ADOPT byte-identical, real-ckpt prefill argmax=12095 " Paris" — correct, not a near-tie). GATES: dgx CUDA -Werror 0-warn; W2 load gate + W3 doctest green; 27B 235/235 + 35B 315/315 UNCHANGED; memcheck 0 err / 0 leak. W4 correctness gate (SACRED, 2026-07-21): paged-engine greedy vs vLLM 0.25.0 (tests/vllm/models/test_qwen3coder_paged_engine.cpp + goldens tests/parity/goldens/qwen3coder_greedy/ + scripts/qwen3coder-oracle-capture.py/qwen3coder-neartie-gap.py). vLLM's own greedy is DETERMINISTIC (K=5, 0 multi-valued cells) → STRICT-where-well-posed near-tie-robust gate (dense methodology). 6/6 prompts PASS (138 assertions): STRICT token-exact 4/6 + near-tie-band 2/6, max teacher-forced gap 0.125 nats (≪ 0.5-nat bar), 0 forward-divergent; the 2 near-tie flips (prompt[3] "largest planet", prompt[4] "chemical symbol") ARE vLLM's OWN argmax on our prefix (gap 0.0) — prefill-vs-decode near-ties, not bugs. Oracle uses vLLM's TRITON MoE backend (FlashInfer-CUTLASS OOMs the 57 GiB model on 119 GiB unified). GATES: -Werror 0-warn; 27B 235/235 + 35B 315/315 UNCHANGED; memcheck 0. W5 fast BF16 grouped-MoE GEMM + SPEED (2026-07-21, LANDED / gate MISSED): NEW op vt::MoeGroupedGemmBf16 (include/vt/ops.h OpId::kMoeGroupedGemmBf16 + src/vt/ops.cpp + src/vt/cuda/cuda_matmul_nvfp4.cu MoeGroupedGemmBf16Naive/MoeGroupedGemmBf16Wmma/LaunchGroupedBf16) — the dtype-native analog of MoeGroupedGemmNvfp4, STRUCTURALLY REUSING its expert counting-sort + ragged per-BM-tile scheduling (MoeHist/MoeOffsets/MoeScatter/MoeTileMap + EnsureMoeScratch) with the fp4 decode replaced by a direct bf16 [K,N] read; 3 regimes (naive P<32, BM=16 decode tile mirroring vLLM fused_moe BLOCK_SIZE_M=16, BM=64 prefill tile). MoeBlockBf16Cuda (qwen3_5.cpp) replaces the per-expert host-gather loop with ~3 on-device grouped launches, DEFAULT ON (VT_MOE_BF16_FAST=0 rolls back) per [[parity-enablers-ship-as-defaults]]; MoeBf16FastLayoutOk GUARDS it to the nk==false [K,N] layout so the nk==true 35B-MTP producer falls through to the reference loop. HOST-MIRROR RELEASE via the existing OwnedTensor::ReleaseHost()/platforms::ShouldReleaseHostWeights mechanism (experts = ~57 GiB = ~94% of the checkpoint; host+device were both resident in ONE 119 GiB unified pool). NEW test tests/vt/test_ops_moe_grouped_bf16.cpp (all 3 launch regimes + both out dtypes + identity/row_map) 4/4. CORRECTNESS RE-VERIFIED AND STRICTER: the grouped GEMM's f32 accumulation order re-resolves bf16 near-ties, so goldens were re-measured by the ratified procedure — test_qwen3coder_paged_engine 6/6, STRICT token-exact 4/6 → 5/6, max teacher-forced gap 0.125 → 0.0000 nats, 0 forward-divergent (a grouped GEMM lands closer to vLLM's own Triton grouped fused_moe than a per-expert loop). SAME-BINARY A/B: median TTFT 3092 → 904 ms (3.4×), TPOT 64.4 → 40.4 ms (1.59×). BINDING GRID vs GRAPHED vLLM 0.25.0 (triton MoE, vllm serve+bench serve, 1024/128, c1/c2/c4/c8): median TTFT 0.41/0.15/0.18/0.24×, median TPOT 0.80/1.02/0.75/0.55×, output tput 0.65/0.75/0.63/0.49× — every-axis parity MISSED (only c2 TPOT). nsys attribution: prefill MoeGroupedGemmBf16Wmma = 56.0% of GPU time (≈855 ms of our 904 ms TTFT; ~4.3 TFLOP/s ≈ 1.7% of GB10 bf16 peak vs vLLM's Triton ~10.0 TFLOP/s) because the tile is untuned (BK=32, no cp.async double-buffer, BN=64); decode MoeGroupedGemmBf16Naive = 21.3% at 151 GB/s (55% of peak) for gate/up, block-starved at N=768 (24 blocks). Named remaining levers: w13 gate+up fusion, deterministic split-K, bf16 decode CUDA graph (fp4-gated today), prefill tile/pipeline rework. GATES: -Werror 0-warn; 6/6; 27B 235/235 + 35B 315/315 UNCHANGED; memcheck 0 errors on the MoE GEMM + engine path (test_llm_engine 5/5, test_runner 14/14). W6 MoE-GEMM tile/pipeline rework + deterministic split-K (2026-07-21): NEW MoeGroupedGemmBf16WmmaPipe (src/vt/cuda/cuda_matmul_nvfp4.cu) replaces the W5 tile's UNCOALESCED [K,N] weight stage (consecutive lanes n_cols elements apart) with a k-major [BK][BN] shared tile read as wmma::matrix_b ROW_MAJOR so the global read runs along contiguous N, plus 16-byte vectorized stages, a 3-deep cp.async multi-stage pipeline, BN 64→128 and +8-half padded shared rows — tile/pipeline shape ported from vLLM fused_moe.py:294 fused_moe_kernel + :1238 get_default_config (bf16 branch block_n/num_stages/num_warps), pipeline structure from CUTLASS mma_multistage.h; shape-guarded (Bf16PipeShapeOk, 8-bf16 row pitches) with the W5 tile retained for ragged pitches; VT_MOE_BF16_PIPE=0 rolls back. NEW MoeGroupedGemmBf16NaiveSplitK + MoeGroupedGemmBf16SplitKReduce + EnsureMoePartials (graph-safe f32 partials, same retire-don't-free contract as EnsureMoeScratch) give the small-P decode GEMM DETERMINISTIC split-K — mirrors vLLM's SPLIT_K (fused_moe.py:338) but with fixed ascending-split partial reduction, explicitly NOT atomicAdd (which would break greedy reproducibility); VT_MOE_SPLIT_K=0 rolls back. KERNEL A/B (same binary, E=128/K=2048/top-8): prefill T=1024 N=768 4.50 → 12.15 TFLOP/s (2.70x), N=2048 4.49 → 12.48 TFLOP/s (2.78x), decode-WMMA P=64 2.84x, split-K P=8 2.63x — vs vLLM's Triton fused_moe ~10.0 TFLOP/s, so the MoE GEMM now runs ~1.2x vLLM's rate and the W5 headline kernel deficit is CLOSED (19.9 TFLOP/s in situ on the c2 2048-token prefill step). CORRECTNESS UNCHANGED — the pipelined tile preserves the per-output K-reduction ORDER exactly, so it is BIT-IDENTICAL to W5: 6/6, STRICT 5/6, max gap 0.0000 nats, 0 forward-divergent, ZERO token movement, goldens md5-verified unchanged. test_ops_moe_grouped_bf16 extended 4 → 7 cases / 19 assertions (aligned-pitch pipelined prefill + decode + down-projection shapes with ragged K/N tails exercising the cp.async zfill, plus a run-to-run BIT-REPRODUCIBILITY assertion on the split-K reduction); a genuine pipeline bug (missing __pipeline_commit() on tail iterations → __pipeline_wait_prior under-count → last k-tiles consumed before their copies landed) was caught by the new decode case and fixed. DENOMINATOR CORRECTION: W5's c2/c4/c8 vLLM numbers were taken from ONE shared vllm serve replaying seed-0 prompts (vLLM RandomDataset builds (offset+index+arange) % vocab_size, datasets/datasets.py:557-566), giving 43.6-63.5% prefix-cache hits and an INFLATED denominator (vLLM's c2 TTFT came out faster than its c1); W6 re-measures with a FRESH server per concurrency, 0.0% hit rate in all four runs (c1 was always clean: 324.04 → 321.88 ms). BINDING GRID vs GRAPHED vLLM 0.25.0 (clean denominator, 1024/128, c1/c2/c4/c8, >1 = we win): median TTFT 1.03/0.82/1.05/1.19x, median TPOT 0.88/1.01/1.22/1.05x, median ITL 0.88/1.03/1.22/1.09x, output tput 0.77/0.90/1.09/0.99x — 12 of 16 cells at/above vLLM; c4 passes EVERY axis; c8 passes 3 of 4. nsys-attributed residual: (a) c1 decode = the MISSING bf16 DECODE CUDA GRAPH, not kernel speed — summed GPU kernel time ~31 ms vs 36.22 ms TPOT ⇒ ~86% GPU-busy, ~5 ms/step host tax ≈ the entire 4.5 ms c1 deficit, while the decode kernels run at 190-211 GB/s = 70-77% of GB10's ~273 GB/s; (b) c2 TTFT is NO LONGER the MoE GEMM (56.0% → 16.6% of GPU time at c1; 374 ms of a 533.6 ms TTFT at 19.9 TFLOP/s) but the NON-MoE prefill glue. GATES: -Werror 0-warn clean full rebuild; 6/6 + 7/7 + NVFP4 9/9 + forward 3/3 on the FINAL binary; 27B 235/235 + 35B 315/315 UNCHANGED; memcheck 0 memory errors W7 bf16 DECODE CUDA GRAPH (2026-07-21) — W6 lever (i): NEW Qwen3MoeDecodeGraph (include/vllm/model_executor/models/qwen3_moe.h + src/vllm/model_executor/models/qwen3_moe.cpp), dispatched from qwen3_moe_registry.cpp::ForwardQwen3MoeForCausalLM on pure_decode && is_cuda(). Third sibling of the SAME in-tree driver design as Qwen3_5DecodeGraph (35B, qwen3_5.cpp:5902) and Qwen3_5DenseDecodeGraph (27B, :6104) — same cold->warm->capture->replay state machine, per-padded-size SizeSlot (own fixed-address host inputs + persistent embed/logits buffers + instantiated graph), cols_changed invalidation, and shared DecodeGraphSizes/PadToCaptureSize. Ported from vllm/v1/worker/gpu_model_runner.py::GPUModelRunner (warm-up then capture, per-step graph dispatch) + vllm/compilation/cuda_graph.py::CUDAGraphWrapper.__call__ (pad-to-nearest-captured-size) @ e24d1b24. TWO differences, both because Coder is pure full attention: attention-only padded-input builder BuildPaddedDecodeAttn (GDN-free analog of qwen3_5.cpp:5843, so EVERY captured size is usable — no GDN state-slot gate), and no defensive copy of the persistent hidden (the Coder layer loop only READS it). ForwardBody split into EmbedInto (OUTSIDE the graph — vt::Embedding allocates a device flag and syncs the stream, cuda_ops.cu:525,535) + the capturable ForwardLayers; refactor verified behavior-preserving (VT_QWEN3MOE_CUDAGRAPH=0 -> 6/6). ONE REAL BUG FOUND+FIXED — a latent graph-safety defect in shared glue: BuildStepInputs (dense_attn_block.h) uploaded the RoPE identity row-index from a stack-local vector; under capture that becomes a memcpy node baking a host source address that is dead stack on return, so every replay re-read freed memory -> wrong RoPE -> a WRONG token (prompt[0] tok=2, 3555 vs 576). Fixed by serving it from a process-persistent per-T table (contents byte-identical; storage created once per T, never resized/moved) — the host-side analog of the EnsureMoeScratch/EnsureMoePartials retire-don't-free contract. Latent, not a regression: the 35B/27B use BuildStepDevInputs and Qwen3-dense had no graph, so Coder is the first caller to capture this glue. MEASURED (nsys --cuda-graph-trace=node, c1): GPU-busy ~86% -> ~92%, host tax ~5 -> ~2.7 ms/step, median TPOT 36.22 -> 34.13 ms; 60 cudaGraphLaunch (one per decode step) vs 3,569 total cudaLaunchKernel in the window (was ~1.4k/step eager). CORRECTNESS: ZERO TOKEN MOVEMENT (as a numerics-neutral graph must be) — 6/6 (138 assertions), STRICT 5/6, max gap 0.0000 nats, goldens md5-verified unchanged. BINDING GRID vs GRAPHED vLLM 0.25.0, FRESH server per concurrency, 0.0% prefix-cache hit VERIFIED in all four serve logs, 2 reps ours with the cold leg discarded (>1 = we win): median TTFT 1.02/0.91/1.05/1.17x, median TPOT 0.93/1.12/1.27/1.08x, median ITL 0.93/1.10/1.28/1.14x, output tput 0.84/0.96/1.11/1.03x — 11/16 at/above vLLM; c4 AND c8 now pass EVERY axis (c8 output tput crossed 0.99 -> 1.03x). Ours vs our own W6: TPOT 36.22->34.13 / 44.12->39.68 / 60.74->57.97 / 81.41->78.22 ms, output tput 22.62->24.60 / 37.24->39.85 / 55.03->56.34 / 86.02->90.10 tok/s, TTFT unchanged within noise (a decode graph cannot move TTFT). Denominator reproducibility: 3 of 4 vLLM cells within 0.5% of W6 (c1 TPOT IDENTICAL at 31.75) — the exception is c2 TTFT, 435.5 -> 485.7 ms (+11.5%), so that cell is a band (0.82-0.91x), not a point. RESIDUAL, attributed: all 5 misses are at c1/c2. c1 = the remaining ~2.7 ms/step HOST tax — vLLM's entire 31.75 ms decode step costs about what our KERNELS ALONE cost (~31.5 ms), so c1 needs no kernel work. REFUTED hypothesis: it is NOT CUDA-API overhead — cuda_api_sum shows cudaStreamSynchronize median 4.5 us and the 19,184 cudaMalloc calls are one-time ResidentWeight uploads, with cudaGraphLaunch at 438 us/step; so <1 ms of the 2.7 ms is CUDA API, and the balance is engine-side per-step host bookkeeping needing a CPU-side profile. c2 TTFT/tput are untouched by a decode lever and remain the non-MoE prefill glue. GATES: -Werror 0-warn clean full rebuild; 6/6 graph-ON + 6/6 graph-OFF + 7/7 + NVFP4 9/9 + forward 3/3 + Qwen3-dense green; 27B 235/235 + 35B 315/315 UNCHANGED; memcheck 0 errors on the runner decode path, the engine path, AND the full Coder decode-graph gate | sweep-qwen3-coder-30b spike | PARTIAL (W0-W7 landed; correctness DONE 6/6 zero-movement; SPEED 11/16 grid cells at/above graphed vLLM — c4 AND c8 now pass EVERY axis — with the 5 residual cells all at c1/c2 and attributed to the remaining ~2.7 ms/step engine-side HOST bookkeeping (c1, ~92% GPU-busy) and the non-MoE prefill glue (c2)) | breadth-sweep Tier-1; dep ENG-RUNNER-MODELSHAPE (composes zero runner change) + Qwen3-dense attn + 35B MoeBlock; registration src/vllm/model_executor/models/qwen3_moe_registry.cpp:181; test tests/vllm/models/test_model_registry.cpp:77 | CLAIM-MODEL-QWEN3-CODER |
MODEL-TEXT-sarvam-sarvam-mo-efor-causal-lm | SarvamMoEForCausalLM | registry.py:194; vllm/model_executor/models/sarvam.py::SarvamMoEForCausalLM | causal generation / text | model loader/forward; FusedMoE/grouped GEMM; MLA/latent KV | ☐ required | INVENTORIED | none | unassigned |
MODEL-TEXT-sarvam-sarvam-mlafor-causal-lm | SarvamMLAForCausalLM | registry.py:195; vllm/model_executor/models/sarvam.py::SarvamMLAForCausalLM | causal generation / text | model loader/forward; FusedMoE/grouped GEMM; MLA/latent KV | ☐ required | INVENTORIED | none | unassigned |
MODEL-TEXT-seed-oss-seed-oss-for-causal-lm | SeedOssForCausalLM | registry.py:196; vllm/model_executor/models/seed_oss.py::SeedOssForCausalLM | causal generation / text | model loader/forward; paged attention/KV; sliding-window attention | ☐ required | INVENTORIED | none | unassigned |
MODEL-TEXT-step1-step1-for-causal-lm | Step1ForCausalLM | registry.py:197; vllm/model_executor/models/step1.py::Step1ForCausalLM | causal generation / text | model loader/forward; paged attention/KV | ☐ required | INVENTORIED | none | unassigned |
MODEL-TEXT-step3-text-step3-text-for-causal-lm | Step3TextForCausalLM | registry.py:198; vllm/model_executor/models/step3_text.py::Step3TextForCausalLM | causal generation / text | model loader/forward; paged attention/KV; FusedMoE/grouped GEMM | ☐ required | INVENTORIED | none | unassigned |
MODEL-TEXT-step3p5-step3p5-for-causal-lm | Step3p5ForCausalLM | registry.py:199; vllm/model_executor/models/step3p5.py::Step3p5ForCausalLM | causal generation / text | model loader/forward; paged attention/KV; FusedMoE/grouped GEMM; sliding-window attention | ☐ required | INVENTORIED | none | unassigned |
MODEL-TEXT-stablelm-stablelm-for-causal-lm | StableLMEpochForCausalLM, StableLmForCausalLM | registry.py:200-201; vllm/model_executor/models/stablelm.py::StablelmForCausalLM | causal generation / text | impl include/vllm/model_executor/models/stablelm.h + src/vllm/model_executor/models/stablelm{,_weights,_registry}.cpp; tokenizer canon src/vllm/tokenizer/tokenizer.cpp; test tests/parity/test_stablelm_paged_engine.cpp (SACRED 16/16, 92 assertions) | recent-dense-batch | PARTIAL | none; registration src/vllm/model_executor/models/stablelm_registry.cpp:144; test tests/vllm/models/test_model_registry.cpp:122 | CLAIM-SWEEP-RECENT-DENSE |
MODEL-TEXT-solar-solar-for-causal-lm | SolarForCausalLM | registry.py:202; vllm/model_executor/models/solar.py::SolarForCausalLM | causal generation / text | model loader/forward; paged attention/KV | ☐ required | INVENTORIED | none | unassigned |
MODEL-TEXT-telechat2-tele-chat2-for-causal-lm | TeleChatForCausalLM, TeleChat2ForCausalLM | registry.py:203-204; vllm/model_executor/models/telechat2.py::TeleChat2ForCausalLM | causal generation / text | model loader/forward | ☐ required | INVENTORIED | none | unassigned |
MODEL-TEXT-teleflm-tele-flmfor-causal-lm | TeleFLMForCausalLM | registry.py:206; vllm/model_executor/models/teleflm.py::TeleFLMForCausalLM | causal generation / text | model loader/forward | ☐ required | INVENTORIED | none | unassigned |
MODEL-TEXT-zamba2-zamba2-for-causal-lm | Zamba2ForCausalLM | registry.py:207; vllm/model_executor/models/zamba2.py::Zamba2ForCausalLM | causal generation / text | model loader/forward; paged attention/KV; Mamba/SSM state | ☐ required | INVENTORIED | none | unassigned |
MODEL-TEXT-qwen3-5-qwen3-5-for-causal-lm | Qwen3_5ForCausalLM | BEYOND-PIN — NOT in 555967922 (the text-only Qwen3.5 arms landed upstream AFTER our parity pin, in PR vllm#50210 at ad5d29db7, which registers both against the SAME qwen3_5 module: upstream registry.py:202-203 @ ad5d29db7). Deliberately NOT written as a pinned module/class target — the pin-derived static invariants below describe 555967922, and a post-pin arm must not silently inflate them, the same convention MuseGlimmerForConditionalGeneration and KimiK3ForConditionalGeneration follow. Upstream class: qwen3_5.py:439-440 @ ad5d29db7 (class Qwen3_5ForCausalLM(Qwen3_5ForCausalLMBase): pass — the shared base UNCHANGED) | causal generation / text | model loader/forward; Mamba/SSM state; GDN/linear-attention state; paged attention | ✅ Qwen3.5/3.8 text-only arms | PARTIAL | LANDED 2026-08-12 (MODEL-QWEN38-TEXT-ONLY, issue #490): the arch loads; NO token gate exists and one is OWED. Additive registration against the EXISTING dense factory — no forward, no KV-cache spec, no loader fork: src/vllm/model_executor/models/qwen3_5_dense.cpp:256 over the text-only _ModelInfo src/vllm/model_executor/models/qwen3_5_common.h:42 (hybrid YES, multimodal NO, mirroring Qwen3_5ForCausalLMBase's IsHybrid-without-SupportsMultiModal). The backbone weight namespace is resolved ONCE per checkpoint — include/vllm/model_executor/models/qwen3_5_weights.h:529, src/vllm/model_executor/models/qwen3_5_weights.cpp:597 — and threaded through the dense loader at src/vllm/model_executor/models/qwen3_5_dense_weights.cpp:676, mirroring upstream's single WeightsMapper(orig_to_new_prefix={"model.language_model.": "model."}) (qwen3_5.py:296-300 @ ad5d29db7) rather than a per-lookup fallback; a MIXED index is REFUSED. Tests tests/vllm/models/test_qwen3_8_text_only.cpp:586 (dispatch), :633 (published flat 3.8 config), :718 (namespace resolution + mixed refusal), :769 (the dense loader reads byte-identical weights through either namespace), :1008 (the VL seam DEFAULT driven with the prefix argument omitted). Registry contract re-pinned at tests/vllm/models/test_model_registry.cpp:150 (40 archs). RUN GATE OWED: no text-only Qwen3.5 checkpoint fits GB10 (2.4T bf16 ≈ 4.8 TB, FP8 ≈ 2.4 TB), so there is NO token-exact oracle run and NO speed number; the row may not advance on dispatch and mapping tests alone. The DENSE gate closes when a fitting Qwen3_5ForCausalLM checkpoint appears — this loader routes BF16/FP8/NVFP4 per projection by tensor presence, so a flat bf16 checkpoint is expected to load. MTP and GGUF arms for 3.8 are NOT implemented and are owed; the sibling MoE row additionally owes the bf16/stacked expert arm (matrix row below, porting-inventory §9 deviation 17(e)). | unassigned |
MODEL-TEXT-qwen3-5-qwen3-5-moe-for-causal-lm | Qwen3_5MoeForCausalLM | BEYOND-PIN — NOT in 555967922 (the text-only Qwen3.5 arms landed upstream AFTER our parity pin, in PR vllm#50210 at ad5d29db7, which registers both against the SAME qwen3_5 module: upstream registry.py:202-203 @ ad5d29db7). Deliberately NOT written as a pinned module/class target — the pin-derived static invariants below describe 555967922, and a post-pin arm must not silently inflate them, the same convention MuseGlimmerForConditionalGeneration and KimiK3ForConditionalGeneration follow. Upstream class: qwen3_5.py:439-440 @ ad5d29db7 (Qwen3_5MoeForCausalLM(Qwen3_5ForCausalLMBase, QwenNextMixtureOfExperts) = that same base plus set_moe_parameters(), qwen3_5.py:443-449 @ ad5d29db7) | causal generation / text | model loader/forward; FusedMoE/grouped GEMM; Mamba/SSM state; GDN/linear-attention state; paged attention | ✅ Qwen3.5/3.8 text-only arms | PARTIAL | LANDED 2026-08-12 (MODEL-QWEN38-TEXT-ONLY, issue #490): the arch loads; NO token gate exists and one is OWED. Additive registration against the EXISTING MoE factory: src/vllm/model_executor/models/qwen3_5_moe.cpp:231 over src/vllm/model_executor/models/qwen3_5_common.h:42. Motivating checkpoint Qwen/Qwen3.8-2.4T-A95B declares Qwen3_5MoeForCausalLM / qwen3_5_moe_text and is the token-exact Qwen3.6-35B-A3B GDN-hybrid MoE backbone at larger scale — hidden 2048→8192, layers 40→92, experts 256→512, top-k 8→10 — every knob config-driven, and the published safetensors names are IDENTICAL modulo the backbone prefix (3D-stacked mlp.experts.gate_up_proj, mlp.shared_expert_gate.weight, top-level lm_head). One namespace decision per checkpoint threaded through the MoE loader AND its deferred per-layer expert closure at src/vllm/model_executor/models/qwen3_5_weights.cpp. Tests tests/vllm/models/test_qwen3_8_text_only.cpp:586 (dispatch), :633 (published flat config), :718 (namespace resolution + mixed refusal), :836 (byte-identical MoE loads on BOTH expert-residency paths), :922 (the unimplemented-layout REFUSAL below), :1008 (the VL seam DEFAULT driven with the prefix argument omitted); 27B/35B/Coder inertness by construction (the VL prefix stays the seam default, include/vllm/model_executor/models/qwen3_5_weights.h) with parity goldens md5-unchanged. CORRECTED 2026-08-12 (review FAIL on records honesty): REGISTRATION IS NOT LOADABILITY, and the debt was recorded INVERTED. The earlier text said the quantized arm was owed; it is the ONLY one implemented. LoadQwen3_5Moe reads exclusively per-expert NVFP4 (LoadMoeExpertsInto -> LoadNvfp4Raw: U8 weight + F8_E4M3 .weight_scale + .weight_scale_2), with no stacked and no bf16 branch. Read live 2026-08-12, Qwen/Qwen3.8-2.4T-A95B publishes 93x mlp.experts.gate_up_proj + 93x .down_proj (3-D STACKED) and ZERO weight_scale/input_scale; Qwen/Qwen3.6-35B-A3B is the same under the VL prefix; our gated 35B row reads the requantized nvidia/Qwen3.6-35B-A3B-NVFP4. At that date the bf16/stacked MoE routed-expert arm was NOT implemented, and such a checkpoint was REFUSED by a message naming it (CheckMoeExpertLayoutSupported) instead of dying on expected U8 for lm_head.weight. SUPERSEDED 2026-08-15 — THE LOADER HALF IS CLOSED AND ORACLE-GATED; ONLY THE HARDWARE HALF REMAINS. #740 (spec) landed the 3-D stacked bf16 routed-expert reader and #864 (spec) the bf16 GDN tower, attention tower, shared expert and lm_head, each selected by tensor presence; both published indices now satisfy the load plan completely, so a published bf16 Qwen MoE repo no longer refuses. The binding token-exact greedy gate RAN on the GB10 against the pinned oracle (0.23.1rc1.dev1511+g555967922, flashinfer 0.6.15.post1) using Qwen/Qwen3.6-35B-A3B bf16 @995ad96eacd98c81ed38be0c5b274b04031597b0 (26 shards, 71,903,645,408 bytes, sha256 recomputed), greedy 7 prompts x 3 repeats x 16 tokens, oracle deterministic 7/7 across repeats: 6/7 prompts STRICT 16/16. The seventh diverges once on a bit-identical logprob (-0.8293954133987427 both sides, top2_gap_mnats = 0.0, our token at oracle rank 2) because torch.argmax takes the lowest maximal index and our on-device argmax the higher — an exact tie, PASS under the ratified near-tie doctrine, filed as #910. Only the FIRST divergence per prompt is validly adjudicable, so the raw 108/112 position count is NOT a quality score and is not recorded as one. SACRED inertness 3 of 3 against the shared loader, GOLDENS_BYTE_IDENTICAL=1 on each: 27B 235/235 @890bdef7a42feba6d83b6e17a03315c694112f2a, 35B 315/315, Coder 138/138 @b2cff646eb4bb1d68355c01b18ae02e7cf42d120, 688 assertions. RUN GATE FOR THIS CLASS STILL OWED, on size alone: that gate ran through Qwen3_5MoeForConditionalGeneration, not through Qwen3_5MoeForCausalLM, whose only published checkpoint (2.4T bf16 ≈ 4.8 TB, FP8 ≈ 2.4 TB vs 128 GB unified) cannot be executed here, so NO token and NO speed number exist for this class; its load plan resolves against the published index, which is name/shape/dtype resolution and not a token. Also owed: MTP and GGUF arms for 3.8. See porting-inventory §9 deviation 17(e). | unassigned |
MODEL-EMBED - Embedding
| ID | Item / architecture aliases | Upstream | Task / modality | Dependencies | Spike/spec | State | Our code + tests/evidence | Owner |
|---|---|---|---|---|---|---|---|---|
MODEL-EMBED-bert-bert-embedding-model | BertModel | registry.py:212; vllm/model_executor/models/bert.py::BertEmbeddingModel | embedding / text | encoder attention; pooler | ☐ required | INVENTORIED | none | unassigned |
MODEL-EMBED-bert-bert-splade-sparse-embedding-model | BertSpladeSparseEmbeddingModel | registry.py:213; vllm/model_executor/models/bert.py::BertSpladeSparseEmbeddingModel | embedding / text | encoder attention; pooler | ☐ required | INVENTORIED | none | unassigned |
MODEL-EMBED-roberta-bge-m3-embedding-model | BgeM3EmbeddingModel | registry.py:214; vllm/model_executor/models/roberta.py::BgeM3EmbeddingModel | embedding / text | encoder attention; pooler | ☐ required | INVENTORIED | none | unassigned |
MODEL-EMBED-nemotron-nas-deci-lmfor-causal-lm | DeciLMForCausalLM | registry.py:215; vllm/model_executor/models/nemotron_nas.py::DeciLMForCausalLM | embedding / text | encoder attention; pooler | ☐ required | INVENTORIED | none | unassigned |
MODEL-EMBED-gemma2-gemma2-for-causal-lm | Gemma2Model | registry.py:216; vllm/model_executor/models/gemma2.py::Gemma2ForCausalLM | embedding / text | encoder attention; pooler; sliding-window attention | ☐ required | INVENTORIED | none | unassigned |
MODEL-EMBED-gemma3-gemma3-model | Gemma3TextModel | registry.py:217; vllm/model_executor/models/gemma3.py::Gemma3Model | embedding / text | encoder attention; pooler; sliding-window attention | ☐ required | INVENTORIED | none | unassigned |
MODEL-EMBED-glm-glm-for-causal-lm | GlmForCausalLM | registry.py:218; vllm/model_executor/models/glm.py::GlmForCausalLM | embedding / text | encoder attention; pooler | ☐ required | INVENTORIED | none | unassigned |
MODEL-EMBED-gritlm-grit-lm | GritLM | registry.py:219; vllm/model_executor/models/gritlm.py::GritLM | embedding / text | encoder attention; pooler; sliding-window attention | ☐ required | INVENTORIED | none | unassigned |
MODEL-EMBED-bert-with-rope-snowflake-gte-new-model | GteModel | registry.py:220; vllm/model_executor/models/bert_with_rope.py::SnowflakeGteNewModel | embedding / text | encoder attention; pooler; FusedMoE/grouped GEMM | ☐ required | INVENTORIED | none | unassigned |
MODEL-EMBED-bert-with-rope-gte-new-model | GteNewModel | registry.py:221; vllm/model_executor/models/bert_with_rope.py::GteNewModel | embedding / text | encoder attention; pooler; FusedMoE/grouped GEMM | ☐ required | INVENTORIED | none | unassigned |
MODEL-EMBED-jina-jina-embeddings-v5-model | JinaEmbeddingsV5Model | registry.py:222; vllm/model_executor/models/jina.py::JinaEmbeddingsV5Model | embedding / text | encoder attention; pooler | ☐ required | INVENTORIED | none | unassigned |
MODEL-EMBED-llama-llama-bidirectional-model | LlamaBidirectionalModel | registry.py:223; vllm/model_executor/models/llama.py::LlamaBidirectionalModel | embedding / text | encoder attention; pooler; sliding-window attention | ☐ required | INVENTORIED | none | unassigned |
MODEL-EMBED-llama-llama-for-causal-lm | LlamaModel, CwmForCausalLM, InternLM3ForCausalLM, IQuestCoderForCausalLM, LlamaForCausalLM, LLaMAForCausalLM, TeleChat3ForCausalLM, MistralModel | registry.py:224-231; vllm/model_executor/models/llama.py::LlamaForCausalLM | embedding / text | encoder attention; pooler; sliding-window attention | embeddings-one-surface | PARTIAL | ARCH-ONE-SURFACE ROW 6 LANDED in PR #137 (2026-08-08): the LlamaModel membership is REGISTERED + LIVE (as_embedding_model mirror, adapters.py:230 — is_pooling_model=true, bare-prefix loader, pooling forward Qwen3DenseModel::ForwardHidden, engine-step pool_tokens, vllm_embed ABI v15 + /v1/embeddings); registration src/vllm/model_executor/models/llama_embedding_registry.cpp:132; fold gate tests/vllm/models/test_llama_embedding_fold.cpp:206 4/4-231 on the committed synthetic fixture. RESIDUALS: the other 7 memberships (incl. MistralModel) unregistered; REAL-checkpoint (e5-mistral class) + LLM(task="embed") oracle cosine gate not run (synthetic-fixture arm only; no cosine-vs-oracle number fabricated) | CLAIM-EMBEDDINGS-ONE-SURFACE |
MODEL-EMBED-modernbert-modern-bert-model | ModernBertModel | registry.py:232; vllm/model_executor/models/modernbert.py::ModernBertModel | embedding / text | encoder attention; pooler; sliding-window attention | ☐ required | INVENTORIED | none | unassigned |
MODEL-EMBED-bert-with-rope-nomic-bert-model | NomicBertModel | registry.py:233; vllm/model_executor/models/bert_with_rope.py::NomicBertModel | embedding / text | encoder attention; pooler; FusedMoE/grouped GEMM | ☐ required | INVENTORIED | none | unassigned |
MODEL-EMBED-phi3-phi3-for-causal-lm | Phi3ForCausalLM | registry.py:234; vllm/model_executor/models/phi3.py::Phi3ForCausalLM | embedding / text | encoder attention; pooler | ☐ required | INVENTORIED | none | unassigned |
MODEL-EMBED-qwen2-qwen2-for-causal-lm | Qwen2Model, Qwen2ForCausalLM | registry.py:235-236; vllm/model_executor/models/qwen2.py::Qwen2ForCausalLM | embedding / text | encoder attention; pooler | ☐ required | INVENTORIED | none | unassigned |
MODEL-EMBED-roberta-roberta-embedding-model | RobertaForMaskedLM, RobertaModel, XLMRobertaModel | registry.py:237-238,245; vllm/model_executor/models/roberta.py::RobertaEmbeddingModel | embedding / text | encoder attention; pooler | ☐ required | INVENTORIED | none | unassigned |
MODEL-EMBED-telechat2-tele-chat2-for-causal-lm | TeleChatForCausalLM, TeleChat2ForCausalLM | registry.py:239-240; vllm/model_executor/models/telechat2.py::TeleChat2ForCausalLM | embedding / text | encoder attention; pooler | ☐ required | INVENTORIED | none | unassigned |
MODEL-EMBED-voyage-voyage-qwen3-bidirectional-embed-model | VoyageQwen3BidirectionalEmbedModel | registry.py:241-244; vllm/model_executor/models/voyage.py::VoyageQwen3BidirectionalEmbedModel | embedding / text | encoder attention; pooler | ☐ required | INVENTORIED | none | unassigned |
MODEL-EMBED-clip-clipembedding-model | CLIPModel | registry.py:247; vllm/model_executor/models/clip.py::CLIPEmbeddingModel | embedding / image+text | encoder attention; pooler | ☐ required | INVENTORIED | none | unassigned |
MODEL-EMBED-colpali-col-pali-model | ColPaliForRetrieval | registry.py:248; vllm/model_executor/models/colpali.py::ColPaliModel | embedding / image+text | encoder attention; pooler | ☐ required | INVENTORIED | none | unassigned |
MODEL-EMBED-nemotron-vl-llama-nemotron-vlfor-embedding | LlamaNemotronVLModel | registry.py:249; vllm/model_executor/models/nemotron_vl.py::LlamaNemotronVLForEmbedding | embedding / image+text | encoder attention; pooler | ☐ required | INVENTORIED | none | unassigned |
MODEL-EMBED-llava-next-llava-next-for-conditional-generation | LlavaNextForConditionalGeneration | registry.py:250-253; vllm/model_executor/models/llava_next.py::LlavaNextForConditionalGeneration | embedding / image+text | encoder attention; pooler | ☐ required | INVENTORIED | none | unassigned |
MODEL-EMBED-phi3v-phi3-vfor-causal-lm | Phi3VForCausalLM | registry.py:254; vllm/model_executor/models/phi3v.py::Phi3VForCausalLM | embedding / image+text | encoder attention; pooler | ☐ required | INVENTORIED | none | unassigned |
MODEL-EMBED-qwen2-vl-qwen2-vlfor-conditional-generation | Qwen2VLForConditionalGeneration | registry.py:255; vllm/model_executor/models/qwen2_vl.py::Qwen2VLForConditionalGeneration | embedding / image+text | encoder attention; pooler | ☐ required | INVENTORIED | none | unassigned |
MODEL-EMBED-siglip-siglip-embedding-model | SiglipModel | registry.py:256; vllm/model_executor/models/siglip.py::SiglipEmbeddingModel | embedding / image+text | encoder attention; pooler | ☐ required | INVENTORIED | none | unassigned |
MODEL-EMBED-terratorch-terratorch | PrithviGeoSpatialMAE, Terratorch | registry.py:260-261; vllm/model_executor/models/terratorch.py::Terratorch | embedding / image+text | encoder attention; pooler | ☐ required | INVENTORIED | none | unassigned |
MODEL-LATE - Late interaction
| ID | Item / architecture aliases | Upstream | Task / modality | Dependencies | Spike/spec | State | Our code + tests/evidence | Owner |
|---|---|---|---|---|---|---|---|---|
MODEL-LATE-colbert-col-bertmodel | HF_ColBERT | registry.py:266; vllm/model_executor/models/colbert.py::ColBERTModel | late interaction / text | token pooler; projection/ranking; Mamba/SSM state | ☐ required | INVENTORIED | none | unassigned |
MODEL-LATE-colbert-col-bertmodern-bert-model | ColBERTModernBertModel | registry.py:267; vllm/model_executor/models/colbert.py::ColBERTModernBertModel | late interaction / text | token pooler; projection/ranking; Mamba/SSM state | ☐ required | INVENTORIED | none | unassigned |
MODEL-LATE-colbert-col-bertjina-roberta-model | ColBERTJinaRobertaModel | registry.py:268; vllm/model_executor/models/colbert.py::ColBERTJinaRobertaModel | late interaction / text | token pooler; projection/ranking; Mamba/SSM state | ☐ required | INVENTORIED | none | unassigned |
MODEL-LATE-colbert-col-bertlfm2-model | ColBERTLfm2Model | registry.py:269; vllm/model_executor/models/colbert.py::ColBERTLfm2Model | late interaction / text | token pooler; projection/ranking; Mamba/SSM state | ☐ required | INVENTORIED | none | unassigned |
MODEL-LATE-jina-jina-for-ranking | JinaForRanking | registry.py:270; vllm/model_executor/models/jina.py::JinaForRanking | late interaction / text | token pooler; projection/ranking | ☐ required | INVENTORIED | none | unassigned |
MODEL-LATE-colmodernvbert-col-modern-vbert-for-retrieval | ColModernVBertForRetrieval | registry.py:272; vllm/model_executor/models/colmodernvbert.py::ColModernVBertForRetrieval | late interaction / image+text | token pooler; projection/ranking | ☐ required | INVENTORIED | none | unassigned |
MODEL-LATE-colpali-col-pali-model | ColPaliForRetrieval | registry.py:273; vllm/model_executor/models/colpali.py::ColPaliModel | late interaction / image+text | token pooler; projection/ranking | ☐ required | INVENTORIED | none | unassigned |
MODEL-LATE-colqwen3-col-qwen3-model | ColQwen3, OpsColQwen3Model, Qwen3VLNemotronEmbedModel | registry.py:274-275,277; vllm/model_executor/models/colqwen3.py::ColQwen3Model | late interaction / image+text | token pooler; projection/ranking | ☐ required | INVENTORIED | none | unassigned |
MODEL-LATE-colqwen3-5-col-qwen3-5-model | ColQwen3_5 | registry.py:276; vllm/model_executor/models/colqwen3_5.py::ColQwen3_5Model | late interaction / image+text | token pooler; projection/ranking | ☐ required | INVENTORIED | none | unassigned |
MODEL-REWARD - Reward
| ID | Item / architecture aliases | Upstream | Task / modality | Dependencies | Spike/spec | State | Our code + tests/evidence | Owner |
|---|---|---|---|---|---|---|---|---|
MODEL-REWARD-internlm2-intern-lm2-for-reward-model | InternLM2ForRewardModel | registry.py:281; vllm/model_executor/models/internlm2.py::InternLM2ForRewardModel | reward scoring / text | backbone; reward pooler/head | ☐ required | INVENTORIED | none | unassigned |
MODEL-REWARD-qwen2-rm-qwen2-for-reward-model | Qwen2ForRewardModel | registry.py:282; vllm/model_executor/models/qwen2_rm.py::Qwen2ForRewardModel | reward scoring / text | backbone; reward pooler/head | ☐ required | INVENTORIED | none | unassigned |
MODEL-REWARD-qwen2-rm-qwen2-for-process-reward-model | Qwen2ForProcessRewardModel | registry.py:283; vllm/model_executor/models/qwen2_rm.py::Qwen2ForProcessRewardModel | reward scoring / text | backbone; reward pooler/head | ☐ required | INVENTORIED | none | unassigned |
MODEL-TOKCLS - Token classification
| ID | Item / architecture aliases | Upstream | Task / modality | Dependencies | Spike/spec | State | Our code + tests/evidence | Owner |
|---|---|---|---|---|---|---|---|---|
MODEL-TOKCLS-bert-bert-for-token-classification | BertForTokenClassification | registry.py:287; vllm/model_executor/models/bert.py::BertForTokenClassification | token classification / text or audio alignment | encoder attention; token head/pooler | ☐ required | INVENTORIED | none | unassigned |
MODEL-TOKCLS-modernbert-modern-bert-for-token-classification | ModernBertForTokenClassification | registry.py:288-291; vllm/model_executor/models/modernbert.py::ModernBertForTokenClassification | token classification / text or audio alignment | encoder attention; token head/pooler; sliding-window attention | ☐ required | INVENTORIED | none | unassigned |
MODEL-TOKCLS-openai-privacy-filter-open-aiprivacy-filter-for-token-classification | OpenAIPrivacyFilterForTokenClassification | registry.py:292-295; vllm/model_executor/models/openai_privacy_filter.py::OpenAIPrivacyFilterForTokenClassification | token classification / text or audio alignment | encoder attention; token head/pooler; sliding-window attention | ☐ required | INVENTORIED | none | unassigned |
MODEL-TOKCLS-qwen3-asr-forced-aligner-qwen3-asrforced-aligner-for-token-classification | Qwen3ASRForcedAlignerForTokenClassification | registry.py:296-299; vllm/model_executor/models/qwen3_asr_forced_aligner.py::Qwen3ASRForcedAlignerForTokenClassification | token classification / text or audio alignment | encoder attention; token head/pooler | ☐ required | INVENTORIED | none | unassigned |
MODEL-SEQCLS - Sequence classification
| ID | Item / architecture aliases | Upstream | Task / modality | Dependencies | Spike/spec | State | Our code + tests/evidence | Owner |
|---|---|---|---|---|---|---|---|---|
MODEL-SEQCLS-bert-bert-for-sequence-classification | BertForSequenceClassification | registry.py:303; vllm/model_executor/models/bert.py::BertForSequenceClassification | sequence classification / text or multimodal | backbone; sequence pooler/head | ☐ required | INVENTORIED | none | unassigned |
MODEL-SEQCLS-gpt2-gpt2-for-sequence-classification | GPT2ForSequenceClassification | registry.py:304; vllm/model_executor/models/gpt2.py::GPT2ForSequenceClassification | sequence classification / text or multimodal | backbone; sequence pooler/head; cross-attention | ☐ required | INVENTORIED | none | unassigned |
MODEL-SEQCLS-bert-with-rope-gte-new-for-sequence-classification | GteNewForSequenceClassification | registry.py:305-308; vllm/model_executor/models/bert_with_rope.py::GteNewForSequenceClassification | sequence classification / text or multimodal | backbone; sequence pooler/head; FusedMoE/grouped GEMM | ☐ required | INVENTORIED | none | unassigned |
MODEL-SEQCLS-jamba-jamba-for-sequence-classification | JambaForSequenceClassification | registry.py:309; vllm/model_executor/models/jamba.py::JambaForSequenceClassification | sequence classification / text or multimodal | backbone; sequence pooler/head; FusedMoE/grouped GEMM; Mamba/SSM state | ☐ required | INVENTORIED | none | unassigned |
MODEL-SEQCLS-llama-llama-bidirectional-for-sequence-classification | LlamaBidirectionalForSequenceClassification | registry.py:310-313; vllm/model_executor/models/llama.py::LlamaBidirectionalForSequenceClassification | sequence classification / text or multimodal | backbone; sequence pooler/head; sliding-window attention | ☐ required | INVENTORIED | none | unassigned |
MODEL-SEQCLS-modernbert-modern-bert-for-sequence-classification | ModernBertForSequenceClassification | registry.py:314-317; vllm/model_executor/models/modernbert.py::ModernBertForSequenceClassification | sequence classification / text or multimodal | backbone; sequence pooler/head; sliding-window attention | ☐ required | INVENTORIED | none | unassigned |
MODEL-SEQCLS-roberta-roberta-for-sequence-classification | RobertaForSequenceClassification, XLMRobertaForSequenceClassification | registry.py:318-322; vllm/model_executor/models/roberta.py::RobertaForSequenceClassification | sequence classification / text or multimodal | backbone; sequence pooler/head | ☐ required | INVENTORIED | none | unassigned |
MODEL-SEQCLS-jina-vl-jina-vlfor-sequence-classification | JinaVLForRanking | registry.py:324; vllm/model_executor/models/jina_vl.py::JinaVLForSequenceClassification | sequence classification / image+text | backbone; sequence pooler/head | ☐ required | INVENTORIED | none | unassigned |
MODEL-SEQCLS-nemotron-vl-llama-nemotron-vlfor-sequence-classification | LlamaNemotronVLForSequenceClassification | registry.py:325-328; vllm/model_executor/models/nemotron_vl.py::LlamaNemotronVLForSequenceClassification | sequence classification / image+text | backbone; sequence pooler/head | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM - Multimodal
| ID | Item / architecture aliases | Upstream | Task / modality | Dependencies | Spike/spec | State | Our code + tests/evidence | Owner |
|---|---|---|---|---|---|---|---|---|
MODEL-MM-aria-aria-for-conditional-generation | AriaForConditionalGeneration | registry.py:333; vllm/model_executor/models/aria.py::AriaForConditionalGeneration | conditional generation / image | MM processor; encoder/merge; FusedMoE/grouped GEMM; cross-attention; vision encoder | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-audioflamingo3-audio-flamingo3-for-conditional-generation | AudioFlamingo3ForConditionalGeneration | registry.py:334-337; vllm/model_executor/models/audioflamingo3.py::AudioFlamingo3ForConditionalGeneration | conditional generation / audio+image | MM processor; encoder/merge; audio/ASR frontend | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-bagel-bagel-for-conditional-generation | BagelForConditionalGeneration | registry.py:338; vllm/model_executor/models/bagel.py::BagelForConditionalGeneration | conditional generation / image | MM processor; encoder/merge; vision encoder | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-bee-bee-for-conditional-generation | BeeForConditionalGeneration | registry.py:339; vllm/model_executor/models/bee.py::BeeForConditionalGeneration | conditional generation / video+image | MM processor; encoder/merge; vision encoder | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-blip2-blip2-for-conditional-generation | Blip2ForConditionalGeneration | registry.py:340; vllm/model_executor/models/blip2.py::Blip2ForConditionalGeneration | conditional generation / image | MM processor; encoder/merge; cross-attention; vision encoder | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-chameleon-chameleon-for-conditional-generation | ChameleonForConditionalGeneration | registry.py:341-344; vllm/model_executor/models/chameleon.py::ChameleonForConditionalGeneration | conditional generation / image | MM processor; encoder/merge; vision encoder | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-cheers-cheers-for-conditional-generation | Cheers, CheersForConditionalGeneration | registry.py:345-346; vllm/model_executor/models/cheers.py::CheersForConditionalGeneration | conditional generation / image | MM processor; encoder/merge; vision encoder | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-cohere2-vision-cohere2-vision-for-conditional-generation | Cohere2VisionForConditionalGeneration | registry.py:347-350; vllm/model_executor/models/cohere2_vision.py::Cohere2VisionForConditionalGeneration | conditional generation / image | MM processor; encoder/merge; vision encoder | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-cosmos3-cosmos3-for-conditional-generation | Cosmos3ForConditionalGeneration | registry.py:351; vllm/model_executor/models/cosmos3.py::Cosmos3ForConditionalGeneration | conditional generation / audio+image | MM processor; encoder/merge; audio/ASR frontend; vision encoder | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-deepseek-vl2-deepseek-vlv2-for-causal-lm | DeepseekVLV2ForCausalLM | registry.py:352; vllm/model_executor/models/deepseek_vl2.py::DeepseekVLV2ForCausalLM | conditional generation / image | MM processor; encoder/merge; vision encoder | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-deepseek-ocr-deepseek-ocrfor-causal-lm | DeepseekOCRForCausalLM | registry.py:353; vllm/model_executor/models/deepseek_ocr.py::DeepseekOCRForCausalLM | conditional generation / image | MM processor; encoder/merge; vision encoder | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-deepseek-ocr2-deepseek-ocr2-for-causal-lm | DeepseekOCR2ForCausalLM | registry.py:354; vllm/model_executor/models/deepseek_ocr2.py::DeepseekOCR2ForCausalLM | conditional generation / image | MM processor; encoder/merge; vision encoder | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-unlimited-ocr-unlimited-ocrfor-causal-lm | UnlimitedOCRForCausalLM | registry.py:355; vllm/model_executor/models/unlimited_ocr.py::UnlimitedOCRForCausalLM | conditional generation / image | MM processor; encoder/merge; vision encoder | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-dots-ocr-dots-ocrfor-causal-lm | DotsOCRForCausalLM | registry.py:356; vllm/model_executor/models/dots_ocr.py::DotsOCRForCausalLM | conditional generation / image | MM processor; encoder/merge; vision encoder; video path | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-eagle2-5-vl-eagle2-5-vlfor-conditional-generation | Eagle2_5_VLForConditionalGeneration | registry.py:357-360; vllm/model_executor/models/eagle2_5_vl.py::Eagle2_5_VLForConditionalGeneration | conditional generation / image | MM processor; encoder/merge; vision encoder | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-ernie45-vl-ernie4-5-vlmoe-for-conditional-generation | Ernie4_5_VLMoeForConditionalGeneration | registry.py:361-364; vllm/model_executor/models/ernie45_vl.py::Ernie4_5_VLMoeForConditionalGeneration | conditional generation / video+image | MM processor; encoder/merge; vision encoder; video path | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-exaone4-5-exaone4-5-for-conditional-generation | Exaone4_5_ForConditionalGeneration | registry.py:365-368; vllm/model_executor/models/exaone4_5.py::Exaone4_5_ForConditionalGeneration | conditional generation / image | MM processor; encoder/merge; vision encoder; video path | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-fireredasr2-fire-red-asr2-for-conditional-generation | FireRedASR2ForConditionalGeneration | registry.py:369-372; vllm/model_executor/models/fireredasr2.py::FireRedASR2ForConditionalGeneration | conditional generation / audio+image | MM processor; encoder/merge; audio/ASR frontend | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-funasr-fun-asrfor-conditional-generation | FunASRForConditionalGeneration | registry.py:373; vllm/model_executor/models/funasr.py::FunASRForConditionalGeneration | conditional generation / audio+image | MM processor; encoder/merge; audio/ASR frontend | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-fireredlid-fire-red-lidfor-conditional-generation | FireRedLIDForConditionalGeneration | registry.py:374-377; vllm/model_executor/models/fireredlid.py::FireRedLIDForConditionalGeneration | conditional generation / audio+image | MM processor; encoder/merge; cross-attention; audio/ASR frontend | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-funaudiochat-fun-audio-chat-for-conditional-generation | FunAudioChatForConditionalGeneration | registry.py:378-381; vllm/model_executor/models/funaudiochat.py::FunAudioChatForConditionalGeneration | conditional generation / audio+image | MM processor; encoder/merge; audio/ASR frontend | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-fuyu-fuyu-for-causal-lm | FuyuForCausalLM | registry.py:382; vllm/model_executor/models/fuyu.py::FuyuForCausalLM | conditional generation / image | MM processor; encoder/merge; vision encoder | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-gemma3-mm-gemma3-for-conditional-generation | Gemma3ForConditionalGeneration | registry.py:383; vllm/model_executor/models/gemma3_mm.py::Gemma3ForConditionalGeneration | conditional generation / image | MM processor; encoder/merge; vision encoder | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-gemma3n-mm-gemma3n-for-conditional-generation | Gemma3nForConditionalGeneration | registry.py:384-387; vllm/model_executor/models/gemma3n_mm.py::Gemma3nForConditionalGeneration | conditional generation / audio+image | MM processor; encoder/merge; audio/ASR frontend; vision encoder | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-diffusion-gemma-diffusion-gemma-for-conditional-generation | DiffusionGemmaForBlockDiffusion | registry.py:388-391; vllm/model_executor/models/diffusion_gemma.py::DiffusionGemmaForConditionalGeneration | conditional generation / image | MM processor; encoder/merge; vision encoder; video path | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-gemma4-mm-gemma4-for-conditional-generation | Gemma4ForConditionalGeneration | registry.py:392; vllm/model_executor/models/gemma4_mm.py::Gemma4ForConditionalGeneration (vision tower :1040, audio tower :1056, projector :908-965, audio fwd :1468-1490) | conditional generation / audio+video+image | MM processor; encoder/merge; audio/ASR frontend; vision encoder; video path | ✅ gemma4-multimodal spec §G1b; backbone characterization sweep-gemma §0.1 | READY | G1b LANDED 2026-07-28 (CLAIM-GEMMA4-G1B) — TEXT PATH STRICT 32/32 TOKEN-EXACT vs vLLM 0.25.0. unsloth/gemma-4-E4B-it loads through our engine (LoadedEngine::FromModelDir) and greedily emits the EXACT 32 golden ids [236776,2455,5192,…] (tests/parity/goldens/gemma4_e4b_text/gen_manifest.json); gate tests/parity/test_gemma4_paged_engine.cpp on the dgx CUDA build under flock. The G1-named blocker is RESOLVED: the runner now carries a PER-LAYER KV head_dim (KVCacheConfig::per_layer_attn_specs, consumed in runner.cpp initialize_kv_cache — the model's MakeGemma4…KVCache publishes sliding 256 / global 512 per layer), so the forward's per-layer VT_CHECK(kv.head_size==Dh) passes on both. Byte-neutral: the field is empty for every uniform-KV model ⇒ old allocation/view/indexing/dispatch (full CPU runner+KV+scheduler+tokenizer+hf_config suite green; OLMo-2 SACRED GPU re-gate 16/16 UNCHANGED on the final binary). Three additive loader gaps fixed en route to the first-ever Gemma-4 forward: nested per-layer rope_parameters (hf_config loader now loads it, model reads raw), the Gemma Replace(" "→"▁") metaspace normalizer (tokenizer, folded onto the validated metaspace path), and reading Gemma-4 scalars from raw["text_config"] (the real 256/512 head_dim / layer_types / PLE dim source — a G1 latent bug where all layers were silently treated as sliding). W0 golden capture unchanged (scripts/mm/g0_gemma4_oracle_capture.py). Row PARTIAL: text e2e gated + speed-pending; the USM-Conformer AUDIO tower (G3) + SigLIP vision (G2, reuses M2a ViT) remain unbuilt; YOCO shared-layer cache DEDUP is a memory-only G1c residual (correctness already right — shared layers read the target's cache in-forward). Row ACTIVE — text e2e gated + speed-pending; mm towers G2/G3 in progress. HW fits (E4B ~15 GiB in the 119 GiB pool). Anchors: include/vllm/v1/kv_cache_interface.h:349 (per_layer_attn_specs), src/vllm/v1/worker/gpu/runner.cpp:600 (per-layer KV alloc+view), src/vllm/model_executor/models/gemma4_registry.cpp:109 (publishes per-layer specs), tests/parity/test_gemma4_paged_engine.cpp:67 (STRICT 32/32 gate). G2 (CLAIM-GEMMA4-G2, 2026-07-28) — IMAGE oracle + SigLIP/NaFlex port map LANDED (honest partial, no C++ tower): IMAGE→text greedy golden STRICT K=5 (tests/parity/goldens/gemma4_e4b_image/gen_manifest.json, 18 tokens, 256 soft tokens, prompt ids 274) + 4 staged vision refs (vision_refs/, M2a unit-gate targets) + full port map (spec §G2) via scripts/mm/g2_gemma4_image_oracle_capture.py + scripts/mm/g2_vision_ref_dump.py. ★ Tower is a custom NaFlex SigLIP2 WITH multidim vision-RoPE + q/k/v-norm + Gemma2 sandwich norms + learned-2D pos-embed + √hidden avg-pool-by-position pooler (§0.1 "no RoPE" REFUTED). Image NOT yet engine-gated — C++ NaFlex tower + Gemma-4 image processor + projector/merge are the named residual. G2-impl (CLAIM-GEMMA4-G2-IMPL, 2026-07-28) — C++ NaFlex SigLIP2 TOWER LANDED + per-stage gates PASS: additive gemma4_vision.{h,cpp} + tests/vllm/multimodal/test_gemma4_vision_tower.cpp prove the tower faithful stage-by-stage vs the transformers-eager refs (patch-embed relL2 2.15e-3, encoder 3.14e-2, pooled 1.36e-2, projected 1.85e-2; 220/220; compute-sanitizer 0; dgx CUDA flock, weights via scripts/mm/g2_vision_weight_dump.py). ★ Found+implemented the FINITE QAT activation clamps (use_clipped_linears=True) on the 7 encoder linears (G2 port-map assumed plain Linear). Multidim vision-RoPE via 2× vt::RopeFromCache; attn scale 1.0; NO new vt op. Inertness: text STRICT 32/32 UNCHANGED on the linking binary; additive-by-construction for Qwen3-VL mm. Image e2e (NaFlex processor + engine merge-plumbing) is the named residual (M2c-equivalent). G3 (CLAIM-GEMMA4-G3, 2026-07-28) — C++ USM-Conformer AUDIO TOWER LANDED + per-stage gates PASS: additive gemma4_audio.{h,cpp} + tests/vllm/multimodal/test_gemma4_audio_tower.cpp prove the tower + audio projector faithful stage-by-stage vs the transformers-eager refs (host f32, 1256/1256): subsample relL2 5.4e-7, posemb 8.9e-8, block0 4.2e-7, block_mid 3.8e-7, block_last 4.4e-6, output_proj 5.9e-6, projected 6.3e-6 — f32-EXACT. Ported 1:1 from modeling_gemma4.py @ 5.13.1: 2×Conv2d subsample + LN-no-bias + ReLU → rel-pos-enc → 12 Conformer layers (half-step FFN, chunked-local attn [chunk 12, past window 12, Transformer-XL rel-shift, softcap 50, per_dim_scale softplus], GLU + depthwise causal conv k5) → output_proj → embed_audio. FINITE QAT clamps (use_clipped_linears=True) implemented. RED-first: wrong sliding window (13 keys) → block_mid 2.8e-2 RED; fixed to dist∈[0,12) (12 keys) → ~1e-6. NO new vt op (host f32). Golden tests/parity/goldens/gemma4_e4b_audio/, weights via scripts/mm/g3_audio_tower_ref.py. Inertness by construction (standalone TU). Audio e2e (feature extractor + engine merge-plumbing) is the named residual — TRI-MODAL TOWERS now all proven. MM-IMAGE-E2E (CLAIM-GEMMA4-MM-E2E, 2026-07-29) — the SigLIP2 vision tower FOLDED into the ENGINE registered forward, image→text dgx-GREEN: Gemma4ForConditionalGeneration now supports_multimodal=true with an mm branch in gemma4_registry.cpp routing ModelForwardInput.mm → Gemma4Model::ForwardMm (additive default-null override on ForwardBody: hidden stream starts from the already-merged inputs_embeds, PLE uses the mm-masked ids ple_token_ids); the driver Gemma4GenerateGreedyViaRegistry (gemma4_mm.cpp) runs every step through ModelRegistry::Forward (embed(prompt)*√H + masked-scatter of the projector output via Qwen3VLMergeMultimodal, per-layer paged KV 256/512 YOCO-aware, 1-D positions, no MRoPE/DeepStack). Gate tests/vllm/multimodal/test_gemma4_registry_e2e.cpp (dgx GB10 sm_121a, flock): 16/18 content tokens BIT-EXACT vs the STRICT gemma4_e4b_image golden (the full sentence "This is a vibrant, abstract background featuring a smooth gradient of bright, blended colors"), the single divergence the terminal "."↔"," bf16 NEAR-TIE (top1-top2 margin ~0.10-0.12 logit) — INVARIANT to vision-input precision (live C++ bf16 SigLIP2 tower AND committed f32 ref_projected.npy diverge identically ⇒ backbone bf16-accumulation on the 256 image rows, NOT the fold; PLE mask + vocab_size_per_layer_input=262144 verified vs E4B config; use_bidirectional_attention=None). The near-tie GATE FORM (ratified rule): content-exact prefix + first divergence must be a bf16 near-tie (margin < 0.5 band) — a structural bug FAILs. Inertness: text SACRED test_gemma4_paged_engine STRICT 32/32 UNCHANGED on the mm binary; supports_multimodal has no engine consumer (byte-neutral flip). Residuals: STRICT-18/18 (bit-match vLLM prefill bf16 on image rows), audio e2e (mel A1 + engine merge), speed | - |
MODEL-MM-gemma4-unified-gemma4-unified-for-conditional-generation | Gemma4UnifiedForConditionalGeneration | registry.py:393-396; vllm/model_executor/models/gemma4_unified.py::Gemma4UnifiedForConditionalGeneration (subclass of gemma4_mm, encoder-free :73,220-291) | conditional generation / audio+video+image | MM processor; encoder/merge; audio/ASR frontend; vision encoder; video path | 📋 gemma4-multimodal spec; backbone characterization sweep-gemma §0.1 | SPIKE | GATEABLE — W0 2026-07-28 (CLAIM-GEMMA4-W0): the sibling Gemma4ForConditionalGeneration W0 RUN-VERIFIED on vLLM 0.25.0 (loads+runs+generates, STRICT greedy golden on unsloth/gemma-4-E4B-it) proves the shared registered mm path executes; this ENCODER-FREE variant (no SigLIP/audio AutoModel tower; Gemma4UnifiedVisionEmbedder factorized-2D-posembed patch pipeline) has an ungated vehicle unsloth/gemma-4-12b-it (23.92 GB, gated=False, fits GB10) — no HF token needed. Oracle block RETIRED; no standalone run this pass (E4B is the smaller W0 vehicle). IMPLEMENTATION-blocked only, staged behind the Gemma-4 backbone; no implementation | CLAIM-GEMMA4-MULTIMODAL |
MODEL-MM-glmasr-glm-asr-for-conditional-generation | GlmAsrForConditionalGeneration | registry.py:397; vllm/model_executor/models/glmasr.py::GlmAsrForConditionalGeneration | conditional generation / audio+image | MM processor; encoder/merge; audio/ASR frontend; vision encoder | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-glm4v-glm4-vfor-causal-lm | GLM4VForCausalLM | registry.py:398; vllm/model_executor/models/glm4v.py::GLM4VForCausalLM | conditional generation / image | MM processor; encoder/merge; vision encoder | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-glm4-1v-glm4v-for-conditional-generation | Glm4vForConditionalGeneration | registry.py:399; vllm/model_executor/models/glm4_1v.py::Glm4vForConditionalGeneration | conditional generation / video+image | MM processor; encoder/merge; vision encoder; video path | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-glm4-1v-glm4v-moe-for-conditional-generation | Glm4vMoeForConditionalGeneration | registry.py:400; vllm/model_executor/models/glm4_1v.py::Glm4vMoeForConditionalGeneration | conditional generation / video+image | MM processor; encoder/merge; vision encoder; video path | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-glm-ocr-glm-ocr-for-conditional-generation | GlmOcrForConditionalGeneration | registry.py:401; vllm/model_executor/models/glm_ocr.py::GlmOcrForConditionalGeneration | conditional generation / image | MM processor; encoder/merge; vision encoder; video path | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-granite-speech-granite-speech-for-conditional-generation | GraniteSpeechForConditionalGeneration | registry.py:402-405; vllm/model_executor/models/granite_speech.py::GraniteSpeechForConditionalGeneration | conditional generation / audio+image | MM processor; encoder/merge; audio/ASR frontend | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-granite-speech-plus-granite-speech-plus-for-conditional-generation | GraniteSpeechPlusForConditionalGeneration | registry.py:406-409; vllm/model_executor/models/granite_speech_plus.py::GraniteSpeechPlusForConditionalGeneration | conditional generation / audio+image | MM processor; encoder/merge; audio/ASR frontend | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-granite4-vision-granite4-vision-for-conditional-generation | Granite4VisionForConditionalGeneration | registry.py:410-413; vllm/model_executor/models/granite4_vision.py::Granite4VisionForConditionalGeneration | conditional generation / image | MM processor; encoder/merge; cross-attention; vision encoder | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-h2ovl-h2-ovlchat-model | H2OVLChatModel | registry.py:414; vllm/model_executor/models/h2ovl.py::H2OVLChatModel | conditional generation / image | MM processor; encoder/merge; vision encoder | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-hunyuan-vision-hun-yuan-vlfor-conditional-generation | HunYuanVLForConditionalGeneration | registry.py:415-418; vllm/model_executor/models/hunyuan_vision.py::HunYuanVLForConditionalGeneration | conditional generation / image | MM processor; encoder/merge; vision encoder; video path | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-internvl-intern-vlchat-model | InternVLChatModel | registry.py:419; vllm/model_executor/models/internvl.py::InternVLChatModel | conditional generation / video+image | MM processor; encoder/merge; vision encoder; video path | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-interns1-intern-s1-for-conditional-generation | InternS1ForConditionalGeneration, InternVLForConditionalGeneration | registry.py:420-427; vllm/model_executor/models/interns1.py::InternS1ForConditionalGeneration | conditional generation / video+image | MM processor; encoder/merge; vision encoder; video path | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-interns1-pro-intern-s1-pro-for-conditional-generation | InternS1ProForConditionalGeneration | registry.py:428-431; vllm/model_executor/models/interns1_pro.py::InternS1ProForConditionalGeneration | conditional generation / image | MM processor; encoder/merge; FusedMoE/grouped GEMM; vision encoder; video path | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-interns2-preview-intern-s2-preview-for-conditional-generation | InternS2PreviewForConditionalGeneration | registry.py:432-435; vllm/model_executor/models/interns2_preview.py::InternS2PreviewForConditionalGeneration | conditional generation / image | MM processor; encoder/merge | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-idefics3-idefics3-for-conditional-generation | Idefics3ForConditionalGeneration | registry.py:436-439; vllm/model_executor/models/idefics3.py::Idefics3ForConditionalGeneration | conditional generation / image | MM processor; encoder/merge; vision encoder | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-isaac-isaac-for-conditional-generation | IsaacForConditionalGeneration | registry.py:440; vllm/model_executor/models/isaac.py::IsaacForConditionalGeneration | conditional generation / image | MM processor; encoder/merge; vision encoder; video path | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-kanana-v-kanana-vfor-conditional-generation | KananaVForConditionalGeneration | registry.py:441; vllm/model_executor/models/kanana_v.py::KananaVForConditionalGeneration | conditional generation / image | MM processor; encoder/merge; vision encoder; video path | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-keye-keye-for-conditional-generation | KeyeForConditionalGeneration | registry.py:442; vllm/model_executor/models/keye.py::KeyeForConditionalGeneration | conditional generation / video+image | MM processor; encoder/merge; vision encoder; video path | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-keye-vl1-5-keye-vl1-5-for-conditional-generation | KeyeVL1_5ForConditionalGeneration | registry.py:443-446; vllm/model_executor/models/keye_vl1_5.py::KeyeVL1_5ForConditionalGeneration | conditional generation / video+image | MM processor; encoder/merge; vision encoder; video path | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-kimi-vl-kimi-vlfor-conditional-generation | KimiVLForConditionalGeneration | registry.py:447; vllm/model_executor/models/kimi_vl.py::KimiVLForConditionalGeneration | conditional generation / image | MM processor; encoder/merge; vision encoder | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-kimi-k25-kimi-k25-for-conditional-generation | KimiK25ForConditionalGeneration | registry.py:448; vllm/model_executor/models/kimi_k25.py::KimiK25ForConditionalGeneration | conditional generation / video+image | MM processor; encoder/merge; vision encoder; video path | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-kimi-audio-kimi-audio-for-conditional-generation | MoonshotKimiaForCausalLM | registry.py:449; vllm/model_executor/models/kimi_audio.py::KimiAudioForConditionalGeneration | conditional generation / audio+image | MM processor; encoder/merge; audio/ASR frontend; vision encoder | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-muse-glimmer-muse-glimmer-for-conditional-generation | MuseGlimmerForConditionalGeneration | BEYOND-PIN — NOT in 555967922 (Muse Glimmer released 2026-08-08, after the pin, and NOT on vLLM main either); the only upstream implementation is the OPEN, approved-but-CI-red PR vllm#51655 at head 075d645af — vllm/model_executor/models/muse_glimmer.py (text attn :1083, decoder layer :1212, vision encoder :692, adapter :1036, MM processor :262), vllm/transformers_utils/configs/muse_glimmer.py, vllm/transformers_utils/processors/muse_glimmer.py, DFlash hooks in qwen3_dflash.py:75-94 + v1/spec_decode/dflash.py:83-95 | conditional generation / image+video | model loader/forward; MM processor; vision encoder + merge; video path; speculative decoding (DFlash); reasoning + tool parsers | muse-glimmer spec, issue #268 | SPIKE | W-SPEC (2026-08-10, row/MODEL-MUSE-GLIMMER, records+spec only, NO code). Traced end-to-end from PR #51655 @ 075d645af. meta-models/Muse-Glimmer-30B, Apache-2.0, bf16, model_type: muse_glimmer; registry maps BOTH MuseGlimmerForConditionalGeneration and MuseGlimmerForCausalLM onto one class. Text tower L=52, H=6656, 32 q-heads / 2 kv-heads (GQA 16:1), head_dim 128, vocab 202048, 131k ctx, rope theta 5e5: Gemma2-style SANDWICH norms (input/post-attn/pre-ffn/post-ffn) computed in fp32 with a BAKED +1 weight offset and SPLIT eps (pre-norms rms_norm_eps, post-norms the smaller post_norm_eps) (muse_glimmer.py:1236-1247); iRoPE — no_rope_layers[i]==1 ⇒ RoPE AND sliding-window, ==0 ⇒ NoPE AND full attention, sliding×3 then full every 4th (:1114-1116,:1167-1168) — the ONE mechanism with no local analogue (we have no Llama-4); WEIGHTLESS QK-norm over head_dim in fp32 applied BEFORE RoPE (:1189-1196); post-QK-norm QUERY PRE-SCALE scale_query_by≈3.87 with softmax scaling left at head_dim**-0.5 (:1112,:1192); per-head sigmoid ATTENTION OUTPUT GATE whose gate reads the LAYER INPUT, not the attn output (:1203-1206); no logit softcapping. Two named correctness traps: (a) the query pre-scale ships under two schemas — native raw qk_scale_factor≈43.784 vs modular PRE-FOLDED ≈3.87 — disambiguated upstream BY MAGNITUDE against sqrt(head_dim) (:472-517); mis-reading it scales every query by 11.3×; (b) use_qk_norm / use_attn_output_gate read as None (not True) in the modular schema and only an explicit False disables them (:456-469), so a naive getattr(...,False) silently drops BOTH mechanisms while still emitting plausible text. Perception encoder L=50, H=1536, 16 heads (head_dim 96), patch 14×14, patch_temporal=2, 32×32 learned pos-emb grid, interleaved window/full per layer_types, projector 4096→6144, image tok 200092 / video tok 200091, placeholders <|patch|>/<|image|>/<|video|>: LINEAR patchify (conv1_linear, NOT a conv) (:696,:710), BILINEAR pos-emb interpolation with per-corner validity masking and a half-pixel +0.5/-0.5 convention (:761-820), 2D RoPE with width-first cat([freq_w,freq_h]) (:741-759), block-windowed attention via a -1-padded permutation whose per-block valid counts become seq_lens (:844-867), pixel-shuffle downsample asserting output_dim == hidden*merge^2 (:822-842), plain LayerNorm (not RMSNorm) at ln_pre/ln_post. DFlash adds NO new drafter — PR #51655 only recognises MuseGlimmerAssistantModel as method dflash and threads the TARGET's is_neox_style into the draft config, because a draft/target RoPE-layout mismatch is SILENT (acceptance collapses, nothing errors, output stays correct). REUSE MAP: sandwich norms → gemma2.cpp/gemma4.cpp; output gate → qwen3_5.cpp; weightless QK-norm → Qwen3 family; windowed vision tower → qwen3_vl_vision.cpp; gated MLP → layers::MlpGateUpMethodBase+vt::MergedGemmGroup; decode → ModelRegistry::Forward+dense_attn::AttnBlock; fusion → vt::FusedChain; DFlash → the existing speculator row. HONESTY: the pinned oracle CANNOT load muse_glimmer (and the checkpoint wants transformers 5.15.0.dev0 vs the pin's 5.14.1), so there is NO gateable oracle and every performance axis is an OPEN GAP by construction — no parity/throughput claim may be made from this row until #51655 merges and the pin advances. Correctness gates against the HF reference instead, with per-mechanism RED-first mutation carrying the weight the missing oracle would have. Weights (~60 GiB bf16) NOT downloaded; GB10 fit needs a quantized arm. W-plan W0-W7 in the spec. W0 CPU SCAFFOLDING LANDED (2026-08-10, CLAIM-MUSE-GLIMMER-W0, CPU-only, no weights, no GPU): additive TUs register BOTH MuseGlimmerForCausalLM and MuseGlimmerForConditionalGeneration onto ONE factory (mirroring registry.py @ #51655); ParseMuseGlimmerParams descends the canonical NESTED layout AND normalizes the older FLAT layout (configs/muse_glimmer.py:186-305 — without it a flat config silently deserializes to an ALL-DEFAULT text config, ignoring every checkpoint value with no error); NormalizeMuseGlimmerWeightName ports the hf_to_vllm_mapper (:1389-1425) for BOTH checkpoint conventions; EnumerateMuseGlimmerTensors is the structural name map, deliberately OMITTING the three WEIGHTLESS modules (embed_norm :1286, per-head qk_norm :1121, perception_emb_norm :1470) that ship no tensor; forward REFUSES-by-name. Gate tests/vllm/models/test_muse_glimmer_scaffold.cpp 11/11 cases, 73/73 assertions, clean CPU -Werror build. RED-FIRST MUTATION-PROVEN, all four named traps (each mutated in-tree, gate went RED, tree restored byte-identical): (1) treating the native raw qk_scale_factor as pre-folded → 3 RED (the 11.3x query blow-up); (2) defaulting the absent use_qk_norm/use_attn_output_gate to FALSE → 4 RED; (3) counting the iRoPE mask FORWARD instead of backward from the last layer → 5 RED; (4) applying the legacy-guac sandwich-norm renames in the wrong order (which SWAPS post-attention with pre-feedforward) → 1 RED. KV-cache spec is a documented W0 placeholder (one full-attention group; the real sliding/full split rides the Gemma-4 per-layer seam at W1). W1-W7 FOLLOWED ON THE SAME BRANCH (row/MODEL-MUSE-GLIMMER, PR #279, NOT merged), and this paragraph's "config/name-map/registry only, NO forward" is superseded: W1 the 52-layer text forward (sandwich norms w/ split eps, iRoPE, weightless QK-norm before RoPE, query pre-scale, attn output gate, SwiGLU, untied lm_head, output_multiplier before the soft-cap) gated vs an independent fp32 transcription of 075d645af plus a property test per mechanism (test_muse_glimmer_text, 21 cases / 487 assertions, and the SAME binary re-run under VT_FUSED_CHAIN_ADOPT=0 so the non-FusedChain fallback arm is gated too); W3 the 50-layer perception encoder (test_muse_glimmer_vision); W4 the mm wiring — tower → adapter → projection → perception_emb_norm → masked scatter onto the image/video placeholder rows, with the released 30B's 1436 tensors accounted 1436/1436 (test_muse_glimmer_wiring, 9/10316); W7 the ATEM reasoning + tool parsers (34 cases / 251 assertions). REAL-WEIGHT EVIDENCE, and its exact limits: on real 30B tensors at reduced depth 4 of 52 our forward's 5 prefill argmax positions are identical to our torch transcription AND to HF's own muse_glimmer (exportable-muse @a9e337e8); max abs diff 0.0889745, cosine 0.999981. NOT established, and not to be implied: the full-depth 52-layer arm NEVER RAN; these are prefill argmax positions, NOT generated tokens, so multi-step decode and the sliding window across steps are untested; the perception encoder has NO reference check of any kind (the wiring gate proves reachability and placement, not that an image produces the right tokens); nothing has run end to end through the server; and the ATEM parsers' channel scoping DOES NOT WORK at server defaults, because the ToolParser/ReasoningParser seam has no adjust_request dispatch site while skip_special_tokens defaults true and the checkpoint marks <|start|>/<|message|>/<|eom|>/<|eot|> special (open gap, spec §6.7). NO vLLM SPEED AXIS ON ANY DIMENSION — unchanged and unchangeable until #51655 merges and the pin advances. A SECONDARY, quant-matched llama.cpp bar now EXISTS and is measured (#333, 2026-08-11, row/MUSE-BENCH-2, spec §14): same 16.76 GB Q4_K_M file both engines on an idle GB10 CPU, 128-token prefill tie at 0.997x (0.898x at 20 threads), decode 0.232x, 512-token prefill 0.170x, peak RSS 1.92x MORE (30.29 vs 15.74 GiB, the §10.2 dequantized qkv/lm_head/embed). It is a secondary reference and is NEVER the denominator; the vLLM cell stays an open gap by construction. Row token stays SPIKE deliberately: advancing it owes docs/STATUS.md and .agents/NOW.md rows and both surfaces sit byte-exactly at their shrink-only ratchet/budget, so the advance belongs to the landing commit | CLAIM-MUSE-GLIMMER-SPEC, CLAIM-MUSE-GLIMMER-W0, CLAIM-MUSE-GLIMMER-W1, CLAIM-MUSE-GLIMMER-W3, CLAIM-MUSE-GLIMMER-W4, CLAIM-MUSE-GLIMMER-W7, CLAIM-MUSE-GLIMMER-FIX |
MODEL-MM-kimi-k3-kimi-k3-for-conditional-generation | KimiK3ForConditionalGeneration | BEYOND-PIN — NOT in 555967922 (K3 released 2026-07-27, after the pin); closest registered = its literal text backbone KimiLinearForCausalLM (registry.py:140; vllm/model_executor/models/kimi_linear.py) + the K2.5 vision wrapper kimi_k25.py:290 / tower kimi_k25_vit.py | conditional generation / image (text-first) | model loader/forward; FusedMoE/grouped GEMM; GDN/linear-attention state (KDA); MLA/latent KV; MXFP4 compressed-tensors quant; MM processor + MoonViT-V2 encoder/merge | kimi-k3 spike | SPIKE | W0 SCOPE (2026-07-28, CLAIM-KIMI-K3-SCOPE, DERIVE-AND-SHIP, records-only). From the HF config.json (fetch-derived): architectures:["KimiK3ForConditionalGeneration"], text_config.architectures:["KimiLinearForCausalLM"] — the text backbone IS the pinned Kimi-Linear hybrid, MASSIVELY scaled: H=7168, L=93 (69 KDA + 24 MLA full-attn), 896 experts / top-16 / 2 shared, moe_intermediate_size=3072; MLA geometry kv_lora=512/q_lora=1536/qk_nope=128/qk_rope=64 (= our landed DeepSeek-V3 dims); KDA head_dim=128/num_heads=96/short_conv=4/gate_lower_bound=-5.0; quant mxfp4-pack-quantized (compressed-tensors, group 32, e8m0) + MXFP8 acts (QAT); vision MoonViT-V2 (~401M, patch 14, 27L). HEAVY REUSE — GDN (KDA's parent, cuda_gdn.cu/gdn_attn.cpp), DeepSeek MLA (deepseek_v2.cpp/mla_attention.*, exact geometry), DeepSeek-style MoE (qwen3_moe.cpp/cuda_moe.cu, scale to 896), and the Qwen3.6-35B GDN-hybrid-MoE model skeleton (qwen3_5_moe.cpp) are the structural twins; Kimi-K2 tokenizer/tool parser (parser/kimi_k2.cpp) reused. NET-NEW: the KDA kernel delta (per-channel [H,D] low-rank decay f_a_proj/f_b_proj + sigmoid-gated output norm + 3 q/k/v convs — already scoped on the Kimi-Linear row), MXFP4 (we have NVFP4 group-16, not MXFP4 group-32/e8m0), AttnRes (report-only, UNCONFIRMED — not in config.json nor pinned kimi_linear.py), and the MoonViT-V2 tower. HW-fit: DOES NOT FIT GB10 — 2.8T MXFP4 ≈ 1.56 TB ≈ ~12× over the 119 GiB pool; no small K3 exists. DERIVE-AND-SHIP (no on-box golden, like the beyond-vLLM CUDA bricks): (a) REAL proxy gate of KDA+MLA+MoE on the FITTING Kimi-Linear-48B-A3B (~89–91 GiB) vs the pinned oracle, (b) build-verify + structural review for the K3 scale-up. The pinned oracle has NO kimi_k3 ⇒ even HW-rich users need a pin advance to oracle-gate K3 itself. CORRECTS the 2026-07-25 sweep note ("loads as DeepseekV3ForCausalLM" — true for K2, NOT K3). W-plan W1-W8 in the spec. W2/W5 CPU SCAFFOLDING LANDED (2026-07-28, CLAIM-KIMI-K3-W2-W5, DERIVED+BUILD-VERIFIED, NOT pushed): additive registry TU registers KimiK3ForConditionalGeneration (info: text-gen + is_hybrid + has_inner_state + supports_multimodal); config-descent ParseKimiK3Params reads the nested text_config (KimiLinear KDA+MLA+MoE scalars — note upstream key num_experts_per_token), vision_config (MoonViT-V2 PARTIAL), quantization_config (MXFP4 detect) grounded in configs/kimi_linear.py:11-148; pure EnumerateKimiK3TextBackboneTensors is the 93-layer KDA/MLA + 896-expert MoE structural name-map grounded 1:1 in kimi_linear.py:104-378,460-554 + kimi_gdn_linear_attn.py:102-226 (per-layer KDA vs MLA-with-qLoRA vs MoE-vs-dense branching); forward REFUSE-by-name (VT_CHECK(false), mirrors deepseek_v4.cpp); loader REFUSES MXFP4 (a real K3 checkpoint's dtype) deferring to the shared DeepSeek-V4 MXFP4 row. NOT-YET-BUILDABLE (correctly deferred): MXFP4 materialization (→ CLAIM-DEEPSEEK-V4-* / quantization-matrix MXFP4), the KDA kernel delta (→ Kimi-Linear row MODEL-TEXT-kimi-linear-*), MoonViT-V2 vision (W7), K3 multimodal-wrapper weight prefix (post-pin). Code: src/vllm/model_executor/models/kimi_k3{,_registry,_weights}.cpp + include/vllm/model_executor/models/kimi_k3.h. Test: tests/vllm/models/test_kimi_k3_scaffold.cpp (6/6, 63 assertions — registry-resolve + config-descent + split logic + enumeration + reject + MXFP4-refuse). Clean CPU build (-DVLLM_CPP_CUDA=OFF). Row stays SPIKE (no on-box e2e; forward not implemented); registration src/vllm/model_executor/models/kimi_k3_registry.cpp:126; test tests/vllm/models/test_kimi_k3_scaffold.cpp:123 | CLAIM-KIMI-K3-SCOPE, CLAIM-KIMI-K3-W2-W5 |
MODEL-MM-dots3-note-dots3-note-for-causal-lm | Dots3NoteForCausalLM | BEYOND-PIN — NOT in 555967922 (the checkout at the pin carries only dots_ocr.py, verified); present on vLLM main as the platform-split package vllm/models/dots3_note/{common,nvidia}/ (15 files, ~5.7k LoC) added by vllm#51255 @ 9035151d6, last touched 170592a93 (2026-08-13, vllm#52172); registry.py:381 (_MULTIMODAL_MODELS) -> ("vllm.models.dots3_note", "Dots3NoteForCausalLM") (re-derived at c205726108df54bb6fbf15b19e725a4a3add2b18; W0 read :375); config transformers_utils/configs/dots3_note.py:7 Dots3NoteConfig(DeepseekV3Config); text tower nvidia/model.py (Dots3NoteMoE:76, Dots3NotePaddedMLAAttention:204, Dots3NoteFullAttention:219, Dots3NoteSlidingAttention:329, Dots3NoteDecoderLayer:481, Dots3NoteModel:549, shared body _forward_note_mla:135); SWA-MLA stack nvidia/attention.py (_gather_swa_kv_kernel:48, _apply_swa_score_mask_kernel:118, Dots3NoteMLAMetadataBuilder:306, Dots3NoteTritonMLAImpl:438, Dots3NotePaddedSparseImpl:689); vision nvidia/vision.py:508 + nvidia/vision_moe.py; audio nvidia/audio_encoder.py:437; MM front end nvidia/multimodal.py:49; processor common/processor.py + common/video.py | conditional generation / text+image+video+audio understanding | model loader/forward; MLA/latent KV; sliding-window MLA (new); heterogeneous/padded MLA KV spec (new); DSA lightning indexer; ungrouped noaux_tc FusedMoE 256/top-8; MM processor + MoE ViT encoder/merge; video path; dots Whisper-variant audio tower; blockwise-FP8 + owed GGUF k-quant arms; MTP | dots3-note spec, issue #699 | SPIKE | W2 LANDED 2026-08-24 (row/MODEL-MM-dots3-note-W2, the WHOLE weight map, CPU-only, no GPU, no tensor byte downloaded). The complete released model.safetensors.index.json @ 1e1e7b0cd37a3a48a6c8d7fa55d5f9d14377006b (sha256 95a364b4…) joined to the safetensors HEADER of all 133 files it names — two HTTP Range requests each, 4770592 header bytes and ZERO tensor bytes against a 576886825984-byte checkpoint. Accounting 38006/38006: 35381 language / 2195 vision / 430 audio, zero unaccounted, zero missing, zero duplicated, zero invented, every bucket asserted BY NUMBER and the whole 38006-name set driven through ModelRegistry::Resolve(...).factory->load_weights as well as through the classifier. Gate test_dots3_note_scaffold 26 cases / 110821 assertions (19/3876 at W1). The two towers stopped being integers: Dots3NoteDeferredTowers() is a table of records — prefix, the ONE file each tower ships in, the brick that owes it (W6 vision, W7 audio) and what it is — the classifier dispatches on it, and the load refusal prints it, so an unknown tensor is distinguishable from a deferred one. THREE facts the W1 slice could not reach. (1) The backbone has EXACTLY FOUR distinct layer shapes — {0} dense+full 19 tensors, the 12 full+MoE {1,5,…,45} at 789, the 33 sliding+MoE at 784, {46} at 18 — so W1's "the remaining 42 layers repeat 1/2 exactly" is now measured and no layer breaks the pattern. (2) The full/sliding split derived from the SHIPPED indexer.wk tensors equals config.layer_types exactly (13 full, q_b_proj [24576,1024] against [16384,1024]), which is two independent released artifacts agreeing. (3) The checkpoint carries 62 F32 tensors in TWO families, not one: the 45 language mlp.gate.e_score_correction_bias W1 predicted, and 17 vision_encoder.blocks.{25..41}.mlp.router_bias the language-only slice could not see, whose widths are the pyramid's own 4,8,…,64,64 — fp32 learned router statistics inside a BF16 tower, and the census closes at 37944 BF16 + 62 F32 with no third dtype. NOT spec R5, which an earlier draft of this cell claimed: R5/§2.4 is the FP32 dynamic ACTIVATION SCALES in note_vision_fused_moe_fp8, and the bf16 checkpoint ships no scale tensors at all — R5 stays wholly owed by W6. Audio is BF16 throughout, and §1.2's vision pyramid is now checkpoint-measured rather than config-inferred. A FOURTH is a finding, not a confirmation (#1846): the index's metadata block declares indexer_rope_layout: "leading" and indexer_rope_converted_from: "tail", and git grep indexer_rope_layout over vLLM origin/main returns NOTHING — upstream reads neither. It agrees with what upstream's code does anyway (DeepseekV2Indexer rotates [..., :rope_dim], deepseek_v2.py:805,:814), and it is NOT §4 trap 2: trap 2 is which PAIRS rope rotates, this is which HALF of the 128-wide head it rotates. W2 pins both values in an assertion and consumes neither; W3 owes the slice. Evidence, the fetch recipe, the tower inventories W6/W7 inherit and the 15-row mutation table (every row RED with compile_err=0 printed beside it, one of them a REPLACEMENT for a guard that came back GREEN and was deleted) are spec §4.4. Row stays SPIKE: reading a weight map is not porting a model. W1 LANDED 2026-08-23 (row/MODEL-MM-dots3-note-W1, config + registry, CPU-only, no GPU, no checkpoint downloaded). Additive TU src/vllm/model_executor/models/dots3_note_registry.cpp with ONE REGISTER_VLLM_MODEL and zero edit to a shared array; dots3_note.{h,cpp} carry Dots3NoteParams, the on-disk name map and a forward that REFUSES BY NAME. Dots3NoteMTPModel is deliberately NOT registered and stays INVENTORIED (W10): a speculator that cannot propose makes the engine accept a config it then dies on mid-run. Gate test_dots3_note_scaffold 19 cases / 3876 assertions (15/3694 as first landed; the #1805 review added four). ALL SIX §4 CONFIG TRAPS RED-FIRST, captured on a RED arm that COMPILED and ran (7/15 cases red, 16 assertions) before the correct values existed: n_group 8→1 and topk_group 4→1 (DeepseekV3Config :168-169 would regroup the router at every MoE layer, and a grouped config is now REFUSED by name); indexer_rope_interleave false→true (deepseek_v2.py:1148 consumes it as is_neox_style = not …, so V3.2's absent-key default rotates split-half NeoX); num_nextn_predict_layers 0→1, whose RED knock-on was 19 UNCLAIMED tensors ("first: model.layers.46.eh_proj.weight"); the two apply_mla_qkv_lora_rescale scalars, which DISAGREE between the geometries (kv sqrt(10) full vs sqrt(5) sliding); and swa_rope_theta 5e4 against the full layers' 8e7. §4 item 6 CORRECTED in place (#1804): is_neox_style=False is NOT sliding-only — Dots3NoteFullAttention inherits the same hard-coded value from deepseek_v2.py:1093-1098, so BOTH MLA ropes are GPT-J and only the theta differs. §1.4 RESOLVED by the checkpoint: exactly ONE nextn layer, carrying the SLIDING attention set (no indexer.*, q_b_proj [16384,1024]) and a DENSE MLP; model.mtp.embed_tokens.weight present and shared_head.head.weight absent, matching has_own_embed_tokens=True/has_own_lm_head=False (mtp.py:141-142). Upstream cannot answer the sliding half — config.layer_types[layer_idx] (model.py:503) has no entry at index 46 — so the checkpoint is the authority and W10 owes the reconciliation. Tensor accounting 1614/1614 both ways over a committed HEADERS-ONLY slice of model.safetensors.index.json @ 1e1e7b0cd37a3a48a6c8d7fa55d5f9d14377006b (root + layer 0 dense/full + layer 1 MoE/full + layer 2 MoE/sliding + nextn 46), out of the 38006 the checkpoint ships; the 2625 vision/audio tensors are NAMED W6/W7 deferrals in the loader's classifier, never silent drops. W2 owes the whole index. Shapes verified against the released safetensors headers (no tensor byte read): full kv row 576 vs sliding 1088, headwise g_proj width = head count, indexer 64x128. KV spec emits ONE MLA group at the PADDED 1088 row (model.py:283, :213), not the full layers' 576. Memory format: mlp.gate.e_score_correction_bias is F32 in an otherwise BF16 tower. The forward refusal is driven through the REAL model the factory returns — a fabricated LoadedModel stub is UB the moment the handle opens (#730/#775) — with a separate foreign-model case for the checked-downcast guarantee alone. Config parsing lives in the model TU, NOT hf_config.cpp as spec §3.2 item 6 proposed: hf_config.cpp is the shared container reader and AGENTS.md forbids a surface every PR must write (spec §4.2); it needs no edit for this checkpoint. Upstream read at vLLM origin/main c205726108df54bb6fbf15b19e725a4a3add2b18 — still BEYOND-PIN. Row stays SPIKE: the §8.1 heading restructure ACTIVE requires belongs to the brick where the forward stops refusing. NO number is claimable on any axis while spec §6.4 option B holds. THREE REVIEW FINDINGS REPAIRED IN THE SAME PR (#1805). (F1) The config readers SUBSTITUTED A DEFAULT for an absent or wrong-typed key, which is the §4 item 4 trap and the one item W1 shipped ungated: deleting apply_mla_qkv_lora_rescale and swa_rope_theta made the parse SUCCEED with all four LoRA scales at 1.0 and 33 of 46 layers at 1e4 instead of 5e4, and neither key is one of the four setdefaults — upstream raises AttributeError. Measured blast radius: 26 of 36 required keys parsed clean when deleted, all 36 when wrong-typed. Every read is now REQUIRED (refuse by name) or OPTIONAL-with-a-named-upstream-default (type-strict), a wrapped text_config layout refuses, and §4.3 records the deliberate stricter-than-upstream call for the DeepseekV3Config-inherited subset. (F2) The tower-deferral subcase asserted only REQUIRE_NOTHROW, so reclassifying all 2625 tower tensors as LANGUAGE stayed GREEN; acc.vision/acc.audio/acc.language are now asserted by count — the mutation reports acc.vision=0, acc.language=35386 against 35381. (F3) docs/FEATURES.md listed this arch as BOTH registered and "not registered", and its 36/40 counts were stale at 37/41. Also repaired: §4.1/§4.3 trap numbering unified on §4's own 1-6, model.py anchors re-derived (:283 not :315, __init__ :222-308/:332-460), §1.1's rope row and §1.4 corrected, the rope_is_neox_style struct default flipped to the never-correct value so deleting the resolution step fails loudly, and the CMake comment orphaning undone. W0 SCOPE (2026-08-14, CLAIM-MODEL-DOTS3-NOTE-W0, records-only — no code, no build, no download, no GPU). dots-studio/dots3-note-prev, 280B total / 16B active, model_type: dots3_note. Measured from config.json + the HF API: L=46, H=5120, intermediate_size 13824, vocab 152064, 524288 positions, rope_theta 8e7; MoE 256 routed + 1 shared, top-8, moe_intermediate_size 1536, first_k_dense_replace 1, sigmoid/noaux_tc. Attention is hybrid with TWO geometries: 13 full_attention layers at indices 0,1,5,9,...,45 (128 heads, kv_lora_rank 512, qk_nope 128, DSA indexer index_n_heads 64 / index_head_dim 128 / index_topk 2048) and 33 sliding_attention layers (window 513, 64 heads, swa_kv_lora_rank 1024, swa_qk_nope_head_dim 192, own rope_theta 5e4, is_neox_style=False, no indexer) => latent rows 576 vs 1088, reconciled upstream by padding the physical row (Dots3NotePaddedMLAAttention.get_kv_cache_spec). Vision = MoE ViT, 42 layers, embed_dim 1536, patch 14, spatial_merge_size 2, use_qk_norm, pre_pixel_shuffle, patch-merger adapter 1536->5120, pyramid pyramid_num_routed (dense to layer 24, then 4->64). Audio = encoder_type: "dots" Whisper variant, d_model 1280 / 32 layers / 128 mel / max_source_positions 6000 / swiglu, conv2d stem, RoPE (partial 0.5), RMSNorm, 60 s chunking, adapter 1280->5120. HEAVY REUSE: upstream literally subclasses DeepSeek — Dots3NoteModel(DeepseekV32Model), Dots3NoteDecoderLayer(DeepseekV32DecoderLayer), Dots3NoteMoE(DeepseekV2MoE), Dots3NoteFullAttention(DeepseekV2MLAAttention) — so our gated MLA (deepseek_v2.cpp, SACRED 8/8), DSA indexer (deepseek_v4_dsa.cpp), noaux_tc router, 256-expert grouped MoE, qwen3_vl_vision.cpp (image+video STRICT 32/32) and voxtral.cpp/whisper_audio.cpp (audio 16/16) carry most of it. NET-NEW: windowed MLA (our every MLA path is full-attention — deepseek_v4_registry.cpp sets is_hybrid=false), the padded/heterogeneous MLA KV spec, the headwise sigmoid attention gate + the extra k_rope_only_layernorm over the 64-dim rope-only k slice + the two apply_mla_qkv_lora_rescale scalars, the MoE ViT (all our ViTs are dense), the dots audio stem, and the GGUF converter (llama.cpp has no dots3_note => no quant-matched llama.cpp bar). FOUR SILENT CONFIG TRAPS the checkpoint's config.json does NOT carry, all set in Dots3NoteConfig.__init__:12-25: n_group=1/topk_group=1 (upstream: "Do not inherit DeepSeek-V3's 8-group/4-group router defaults ... A different grouping changes the selected experts at every MoE layer" — our router is gated at V3's grouped dims), indexer_rope_interleave=True (GPT-J adjacent pairs, where V3.2 and therefore our indexer default to split-half NeoX), and num_nextn_predict_layers=1. A fifth, in-JSON: apply_mla_qkv_lora_rescale => sqrt(hidden/lora_rank) after each layernorm. HW-fit: DOES NOT FIT ANY HOST WE OWN. Measured via the HF API: 131 language shards + model-vision.safetensors 13.7 GB + model-audio.safetensors 1.77 GB = ~576 GB bf16; the -fp8 sibling ~290 GB; upstream's own recipe is --tensor-parallel-size 8 on H100s. GB10 is ~119 GiB and Thor ~122 GiB RAM / 123 GiB free disk, so the vLLM oracle cannot run this model here at any published precision and there is no smaller dots3-note in the dots-studio org — the same wall deepseek-v4-flash hit at 156.7 GiB, three times worse. Our own arm is plausible at <=2 bpw (~75-90 GiB, the ds4flash IQ2_XXS pattern) but an arm with no oracle beside it yields NO parity number, so every performance axis is an open gap by construction. Beyond-pin on top of that: gating needs a pin advance, and upstream is still moving (#52172 landed the day before this scope). Thor 192.168.68.23 is the developer-designated e2e CUDA host (2026-08-14) and probed the same day — 14 cores, 122 GB RAM, 123 GiB free, but no nvcc, no cmake, no ninja, no venv, and nvidia-smi refuses under non-interactive ssh — so W0.5 provisions it; it can carry our low-bit arm, the sm_110 axis and every unit gate, but not a 290 GB oracle. DECISION 2026-08-15: option B (developer) — port brick by brick against independent references, ship our own low-bit arm on Thor, and carry the e2e parity gate as an OPEN GAP. Consequence recorded on the row, not just in the spec: no performance number for this model is claimable on any axis while B holds, because there is no denominator; and a brick gated against an in-test reference proves two implementations AGREE, never that either matches vLLM. The owed gate lives in the spec's ## Owed. W0.5 (provision Thor) is in flight and is NOT dots3-specific — Thor is the only non-GB10 CUDA host, and it runs an immutable OS with / READ-ONLY, so the recipe goes in environment.md. W-plan W0-W11 in the spec | CLAIM-MODEL-DOTS3-NOTE-W0 |
MODEL-DIFFUSION-minimax-h3-mini-max-h3-dit | MiniMaxH3DiTModel | BEYOND-PIN AND OUT-OF-REPO — not in 555967922 (H3 released after the pin) and not in the vLLM repository at all: it lives in vllm-project/vllm-omni, vllm_omni/diffusion/models/minimax_h3/ (minimax_h3_transformer.py, packed_sequence.py, packed_tokens.py, scheduling_minimax_h3_euler_ancestral.py, denoise_loop.py, vae.py, encoder.py, pipeline_minimax_h3.py) | diffusion generation / video + audio (text/image/video/audio in) | flow-matching denoise loop; packed varlen NON-CAUSAL attention; AdaLN modulation; 3D MM-RoPE; video VAE + audio VAE (checkpoint REMOTE CODE); Qwen3-VL-derived encoder; MP4 muxing; Ulysses sequence parallelism | minimax-h3 spike | PARTIAL | W0-W2 LANDED (2026-08-03, CLAIM-MINIMAX-H3-W0-W2, DERIVE-AND-SHIP). H3 is NOT an autoregressive LLM: it is a CFG-distilled joint video+audio DIFFUSION transformer (50 blocks, H=5376, 56 MHA heads x 128, SwiGLU 14336, AdaLN 6x3xH, 3D RoPE rotating 96 of 128 dims, video row width 96, audio latent 32) forwarded ONCE PER STEP of a 50-step flow-matching loop — no KV cache, no sampler, no logits, so the SACRED token-exact methodology does not apply. HW VERDICT: e2e is IMPOSSIBLE on this project's hardware — ~354 GB checkpoint (DiT 66.3 GB + Qwen3-VL-derived encoder 51.5 GB + video VAE ~10 GB + audio VAE ~0.6 GB), upstream validates on 4x NVIDIA B300 at ~133 GB peak per rank, vs ONE GB10 with 119 GiB UNIFIED memory; CPU offload does not help because the pool IS host RAM. WHAT IS GATED (and it is exact): upstream's modules are pure Python, so they are executed at REDUCED DIMENSIONS on CPU as the oracle (scripts/gen-minimax-h3-goldens.py imports them by file path and freezes their outputs; both sides rebuild weights/inputs from an identical FNV-1a + splitmix64 stream, so no weight byte is checked in). Results: fl2va + ref2va packed layouts EXACT including the fp64 position grid BIT-EXACT (it feeds RoPE — the port reproduces numpy's linspace(endpoint=False) evaluation order and upstream's deliberately-split pairwise vs sequential span summations, packed_sequence.py:101-113); patchify/unpatchify/audio pack EXACT + round-trip; scheduler EXACT; full DiT forward max abs diff 1.6e-7 (video) / 1.5e-7 (audio) — f32 round-off. REUSE: the packed varlen non-causal attention routes through the SHARED vt::DFlashBlockAttention(causal=false) (its per-document bidirectional contract IS upstream's varlen FA call) and every projection through vt::MatmulBT — NO new kernel was added. NOT-YET-BUILT (honest): device-resident/bf16 forward + fusion folds (W2b), H3-Encoder on our existing Qwen3-VL tower (W3), the two VAEs — which are checkpoint REMOTE CODE under trust_remote_code and must be reimplemented in C++, not adapted (W4/W5) — pipeline/tasks (W6), /v1/videos + MP4 muxing, which needs a NEW dependency decision (W7), and USP multi-GPU (W8). No speed number is claimed; upstream reports the DiT at 88% of request latency. Code: minimax_h3.h, minimax_h3.cpp (MiniMaxH3DitForward), minimax_h3_packing.cpp (BuildMiniMaxH3PackedSequence). Test: test_minimax_h3.cpp (DiT forward parity; 10/10 cases / 2539 assertions, clean CPU build) + generator gen-minimax-h3-goldens.py; ledger. W6A+W9 LANDED (2026-08-03, CLAIM-MINIMAX-H3-W6A-W9) + HW VERDICT CORRECTED. (a) the bf16 PRODUCTION dtype policy now runs (upstream's cast points with the fp32 islands preserved; gated vs a bf16 upstream golden at max abs diff 2.4e-3); (b) request planning — 17n+5 frame snapping, video/audio latent shapes, the rectified-flow time-shift sigma schedule, canvas resolution and t2va/fl2va/ref2va dispatch — ported and EXACT vs time_request.py + pipeline_minimax_h3.py:121-122,207-222,374-434; (c) the ComfyUI-GGUF arm: the name map is the IDENTITY and every one of the 535 real tensors of MiniMax-H3-FL2VA-Q3_K_M.gguf resolves onto our contract, with the geometry derived from SHAPES ALONE equal to the shipped config (gated on a manifest read from the file's own header by range request — no payload downloaded). Two shape rules recorded: GGUF ne is reversed vs torch, and comfy.gguf.orig_shape.<name> overrides it where ComfyUI reshaped a tensor for quant-block alignment (the 50 AdaLN projections: logical [96768, 2688], 2688 not a multiple of the 256-element Q3_K block). ★ HARDWARE VERDICT CORRECTED — the earlier 'e2e is IMPOSSIBLE on this hardware' was WRONG because it reasoned from the bf16 release alone: quantized H3 checkpoints exist and FIT (GGUF DiT Q3_K_M 15.6 GB + Qwen3-VL encoder Q4_K_M 14.6 GB + VAEs ~11 GB ~= 41 GB in a 119 GiB pool; lilcheaty/MiniMax-H3-NVFP4 likewise). So e2e AND a speed comparison are REACHABLE; NVFP4 is the likely speed path (sm_121 native FP4 tensor cores + our tuned NVFP4 stack). W5 LANDED (audio VAE): H3's VAEs are checkpoint REMOTE CODE under trust_remote_code, so a no-Python engine must REIMPLEMENT them — the DAC-lineage BigVGAN audio decoder (weight-norm materialization (w = g*v divided by the row norm), anti-aliased SnakeBeta with kaiser-sinc up/down resampling, replicate padding, final clamp) is ported and gated against the checkpoint's OWN modules at max abs diff 4.2e-9. The VIDEO VAE (klvae.py ~48 KB + CNN/ViT + tiling) is the largest remaining brick. W10 GROUNDED + W4 SCOPED from REAL manifests (safetensors headers captured by range request, no payload downloaded): the NVFP4 checkpoint's 1051 tensors are textbook compressed-tensors NVFP4 (U8 packed 2-per-byte + E4M3 weight_scale at group 16 + F32 scalar weight_scale_2; 258 quantized projections; fp32/bf16 islands left unquantized; names identical to our contract) — i.e. EXACTLY the layout our tuned NVFP4 stack already consumes, so W10 is loader wiring, not a new quant scheme. The video VAE's 560 tensors show its ENCODER is the 3D CNN (rank-5 Conv3d) while its DECODER — the half generation needs — is a 36-block TRANSFORMER (to_qkv/to_out, ff.w1/w2, 2 norms + 2 learned residual scales per block, x_embedder/mask_token/register_tokens/proj_out), materially smaller than klvae.py's 48 KB suggested. W4 BLOCK LANDED: the video-VAE decoder's repeated TransformerBlock is ported and gated at 6.0e-8 against the checkpoint's OWN remote code — RMSNorm + per-head RMS qk-norm (no affine) + full attention + gated-SiLU FF + LEARNED PER-CHANNEL residual scales, and critically the PER-HEAD-INTERLEAVED qkv layout ([head][q,k,v], NOT the DiT's [q_all,k_all,v_all]) that would otherwise produce a plausible-but-wrong image. W4 DECODER DONE: the FULL ViT3D video-VAE decoder — pack, x_embedder, register/cls tokens, 3D RoPE (RotaryEmbeddingND, length-normalized ids, angle scale 2pi, tiled freq blocks), the 36-block stack, LayerNorm norm_out, proj_out and unpatchify — is ported and gated at 8.9e-8 against the checkpoint's own ViT3DDecoder, at its real hyperparameters (36 layers, 32 heads x 64, rope_theta 100, rope_dim_ratio 0.75). BOTH VAE decoders are now done (audio 4.2e-9). W3 TEXT TOWER DONE: the H3-Encoder's truncated Qwen3-VL text tower — gated at 1.2e-7 vs upstream — with all three H3 deltas exercised: layer truncation (min(num_hidden_layers, 50)), the UNNORMALIZED layer-49 output (NO final RMSNorm, unlike stock Qwen3-VL — applying one silently shifts every conditioning vector), and DeepStack visual injection into the first N layers; plus interleaved M-RoPE, fused QKV, per-head q/k RMSNorm, causal GQA and the gated-SiLU MLP. W6 t2va ASSEMBLED — the WHOLE PATH COMPOSES: MiniMaxH3GenerateT2va wires packed layout -> rectified-flow sigma schedules -> the multi-step denoise loop of DiT forwards -> unpatchify + audio unpack -> per-channel denormalize -> BOTH VAE decoders, producing correctly-shaped, finite frames and a stereo waveform in [-1, 1]; gated by a structural end-to-end test at reduced dimensions with random weights (NOT a quality result). Assembling it also caught a real gap: the audio decode needed the checkpoint's dec_in_proj (Conv1d k=1, vae_latent_channels -> num_mels) ahead of BigVGAN. W9 GGUF ARM DONE: LoadMiniMaxH3DitFromGguf materializes the DiT from a ComfyUI-format GGUF — dequantizing every tensor through the SHARED GGUF dequant entry point (so the Q2_K/Q3_K/Q4_K families the H3 GGUFs use are covered by the same code every other GGUF model uses), recovering the geometry from shapes alone, and binding the forward's views with missing tensors throwing BY NAME rather than reading as zeros; gated by a synthetic-file load-AND-RUN test (a real DiT forward executes off the loaded weights). W10 LOADER DONE: LoadMiniMaxH3DitFromNvfp4 materializes the DiT from an NVFP4 compressed-tensors checkpoint — the U8-packed [out, in/2] weight plus its E4M3 group-16 weight_scale and F32 scalar weight_scale_2 go through the project's EXISTING NVFP4 dequant (no new quant code), sidecars are excluded from the model tensor set, and the logical [out, in] shape is recovered; gated by a synthetic-file load-AND-RUN test. BOTH quantized loaders are now done. W3 VISION BLOCK DONE (6.0e-8): the repeated unit of the encoder's Qwen3-VL vision tower, which differs from the text tower in every way that matters numerically — LayerNorm WITH BIAS (not RMSNorm), a [q_all, k_all, v_all] qkv layout (not the video VAE ViT's per-head interleave), fp32 rotary, NON-CAUSAL attention segmented by cu_seqlens (the test asserts a perturbation in one packed image leaves the other's outputs BIT-IDENTICAL), and the TANH-approximate GELU. W3 ENCODER COMPLETE: the FULL vision tower also lands — Conv3d patch embed (kernel == stride, so a linear over the flattened patch), BILINEAR resampling of the learned position grid into spatial-merge order, the 2D rotary table, per-frame cu_seqlens, and both merger flavours (the final merger norms the PRE-shuffle width while the DeepStack mergers norm the POST-shuffle width, and both use exact-erf GELU unlike the block MLP's tanh approximation), gated over a RAGGED two-image batch. Only the MM processor remains on the encoder. CONDITION-NOISE augmentation DONE (fl2va/ref2va): the noised-anchor mix plus its ROW ACCOUNTING — each visual condition draws noise of length target_latent_t + imgvid_cond_num_frames and slices the PREFIX matching its own latent_t, every condition restarts the SAME seed (so concatenating and drawing once would differ for multi-reference requests), and rows advance by that condition's own patchified count. Gated EXACT with the noise supplied, so the comparison isolates the accounting from torch's RNG. REFERENCE-VIDEO geometry + FRAME SCHEDULE DONE (the pure-math half of reference_video.py): the canvas pipeline (aspect clamp -> 768 short edge -> max-pixel rescale -> nearest multiple of 32) and the 24-to-2 FPS frame resample with per-temporal-patch block timestamps, both EXACT. NOTE the rest of that module (probe, transcode, frame extraction, audio decode) shells out to ffmpeg and is blocked on the SAME external dependency decision as /v1/videos MP4 muxing — one decision unlocks reference-video INPUT decode and generated-video OUTPUT encode together. VIDEO VAE TILING DONE: the tile plan (smallest tile count whose MINIMUM overlaps still cover the axis, leftover slack distributed in whole vae_ratio units ROUND-ROBIN across the seams) plus the linear seam cross-fade, both EXACT. Shipped config tile 256 / overlap 64 / vae_ratio 16 (= prod(space_down), the 'f16' in f16t4). Getting the slack distribution wrong shifts every tile after the first and surfaces as seam artifacts rather than an error. PRESENTATION TOKEN TAGS DONE — the fl2va vision-span override the denoise loop requires callers to have applied: a vision block is vision_start + pad*count + vision_end and the WHOLE block, MARKERS INCLUDED, is tagged VIDEO; tagging only the pads would leave two markers as TEXT and shift every AdaLN modulation index after them. Gated EXACT, with the test proving each VIDEO run is a whole vision block. VAE 3D-CNN ENCODER PRIMITIVES DONE: causal Conv3d (all temporal padding on the LEFT so a frame never sees the future, reflect spatial padding), GroupNorm3D (32 groups, eps 1e-6, statistics spanning TIME as well as space) and ResnetBlock3D, gated EXACT — with CAUSALITY proven directly on the bare convolution (a change to the last frame provably cannot reach earlier frames). Downsample3D DONE too: the strided inter-level conv, whose subtlety is the ASYMMETRIC pre-pad — one pixel on the RIGHT of W and the BOTTOM of H before a stride-2 conv with padding (1,0,0); padding symmetrically instead shifts the whole sampling lattice by half a pixel, which is a silent wrong latent rather than an error. Only the EncoderFCN3D level-loop assembly remains on the VAE encoder. VIDEO VAE COMPLETE — encoder AND decoder: the whole 3D-CNN encoder level loop (conv_in -> per level [ResnetBlock3D x N then Downsample3D or a 1x1x1 channel match] -> GroupNorm -> SiLU -> conv_out) is gated EXACT. MM PROCESSOR = REUSE, gated: H3's FL2VA/processor is a stock Qwen3VLProcessor, so the multimodal front end is this project's EXISTING Qwen3-VL processor rather than a new port; H3's own config is parsed and driven through it (patch 16 / temporal 2 / merge 2, 0.5 normalization rather than CLIP statistics, a 32-pixel grid, the 768x1344 default canvas proven an IDENTITY under smart_resize, and VIDEO bounds deliberately looser than the image ones). With this every PORTABLE piece of the lane is done; what remains is one dependency decision (ffmpeg media I/O + MP4 muxing) and GPU-blocked work (the device-resident FP4 forward and any speed number). WAV OUTPUT added: the decoded stereo waveform serializes to RIFF/WAVE 16-bit PCM, converting the VAE's CHANNEL-MAJOR layout to INTERLEAVED (getting that backwards yields audio that plays but with the channels time-smeared) and clamping rather than wrapping. Deliberately dependency-free, and required under EITHER outcome of the open MP4/muxer decision. VIDEO OUTPUT PATH DONE: PPM frame serialization (planar [C,T,H,W] -> row-major interleaved RGB, [-1,1] -> [0,255] clamped) plus the MP4 mux argv (h264/yuv420p + AAC, -shortest, +faststart). The built argv was RUN through real ffmpeg 6.1.1 and produced a VALID MP4 (ffprobe: h264 yuv420p video + AAC stereo at 32 kHz). The library never spawns a process — src/vllm/ has no subprocess precedent — so it builds the artifacts and the command while the example/server layer invokes it. /v1/videos API LOGIC DONE: the request contract (H3 defaults — 50 steps, flow shift 12 video / 3 audio; both the vLLM-Omni extra_params nesting and a flat top-level spelling accepted; malformed input rejected with a reason rather than silently defaulted) and the job store (queued -> running -> succeeded/failed, illegal transitions throw, unknown ids reported so the route can 404, status JSON omits fields that do not apply, and concurrent creation is thread-safe). Remaining is mechanical glue: route registration + runner injection, with the ffmpeg call in examples/ per the ratified boundary. Test: 34/34 + video-api 4/4 (9233 + 63 assertions). OPEN: there is no vllm-omni parity PIN — the upstream-sync protocol covers only the vLLM repo | CLAIM-MINIMAX-H3-W0-W2, CLAIM-MINIMAX-H3-W6A-W9 |
MODEL-MM-moss-tts-moss-tts-delay-talker-for-generation | MossTTSDelayModel | OUT-OF-REPO — not in 555967922 and not in the vLLM repository at all (absent from every dictionary of registry.py on main as well): it is registered by the separate vllm-project/vllm-omni repository, vllm_omni/model_executor/models/registry.py:350-354 @ bbe6ccc512a404a2df8c977ea29003002f2683e8 → module vllm_omni/model_executor/models/moss_tts/modeling_moss_tts_talker.py, class MossTTSDelayTalkerForGeneration, with the shared stage-1 codec decoder at registry.py:368-372 → module moss_tts/modeling_moss_tts_codec.py, class MossTTSCodecDecoder. Module and class are given as separate fields rather than in the compact colon-pair form, as the MiniMaxH3DiTModel row does: that form marks a target inside the PINNED vLLM registry inventory, and this target is in another repository entirely. Documented at vllm-omni docs/models/supported_models.md:78. Official recipes: OpenMOSS-Team/MOSS-TTS, MOSS-TTSD-v1.0, MOSS-SoundEffect, MOSS-VoiceGenerator — four of the 157 recipes at vllm-project/recipes 86c7777a, all four declaring architectures: ["MossTTSDelayModel"]. Tracked by #610 | conditional generation / text+audio in, AUDIO OUT (TTS) | MM processor; audio detokenizer / codec decoder; delay-pattern multi-codebook decoding; reference-audio speaker encoder; streaming audio output. We serve no audio-generation output modality at all today | ☐ required | INVENTORIED | none. There is no vllm-omni parity PIN — the upstream-sync protocol covers only the vLLM repository — so an oracle for this row has to be established before it can advance | unassigned |
MODEL-MM-moss-tts-moss-tts-realtime-talker-for-generation | MossTTSRealtime | OUT-OF-REPO — not in 555967922, and absent from registry.py on vLLM main; registered by vllm-project/vllm-omni at vllm_omni/model_executor/models/registry.py:356-360 @ bbe6ccc512a404a2df8c977ea29003002f2683e8 → module vllm_omni/model_executor/models/moss_tts/modeling_moss_tts_talker.py, class MossTTSRealtimeTalkerForGeneration (separate module and class fields rather than the compact colon-pair form, per the out-of-repo convention) — the SAME module as MossTTSDelayModel but a different class, so the two are separate targets rather than aliases. Documented at vllm-omni docs/models/supported_models.md:79. Official recipe: OpenMOSS-Team/MOSS-TTS-Realtime (1.7B). Tracked by #610 | conditional generation / text+audio in, AUDIO OUT (realtime/streaming TTS) | MM processor; audio detokenizer / codec decoder; realtime full-duplex streaming; incremental audio decode | ☐ required | INVENTORIED | none. Same missing-vllm-omni-pin blocker as the other omni rows | unassigned |
MODEL-MM-indextts2-index-tts2-talker-for-conditional-generation | IndexTTS2TalkerForConditionalGeneration | OUT-OF-REPO — not in 555967922 and not in the vLLM repository at all: registered by the separate vllm-project/vllm-omni at vllm_omni/model_executor/models/registry.py @ bbe6ccc512a404a2df8c977ea29003002f2683e8 (separate module and class fields rather than the compact colon-pair form, per the out-of-repo convention) → module vllm_omni/model_executor/models/indextts2/indextts2_talker.py, class IndexTTS2TalkerForConditionalGeneration. Deploy config vllm_omni/deploy/indextts2_5.yaml (selected by model_type); offline entry point indextts2.end2end; served as vllm-omni serve IndexTeam/IndexTTS-2.5 --omni --trust-remote-code over /v1/audio/speech + /v1/audio/voices. Official recipe: IndexTeam/IndexTTS-2.5 (~0.8B talker, ~6 GB VRAM). Tracked by #634 | conditional generation / text + REFERENCE AUDIO in, mel codes out (stage 0 of a 2-stage TTS pipeline) | GPT-2 backbone (opt.cpp is the same family: learned absolute positions + LayerNorm); reference-audio conditioning via w2v-bert-2.0 + MaskGCT semantic codec + CAMPPlus speaker embedding — MANDATORY, since upstream states IndexTTS-2 does not support text-only synthesis | indextts-2-5 spec | INVENTORIED | none. Blocked on the UNPINNED vllm-omni oracle (.agents/oracles/vllm-omni.md, pin = UNPINNED / gateable = no / evidence = #633), which vLLM-Omni compounds by requiring vLLM 0.27.0+ against our 0.26.0.dev0 parity pin | unassigned |
MODEL-MM-indextts2-index-tts2-s2-mel-decoder | IndexTTS2S2MelDecoder | OUT-OF-REPO — not in 555967922 and not in the vLLM repository at all: registered by the separate vllm-project/vllm-omni at vllm_omni/model_executor/models/registry.py @ bbe6ccc512a404a2df8c977ea29003002f2683e8 (separate module and class fields rather than the compact colon-pair form, per the out-of-repo convention) → module vllm_omni/model_executor/models/indextts2/indextts2_s2mel_decoder.py, class IndexTTS2S2MelDecoder. The stage-1 half of the same recipe as the talker row; 2.5 replaces IndexTTS-2's RepCodec with EnhancedCodec and sets use_gpt_latent=false. Tracked by #634 | mel codes in, 22.05 kHz mono WAV out (stage 1) | EnhancedCodec; S2Mel CFM/DiT flow-matching decoder; BigVGAN vocoder. REUSE: the BigVGAN 1-D core (Conv1d/ConvTranspose1d, pads, Snake/SnakeBeta, alias-free Activation1d) is ALREADY shared by H3 and LTX-2.5, published from minimax_h3.h and gated by both suites (ltx2_audio_vae.cpp:223-230 explains why it was not copied); this row would be the THIRD consumer, so the work is relocating that core to a neutral home, not generalizing it. WAV at minimax_h3_wav.cpp; denoise loop + AdaLN from the H3 lane | indextts-2-5 spec | INVENTORIED | none. Blocked on the UNPINNED vllm-omni oracle (.agents/oracles/vllm-omni.md, pin = UNPINNED / gateable = no / evidence = #633), which vLLM-Omni compounds by requiring vLLM 0.27.0+ against our 0.26.0.dev0 parity pin. NOTE the shared vocoder core is gated at H3's and LTX-2.5's hyperparameters, not this model's, so it is re-gated here: a gate passing because both arms call the same helper proves consistency, not correctness | unassigned |
MODEL-MM-qwen3-tts-qwen3-tts-talker-for-conditional-generation | Qwen3TTSForConditionalGeneration, Qwen3TTSTalkerForConditionalGeneration | OUT-OF-REPO — not in 555967922, and absent from registry.py on vLLM main; registered by vllm-project/vllm-omni at vllm_omni/model_executor/models/registry.py:152-156 (and the explicit Qwen3TTSTalkerForConditionalGeneration spelling at :157) @ bbe6ccc512a404a2df8c977ea29003002f2683e8 → module vllm_omni/model_executor/models/qwen3_tts/qwen3_tts_talker.py, class Qwen3TTSTalkerForConditionalGeneration (separate module and class fields rather than the compact colon-pair form, per the out-of-repo convention), with the code2wav stage at registry.py:162-166 → module qwen3_tts/qwen3_tts_code2wav.py, class Qwen3TTSCode2Wav and 12 Hz / 25 Hz tokenizers under qwen3_tts/tokenizer_12hz/ and tokenizer_25hz/. Documented at vllm-omni docs/models/supported_models.md:72-74. Official recipe: Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice (published at recipes.vllm.ai). The two architecture strings resolve to the identical (module, class) target so they are grouped on one row, per the alias rule | conditional generation / text in, AUDIO OUT (TTS, custom voice) | MM processor; audio detokenizer / code2wav; speaker-embedding conditioning; word timestamps; streaming audio output | ☐ required | INVENTORIED | none. Its text tower is Qwen3-derived, which is the one piece of this row we already ship; everything downstream of the talker (code2wav, audio tokenizer, speaker embeddings) is unported | unassigned |
MODEL-MM-higgs-audio-v3-higgs-audio-v3-talker-for-conditional-generation | HiggsMultimodalQwen3ForConditionalGeneration, HiggsAudioV3TalkerForConditionalGeneration | OUT-OF-REPO — not in 555967922, and absent from registry.py on vLLM main; registered by vllm-project/vllm-omni at vllm_omni/model_executor/models/registry.py:213-217 (and the explicit HiggsAudioV3TalkerForConditionalGeneration spelling at :218) @ bbe6ccc512a404a2df8c977ea29003002f2683e8 → module vllm_omni/model_executor/models/higgs_audio_v3/higgs_audio_v3_talker.py, class HiggsAudioV3TalkerForConditionalGeneration (separate module and class fields rather than the compact colon-pair form, per the out-of-repo convention), with code2wav at registry.py:223-227 → module higgs_audio_v3/higgs_audio_v3_code2wav.py, class HiggsAudioV3Code2WavForConditionalGeneration. Documented at vllm-omni docs/models/supported_models.md:81. Official recipe: bosonai/higgs-audio-v3-tts-4b (published at recipes.vllm.ai), whose config.json declares architectures: ["HiggsMultimodalQwen3ForConditionalGeneration"]. The distinct v2 lineage (HiggsAudioV2ForConditionalGeneration, higgs_audio_v2/) is a DIFFERENT target and has no recipe in the audited set, so it gets no row here | conditional generation / text+audio in, AUDIO OUT (TTS) | MM processor; audio detokenizer / code2wav; audio tokenizer; Qwen3-derived multimodal decoder; streaming audio output | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-voxtral-realtime-voxtral-realtime-for-conditional-generation | VoxtralRealtimeForConditionalGeneration (target-pending) | TARGET-PENDING — the architecture string is registered NOWHERE we can find. Recipe mistralai/Voxtral-Mini-4B-Realtime-2602 (one of the 157 recipes at vllm-project/recipes 86c7777a) declares architectures: ["VoxtralRealtimeForConditionalGeneration"] with an inner audio_config.model_type: "voxtral_realtime_encoder", but that exact string appears in none of: (a) the pinned registry 555967922; (b) any of the ten dictionaries of vllm/model_executor/models/registry.py on vLLM main; (c) _OMNI_MODELS in vllm_omni/model_executor/models/registry.py @ bbe6ccc512a404a2df8c977ea29003002f2683e8; (d) vllm-omni docs/models/supported_models.md. Related but NOT equal, recorded so nobody re-derives it: VoxtralRealtimeGeneration — a DIFFERENT string — is registered at the pin (registry.py:585, main :603) and already has its own row here, MODEL-MM-voxtral-realtime-voxtral-realtime-generation; VoxtralForConditionalGeneration (pin registry.py:584) is the audio→TEXT model; and vllm-omni registry.py:305-309 registers VoxtralTTSForConditionalGeneration for the SEPARATE mistralai/Voxtral-4B-TTS-2603 recipe. The most likely reading is that this checkpoint's config.json names a renamed or not-yet-landed spelling of the existing VoxtralRealtimeGeneration target, but that is a hypothesis: no anchor is asserted until one of those is shown to be what actually resolves. Tracked by #610 | conditional generation / audio in, AUDIO OUT (realtime speech-to-speech) — modality inferred from the recipe and config, NOT from an upstream implementation | unresolved until the target is located | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-bailing-mm-native-bailing-mm-native-for-conditional-generation | BailingMMNativeForConditionalGeneration (target-pending) | TARGET-PENDING — the architecture string is registered NOWHERE we can find. Recipe inclusionAI/Ming-omni-tts-0.5B (one of the 157 recipes at vllm-project/recipes 86c7777a) declares architectures: ["BailingMMNativeForConditionalGeneration"], but that exact string appears in none of: (a) the pinned registry 555967922; (b) any dictionary of vllm/model_executor/models/registry.py on vLLM main; (c) _OMNI_MODELS in vllm_omni/model_executor/models/registry.py @ bbe6ccc512a404a2df8c977ea29003002f2683e8; (d) vllm-omni docs/models/supported_models.md. Related but NOT equal: vllm-omni docs/models/supported_models.md:75 lists that SAME checkpoint under a different architecture name, MingTTSForConditionalGeneration (registry.py:250-254 → module ming_tts/ming_tts.py, class MingTTSForConditionalGeneration); and vllm-omni registry.py:417-421 carries a near-miss alias BailingMM2NativeForConditionalGeneration — note the 2 — commented "HF repo currently ships this architecture name in config.json", pointing at module ming_flash_omni/ming_flash_omni.py, class MingFlashOmniForConditionalGeneration, for the DIFFERENT Ming-flash-omni-2.0 checkpoint. So the shape of the gap is a missing config-string alias upstream, but that is an observation, not an anchor, and none is asserted here. Tracked by #610 | conditional generation / text+image+audio in, AUDIO OUT (TTS) — modality from the recipe and config, NOT from a resolved upstream target | unresolved until the target is located | ☐ required | INVENTORIED | none | unassigned |
MODEL-DIFFUSION-ltx-2-5-ltx2-video-transformer-3d-model | LTX2VideoTransformer3DModel | BEYOND-PIN AND OUT-OF-REPO — not in 555967922 (LTX-2.5 released 2026-08) and not in the vLLM repository at all. Architecture reference is Lightricks' own LTX-2 (packages/ltx-core/src/ltx_core/: model/transformer/{transformer,model,attention,rope,model_configurator}.py, model/audio_vae/, components/, guidance/). vLLM-Omni carries an ltx2 module but its recipes stop at 2.3 (vllm_omni/diffusion/models/ltx2/ltx2_recipes.py:162-166); 2.5 is upstream-OPEN at vllm-omni#6066 and #4985. Serving oracle reaches 2.5 through the GENERIC DiffusersAdapterPipeline (pipeline_diffusers_adapter.py:116) against Lightricks/LTX-2.5-Diffusers. | diffusion generation / video + audio (text/image/video in, video+audio out) | flow-matching denoise loop; TWO coupled modality streams with audio<->video cross-attention; per-head gated attention; AdaLN-single with 9-vector per-block tables; timestep-CONDITIONED prompt K/V via prompt_adaln_single (model.py:223-227, transformer.py:441-446), so the K/V cache is inapplicable on the shipped DiT -- the row's first claim of cacheability was RETRACTED 2026-08-12 and the module PORTED 2026-08-13 (#644, spec); on the real weights the timestep term DOMINATES the static table, RMS ratio 1347% video / 1583% audio; split/interleaved 3D RoPE; gelu-approximate FFN; Gemma-4 12B text encoder with dual caption projections (4096 video / 2048 audio); Conv video VAE + DiffVAE; audio VAE + vocoder; latent spatial/temporal x2 upsamplers; duration head; distilled two-stage recipe | ltx-2.5 | SPIKE | L0 LANDED 2026-08-11 (#435). Spec committed BEFORE implementation per AGENTS.md. Geometry measured from the ungated vonkaiser/LTX-2.5-FP8-NVFP4 FP8 checkpoint header by range request — the same no-payload technique used for H3's manifests: 21.00B params, 48 blocks, 1775 F8_E4M3 tensors each with a per-tensor F32 weight_scale, biases/norms BF16. Cross-modal projection shapes are ASYMMETRIC and are a named test trap: audio_to_video_attn.to_q is [2048, 4096] while to_k/to_v are [2048, 2048] and to_out is [4096, 2048], so a transposition still type-checks against a square assumption. CHECKPOINT ACCESS (verified against the HF API 2026-08-11): Lightricks/LTX-2.5 is gated: auto (accepting the license opens it; holds the first-party NVFP4 DiT at 18.72 GB), Lightricks/LTX-2.5-Diffusers is RESTRICTED behind manual approval (needed for the binding oracle), and vonkaiser/LTX-2.5-FP8-NVFP4 is UNGATED and unblocks L1-L2 today. All artifacts land under $CHECKPOINT_ROOT (/mnt/nas_share/checkpoints) so dgx.casa and the cluster nodes share one copy. PHASES (one PR, developer-directed): L1 vllm::multimodal::VideoEngine interface + checkpoint-detected registry with MiniMax-H3 moved behind it UNCHANGED (gated on frames+WAV byte-identical to the pre-refactor fold fixture) and ABI v13 by ADDING fields only; L2 DiT forward; L3 Gemma-4 TE reusing gemma4.cpp; L4 Conv video VAE + audio VAE + vocoder; L5 pipeline/recipes/upsampler/duration head; L6 NVFP4 arms + GB10 load-time residency; L7 e2e on dgx.casa under flock $HOME/gpu.lock. OPEN, same as H3: there is no vllm-omni parity PIN — .agents/upstream-sync.md covers the vLLM repo only, so each golden records the vllm-omni SHA inline. | CLAIM-LTX25-L0-L7 |
MODEL-MUSIC-minimax-music3-mini-max-music3-for-conditional-generation | MiniMaxMusic3ForConditionalGeneration | BEYOND-PIN AND OUT-OF-REPO — not in 555967922 and not in the vLLM repository at all; absent from vllm-omni too. Its reference implementation is the OPEN diffusers PR #14456 head c6da9936 (src/diffusers/models/transformers/transformer_minimax_music3.py, .../minimax_music3_rvq_depth_decoder.py, src/diffusers/models/autoencoders/minimax_music3_vocoder.py, src/diffusers/models/condition_embedders/condition_embedder_minimax_music3.py, src/diffusers/modular_pipelines/minimax_music3/), with SGLang-Omni 748a0b43 (sglang_omni/models/minimax_music3/) serving the NATIVE layout as the e2e cross-check. The first row to exercise the fallback-oracle rule (#647) | music generation / text -> audio (lyrics + structured description in, multi-minute stereo song out) | Qwen3 dense forward + paged KV (LANDED); flow-matching denoise loop (H3/LTX-2.5 precedent); DAC-style audio VAE decode + WAV writer (H3/LTX-2 precedent); NEW: 8-codebook RVQ frame path, 4-layer depth decoder, learned 8-layer condition mix, snake activations with weight-norm folding, and an LLM->diffusion handoff on CONTINUOUS hidden states rather than discrete tokens | minimax-music3 | ACTIVE | W0 DONE + W1 LANDED (#672). Oracle GATEABLE: tools/oracle/music3_oracle.py generated 0.998 s of 44100 Hz stereo at diffusers c6da9936 (identity asserted at runtime, proved red first), with 13 per-stage goldens under tests/parity/goldens/minimax_music3_oracle/ (#708). W1 modular loader minimax_music3_loader.{h,cpp}: 1413/1413 assertions against the real 27 GB tree, all 1012 tensors accounted, 50 mutations proven to fire, native arm refused by name; MaterializeWeightNorm relocated into the shared vocoder1d primitives (H3 79/79 cases, 57,395 assertions unchanged) rather than forked (#714). Spec committed BEFORE implementation per AGENTS.md. Geometry MEASURED, not inferred: every component config.json read, plus safetensors headers by HTTP range request — transformer 231 tensors/1.240B per shard all F32 (so the card's 2.4B is right and 9.73 GB is fp32 storage, NOT a 4.9B bf16 model), rvq_depth_decoder 47/0.646B BF16, vocoder 121/0.054B F32 with weight_g/weight_v weight-norm, condition_encoder 4 tensors (layer_scale, layer_weight_logits, proj) which makes it a learned weighted mix over 8 LLM layers rather than an encoder tower. language_model is our LANDED Qwen3ForCausalLM architecture exactly (36L/4096/32-8/head_dim 128) at vocab_size 200000. The repo ships TWO packagings of ONE set of weights (native qwen_7B/+flowmatching_vae.pth+dav.pth vs the diffusers six); scripts/convert_minimax_music3_to_diffusers.py@c6da9936 :29-38,47-189 proves the conversion is a pure re-layout, so the diffusers arm is the port target at ~28.5 GB resident (no pickle, no trust_remote_code) and SGLang-Omni stays a valid e2e cross-check. Sample rate RESOLVED 2026-08-13 (spec §1.1): not a contradiction but a stage boundary. The vocoder natively emits 44100 stereo (86.133 Hz latent frames x 512 upsampling, derived from minimax_music3_vocoder.py:84,92-95 and confirmed by dav.py:94,115); diffusers returns that unresampled, while SGLang-Omni's SERVER resamples 44100 -> 32000 on delivery (constants.py:18-19, acoustic.py:55-58,422-431). Goldens are captured at 44100 stereo, the native resample-free rate; 32 kHz is a downstream delivery transform gated separately, and NOT free — reproducing its bytes needs torchaudio's default sinc filter, and a latent-parity check sits upstream of that call and cannot see it. W2/W3 landed 2026-08-14 (#672): the autoregressive half's compute — prompt assembly, the semantic CFG logit pipeline, the learned 8-layer condition mix and the 4-layer RVQ depth decoder — gated at reduced dimensions in float32 against upstream's own executed classes (25 cases / 338 assertions, no checkpoint) AND at full scale in bf16 against the committed oracle capture (condition mix 175 989 of 176 128 values BIT-IDENTICAL; depth decoder 716 800 values over 25 frames x 7 depth steps). Spec §5's token-exact claim is WITHDRAWN and the reason is measured: upstream's AR stage has NO greedy path (_sample_top_k ends in torch.multinomial against a seeded generator, encoders.py:94-103) and both stages sample from a CFG mix whose unconditional row the goldens do not store, so rvq_codes.npy is a seeded sample consumed as an INPUT. The full-scale bound is calibrated against a MATCHED CONTROL — torch's own sdpa_kernel(MATH) arm reproduces the goldens to 46.34% bit-identical at mean absolute error 1.659e-03, ours 43.61% / 1.824e-03 — because torch's CPU attention kernel runs a blocked online softmax no closed-form rounding model reproduced (its bf16 Linear reproduces 32 759 of 32 768). W4/W5 landed 2026-08-14 (#672): the ACOUSTIC half — minimax_music3_acoustic.{h,cpp} — carrying the 2.4B fp32 flow-matching DiT (partial rotary over 32 of 64 head dims, a prepended timestep token, two RESIDUAL 1x1 convolutions, a gated FFN whose FIRST half is the value), the FlowMatchEulerDiscreteScheduler with invert_sigmas (shift -> invert -> train-scale, in float32 because the shift3 golden can tell float32 from double), ClassifierFreeGuidance at scale 1.7 based on the UNCONDITIONAL row, the denoise loop's 200-frame/100-hop window bookkeeping, and the DAC Flow-VAE vocoder routed through the shared vocoder1d primitives with AliasFreeActivation1d explicitly NOT applied (Music3 uses plain snake). Gated at reduced dimensions in float32 against upstream's own EXECUTED classes (27 cases / 265 assertions, no checkpoint; 16 of 18 mutations fire) AND at full scale against the committed capture: the Euler step reproduces denoise_{first,last}_latents_out BIT-EXACTLY (22 016 of 22 016), denoise_last_latents_out is bit-identical to vocoder_input_chunk0, and the vocoder reproduces waveform.npy over 88 064 samples with 0 outside tolerance (mean abs-diff 3.19e-08). Bounds calibrated against a MEASURED control — upstream's own modules at torch.set_num_threads(1) against goldens captured at 20 threads reproduce to 1.911 % bit-identical (vocoder) and 15.416 %/5.596 % (DiT first/last), so the ABSOLUTE floor binds and the relative one does not (the control's own max relative deviation is 7.4e-02 on near-zero samples). Three findings: W1's header documented folded == 20 where the checkpoint yields 30 (corrected in the same change); a 1e-5 relative tolerance HIDES upstream's (1 - 1e-6) overlap-blend factor, so that assertion is now bit-exact; and CFG at scale 1 does not recover the conditional row bit-for-bit in float32, so the exact discriminator is scale 0 against the unconditional row. W6 landed 2026-08-14 (#672): the model reaches the SHARED SURFACE. minimax_music3_speech.{h,cpp} registers it as a SpeechRegistry family named minimax-music3 whose detector INSPECTS the artifact (modular_model_index.json must name MiniMaxMusic3ModularPipeline AND all seven component directories must exist), declares 44100 Hz stereo and requires_reference_audio() == false, and composes upstream's four modular-pipeline blocks that nothing had composed: before_denoise.py -> Music3ChunkPlan, denoise.py -> Music3DenoiseChunks, decoders.py -> Music3DecodeChunks. multimodal::SpeechGenParams grew ADDITIVELY by lyrics, description, audio_duration_s, num_inference_steps and guidance_scale — two texts because upstream normalizes lyrics and description differently (encoders.py:54-91), so one text behind a separator would be a private protocol inside a shared struct — and IndexTTS-2.5 is BYTE-IDENTICAL (0 lines changed in indextts2.{h,cpp} or speech_engine.cpp; its 4 cases / 8 assertions and the seam's 7 cases / 20 assertions unchanged). The new C-ABI surface is v20, not v19: origin/main took v19 for the multimodal input limits (#607 L2) while this was in flight, so the number was renumbered rather than contested. POST /v1/audio/speech is OpenAI's createSpeech shape with the two music inputs as additional named fields; it is registered ONLY when a synthesizer is attached, gated over a REAL SOCKET in both directions (absent => 404 from the route table with no envelope leaked; present => audio/wav bytes at 44100 Hz stereo), and voice, speed, streaming and any non-wav response_format are refused BY NAME rather than ignored. The requires_reference_audio() refusal happens BEFORE the runner is called, which is the reason that method exists. Gated full-scale against the capture: the vocoder + crop + clamp + stereo split reproduce waveform.npy over 88 064 values with 0 outside W5's own bound, the WAV payload is BIT-EXACT against the quantization of that golden (88 064 int16 samples, 0 mismatched), and the WHOLE TAIL — condition mix -> 4 guided 2.4B DiT steps -> vocoder, driven from frame_hiddens.npy and the capture's own denoise_first_sample_in.npy — lands at max abs-diff 4.523e-06 / mean abs-diff 1.225e-07 on the waveform and max abs-diff 2.396e-05 on the latents, exactly where the DiT's measured per-step error propagated over four Euler steps says it should. No request's waveform can ever equal the golden, twice over: the AR codes are a seeded torch.multinomial draw and the denoise loop's initial latents are a seeded randn_tensor, so Music3NoiseSource is a PARAMETER and the gate supplies the capture's own noise — that is the only entry at which this loop is comparable at all. Formerly owed on W2, now LANDED (this PR). The dense path grew Qwen3DenseModel::ForwardEmbeds - the inputs_embeds door upstream's own Qwen3Model.forward has and that only the DENSE registration had never wired, already present on qwen3_vl.h:145,159, gemma4.h:210-218 and muse_glimmer.h:369-380. It is needed because _embed_audio_frame (encoders.py:106-115) is a SUM of one embedding row and seven depth-decoder rows scaled by num_codebooks^-0.5, a continuous vector no token id spells. ADDITIVE and proved so: the embedding OF THE SAME IDS reproduces Forward bit for bit in the logits AND in the paged KV, which matters because five registrations ride that one forward. num_condition_layers: 8 is NOT eight transformer layers - the eight rows are cat(last_hidden, depth_hidden_1..7) (encoders.py:343), so no per-layer capture is needed and none is taken. Gated: 102 400 values, 9337 bit-identical, mean abs-diff 1.763e-02, inside a MATCHED CONTROL (12 036 / 1.475e-02) and with a NEGATIVE CONTROL 46x outside; the oracle's own codes rank 2.48 where chance is 8191.5. End to end: OBSERVED TO PASS 2026-08-15 (#852) - POST /v1/audio/speech returns a real 44100 Hz stereo WAV, 3072 samples/channel, 21 assertions in 7:54; the whole file is 5 cases / 535 assertions in 31:14. The gate had never completed because its body spelled the duration audio_duration_s (the FIELD name) where the parser reads audio_duration, so it silently ran the family's 60 s default: a ~750x job it would then have failed on its own payload length. Fixed, and the near-miss key REFUSED (#925). The LoadQwen3ForCausalLMWeights stall recorded here was NOT REAL - the load is 180 s of I/O in both binaries; open shard fds are a borrow artifact, not a program counter. Previously owed: the 8.6B Qwen3ForCausalLM forward teacher-forced on the golden codes, which needs an inputs_embeds entry the landed Qwen3 dense path does not have — it is REFUSED BY NAME from Synthesize, so a real request gets that refusal rather than silence. W7 landed 2026-08-14 (#672): ONE arm implemented and value-gated, the rest refused BY NAME. The survey (spec §9.1, every query with its result count) found MiniMaxAI ships bf16/fp32 ONLY while the community published 14 quantized repositories in 5 formats within days. NOT found by those queries: AWQ, GPTQ, compressed-tensors, fp8 or bitsandbytes. Ten published GGUF headers were then MEASURED by HTTP range request (56 MiB, no weight bytes), which is what turned a repo list into a contract and produced the finding that "the GGUF arm" is THREE MUTUALLY INCOMPATIBLE LINEAGES: the ComfyUI lineage ships the DiT + condition encoder ONLY so it can never generate audio; mm3 (scragnog) needs a rename table PLUS fused QKV to split and folded weight-norm to invert; comfy.gguf.orig_shape.*, which the H3 GGUF arm depends on, is absent from all ten. The IMPLEMENTED arm is the audio-cpp native lineage's rvq_depth_decoder_q4_k.gguf — chosen because it is the ONLY component whose bound can be DERIVED today (W3 already gates that tensor full-scale and its bf16 control is already measured), and because audiocpp.tensor_name_format = native means all 47 names bind to EnumerateMiniMaxMusic3RvqDepthDecoderTensors EXACTLY, so no rename table can hide a mis-binding. Everything else — the other 4 components, the other 2 lineages, NVFP4, MXFP4, FP8, INT8, AWQ/GPTQ, bitsandbytes, MLX and an UNIDENTIFIED case naming all 3 candidate schemes rather than picking one — is refused with the format, the evidence, the missing piece, the supported arm, the phase and the issue. Before it, a GGUF tree was told it was "missing transformer, condition_encoder, ..." (seven directories its owner does not have) and a real NVFP4 condition_encoder refused on layer_scale, a tensor that is not quantized, is not wrong, and merely sorts first. Gated: 29 cases / 125 assertions with NO checkpoint and no network, plus 6 cases / 319 assertions against the pinned artifact. RED captured first at 8 of 8 probe checks. 18 mutations, all 18 fire, and three are recorded rather than counted — one STAYED GREEN (hardcoding matched = 1 passed every case because each carried exactly one marker, so the count was reported but never DISCRIMINATED; a 36-marker case closed it), one was a BAD MUTATION (deleting the config hook's only caller tripped -Werror unused-function, so the COMPILER refused it and the gate never spoke — a build failure is not a red gate), and QM4 (neutering the GGUF lineage guard) exposed a real COVERAGE HOLE: the suite's only GGUF was the accepted lineage, so nothing proved the guard rejected the other two — closed with synthetic in-memory GGUF fixtures, after which it fires. NO speed number is claimed and none was measured. The negatives are gated too, because a detector that fired on the shipped checkpoint would refuse every real load: the bf16/fp32 dtypes, the real transformer config, a null quantization_config, and the vocoder's 30 legacy weight_g/weight_v weight-norm pairs, which are a PARAMETERIZATION and not a quantization. DEVICE ARM LANDED (#672, this PR) — PARTIAL, and the split is the point. SpeechModelParams grew device (0 = CPU, 1 = the accelerator this build resolves; the VideoModelParams::device polarity, NOT vllm_model_params.device's 0=auto, because every Music3 gate was taken on the CPU arm and auto would move every zero-filled caller off it), resolved through a NEW SHARED multimodal::SpeechEngineDeviceType rather than a third copy of a mapping the tree already carried twice (minimax_h3_video.cpp:255, ltx2_video.cpp:706). The engine's vt::Queue was a COMPILE-TIME CONSTANT (minimax_music3_speech.cpp:492, whose own comment said "a device arm is a queue, not a fork"); it is now that queue, built once in the constructor BEFORE any weight is read (the unified-memory recipe). Music3LmSession allocates its 36-layer paged KV on the queue's device, because dense_attn::KvSlice labels the pointer with d.q.device and a host std::vector handed to a CUDA forward is a host pointer wearing a device tensor's label. C ABI v21: vllm_speech_model_params.device + vllm_speech_engine_device, which reports what was GRANTED not what was requested; --speech-device on the server, --device on minimax-music3-gen. WHAT MOVES is the 8.6B Qwen3ForCausalLM half and nothing else: it already rides the shared Qwen3DenseModel::ForwardEmbeds five text registrations use, so no model was forked. WHAT DOES NOT, named rather than implied: the 0.646B RVQ depth decoder and the 2.4B fp32 DiT are host std::vector<float> scalar loops under -ffp-contract=off (kept reproducible against torch by W2-W5) and are OWED; the DAC Flow-VAE vocoder needs ConvTranspose1d, and vt HAS NO SUCH OP — vt::Conv2d and vt::DepthwiseConv1d exist but are registered for the CPU only and neither expresses a transposed convolution — so it is recorded as owed rather than hand-rolled outside the seam. The CPU arm is BIT-IDENTICAL: it takes the same host code with the same std::vector KV, and every Music3 gate is unchanged. | CLAIM-MODEL-MUSIC3-W0 |
MODEL-MM-moss-transcribe-diarize-moss-transcribe-diarize-for-conditional-generation | MossTranscribeDiarizeForConditionalGeneration (v0.25.0 target-pending) | v0.25.0 target registry.py:450-453; vllm/model_executor/models/moss_transcribe_diarize.py::MossTranscribeDiarizeForConditionalGeneration @ 702f481 | conditional generation / audio | MM processor; Whisper encoder; VQ adaptor; Qwen3 decoder; speech-to-text/diarization frontend | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-lightonocr-light-on-ocrfor-conditional-generation | LightOnOCRForConditionalGeneration | registry.py:450-453; vllm/model_executor/models/lightonocr.py::LightOnOCRForConditionalGeneration | conditional generation / image | MM processor; encoder/merge; vision encoder | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-lfm2-vl-lfm2-vlfor-conditional-generation | Lfm2VlForConditionalGeneration | registry.py:454; vllm/model_executor/models/lfm2_vl.py::Lfm2VLForConditionalGeneration | conditional generation / image | MM processor; encoder/merge; Mamba/SSM state; vision encoder; video path | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-mllama4-llama4-for-conditional-generation | Llama4ForConditionalGeneration | registry.py:455; vllm/model_executor/models/mllama4.py::Llama4ForConditionalGeneration | conditional generation / image | MM processor; encoder/merge; FusedMoE/grouped GEMM; vision encoder | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-nemotron-vl-llama-nemotron-vlchat-model | Llama_Nemotron_Nano_VL | registry.py:456; vllm/model_executor/models/nemotron_vl.py::LlamaNemotronVLChatModel | conditional generation / image | MM processor; encoder/merge; vision encoder | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-llava-llava-for-conditional-generation | LlavaForConditionalGeneration | registry.py:457; vllm/model_executor/models/llava.py::LlavaForConditionalGeneration | conditional generation / image | MM processor; encoder/merge; vision encoder | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-llava-next-llava-next-for-conditional-generation | LlavaNextForConditionalGeneration | registry.py:458-461; vllm/model_executor/models/llava_next.py::LlavaNextForConditionalGeneration | conditional generation / image | MM processor; encoder/merge; vision encoder | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-llava-next-video-llava-next-video-for-conditional-generation | LlavaNextVideoForConditionalGeneration | registry.py:462-465; vllm/model_executor/models/llava_next_video.py::LlavaNextVideoForConditionalGeneration | conditional generation / video+image | MM processor; encoder/merge; vision encoder; video path | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-llava-onevision-llava-onevision-for-conditional-generation | LlavaOnevisionForConditionalGeneration | registry.py:466-469; vllm/model_executor/models/llava_onevision.py::LlavaOnevisionForConditionalGeneration | conditional generation / video+image | MM processor; encoder/merge; vision encoder; video path | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-llava-onevision2-llava-onevision2-for-conditional-generation | LlavaOnevision2ForConditionalGeneration | registry.py:470-473; vllm/model_executor/models/llava_onevision2.py::LlavaOnevision2ForConditionalGeneration | conditional generation / video+image | MM processor; encoder/merge; vision encoder; video path | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-midashenglm-mi-dasheng-lmmodel | MiDashengLMModel | registry.py:474; vllm/model_executor/models/midashenglm.py::MiDashengLMModel | conditional generation / audio+image | MM processor; encoder/merge; audio/ASR frontend | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-mimo-v2-omni-mi-mo-v2-omni-for-causal-lm | MiMoV2OmniForCausalLM | registry.py:475; vllm/model_executor/models/mimo_v2_omni.py::MiMoV2OmniForCausalLM | conditional generation / audio+video+image | MM processor; encoder/merge; audio/ASR frontend; vision encoder; video path | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-minimax-m3-mini-max-m3-sparse-for-conditional-generation | MiniMaxM3SparseForConditionalGeneration | registry.py:476-479; vllm/models/minimax_m3/__init__.py::MiniMaxM3SparseForConditionalGeneration | conditional generation / image | MM processor; encoder/merge; FusedMoE/grouped GEMM; sparse-attention indexer (NOT MLA — GQA + a learned indexer, minimax_m3/nvidia/model.py:286,386,673-690; corrected by mla-deepseek-campaign §0.6); vision encoder; video path | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-minicpmo-mini-cpmo | MiniCPMO | registry.py:480; vllm/model_executor/models/minicpmo.py::MiniCPMO | conditional generation / audio+video+image | MM processor; encoder/merge; audio/ASR frontend; vision encoder; video path | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-minicpmv-mini-cpmv | MiniCPMV | registry.py:481; vllm/model_executor/models/minicpmv.py::MiniCPMV | conditional generation / video+image | MM processor; encoder/merge; audio/ASR frontend; vision encoder; video path | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-minicpmv4-6-mini-cpmv4-6-for-conditional-generation | MiniCPMV4_6ForConditionalGeneration | registry.py:482-485; vllm/model_executor/models/minicpmv4_6.py::MiniCPMV4_6ForConditionalGeneration | conditional generation / video+image | MM processor; encoder/merge; Mamba/SSM state; GDN/linear-attention state; vision encoder; video path | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-mistral3-mistral3-for-conditional-generation | Mistral3ForConditionalGeneration | registry.py:486-489; vllm/model_executor/models/mistral3.py::Mistral3ForConditionalGeneration | conditional generation / image | MM processor; encoder/merge; vision encoder | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-molmo-molmo-for-causal-lm | MolmoForCausalLM | registry.py:490; vllm/model_executor/models/molmo.py::MolmoForCausalLM | conditional generation / image | MM processor; encoder/merge; vision encoder | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-molmo2-molmo2-for-conditional-generation | Molmo2ForConditionalGeneration | registry.py:491; vllm/model_executor/models/molmo2.py::Molmo2ForConditionalGeneration | conditional generation / video+image | MM processor; encoder/merge; vision encoder; video path | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-moondream3-moondream3-for-causal-lm | Moondream3ForCausalLM, HfMoondream | registry.py:492,494; vllm/model_executor/models/moondream3.py::Moondream3ForCausalLM | conditional generation / image | MM processor; encoder/merge; FusedMoE/grouped GEMM; vision encoder | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-moss-audio-moss-audio-model | MossAudioModel | registry.py:493; vllm/model_executor/models/moss_audio.py::MossAudioModel | conditional generation / audio+image | MM processor; encoder/merge; audio/ASR frontend; vision encoder | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-nano-nemotron-vl-nemotron-h-nano-vl-v2 | NemotronH_Nano_VL_V2, NemotronH_Nano_Omni_Reasoning_V3, NemotronH_Super_Omni_Reasoning_V3 | registry.py:495-497; vllm/model_executor/models/nano_nemotron_vl.py::NemotronH_Nano_VL_V2 | conditional generation / audio+video+image | MM processor; encoder/merge; Mamba/SSM state; audio/ASR frontend; vision encoder; video path | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-nvlm-d-nvlm-d-model | NVLM_D | registry.py:498; vllm/model_executor/models/nvlm_d.py::NVLM_D_Model | conditional generation / image | MM processor; encoder/merge; vision encoder | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-opencua-open-cuafor-conditional-generation | OpenCUAForConditionalGeneration | registry.py:499; vllm/model_executor/models/opencua.py::OpenCUAForConditionalGeneration | conditional generation / video+image | MM processor; encoder/merge; vision encoder; video path | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-openpangu-vl-open-pangu-vlfor-conditional-generation | OpenPanguVLForConditionalGeneration | registry.py:500-503; vllm/model_executor/models/openpangu_vl.py::OpenPanguVLForConditionalGeneration | conditional generation / image | MM processor; encoder/merge; vision encoder; video path | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-openvla-open-vlafor-action-prediction | OpenVLAForActionPrediction | registry.py:504; vllm/model_executor/models/openvla.py::OpenVLAForActionPrediction | conditional generation / image | MM processor; encoder/merge; vision encoder | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-ovis-ovis | Ovis | registry.py:505; vllm/model_executor/models/ovis.py::Ovis | conditional generation / image | MM processor; encoder/merge; vision encoder | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-ovis2-5-ovis2-5 | Ovis2_5, Ovis2_6ForCausalLM, Ovis2_6_MoeForCausalLM | registry.py:506-508; vllm/model_executor/models/ovis2_5.py::Ovis2_5 | conditional generation / video+image | MM processor; encoder/merge; vision encoder; video path | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-paddleocr-vl-paddle-ocrvlfor-conditional-generation | PaddleOCRVLForConditionalGeneration | registry.py:509-512; vllm/model_executor/models/paddleocr_vl.py::PaddleOCRVLForConditionalGeneration | conditional generation / image | MM processor; encoder/merge; vision encoder; video path | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-paligemma-pali-gemma-for-conditional-generation | PaliGemmaForConditionalGeneration | registry.py:513-516; vllm/model_executor/models/paligemma.py::PaliGemmaForConditionalGeneration | conditional generation / image | MM processor; encoder/merge; vision encoder | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-phi3v-phi3-vfor-causal-lm | Phi3VForCausalLM | registry.py:517; vllm/model_executor/models/phi3v.py::Phi3VForCausalLM | conditional generation / image | MM processor; encoder/merge; vision encoder | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-phi4siglip-phi4-for-causal-lmv | Phi4ForCausalLMV | registry.py:518; vllm/model_executor/models/phi4siglip.py::Phi4ForCausalLMV | conditional generation / image | MM processor; encoder/merge; vision encoder | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-phi4mm-phi4-mmfor-causal-lm | Phi4MMForCausalLM | registry.py:519; vllm/model_executor/models/phi4mm.py::Phi4MMForCausalLM | conditional generation / audio+image | MM processor; encoder/merge; audio/ASR frontend; vision encoder; video path | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-pixtral-pixtral-for-conditional-generation | PixtralForConditionalGeneration | registry.py:520; vllm/model_executor/models/pixtral.py::PixtralForConditionalGeneration | conditional generation / image | MM processor; encoder/merge; vision encoder | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-qianfan-ocr-qianfan-ocrfor-conditional-generation | QianfanOCRForConditionalGeneration | registry.py:521-524; vllm/model_executor/models/qianfan_ocr.py::QianfanOCRForConditionalGeneration | conditional generation / video+image | MM processor; encoder/merge; vision encoder; video path | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-qwen2-vl-qwen2-vlfor-conditional-generation | Qwen2VLForConditionalGeneration | registry.py:525; vllm/model_executor/models/qwen2_vl.py::Qwen2VLForConditionalGeneration | conditional generation / video+image | MM processor; encoder/merge; vision encoder; video path | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-qwen2-5-vl-qwen2-5-vlfor-conditional-generation | Qwen2_5_VLForConditionalGeneration | registry.py:526-529; vllm/model_executor/models/qwen2_5_vl.py::Qwen2_5_VLForConditionalGeneration | conditional generation / image | MM processor; encoder/merge; vision encoder; video path | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-qwen2-audio-qwen2-audio-for-conditional-generation | Qwen2AudioForConditionalGeneration | registry.py:530-533; vllm/model_executor/models/qwen2_audio.py::Qwen2AudioForConditionalGeneration | conditional generation / audio+image | MM processor; encoder/merge; audio/ASR frontend | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-qwen2-5-omni-thinker-qwen2-5-omni-thinker-for-conditional-generation | Qwen2_5OmniModel, Qwen2_5OmniForConditionalGeneration | registry.py:534-541; vllm/model_executor/models/qwen2_5_omni_thinker.py::Qwen2_5OmniThinkerForConditionalGeneration | conditional generation / audio+video+image | MM processor; encoder/merge; audio/ASR frontend; vision encoder; video path | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-qwen3-omni-moe-thinker-qwen3-omni-moe-thinker-for-conditional-generation | Qwen3OmniMoeForConditionalGeneration | registry.py:542-545; vllm/model_executor/models/qwen3_omni_moe_thinker.py::Qwen3OmniMoeThinkerForConditionalGeneration | conditional generation / audio+image | MM processor; encoder/merge; audio/ASR frontend; vision encoder; video path | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-qwen3-asr-qwen3-asrfor-conditional-generation | Qwen3ASRForConditionalGeneration | registry.py:546-549; vllm/model_executor/models/qwen3_asr.py::Qwen3ASRForConditionalGeneration | conditional generation / audio+image | MM processor; encoder/merge; audio/ASR frontend; vision encoder; video path | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-qwen3-asr-realtime-qwen3-asrrealtime-generation | Qwen3ASRRealtimeGeneration | registry.py:550; vllm/model_executor/models/qwen3_asr_realtime.py::Qwen3ASRRealtimeGeneration | conditional generation / audio+image | MM processor; encoder/merge; audio/ASR frontend | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-qwen3-vl-qwen3-vlfor-conditional-generation | Qwen3VLForConditionalGeneration | registry.py:551; vllm/model_executor/models/qwen3_vl.py::Qwen3VLForConditionalGeneration | conditional generation / video+image | IMAGE e2e WORKING (M2 CLOSED) — STRICT image→text token-exact 32/32 vs vLLM 0.25.0 on the fixture (image, prompt): full pipeline MM processor (M1) → vision tower (M2a) → merge → forked VL decode (inputs_embeds merge + 3-section MRoPE + DeepStack inject at layers 0/1/2) → paged greedy. VIDEO e2e WORKING (M3c, NEAR-TIE-ROBUST PASS 2026-07-25) — teacher-forcing vLLM 0.25.0 on OUR exact tokens proves the sole divergence is ONE genuine bf16 near-tie at tok22 (gap 0.125 nats, our token vLLM's 2nd of 4 tied choices) with every downstream token == vLLM's teacher-forced argmax at gap 0.0; tower already f32-accumulates everywhere (matches vLLM) so the residual is the irreducible bf16 envelope (no kernel fix). Both modalities correct; speed pending (not yet DONE). Qwen3.6 image+video (M3/M3b) reuse this tower/loader. ENGINE MM-FORWARD (2026-07-28, CLAIM-ENGINE-MM-FORWARD): now REGISTERED + driven by ModelRegistry::Forward (not only the standalone driver) — additive default-nullopt ModelForwardInput.mm field + REGISTER_VLLM_MODEL + the registered forward folds the M2c decode via the SHARED Qwen3VLForwardStepLastLogits. Text inertness green (runner/scheduler/registry/chat_mm/serving); engine mm-forward token-exact gate test_qwen3vl_registry_e2e (THROUGH ModelRegistry::Forward == M2c golden, dgx). Residual: in-runner scheduler-fed tower + server /v1/chat/completions GPU e2e. | ☐ required | PARTIAL | VL loader + forked decode src/vllm/model_executor/models/qwen3_vl.cpp:1 + include/vllm/model_executor/models/qwen3_vl.h:1; STRICT e2e image gate tests/vllm/multimodal/test_qwen3vl_e2e.cpp:1 (32/32); NEAR-TIE-ROBUST e2e video gate tests/vllm/multimodal/test_qwen3vl_video_e2e.cpp:1 (anchor our_ids_i32.bin + neartie_gap_mnats_i32.bin from scripts/mm/m3c_video_neartie_gap.py, max gap 0.125 nats); vision tower src/vllm/model_executor/models/qwen3_vl_vision.cpp:1 + M2a tower gate tests/vllm/multimodal/test_qwen3vl_tower.cpp:1; M2b helpers src/vllm/model_executor/models/qwen3_vl_text.cpp:1; M2b/M2c unit gate tests/vllm/multimodal/test_qwen3vl_text.cpp:1 (85/85); engine registration src/vllm/model_executor/models/qwen3_vl_registry.cpp:1 + REGISTER_VLLM_MODEL; engine mm-forward field include/vllm/model_executor/models/model_registry.h (MultiModalForwardInput/ModelForwardInput.mm); engine mm-forward token-exact gate tests/vllm/multimodal/test_qwen3vl_registry_e2e.cpp:1 | CLAIM-MULTIMODAL-TOWER-FIDELITY |
MODEL-MM-qwen3-vl-moe-qwen3-vlmoe-for-conditional-generation | Qwen3VLMoeForConditionalGeneration | registry.py:552-555; vllm/model_executor/models/qwen3_vl_moe.py::Qwen3VLMoeForConditionalGeneration | conditional generation / image | MM processor; encoder/merge; vision encoder; video path | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-qwen3-5-qwen3-5-for-conditional-generation | Qwen3_5ForConditionalGeneration | registry.py:556; vllm/model_executor/models/qwen3_5.py::Qwen3_5ForConditionalGeneration | conditional generation / image | MM processor; encoder/merge; Mamba/SSM state; GDN/linear-attention state; vision encoder; video path | 🚧 family scoping; plain-BF16 loader leaf; multimodal-track W-plan; full target spec required | PARTIAL (text-only) | text-only: include/vllm/model_executor/models/qwen3_5_dense.h:40-105,146-171; plain BF16/F32 + stacked/tied load src/vllm/model_executor/models/qwen3_5_dense_weights.cpp:52-133,187-246,334-472; plain execution src/vllm/model_executor/models/qwen3_5.cpp:1993-2003,4579-4585,5249-5255,5533-5545; loader route/queue reuse src/vllm/entrypoints/model_loader.cpp:364-400; real 4B gate tests/vllm/models/test_qwen35_plain_weights.cpp:80-196: CPU topology/load 1656/1656, AOT CUDA direct OFF/ON full-engine token equivalence 1664/1664. Existing W3-G immutable ae9e8ff default/fallback each pass 235/235 + 16/16 with the frozen 64 plans. Corrected root /tmp/qwen35-transplant-4b-aot-557ab41d proves ON=OFF 128/128 and records ON/OFF/vLLM total 6155.10/6064.06/6730.46 tok/s, peak PSS 2.405/8.571/7.569 GiB; current ON is 0.9316x historical AOT ON. Current-v0.25 oracle, sanitizer, vision, strict VRAM and external 27B/35B regressions remain unverified, with no new support claim. MM-completion plan (multimodal-track.md, CLAIM-MULTIMODAL-TRACK, 2026-07-25): modalities = image + video (NO audio); reuses the landed GDN-hybrid text path — the mm half is the shared Qwen3_VisionTransformer (DeepStack, qwen3_vl.py:519) stood up on Qwen3-VL-4B first (M2) then attached to this wrapper (M3). Oracle-runnable (0.25.0 ships qwen3_5.py+qwen3_vl.py); NOT HW/oracle-blocked but CHECKPOINT-gated — the cached unsloth/Qwen3.6-27B-NVFP4 quant is TEXT-ONLY (2111 tensors, ZERO visual.*; vision_config declared but weights absent), so a vision-inclusive checkpoint download is required (M0). Tower ~0.5-0.7 B params (~1-1.4 GiB bf16) fits GB10 trivially alongside the 27B. Plan owner CLAIM-MULTIMODAL-TRACK (row stays PARTIAL/narrative-only; the mm work re-claims it at M3). M3-b LANDED 2026-07-25 (CLAIM-MULTIMODAL-M3B): IMAGE e2e WORKING — Qwen3_5VLGenerateGreedy (src/vllm/model_executor/models/qwen3_5.cpp) forks the GDN-hybrid forward on inputs_embeds(scatter tower merger [196,5120] into image_token 248056 rows, no deepstack) + 3-section MRoPE [11,11,10] interleaved in the 16 full-attn layers (host BuildMropeCosSinHost → the mrope_cos_sin param on DenseForwardLayers, nullptr on text ⇒ byte-identical); vision loader LoadQwen3VLVisionWeights (src/vllm/model_executor/models/qwen3_vl.cpp, 27B config) + M2a tower + LoadQwen3_5Dense bf16 LLM. STRICT gate tests/vllm/multimodal/test_qwen3_5_vl_e2e.cpp 32/32 token-exact vs vLLM 0.25.0 (sha256 ead4b484…); text-inertness re-run cutlass-ON 27B/35B/Coder 235/315/138. M3d LANDED 2026-07-25 (CLAIM-MULTIMODAL-M3D): VIDEO e2e WORKING — Qwen3_5VLGenerateGreedyVideo (src/vllm/model_executor/models/qwen3_5.cpp) reuses the M3-b image driver via a shared VLGenerateCoreGdn (video merge mask on video_token 248057 + Qwen3VLGetRopeIndexVideo per-frame temporal MRoPE; M3c processor/windowed-tower reused verbatim; no deepstack). STRICT gate tests/vllm/multimodal/test_qwen3_5_vl_video_e2e.cpp 32/32 token-exact vs vLLM 0.25.0 (oracle scripts/mm/m3d_video_oracle_capture.py, K=5 deterministic, near-tie gaps 0.0000); image e2e re-run STRICT 32/32 (refactor-safe); text SACRED byte-identical by construction (shared forward untouched). Qwen video modalities COMPLETE (image+video e2e; audio N/A); speed still pending (row stays PARTIAL). Speed lever #2 CLOSED 2026-07-27 (CLAIM-MULTIMODAL-SPEED-DECODE, multimodal-speed.md §8): on-GPU greedy argmax + decode embed round-trip removed on the shared VLGenerateCoreGdn; bit-exact (image+video STRICT 32/32 held, goldens md5-identical); 27B decode TPOT NEUTRAL (223 ms, ~222 ms bandwidth floor, at vLLM parity). Speed lever #3 FIRST BRICK 2026-07-27 (CLAIM-MULTIMODAL-SPEED-GRAPH, multimodal-speed.md §9): the shared VLGenerateCoreGdn decode step now routes through the production Qwen3_5DenseDecodeGraph (cold→warm→replay captured decode) — mm decode is GRAPH-CAPTURABLE (was eager per-step). S==B==1 bit-identical rebuild; the decode-time 1-D device RoPE at p reproduces the degenerate MRoPE {p,p,p} → token-exact HELD (image+video STRICT 32/32, 30 graph replays confirmed); A/B graphed 232.5 vs eager 233.4 ms/tok = NEUTRAL at the 27B bandwidth floor. Structural gap closed; W-plan = Voxtral decode-graph (audio 1.52× gap-closer) + batched c2+ + serving ingestion. Row stays PARTIAL/speed-pending. | unassigned |
MODEL-MM-qwen3-5-qwen3-5-moe-for-conditional-generation | Qwen3_5MoeForConditionalGeneration | registry.py:557-560; vllm/model_executor/models/qwen3_5.py::Qwen3_5MoeForConditionalGeneration | conditional generation / image | MM processor; encoder/merge; Mamba/SSM state; GDN/linear-attention state; vision encoder; video path | 🚧 family scoping; multimodal-track W-plan; target spec required | PARTIAL (text gated, vision NOT gated) | text-only: include/vllm/model_executor/models/qwen3_5.h:1-17,97; direct registry src/vllm/model_executor/models/registry.cpp:10-20; gate tests/parity/test_qwen36_paged_engine.cpp:78,140. W3-G immutable ae9e8ff correctness-only ratio-8 inertness passes 2/2 + 315/315; no 35B performance claim; vision not implemented AT THAT DATE (it landed later, see M2/M3 below, and is still NOT gated). Disk load now DEFERS the routed-expert host copies and streams+frees them per layer during PrepareMarlinResident to bound load-phase peak PSS (ENG-MOE-LOADSTREAM, engine-matrix; CPU-gated, DGX pending) — device residents byte-identical. MM-completion plan (multimodal-track.md, CLAIM-MULTIMODAL-TRACK, 2026-07-25): image + video (NO audio); same shared Qwen3_VisionTransformer as the 27B row, attached to the landed MoE GDN-hybrid text path (M3). Oracle-runnable (0.25.0); CHECKPOINT-gated — the cached nvidia/Qwen3.6-35B-A3B-NVFP4 quant is TEXT-ONLY (vision_config declared, visual.* weights absent); vision-inclusive download required (M0). Tower fits GB10 alongside the 35B MoE per the landed text run. Plan owner CLAIM-MULTIMODAL-TRACK (row stays PARTIAL/narrative-only; the mm work re-claims it at M3). M2/M3 LANDED (#891, .agents/specs/moe-vision-tower.md): the loader no longer drops the checkpoint's 333 model.visual.* tensors (LoadQwen3_5MoeVision -> the SHARED LoadQwen3VLVisionWeights the dense arm is gated on; their ABSENCE is refused by name), and Qwen3_5MoeVLGenerateGreedy[Video] forks the forward gated on mm input over a greedy core now TEMPLATED on the weights arm rather than copied. Evidence: CPU suite 479/479 serial; the new test_qwen3_5_moe_vision proves the forked forward reduces EXACTLY to the text forward over the tower row (one visual token, 1x1x1 LLM grid) and that MRoPE is applied (8x8 grid must DIFFER from the 1-D run), with 4 mutations driven RED and restored byte-exact; on Thor (sm_110, FALLBACK attention) test_qwen3_5_moe_vision_hw loads the real 333 tensors and runs the tower on the fixture image. OWED: the binding image and video token-exact gates vs the pinned oracle at 35B. Not runnable on Thor -- vLLM cannot import there (libcuda.so.1 absent on the host, torch.cuda.is_available() False) and the bf16 35B is ~67 GiB against this box's documented 25 GB single-model reboot ceiling; dgx.casa was off-limits mid-run for a sibling row. TEXT ARM ORACLE-GATED ON THE PUBLISHED BF16 REPO 2026-08-15 (#740 + #864), and this changes NOTHING about the vision claim: greedy 7 prompts x 3 repeats x 16 tokens on Qwen/Qwen3.6-35B-A3B bf16 @995ad96eacd98c81ed38be0c5b274b04031597b0 vs the pinned oracle gave 6/7 prompts STRICT 16/16, the seventh one exact logit tie (top2_gap_mnats = 0.0) our on-device argmax breaks toward the higher id (#910); only the FIRST divergence per prompt is adjudicable, so the raw 108/112 position count is NOT a quality score. SACRED inertness 3/3, goldens byte-identical (27B 235/235, 35B 315/315, Coder 138/138). NO throughput, latency or memory number exists for this checkpoint. The row therefore stays PARTIAL: the binding image and video token-exact gates at 35B are still OWED, the vision claim remains "the tower loads and computes" rather than "produces correct tokens", and the sm_110 run that proved it used the FALLBACK attention path, which is not coverage of the shipped GB10 path. #908's dense regression check is PARTIAL too: dense TEXT is 235/235 at 2f2bce926, a true before/after (binary md5 db889909d4… vs 49ded1ece8…, 500 TUs recompiled), while dense image/video stays UNVERIFIED (network-blocked) | unassigned |
MODEL-MM-rvl-rfor-conditional-generation | RForConditionalGeneration | registry.py:561; vllm/model_executor/models/rvl.py::RForConditionalGeneration | conditional generation / video+image | MM processor; encoder/merge; vision encoder | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-skyworkr1v-skywork-r1-vchat-model | SkyworkR1VChatModel | registry.py:562; vllm/model_executor/models/skyworkr1v.py::SkyworkR1VChatModel | conditional generation / image | MM processor; encoder/merge; vision encoder | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-smolvlm-smol-vlmfor-conditional-generation | SmolVLMForConditionalGeneration | registry.py:563; vllm/model_executor/models/smolvlm.py::SmolVLMForConditionalGeneration | conditional generation / image | MM processor; encoder/merge; vision encoder | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-step-vl-step-vlfor-conditional-generation | StepVLForConditionalGeneration | registry.py:564; vllm/model_executor/models/step_vl.py::StepVLForConditionalGeneration | conditional generation / image | MM processor; encoder/merge; vision encoder | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-step3-vl-step3-vlfor-conditional-generation | Step3VLForConditionalGeneration | registry.py:565; vllm/model_executor/models/step3_vl.py::Step3VLForConditionalGeneration | conditional generation / image | MM processor; encoder/merge; vision encoder | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-step3p7-step3p7-for-conditional-generation | Step3p7ForConditionalGeneration | registry.py:566; vllm/model_executor/models/step3p7.py::Step3p7ForConditionalGeneration | conditional generation / image | MM processor; encoder/merge; vision encoder | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-ultravox-ultravox-model | UltravoxModel | registry.py:567; vllm/model_executor/models/ultravox.py::UltravoxModel | conditional generation / audio+image | MM processor; encoder/merge; audio/ASR frontend; vision encoder | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-voxtral-voxtral-for-conditional-generation | VoxtralForConditionalGeneration | registry.py:568; vllm/model_executor/models/voxtral.py::VoxtralForConditionalGeneration | conditional generation / audio→text (audio-track A3) | A1 log-mel (landed) + A2 Whisper-class encoder tower (reused at Voxtral config) + LANDED Mistral/Llama decoder | audio-track.md §0c/§1 (A3) | READY | audio→text e2e gate PASS 14/14 (near-tie-robust, GPU dgx-only): include/vllm/model_executor/models/voxtral.h + src/vllm/model_executor/models/voxtral.cpp + tests/vllm/multimodal/test_voxtral_e2e.cpp; oracle scripts/mm/a3_voxtral_oracle_capture.py + scripts/mm/a3_voxtral_neartie_gate.py; fixtures tests/vllm/multimodal/fixtures/voxtral_audio/. Speed lever #2 CLOSED 2026-07-27 (CLAIM-MULTIMODAL-SPEED-DECODE, multimodal-speed.md §8): on-GPU greedy argmax + decode embed round-trip removed in VoxtralGenerateGreedy; bit-exact (14/14 held, near-tie seq 48/48); decode TPOT 61.85 vs 62.08 ms host (~0.4% win); still 1.52× vs vLLM graphed = eager-launch (lever #3). Speed lever #3 W1 LANDED 2026-07-27 (CLAIM-MM-SPEED-GRAPH-W1, multimodal-speed.md §10): the audio decode now routes through a NEW VoxtralDecodeGraph (Voxtral's Mistral/Llama stack was the last mm text stack with no decode-graph class — sibling of Qwen3MoeDecodeGraph, pure full-attention over dense_attn::AttnBlock, no GDN); VoxtralGenerateGreedy decode is now graph-captured, eager fallback VT_MM_DECODE_EAGER. Bit-exact HELD (14/14, near-tie seq 48/48; 46 replays confirmed; goldens md5 unchanged). A/B steady TPOT graphed 60.94 (60.79–61.07) vs eager 61.71 (61.57–61.88) = −0.77 ms/tok (~1.25%, NON-OVERLAPPING) — real clean win, but NARROWS the gap 1.52×→1.49× vs vLLM 40.8 ms, does NOT close it; the residual is per-step compute/kernel efficiency, not launch overhead. Structural: Voxtral now graph-capturable (prereq for c2+). Row stays PARTIAL (speed-pending). Encoder flash-tiled kernel 2026-07-28 (CLAIM-MM-SPEED-AUDIO-ENC-KERNEL, multimodal-speed.md §14): §13 lever #1 — vt::AttentionDenseFlash shared-memory-TILES K/V (FA2 tiling), BIT-IDENTICAL per-warp math; encoder attention 35.11→19.29 ms/layer (1.82×), encoder forward ~1834→~1375 ms (1.33×); 16/16, token md5-identical, goldens unchanged, sanitizer 0, RED held. NOT at parity (~32× vs vLLM 43 ms); gap-closer = tensor-core MMA hd-64 FA2 + resident weights. Row stays PARTIAL (speed-pending). — anchor tests/vllm/multimodal/test_voxtral_e2e.cpp:87 | - |
MODEL-MM-voxtral-realtime-voxtral-realtime-generation | VoxtralRealtimeGeneration | registry.py:569; vllm/model_executor/models/voxtral_realtime.py::VoxtralRealtimeGeneration | conditional generation / audio+image | MM processor; encoder/merge; audio/ASR frontend | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-cohere-asr-cohere-asr-for-conditional-generation | CohereAsrForConditionalGeneration | registry.py:571-574; vllm/model_executor/models/cohere_asr.py::CohereAsrForConditionalGeneration | conditional generation / audio+image | MM processor; encoder/merge; cross-attention; audio/ASR frontend; vision encoder | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-nemotron-parse-nemotron-parse-for-conditional-generation | NemotronParseForConditionalGeneration | registry.py:575-578; vllm/model_executor/models/nemotron_parse.py::NemotronParseForConditionalGeneration | conditional generation / audio+image | MM processor; encoder/merge; cross-attention; audio/ASR frontend; vision encoder | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-whisper-whisper-for-conditional-generation | WhisperForConditionalGeneration | registry.py:579; vllm/model_executor/models/whisper.py::WhisperForConditionalGeneration | conditional generation / audio+image | MM processor; encoder/merge; sliding-window attention; cross-attention; audio/ASR frontend | ☐ required | INVENTORIED | none | unassigned |
MODEL-AUDIO - Audio encoder components (not vLLM registry architectures)
Rows here are audio TOWERS that vLLM ships as a component of some registry
architecture rather than as one themselves, so they have no registry.py entry
of their own and are NOT part of the 366 architecture rows counted above.
| ID | Item / architecture aliases | Upstream | Task / modality | Dependencies | Spike/spec | State | Our code + tests/evidence | Owner |
|---|---|---|---|---|---|---|---|---|
MODEL-AUDIO-PARAKEET-ENCODER | ParakeetEncoder + ParakeetForCTC (Parakeet / FastConformer conformer audio encoder + CTC head) | NOT a registry architecture, and NOT vLLM-implemented: vllm/model_executor/models/parakeet.py:14,61 imports and instantiates transformers.ParakeetEncoder as the audio component of nano_nemotron_vl.py (registry.py:511-513). The honest mirror source is therefore transformers 5.3.0 transformers/models/parakeet/modeling_parakeet.py (ParakeetEncoderRelPositionalEncoding:51, ParakeetEncoderFeedForward:101, ParakeetEncoderConvolutionModule:116, ParakeetEncoderAttention:259, ParakeetEncoderSubsamplingConv2D:357, ParakeetEncoderBlock:426, ParakeetEncoder:549, ParakeetForCTC:675, plus tokenization_parakeet.py ParakeetTokenizer._decode:28-49); the vLLM-NATIVE halves are ParakeetProjection:27, ProjectedParakeet:48 and ParakeetExtractor:138, with vllm/transformers_utils/configs/parakeet.py ParakeetConfig:8 / ExtractorConfig:41 | ASR / audio -> text (CTC greedy) | vt::Conv2d, vt::DepthwiseConv1d, vt::AttentionRelPos (spike P1-P3, LANDED 1ace9e27) | parakeet-conformer-encoder work item P4 | ACTIVE | Encoder + CTC head + log-mel front end LANDED (CPU, f32) 2026-08-07. Code: include/vllm/model_executor/models/parakeet_encoder.h + src/vllm/model_executor/models/parakeet_encoder.cpp (subsampling -> rel-pos table -> N x block{FF/2, rel-pos attention, convolution module, FF/2, norm} -> CTC head -> greedy collapse), src/vllm/model_executor/models/parakeet_weights.cpp (HF-format safetensors loader), include/vllm/multimodal/parakeet_audio_processor.h + src/vllm/multimodal/parakeet_audio_processor.cpp (preemphasis -> STFT -> slaney mel -> log -> per-bin normalisation). Gates, all CPU x86-64 -DVLLM_CPP_CUDA=OFF: tests/vllm/models/test_parakeet_encoder.cpp 7 cases / 543 assertions vs an INDEPENDENT in-test double-precision reference that performs _rel_shift LITERALLY (encoder rel-L2 1.9e-7 to 2.4e-7, CTC logits 2.0e-7, greedy + collapsed ids EXACT); tests/vllm/models/test_parakeet_ctc_engine.cpp 2 / 12485 vs a dumped HuggingFace ParakeetForCTC oracle stage by stage (subsampling / ff1 / attn / conv / block0 / block1 / hidden / logits all rel-L2 <= 3.5e-7) with greedy and collapsed token ids EXACT on a full-length AND a PADDED batch item; tests/vllm/multimodal/test_parakeet_audio_processor.cpp 6 / 41054 (mel bank BIT-IDENTICAL to transformers mel_filter_bank, log-mel rel-L2 4.9e-6 vs ParakeetFeatureExtractor). 20 single-line mutations compiled and run, 20 killed, 0 survivors. NOT verified: no pretrained nvidia/parakeet-* checkpoint was downloaded, so NO transcript is claimed (the oracle is a seeded, randomly-initialised ParakeetForCTC; the pretrained arm is wired behind VLLM_PARAKEET_CKPT and SKIPS when unset); no CUDA provider exists for the three kernels, no GPU suite was run, no aarch64 run, no speed or memory number measured, claimed or owed. Pretrained transcript ADDED 2026-08-07 on nvidia/parakeet-ctc-0.6b and nvidia/parakeet-ctc-1.1b (the 1.1b is the identical architecture at 42 layers instead of 24; downloaded, transcribed, then deleted to give the shared disk back). The "RNN-T / TDT has no upstream in either vLLM or HF" line this row used to carry was WRONG: it measured the locally installed transformers 5.3.0, not main; see MODEL-AUDIO-PARAKEET-TRANSDUCER below. ONE-SURFACE ROW 1 LANDED 2026-08-07 (row/PARAKEET-ONE-SURFACE): the "NOT a registry architecture" clause above is SUPERSEDED — parakeet_registry.cpp registers ParakeetForCTC (+RNNT/TDT) as SupportsTranscription-ONLY (interfaces.py:1110-1118 mirror; every text hook refuses actionably), the pipeline moved into vllm::multimodal::ParakeetTranscriber, vllm_transcribe landed on ABI v11, /v1/audio/transcriptions serves task-conditionally, the example is a thin vllm.h client, and the fold gate holds ids+text byte-identical to the pre-fold binary on committed synthetic fixtures (tests/vllm/models/fixtures/parakeet_e2e, goldens captured from main@f98e1e48) | Claude Code (opus-5), CLAIM-PARAKEET-MODEL-P4; ROW 1 fold: Claude Code (fable-5), PR #121 |
MODEL-AUDIO-PARAKEET-TRANSDUCER | ParakeetForRNNT + ParakeetForTDT (Parakeet RNN-T / TDT transducer heads over the same FastConformer encoder) | NOT a registry architecture, and NOT vLLM-implemented: vLLM has no transducer call site at all (it wraps only the ENCODER, as the audio tower of nano_nemotron_vl.py). The mirror source is transformers main transformers/models/parakeet/modeling_parakeet.py (ParakeetRNNTDecoder:831, ParakeetRNNTJointNetwork:879, ParakeetForRNNT:922, ParakeetTDTJointNetwork:1035, ParakeetForTDT:1052) plus generation_parakeet.py (ParakeetRNNTDecoderCache:23, ParakeetRNNTGenerationMixin:125, ParakeetTDTGenerationMixin:271) and configuration_parakeet.py (ParakeetRNNTConfig:136, ParakeetTDTConfig:188) | ASR / audio -> text (greedy transducer decode, with TDT frame skipping) | MODEL-AUDIO-PARAKEET-ENCODER (composes over its forward), vt::MatmulBT | parakeet-conformer-encoder work item P6 | ACTIVE | LANDED (CPU, f32) 2026-08-07. Code: include/vllm/model_executor/models/parakeet_transducer.h + src/vllm/model_executor/models/parakeet_transducer.cpp (embedding -> stacked LSTM prediction network with the cache's blank fast path -> decoder projector; joint head(relu(encoder_projected + decoder)); the greedy loop with the RNN-T max_symbols_per_step forced advance and the TDT duration-driven frame skip), plus the transducer half of src/vllm/model_executor/models/parakeet_weights.cpp. Gates, all CPU x86-64 -DVLLM_CPP_CUDA=OFF: tests/vllm/models/test_parakeet_transducer.cpp 3 cases / 777 assertions: the LSTM cell vs an INDEPENDENT in-test scalar reference over 5 shapes x bias/no-bias x 6 sequential steps; decoder and joint tensors rel-L2 <= 1.2e-7 vs a dumped HF oracle over a fixed token walk that drives the blank fast path; the emitted sequence and the per-step durations EXACT for BOTH ParakeetForRNNT and ParakeetForTDT, from the oracle's own projected encoder output AND end to end from mel features. The fixture asserts its own branch coverage (blank emissions, non-blank emissions, tokens held at one frame, RNN-T forced advances), so a regeneration that degenerated to one branch fails. PRETRAINED, and this is the binding claim: on nvidia/parakeet-rnnt-0.6b and nvidia/parakeet-tdt-0.6b-v3 our emitted token ids are EXACT against a real HF generate() run on a LibriSpeech clip, both through the reference's own input_features and end to end through our own log-mel front end. TRACED, not read (T0): a forward hook over a full generate() run records that every decoder call took input_ids of shape [1,1]; the gate asserts it, because the whole greedy loop rests on it. NOT verified: any GPU path, aarch64, and any speed or memory number. Checkpoint reach: the HF-safetensors models are in; .nemo-only ones need upstream's convert_nemo_to_hf.py, and the hybrid TDT-CTC and streaming EOU models stay OUT (spike § Checkpoint reach) | Claude Code (opus-5), CLAIM-PARAKEET-MODEL-P4 |
MODEL-SPEC - Speculative decoding models
| ID | Item / architecture aliases | Upstream | Task / modality | Dependencies | Spike/spec | State | Our code + tests/evidence | Owner |
|---|---|---|---|---|---|---|---|---|
MODEL-SPEC-extract-hidden-states-extract-hidden-states-model | ExtractHiddenStatesModel | registry.py:583; vllm/model_executor/models/extract_hidden_states.py::ExtractHiddenStatesModel | speculative draft / target-dependent | draft runner; acceptance/sampling | ☐ required | INVENTORIED | none | unassigned |
MODEL-SPEC-mimo-mtp-mi-mo-mtp | MiMoMTPModel | registry.py:584; vllm/model_executor/models/mimo_mtp.py::MiMoMTP | speculative draft / target-dependent | draft runner; acceptance/sampling; MTP | ☐ required | INVENTORIED | none | unassigned |
MODEL-SPEC-mimo-v2-mtp-mi-mo-v2-mtp | MiMoV2MTPModel | registry.py:585; vllm/model_executor/models/mimo_v2_mtp.py::MiMoV2MTP | speculative draft / target-dependent | draft runner; acceptance/sampling; sliding-window attention; MTP | ☐ required | INVENTORIED | none | unassigned |
MODEL-SPEC-mimo-v2-mtp-mi-mo-v2-omni-mtp | MiMoV2OmniMTPModel | registry.py:586; vllm/model_executor/models/mimo_v2_mtp.py::MiMoV2OmniMTP | speculative draft / target-dependent | draft runner; acceptance/sampling; sliding-window attention; MTP | ☐ required | INVENTORIED | none | unassigned |
MODEL-SPEC-cohere-eagle-eagle-cohere-for-causal-lm | EagleCohereForCausalLM | registry.py:587; vllm/model_executor/models/cohere_eagle.py::EagleCohereForCausalLM | speculative draft / target-dependent | draft runner; acceptance/sampling; EAGLE/EAGLE3 | ☐ required | INVENTORIED | none | unassigned |
MODEL-SPEC-llama-eagle-eagle-llama-for-causal-lm | EagleLlamaForCausalLM | registry.py:588; vllm/model_executor/models/llama_eagle.py::EagleLlamaForCausalLM | speculative draft / target-dependent | draft runner; acceptance/sampling; EAGLE/EAGLE3 | ☐ required | INVENTORIED | none | unassigned |
MODEL-SPEC-llama4-eagle-eagle-llama4-for-causal-lm | EagleLlama4ForCausalLM | registry.py:589; vllm/model_executor/models/llama4_eagle.py::EagleLlama4ForCausalLM | speculative draft / target-dependent | draft runner; acceptance/sampling; EAGLE/EAGLE3 | ☐ required | INVENTORIED | none | unassigned |
MODEL-SPEC-minicpm-eagle-eagle-mini-cpmfor-causal-lm | EagleMiniCPMForCausalLM | registry.py:590; vllm/model_executor/models/minicpm_eagle.py::EagleMiniCPMForCausalLM | speculative draft / target-dependent | draft runner; acceptance/sampling; EAGLE/EAGLE3 | ☐ required | INVENTORIED | none | unassigned |
MODEL-SPEC-qwen3-dflash-dflash-qwen3-for-causal-lm | DFlashDraftModel | registry.py:591; vllm/model_executor/models/qwen3_dflash.py::DFlashQwen3ForCausalLM | speculative draft / target-dependent | draft runner; acceptance/sampling; sliding-window attention; DFlash | ✅ DFlash spec §0 D14 | DONE | D0-redo (CLAIM-DFLASH-D0D1, 2026-07-26) UNBLOCKED on the advanced pin 555967922/vLLM 0.26.0.dev0: the mixed-SWA/full z-lab 27B draft CONSTRUCTS + drafter ALIVE under VLLM_USE_V2_MODEL_RUNNER=1 (acceptance > 1); D0+D1 (DF-AUX-TAPS target-side multi-tap) landed. D2 (DF-DRAFT-MODEL, CLAIM-DFLASH-D2, 2026-07-26) CODE LANDED + CPU-GATED: drafter model qwen3_dflash.{h,cpp} (plain 5-layer Qwen3-dense reusing the landed dense block ops, routing attention through the NEW vt::DFlashBlockAttention non-causal in-block primitive), the fc aux-combine, mask-embed, per-layer SWA/full resolution (ResolveQwen3DFlashAttnModes), and the z-lab loader. CPU gate GREEN: test_ops_dflash_block_attn 5/12 (hand-checked non-causal + RED causal-vs-non-causal + block isolation + SWA window + GQA), test_qwen3_dflash_forward 5/95 (forward runs + RED full-layer-causal-flip + block isolation + fc-combine ref + RED reversed-tap-order + attn-mode resolution). Existing causal test_ops_attention 9/9·23 + test_qwen3_forward 5/1028 UNCHANGED (new op is SEPARATE; causal path byte-identical). ACTIVE — GPU promotion GREEN on dgx 2026-07-26 (GB10 sm_121a): CUDA -Werror build clean (kernel compiles as-written); CUDA==CPU parity test_ops_dflash_block_attn 198412/198412 + compute-sanitizer 0; draft-forward parity test_qwen3_dflash_draft_parity vs the REAL vLLM DFlash draft (via collective_rpc) — fc rel-L2 0.46%, per-layer hidden ≤1.30%, final 0.88%, 11 deterministic rows STRICT top-1 + 5 bf16-near-tie cluster-matched; 27B SACRED 235/235 + MTP 9/9 byte-identical. Loader now tolerates the draft ckpt's omitted embed_tokens/lm_head (target-shared). Capture harness scripts/spec/d2_dflash_draft_ref.py. D3 DF-DRAFT-KV-PREP CODE LANDED + CPU-GATED 2026-07-26 (CLAIM-DFLASH-D3): PrepareDflashInputs (pure-integer HOST port of _prepare_dflash_inputs_kernel) + PrecomputeContextKV + ForwardBlockLogitsWithContext in qwen3_dflash.{h,cpp} reuse the landed MatmulBT/RmsNorm/RopeNeox + the UNCHANGED D2 vt::DFlashBlockAttention via a [context;block] combined sequence (NO new kernel); CPU gate tests/vllm/v1/spec_decode/test_dflash_kvprep.cpp 6/114 (prepare INTEGER bit-exact + RED valid_ctx_end; context-KV envelope + RED hidden_norm/k_norm/pos; context forward degenerates to D2 at empty ctx + diverges with ctx + block isolation); additive-only diff so D2/MTP/SACRED byte-identical. D3 DONE 2026-07-26 — GPU numeric-parity GREEN on dgx (GB10 sm_121a): CUDA -Werror 0 warnings; test_qwen3_dflash_kvprep_parity 61/61 vs the REAL vLLM draft — prepare_dflash_inputs INTEGER bit-exact vs vLLM's ACTUAL Triton _prepare_dflash_inputs_kernel (kernel_matches_numpy), context-KV worst K/V rel-L2 0.31%/0.26% across all 5 layers (proves per-layer qkv_proj slicing == vLLM's fused KV weight), block-proposal 13 STRICT + 3 near-tie = 16/16; CPU test_dflash_kvprep 114/114 re-passed (RED-proven); 27B SACRED 235/235 + MTP 9/9 + D2 parity 37/37 byte-identical. Capture harness scripts/spec/d3_dflash_kvprep_ref.py. D4 DF-ENGINE-INTEGRATION propose brick + dflash config-select CODE LANDED + CPU-GATED 2026-07-26 (CLAIM-DFLASH-D4D5): NEW src/vllm/v1/worker/gpu/spec_decode/dflash/speculator.{h,cpp} DflashProposeBlock/SampleDflashBlockDrafts — the non-autoregressive whole-block propose swapping in for the MTP k=1 MtpProposePrefill, composing D3 ForwardBlockLogitsWithContext + greedy per-mask argmax (anchor not sampled, dflash/speculator.py:300-413); ParseSpeculativeConfigJson/ResolveDflash accept method:"dflash" (lookahead k+1). CPU gate tests/vllm/v1/spec_decode/test_dflash_propose.cpp 5/19 GREEN (RED-first: sampler reading the anchor row fails 4/5 cases; brick == forward+sampler; empty-ctx degenerates to D2 context-free argmax; config parses; dspark throws), -Werror-clean TU. Additive + config-gated ⇒ MTP/non-spec byte-identical BY CONSTRUCTION (no runner/model/loader/scheduler edit). D5 DF-ENGINE-INTEGRATION runner-loop LANDED + e2e RUNS on dgx 2026-07-26 (CLAIM-DFLASH-D5): the verify/propose loop is wired — loader loads the SEPARATE z-lab draft (LoadDflashDraft, host bf16 + target-SHARED bf16 embed/lm_head) via a --speculative-config model key + ResolveSpecConfig dflash branch + runner.set_dflash_draft; the verify forward captures the D1 multi-tap (aux_tap→ForwardDeviceMultiTap); propose_drafts_dflash ACCUMULATES the per-request combined-feature context (CombineAuxFeatures(aux_tap)) and honors the num_rejected rollback by appending only the accepted-prefix features, then runs DflashProposeBlock (k=16). e2e test_qwen27_dflash_spec_decode (4×32 tok vs the committed vLLM-DFlash-ON golden): 2/4 STRICT token-exact (fibonacci, three-laws) + acceptance ~ vLLM on ALL 4 (accepted 19/39/29/25 vs golden 17/39/30/25 — the MANDATORY dead-drafter-trap condition MET). The 2 divergences (France tok11, 1723 tok12) are SINGLE bf16 near-tie flips (1723 RE-CONVERGES = proven near-tie; France cascades from one flip) — the ratified near-tie ROOT, rooted in the D3-documented inline bf16 context-KV recompute envelope (~0.3-1.3% rel-L2), NOT a wiring bug (proven by the 2 exact prompts + near-exact acceptance). Inertness GREEN (SACRED 235/235 + MTP 9/9 byte-identical); CUDA -Werror clean; NO new CUDA kernel (host orchestration reusing D1/D2/D3-sanitized ops). NOT a clean strict-4/4 pass — STRICT 4/4 token-identity + speed A/B = D6 (persistent paged draft-KV bit-matching vLLM's fused context-KV projections). D6 CLAIM-DFLASH-D6 2026-07-27 (records-only, NO code) — c1 SPEED A/B DONE + STRICT proven bf16-IRREDUCIBLE + CG feasibility: our DFlash-ON = 2.50x TPOT (40.4 vs 101.2 ms) / 2.48x output-tput (24.4 vs 9.86 tok/s) over our OFF at c1 (8 prose+code prompts×256 tok greedy, 2 reps, acceptance 0.22 = 3.56/16, rep-stable <1.5%), benchmark_binding=true; vs vLLM-DFlash-ON graphed vLLM-DFlash-ON graphed = 28.5 tok/s / 35.1 ms TPOT / acceptance_len 4.30 (same 8 prompts, VLLM_USE_V2_MODEL_RUNNER=1, mm-off, gpu_util 0.30), so OURS IS ~14% BELOW vLLM-DFlash-ON on output throughput (24.4 vs 28.5 tok/s) - both ~on-par at spec-OFF (9.86 vs 9.83 tok/s), but vLLM extracts a larger DFlash speedup (2.90x vs our 2.47x) because its draft step is fully device-resident + CUDA-graphed (ours host-orchestrates 13 downloads/step) + slightly higher acceptance (~4.3 vs ~3.6 draft tokens/step). The DONE speed bar (ours >= vLLM) is NOT met; closing it = the device-resident draft rewrite + FULL CG (D6 part 2). STRICT-4/4 is bf16-IRREDUCIBLE — the draft KV cache is bf16 not fp8 (torch_utils.py:398 auto→model dtype; the D0 "fp8-KV" was the backend name), the D3 golden already compares pre-storage bf16 (residual = sub-ULP kernel noise), and a fused multi-layer KV GEMM is per-element invariant to our per-layer GEMMs (bit-exact needs vLLM's exact kernels) ⇒ the ratified near-tie gate is the FINAL correctness form (no fused-KV code landed). FULL CG + persistent-paged-KV BLOCKED on a device-resident draft-path rewrite (the D5 path does 13 device→host downloads/step) — the remaining throughput-parity increment. Inertness by construction (D5 binary; SACRED 235/235 + MTP 9/9 stand). Row STAYS ACTIVE (correctness-final + c1-speed-measured). Evidence tool scripts/spec/vllm_dflash_timing.py. D7 CLAIM-DFLASH-D7 2026-07-27 (source-owning) — within-step draft forward DEVICE-RESIDENT: PrecomputeContextKVDevice keeps per-layer K/V on device + ForwardBlockLogitsWithContext builds [context;block] with vt::IndexCopy/IndexSelect (removes ~30 D→H Downloads/step), bit-identical (identity bf16↔f32 round-trips replaced, no float op changed) — e2e test_qwen27_dflash_spec_decode 27/27 SAME tokens (2/4 STRICT + 2/4 near-tie at identical divergence points, acceptance 19/39/29/25), SACRED 235/235 + MTP 9/9, CUDA -Werror clean, compute-sanitizer 0 (198412; no new kernel). The direct old-vs-new A/B = +2.0% output-tput (IN-NOISE) ⇒ D6's "downloads = the ~14% gap" REFUTED by measurement; ours 19.68 tok/s STILL ~33% BELOW vLLM-DFlash-ON 29.2 (reconstructed 8-prompt set; OFF parity our 9.97 ≥ vLLM 9.66); residual re-attributed to acceptance (ours 2.49 vs vLLM ~3.13 accepted draft-tok/step, bf16-irreducible) + per-step context-KV RECOMPUTE (O(context²); needs the cross-step persistent paged draft-KV store) + eager-vs-graphed. SPEED BAR NOT met; STAYS ACTIVE D9 CLAIM-DFLASH-D9 2026-07-27 (source-owning) — PERSISTENT PAGED DRAFT-KV LANDED (bit-identical, +22.7%): qwen3_dflash.{h,cpp} AppendContextKVHost (project ONLY newly-accepted rows → per-layer bf16 K/V) + ForwardBlockLogitsWithPrecomputedKV (upload the persistent store, NO re-projection) share the core ForwardWithCtxKVDev with the old recompute; runner.cpp::propose_drafts_dflash swaps the O(context²) per-step recompute (dflash_ctx_feats_) for an append-only per-request dflash_kv_store_ (rollback=don't-append). NO new CUDA kernel; config-gated. Bit-identity: CPU test_dflash_propose 2 new D9 cases = exact float equality vs full recompute; GPU e2e test_qwen27_dflash_spec_decode 27/27 SAME tokens (acceptance 19/39/29/25, same divergences); SACRED 235/235 + MTP 9/9 byte-identical; CUDA -Werror clean. A/B (c1, 8 prose+code×256 tok, benchmark_binding=true): ours-ON 25.75 tok/s (was 20.99, +22.7%) / 38.40 ms / acc 3.68/step vs vLLM-ON graphed 28.09 / 35.60 / acc 3.31 = 0.917× (was 0.69×). Part 1: same-trajectory per-step acceptance == vLLM EXACTLY (ratio 1.00 on the 2 token-identical prompts) AND ours realized acceptance (3.68) > vLLM (3.31) ⇒ D8's "bf16 acceptance ceiling" REFUTED (confound). Residual ~8% = eager-vs-graphed ONLY; the FULL uniform-(1+k) CG (device paged-KV store + paged attn, new-CUDA) is the SOLE un-landed increment. D12/D13 CLAIM-DFLASH-D12/D13 2026-07-27 — fixed-capacity paged draft-KV store (vt::DFlashPagedBlockAttention) + the draft-step CUDA graph landed (capture-correctness PROVEN replayed==eager bit-identical); c1 0.917×→0.978×. D14 CLAIM-DFLASH-D14 2026-07-27 — SPEED GATE MET → DONE: nsys attributed the D13 ~2% residual to the from-scratch DFlashPagedBlockAttentionKernel draft attention (1.8% of GPU time, ~460 us/call; the draft bf16 GEMMs run cutlass_80_wmma in BOTH engines, NOT the gap); ported it to a WARP-scoped __shfl_xor online-softmax variant (mirrors the shipped AttentionWarpKernel; src/vt/cuda/cuda_ops.cu DFlashPagedBlockAttentionWarpKernel, default ON, VT_DFLASH_ATTN_BLOCK=1 = the bit-identical D12/D13 block kernel) → draft attn 242.9→77.9 ms (3.1×), our-ON c1 28.60→29.32 tok/s. FINAL same-session 3-rep A/B: our-ON 29.42/29.27/29.32 vs vLLM-ON 29.240/29.247/29.233 — our WORST > vLLM's BEST, 1.003× ⇒ ≥vLLM MET. Correctness UNCHANGED (e2e 27/27 graph==eager, acceptance 19/39/29/25 identical, 1629 draft accepted identical warp-vs-block; CUDA==CPU 795648/795648 + compute-sanitizer 0), inertness SACRED 235/235 + MTP 9/9. Correctness-complete (ratified near-tie) AND at/above vLLM throughput. Anchors: cuda_ops.cu + test_ops_dflash_paged_block_attn + ledger; closing commit 164453a2 (claims CLAIM-DFLASH-D0D1..D14 recorded in coordination.md + ledger). | 489a7544 |
MODEL-SPEC-deepseek-v4-dspark-deepseek-v4-for-causal-lm | DSparkDraftModel | registry.py:592; vllm/models/deepseek_v4/__init__.py::DSparkDeepseekV4ForCausalLM | speculative draft / target-dependent | draft runner; acceptance/sampling; FusedMoE/grouped GEMM; MLA/latent KV; sliding-window attention; DSpark | ☐ required | INVENTORIED | The DeepSeek-V4 destination stays INVENTORIED and unimplemented, but the STRING now has two: BEYOND-PIN vllm#52197 (merged 2026-08-17 at 7075ddac) makes DSparkDraftModel name a QWEN3 draft whenever the draft's model_type is qwen3, which is what RadixArk/Qwen3.8-27B-DSpark declares. SPEC-DSPARK-QWEN3-ROUTING (#1193, spec) routes that pair to the landed Qwen3DSparkModel lane in SpeculativeConfig::ResolveDsparkArchitecture and REFUSES this DeepSeek-V4 destination by name from LoadedEngine::ResolveSpecConfig, rather than mirroring upstream's silent rewrite into a stub (§7 R2). Evidence: test_speculative_dspark.cpp, test_dspark_draft_routing.cpp | unassigned |
MODEL-SPEC-qwen3-dspark-qwen3-dspark-for-causal-lm | Qwen3DSparkModel | registry.py:593; vllm/model_executor/models/qwen3_dspark.py::Qwen3DSparkForCausalLM | speculative draft / target-dependent | draft runner; acceptance/sampling; DSpark | ☐ required | INVENTORIED | none | unassigned |
MODEL-SPEC-laguna-dflash-dflash-laguna-for-causal-lm | DFlashLagunaForCausalLM (v0.25.0 target-pending) | v0.25.0 target registry.py:598; vllm/model_executor/models/laguna_dflash.py::DFlashLagunaForCausalLM @ 702f481 | speculative draft / Laguna targets | draft runner; acceptance/sampling; full/sliding attention; DFlash | ☐ required | INVENTORIED | none | unassigned |
MODEL-SPEC-llama-eagle3-eagle3-llama-for-causal-lm | PEagleDraftModel, PeagleLlamaForCausalLM, Eagle3LlamaForCausalLM, Eagle3MiniMaxM2ForCausalLM, LlamaForCausalLMEagle3, Eagle3Qwen2_5vlForCausalLM, Eagle3Qwen3vlForCausalLM | registry.py:594-600; vllm/model_executor/models/llama_eagle3.py::Eagle3LlamaForCausalLM | speculative draft / target-dependent | draft runner; acceptance/sampling; EAGLE/EAGLE3 | ☐ required | INVENTORIED | none | unassigned |
MODEL-SPEC-qwen3-eagle3-eagle3-qwen3-for-causal-lm | Eagle3Qwen3ForCausalLM, PeagleQwen3ForCausalLM | registry.py:601-602; vllm/model_executor/models/qwen3_eagle3.py::Eagle3Qwen3ForCausalLM | speculative draft / target-dependent | draft runner; acceptance/sampling; EAGLE/EAGLE3 | ☐ required | INVENTORIED | none | unassigned |
MODEL-SPEC-mistral-eagle-eagle-mistral-for-causal-lm | EagleMistralForCausalLM | registry.py:603; vllm/model_executor/models/mistral_eagle.py::EagleMistralForCausalLM | speculative draft / target-dependent | draft runner; acceptance/sampling; EAGLE/EAGLE3 | ☐ required | INVENTORIED | none | unassigned |
MODEL-SPEC-mistral-large-3-eagle-eagle-mistral-large3-for-causal-lm | EagleMistralLarge3ForCausalLM | registry.py:604-607; vllm/model_executor/models/mistral_large_3_eagle.py::EagleMistralLarge3ForCausalLM | speculative draft / target-dependent | draft runner; acceptance/sampling; EAGLE/EAGLE3 | ☐ required | INVENTORIED | none | unassigned |
MODEL-SPEC-deepseek-eagle3-eagle3-deepseek-v2-for-causal-lm | Eagle3DeepseekV2ForCausalLM, Eagle3DeepseekV3ForCausalLM | registry.py:608-609; vllm/model_executor/models/deepseek_eagle3.py::Eagle3DeepseekV2ForCausalLM | speculative draft / target-dependent | draft runner; acceptance/sampling; EAGLE/EAGLE3 | ☐ required | INVENTORIED | none | unassigned |
MODEL-SPEC-deepseek-eagle-eagle-deepseek-v3-for-causal-lm | EagleDeepSeekMTPModel | registry.py:610; vllm/model_executor/models/deepseek_eagle.py::EagleDeepseekV3ForCausalLM | speculative draft / target-dependent | draft runner; acceptance/sampling; FusedMoE/grouped GEMM; EAGLE/EAGLE3 | ☐ required | INVENTORIED | none | unassigned |
MODEL-SPEC-deepseek-mtp-deep-seek-mtp | DeepSeekMTPModel | registry.py:611; vllm/model_executor/models/deepseek_mtp.py::DeepSeekMTP | speculative draft / target-dependent | draft runner; acceptance/sampling; FusedMoE/grouped GEMM; MTP | ☐ required | INVENTORIED | none | unassigned |
MODEL-SPEC-deepseek-v4-deep-seek-v4-mtp | DeepSeekV4MTPModel | registry.py:630; vllm/models/deepseek_v4/__init__.py::DeepSeekV4MTP (impl vllm/models/deepseek_v4/nvidia/mtp.py:72) | speculative draft / target-dependent | draft runner; acceptance/sampling; FusedMoE/grouped GEMM; DSA sparse-MLA; sliding-window attention; MTP — W0-scoped (2026-07-28): V4 MTP is NOT V3's fused eh_proj — it keeps separate e_proj/h_proj (fp8), adds enorm/hnorm + an hc_head vocab collapse in compute_logits, and its mtp_block is a full DeepseekV4DecoderLayer so it inherits the WHOLE V4 stack (MHC hyper-connections + DSA indexer/compressor + sqrtsoftplus/hash MoE). REUSES our landed MTP spec-decode engine loop (Qwen3_5MTP/Qwen3_5MoeMTP verify/reject/take_draft_token_ids); only the draft FORWARD is new, and it is gated behind the target model (W3-W8) landing first. HW-fit follows the target: nvidia/DeepSeek-V4-Flash-NVFP4 ~83 GiB fits GB10. W1-WIRED (2026-07-30, CLAIM-DEEPSEEK-V4-MTP): the draft FORWARD landed as a tiny-config host oracle (DeepseekV4MtpDraftLogitsHost) reusing the DS4 AttentionBlock/MoeBlock/MHC helpers; lossless self-spec proven via the SHARED RejectionSampler (MTP-on == MTP-off greedy). Correction to the W0 note: the DS4 forward is NOT the Qwen paged-runner path, so MtpProposePrefill is NOT reused directly — only the rejection sampler is shared; the DS4-native decode-loop propose/verify + engine spec-config registration are residuals (R2/R3). Real-model gate WEIGHT-BLOCKED — both shipped deepseek4 GGUFs advertise nextn_predict_layers=1 but carry ZERO nextn tensors (DeepseekV4GgufHasMtp==false, verified 2026-07-30) | deepseek-v4-mtp spec | ACTIVE | draft forward deepseek_v4.cpp + absence guard deepseek_v4_weights.cpp; gate test_deepseek_v4_mtp.cpp 5/5 | CLAIM-DEEPSEEK-V4-MTP |
MODEL-SPEC-minimax-m3-mini-max-m3-mtp | MiniMaxM3MTP | registry.py:613; vllm/models/minimax_m3/__init__.py::MiniMaxM3MTP | speculative draft / target-dependent | draft runner; acceptance/sampling; FusedMoE/grouped GEMM; MLA/latent KV; MTP | ☐ required | INVENTORIED | none | unassigned |
MODEL-SPEC-bailing-moe-mtp-bailing-moe-v25-mtp-model | BailingMoeV25MTPModel (v0.25.0 target-pending) | v0.25.0 target registry.py:619; vllm/model_executor/models/bailing_moe_mtp.py::BailingMoeV25MTPModel @ 702f481 | speculative draft / Bailing MoE v2.5 | draft runner; acceptance/sampling; FusedMoE/grouped GEMM; MLA/latent KV; MTP | ☐ required | INVENTORIED | none | unassigned |
MODEL-SPEC-gemma4-mtp-gemma4-mtp | Gemma4MTPModel | registry.py:614; vllm/model_executor/models/gemma4_mtp.py::Gemma4MTP | speculative draft / target-dependent | draft runner; acceptance/sampling; sliding-window attention; MTP | ☐ required | INVENTORIED | none | unassigned |
MODEL-SPEC-ernie-mtp-ernie-mtp | ErnieMTPModel | registry.py:615; vllm/model_executor/models/ernie_mtp.py::ErnieMTP | speculative draft / target-dependent | draft runner; acceptance/sampling; MTP | ☐ required | INVENTORIED | none | unassigned |
MODEL-SPEC-exaone-moe-mtp-exaone-moe-mtp | ExaoneMoeMTP | registry.py:616; vllm/model_executor/models/exaone_moe_mtp.py::ExaoneMoeMTP | speculative draft / target-dependent | draft runner; acceptance/sampling; MTP | ☐ required | INVENTORIED | none | unassigned |
MODEL-SPEC-exaone4-5-mtp-exaone4-5-mtp | Exaone4_5_MTP | registry.py:617; vllm/model_executor/models/exaone4_5_mtp.py::Exaone4_5_MTP | speculative draft / target-dependent | draft runner; acceptance/sampling; MTP | ☐ required | INVENTORIED | none | unassigned |
MODEL-SPEC-nemotron-h-mtp-nemotron-hmtp | NemotronHMTPModel | registry.py:618; vllm/model_executor/models/nemotron_h_mtp.py::NemotronHMTP | speculative draft / target-dependent | draft runner; acceptance/sampling; FusedMoE/grouped GEMM; MTP | ☐ required | INVENTORIED | none | unassigned |
MODEL-SPEC-longcat-flash-mtp-long-cat-flash-mtp | LongCatFlashMTPModel | registry.py:619; vllm/model_executor/models/longcat_flash_mtp.py::LongCatFlashMTP | speculative draft / target-dependent | draft runner; acceptance/sampling; MTP | ☐ required | INVENTORIED | none | unassigned |
MODEL-SPEC-glm4-moe-mtp-glm4-moe-mtp | Glm4MoeMTPModel | registry.py:620; vllm/model_executor/models/glm4_moe_mtp.py::Glm4MoeMTP | speculative draft / target-dependent | draft runner; acceptance/sampling; FusedMoE/grouped GEMM; MTP | ☐ required | INVENTORIED | none | unassigned |
MODEL-SPEC-glm4-moe-lite-mtp-glm4-moe-lite-mtp | Glm4MoeLiteMTPModel | registry.py:621; vllm/model_executor/models/glm4_moe_lite_mtp.py::Glm4MoeLiteMTP | speculative draft / target-dependent | draft runner; acceptance/sampling; FusedMoE/grouped GEMM; MTP | ☐ required | INVENTORIED | none | unassigned |
MODEL-SPEC-glm-ocr-mtp-glm-ocr-mtp | GlmOcrMTPModel | registry.py:622; vllm/model_executor/models/glm_ocr_mtp.py::GlmOcrMTP | speculative draft / target-dependent | draft runner; acceptance/sampling; MTP | ☐ required | INVENTORIED | none | unassigned |
MODEL-SPEC-medusa-medusa | MedusaModel | registry.py:623; vllm/model_executor/models/medusa.py::Medusa | speculative draft / target-dependent | draft runner; acceptance/sampling; Medusa | ☐ required | INVENTORIED | none | unassigned |
MODEL-SPEC-openpangu-mtp-open-pangu-mtp | OpenPanguMTPModel | registry.py:624; vllm/model_executor/models/openpangu_mtp.py::OpenPanguMTP | speculative draft / target-dependent | draft runner; acceptance/sampling; FusedMoE/grouped GEMM; MTP | ☐ required | INVENTORIED | none | unassigned |
MODEL-SPEC-qwen3-next-mtp-qwen3-next-mtp | Qwen3NextMTP | registry.py:625; vllm/model_executor/models/qwen3_next_mtp.py::Qwen3NextMTP | speculative draft / target-dependent | draft runner; acceptance/sampling; MTP | ☐ required | INVENTORIED | none | unassigned |
MODEL-SPEC-step3p5-mtp-step3p5-mtp | Step3p5MTP | registry.py:626; vllm/model_executor/models/step3p5_mtp.py::Step3p5MTP | speculative draft / target-dependent | draft runner; acceptance/sampling; MTP | ☐ required | INVENTORIED | none | unassigned |
MODEL-SPEC-qwen3-5-mtp-qwen3-5-mtp | Qwen3_5MTP | registry.py:627; vllm/model_executor/models/qwen3_5_mtp.py::Qwen3_5MTP | speculative draft / target-dependent | draft runner; acceptance/sampling; GDN verify path; MTP | ✅ Qwen3.5 MTP spec | DONE | DONE 2026-07-26 (CLAIM-SPEC-MTP-DONE, closing commit records the I5e/I6/I7 gate evidence): k=1 MTP spec-decode e2e on the 27B GDN hybrid — three-way token-exact at c1 (our-ON == vLLM --speculative-config mtp == our-OFF, acceptance 16/16), c1 above vLLM every-axis (I6), c2-c8 on-par-or-above (I7), mixed-batch concurrency model-independently bit-exact (I7), server/CLI/C-ABI --speculative-config; spec-OFF byte-identical (27B 235/235). dense safetensors loader src/vllm/model_executor/models/qwen3_5_mtp.cpp:271; forward src/vllm/model_executor/models/qwen3_5.cpp:3359; mixed batch qwen3_5.cpp:3301 (GdnBlockPagedMixedSpec); tests tests/vllm/v1/spec_decode/test_mtp_speculator.cpp:201,299, tests/parity/test_qwen27_spec_decode.cpp, tests/parity/test_qwen27_spec_decode_concurrent.cpp, tests/vllm/models/test_qwen3_5_gdn_spec_routing.cpp; DGX oracle runner tests/parity/test_op_parity.cpp:1226; DONE closure ledger | 72f9fb1 |
MODEL-SPEC-qwen3-5-mtp-qwen3-5-moe-mtp | Qwen3_5MoeMTP | registry.py:628; vllm/model_executor/models/qwen3_5_mtp.py::Qwen3_5MoeMTP | speculative draft / target-dependent | draft runner; acceptance/sampling; GDN verify path; MTP | ✅ Qwen3.5 MTP spec | DONE | DONE 2026-07-26 (CLAIM-SPEC-MTP-M-MTP-2, M-mtp-2): k=1 MoE-MTP spec-decode full e2e three-way token gate on the real 35B NVFP4 — our spec-ON == our spec-OFF == vLLM 0.25.0 --speculative-config {"method":"mtp","num_speculative_tokens":1} greedy == vLLM 0.25.0 spec-OFF, token-for-token 16/16 vs the qwen36_logits_35b/greedy_ids anchor (STRICT, deterministic at c1); acceptance 16/16 both sides (vLLM rate 1.0 == ours); RAN on dgx (9/9 assertions). c1 our-engine A/B spec-ON vs spec-OFF: TPOT 11.80 vs 14.03 ms (1.19x), output-tput 78.73 vs 67.68 tok/s (+16.3%), acceptance 0.908 — MoE speedup transfers. spec-OFF byte-identical (test+docs-only diff; 35B SACRED 315/315 unaffected). fused-stack safetensors split src/vllm/model_executor/models/qwen3_5_mtp.cpp:83,221,271; forward src/vllm/model_executor/models/qwen3_5.cpp:3347,3359; MoE kind select src/vllm/entrypoints/model_loader.cpp:582; tests tests/vllm/v1/spec_decode/test_mtp_speculator.cpp:225,331, tests/parity/test_qwen36_spec_decode.cpp; oracle capture tools/parity/capture_qwen36_spec_greedy.py; DGX oracle runner tests/parity/test_op_parity.cpp:1226; ledger | 3662ec5 |
MODEL-SPEC-hy-v3-mtp-hyv3-mtp | HYV3MTPModel | registry.py:629; vllm/model_executor/models/hy_v3_mtp.py::HYV3MTP | speculative draft / target-dependent | draft runner; acceptance/sampling; FusedMoE/grouped GEMM; MTP | ☐ required | INVENTORIED | none | unassigned |
MODEL-HFALIAS - Static Transformers aliases
| ID | Item / architecture aliases | Upstream | Task / modality | Dependencies | Spike/spec | State | Our code + tests/evidence | Owner |
|---|---|---|---|---|---|---|---|---|
MODEL-HFALIAS-transformers-transformers-for-causal-lm | GPTBigCodeForCausalLM, SmolLM3ForCausalLM, Starcoder2ForCausalLM | registry.py:637-639; vllm/model_executor/models/transformers/__init__.py::TransformersForCausalLM | Transformers-backed generation / text or multimodal | Transformers compatibility adapter | ☐ required | INVENTORIED | none | unassigned |
MODEL-HFALIAS-transformers-transformers-multi-modal-for-causal-lm | Emu3ForConditionalGeneration | registry.py:641-644; vllm/model_executor/models/transformers/__init__.py::TransformersMultiModalForCausalLM | Transformers-backed generation / text or multimodal | Transformers compatibility adapter | ☐ required | INVENTORIED | none | unassigned |
MODEL-HFBACKEND - Generic Transformers backend
| ID | Item / architecture aliases | Upstream | Task / modality | Dependencies | Spike/spec | State | Our code + tests/evidence | Owner |
|---|---|---|---|---|---|---|---|---|
MODEL-HFBACKEND-transformers-transformers-for-causal-lm | TransformersForCausalLM | registry.py:649; vllm/model_executor/models/transformers/__init__.py::TransformersForCausalLM | causal generation / text backend | Transformers compatibility adapter | ☐ required | INVENTORIED | none | unassigned |
MODEL-HFBACKEND-transformers-transformers-mo-efor-causal-lm | TransformersMoEForCausalLM | registry.py:650; vllm/model_executor/models/transformers/__init__.py::TransformersMoEForCausalLM | causal generation / text backend | Transformers compatibility adapter | ☐ required | INVENTORIED | none | unassigned |
MODEL-HFBACKEND-transformers-transformers-multi-modal-for-causal-lm | TransformersMultiModalForCausalLM | registry.py:652-655; vllm/model_executor/models/transformers/__init__.py::TransformersMultiModalForCausalLM | causal generation / multimodal backend | Transformers compatibility adapter | ☐ required | INVENTORIED | none | unassigned |
MODEL-HFBACKEND-transformers-transformers-multi-modal-mo-efor-causal-lm | TransformersMultiModalMoEForCausalLM | registry.py:656-659; vllm/model_executor/models/transformers/__init__.py::TransformersMultiModalMoEForCausalLM | causal generation / multimodal backend | Transformers compatibility adapter | ☐ required | INVENTORIED | none | unassigned |
MODEL-HFBACKEND-transformers-transformers-embedding-model | TransformersEmbeddingModel | registry.py:661; vllm/model_executor/models/transformers/__init__.py::TransformersEmbeddingModel | embedding / backend-generic | Transformers compatibility adapter | ☐ required | INVENTORIED | none | unassigned |
MODEL-HFBACKEND-transformers-transformers-mo-eembedding-model | TransformersMoEEmbeddingModel | registry.py:662; vllm/model_executor/models/transformers/__init__.py::TransformersMoEEmbeddingModel | embedding / backend-generic | Transformers compatibility adapter | ☐ required | INVENTORIED | none | unassigned |
MODEL-HFBACKEND-transformers-transformers-multi-modal-embedding-model | TransformersMultiModalEmbeddingModel | registry.py:663-666; vllm/model_executor/models/transformers/__init__.py::TransformersMultiModalEmbeddingModel | embedding / backend-generic | Transformers compatibility adapter | ☐ required | INVENTORIED | none | unassigned |
MODEL-HFBACKEND-transformers-transformers-for-sequence-classification | TransformersForSequenceClassification | registry.py:668-671; vllm/model_executor/models/transformers/__init__.py::TransformersForSequenceClassification | sequence classification / backend-generic | Transformers compatibility adapter | ☐ required | INVENTORIED | none | unassigned |
MODEL-HFBACKEND-transformers-transformers-mo-efor-sequence-classification | TransformersMoEForSequenceClassification | registry.py:672-675; vllm/model_executor/models/transformers/__init__.py::TransformersMoEForSequenceClassification | sequence classification / backend-generic | Transformers compatibility adapter | ☐ required | INVENTORIED | none | unassigned |
MODEL-HFBACKEND-transformers-transformers-multi-modal-for-sequence-classification | TransformersMultiModalForSequenceClassification | registry.py:676-679; vllm/model_executor/models/transformers/__init__.py::TransformersMultiModalForSequenceClassification | sequence classification / backend-generic | Transformers compatibility adapter | ☐ required | INVENTORIED | none | unassigned |
MODEL-EMBED-bert-bert-for-masked-lm | BertForMaskedLM | registry.py:217; vllm/model_executor/models/bert.py | embedding / pooling | upstream-derived 2026-08-05, never previously inventoried | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-cheers-cheers | Cheers | registry.py:354; vllm/model_executor/models/cheers.py | multimodal generation | upstream-derived 2026-08-05, never previously inventoried | ☐ required | INVENTORIED | none | unassigned |
MODEL-EMBED-colqwen3-col-qwen3 | ColQwen3 | registry.py:278; vllm/model_executor/models/colqwen3.py | late-interaction retrieval | upstream-derived 2026-08-05, never previously inventoried | ☐ required | INVENTORIED | none | unassigned |
MODEL-EMBED-colqwen3-5-col-qwen3_5 | ColQwen3_5 | registry.py:280; vllm/model_executor/models/colqwen3_5.py | late-interaction retrieval | upstream-derived 2026-08-05, never previously inventoried | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-cosmos3edgeforconditionalgeneration-cosmos3-edge-for-conditional-generation | Cosmos3EdgeForConditionalGeneration | registry.py:361; vllm/model_executor/models/cosmos3edgeforconditionalgeneration.py | multimodal generation | upstream-derived 2026-08-05, never previously inventoried | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-diffusiongemmaforblockdiffusion-diffusion-gemma-for-block-diffusion | DiffusionGemmaForBlockDiffusion | registry.py:400; vllm/model_executor/models/diffusiongemmaforblockdiffusion.py | multimodal generation | upstream-derived 2026-08-05, never previously inventoried | ☐ required | INVENTORIED | none | unassigned |
MODEL-SPEC-exaone4-5-mtp-exaone4_5_mtp | Exaone4_5_MTP | registry.py:637; vllm/model_executor/models/exaone4_5_mtp.py | speculative draft head | upstream-derived 2026-08-05, never previously inventoried | ☐ required | INVENTORIED | none | unassigned |
MODEL-SPEC-gemma4-dspark-gemma4-dspark-model | Gemma4DSparkModel | registry.py:611; vllm/model_executor/models/gemma4_dspark.py | speculative draft head | upstream-derived 2026-08-05, never previously inventoried | ☐ required | INVENTORIED | none | unassigned |
MODEL-EMBED-colbert-hf_col-bert | HF_ColBERT | registry.py:270; vllm/model_executor/models/colbert.py | late-interaction retrieval | upstream-derived 2026-08-05, never previously inventoried | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-moondream3-hf-moondream | HfMoondream | registry.py:510; vllm/model_executor/models/moondream3.py | multimodal generation | upstream-derived 2026-08-05, never previously inventoried | ☐ required | INVENTORIED | none | unassigned |
MODEL-TEXT-vllm.models.inkling-inkling-for-causal-lm | InklingForCausalLM | registry.py:163; vllm/model_executor/models/vllm.models.inkling.py | causal generation / text | upstream-derived 2026-08-05, never previously inventoried | ☐ required | INVENTORIED | none | unassigned |
MODEL-TEXT-inklingforconditionalgeneration-inkling-for-conditional-generation | InklingForConditionalGeneration | registry.py:164; vllm/model_executor/models/inklingforconditionalgeneration.py | causal generation / text | upstream-derived 2026-08-05, never previously inventoried | ☐ required | INVENTORIED | none | unassigned |
MODEL-SPEC-vllm.models.inkling-inkling-mtpmodel | InklingMTPModel | registry.py:633; vllm/model_executor/models/vllm.models.inkling.py | speculative draft head | upstream-derived 2026-08-05, never previously inventoried | ☐ required | INVENTORIED | none | unassigned |
MODEL-EMBED-jina-jina-for-ranking | JinaForRanking | registry.py:274; vllm/model_executor/models/jina.py | late-interaction retrieval | upstream-derived 2026-08-05, never previously inventoried | ☐ required | INVENTORIED | none | unassigned |
MODEL-EMBED-jina-vl-jina-vlfor-ranking | JinaVLForRanking | registry.py:333; vllm/model_executor/models/jina_vl.py | sequence classification | upstream-derived 2026-08-05, never previously inventoried | ☐ required | INVENTORIED | none | unassigned |
MODEL-SPEC-llama-eagle3-llama-for-causal-lmeagle3 | LlamaForCausalLMEagle3 | registry.py:616; vllm/model_executor/models/llama_eagle3.py | speculative draft head | upstream-derived 2026-08-05, never previously inventoried | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-nemotron-vl-llama_nemotron_nano_vl | Llama_Nemotron_Nano_VL | registry.py:472; vllm/model_executor/models/nemotron_vl.py | multimodal generation | upstream-derived 2026-08-05, never previously inventoried | ☐ required | INVENTORIED | none | unassigned |
MODEL-TEXT-longcatflashngramforcausallm-longcat-flash-ngram-for-causal-lm | LongcatFlashNgramForCausalLM | registry.py:149; vllm/model_executor/models/longcatflashngramforcausallm.py | causal generation / text | upstream-derived 2026-08-05, never previously inventoried | ☐ required | INVENTORIED | none | unassigned |
MODEL-SPEC-vllm.models.minimax-m3-mini-max-m3-mtp | MiniMaxM3MTP | registry.py:631; vllm/model_executor/models/vllm.models.minimax_m3.py | speculative draft head | upstream-derived 2026-08-05, never previously inventoried | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-nvlm-d-nvlm_d | NVLM_D | registry.py:514; vllm/model_executor/models/nvlm_d.py | multimodal generation | upstream-derived 2026-08-05, never previously inventoried | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-nano-nemotron-vl-nemotron-h_nano_omni_reasoning_v3 | NemotronH_Nano_Omni_Reasoning_V3 | registry.py:512; vllm/model_executor/models/nano_nemotron_vl.py | multimodal generation | upstream-derived 2026-08-05, never previously inventoried | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-nano-nemotron-vl-nemotron-h_nano_vl_v2 | NemotronH_Nano_VL_V2 | registry.py:511; vllm/model_executor/models/nano_nemotron_vl.py | multimodal generation | upstream-derived 2026-08-05, never previously inventoried | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-nano-nemotron-vl-nemotron-h_super_omni_reasoning_v3 | NemotronH_Super_Omni_Reasoning_V3 | registry.py:513; vllm/model_executor/models/nano_nemotron_vl.py | multimodal generation | upstream-derived 2026-08-05, never previously inventoried | ☐ required | INVENTORIED | none | unassigned |
MODEL-MM-ovis2-5-ovis2_5 | Ovis2_5 | registry.py:522; vllm/model_executor/models/ovis2_5.py | multimodal generation | upstream-derived 2026-08-05, never previously inventoried | ☐ required | INVENTORIED | none | unassigned |
MODEL-EMBED-terratorch-prithvi-geo-spatial-mae | PrithviGeoSpatialMAE | registry.py:264; vllm/model_executor/models/terratorch.py | embedding / pooling | upstream-derived 2026-08-05, never previously inventoried | ☐ required | INVENTORIED | none | unassigned |
MODEL-SPEC-qwen3-5-mtp-qwen3_5-mtp | Qwen3_5MTP | registry.py:647; vllm/model_executor/models/qwen3_5_mtp.py | speculative draft head | upstream-derived 2026-08-05, never previously inventoried | ☐ required | INVENTORIED | none | unassigned |
MODEL-SPEC-qwen3-5-mtp-qwen3_5-moe-mtp | Qwen3_5MoeMTP | registry.py:648; vllm/model_executor/models/qwen3_5_mtp.py | speculative draft head | upstream-derived 2026-08-05, never previously inventoried | ☐ required | INVENTORIED | none | unassigned |
MODEL-SPEC-dots3-note-dots3-note-mtp | Dots3NoteMTPModel | BEYOND-PIN — NOT in 555967922; on vLLM main at registry.py:670 (_SPECULATIVE_DECODING_MODELS) -> ("vllm.models.dots3_note", "Dots3NoteMTP") (re-derived at c205726108df54bb6fbf15b19e725a4a3add2b18; the W0 record read :662 at the revision it was taken from), implemented in vllm/models/dots3_note/nvidia/mtp.py (Dots3NoteMultiTokenPredictorLayer:31, Dots3NoteMultiTokenPredictor(DeepseekV32MultiTokenPredictor):88, Dots3NoteMTP(DeepseekV32MTP):141 with has_own_embed_tokens=True / has_own_lm_head=False) | speculative draft / target-dependent | draft runner; acceptance/sampling; MTP; the whole MODEL-MM-dots3-note-dots3-note-for-causal-lm target stack | dots3-note spec §1.4/W10, issue #699 | INVENTORIED | none. Scoped only: Dots3NoteConfig defaults num_nextn_predict_layers to 1 (transformers_utils/configs/dots3_note.py:24), so the model card's "three-token speculative decoding" reads as the speculation depth the single nextn head is driven at, not a head count — inferred from the config default, NOT verified against the checkpoint; RESOLVED at W1 (2026-08-23) from the released shard index: the checkpoint ships EXACTLY ONE nextn layer, model.layers.46.* (18 tensors), so the reading holds — one head driven at depth three, not three heads. It carries the SLIDING attention tensor set (no indexer.*, q_b_proj [16384,1024] = 64 x (192+64)) and a DENSE MLP; model.mtp.embed_tokens.weight is present and shared_head.head.weight is absent, matching has_own_embed_tokens=True / has_own_lm_head=False (mtp.py:141-142). The sliding half is the CHECKPOINT's answer, not upstream's: Dots3NoteDecoderLayer picks its attention class from config.layer_types[layer_idx] (model.py:503) and layer_types has no entry at index 46, so W10 owes that reconciliation. Still NOT registered as a speculator — registering one that cannot propose makes the engine accept a config it then dies on mid-run. Blocked behind the target row's oracle and hardware gaps (spec §6) | unassigned |
MODEL-EMBED-roberta-roberta-for-token-classification | RobertaForTokenClassification | registry.py:304; vllm/model_executor/models/roberta.py | token classification | upstream-derived 2026-08-05, never previously inventoried | ☐ required | INVENTORIED | none | unassigned |
MODEL-HFBACKEND-transformers-vault-gemma-for-causal-lm | VaultGemmaForCausalLM | registry.py:662; vllm/model_executor/models/transformers.py | backend-generic adapter | upstream-derived 2026-08-05, never previously inventoried | ☐ required | INVENTORIED | none | unassigned |
MODEL-HFBACKEND-vibevoiceasrforconditionalgeneration-vibe-voice-asr-for-conditional-generation | VibeVoiceAsrForConditionalGeneration | registry.py:668; vllm/model_executor/models/vibevoiceasrforconditionalgeneration.py | backend-generic adapter | upstream-derived 2026-08-05, never previously inventoried | ☐ required | INVENTORIED | none | unassigned |
MODEL-EMBED-xlmrobertafortokenclassification-xlmroberta-for-token-classification | XLMRobertaForTokenClassification | registry.py:305; vllm/model_executor/models/xlmrobertafortokenclassification.py | token classification | upstream-derived 2026-08-05, never previously inventoried | ☐ required | INVENTORIED | none | unassigned |
Dynamic Transformers compatibility
| ID | Item / surface | Upstream | Task / modality | Dependencies | Spike/spec | State | Our code + tests/evidence | Owner |
|---|---|---|---|---|---|---|---|---|
MODEL-HFDYNAMIC-capability-factory | Any Transformers or remote auto_map architecture whose model class reports backend compatibility; not a finite architecture list | registry.py:1096-1164; vllm/model_executor/models/transformers/ | task inferred from config and generic backend class | Transformers graph/model adapter; remote-code policy; attention replacement; weight loader; pooler/MM/MoE adapters | ☐ define pure-C++ compatibility boundary | INVENTORIED | none | unassigned |
Completion for this row is a capability contract and compatibility suite, not a claim that a finite snapshot enumerates every future Transformers architecture.
Corrections exposed by the inventory
.agents/porting-inventory.md:129says 292 registered upstream models; the pinned static registry has 353 unique architecture IDs..agents/feature-matrix.md:77marks the model registry✅and says dense/MoE dispatch is included. The direct registry accepts onlyQwen3_5MoeForConditionalGenerationatsrc/vllm/model_executor/models/registry.cpp:10-20.include/vllm/model_executor/models/registry.h:18-21hardcodesQwen3_5MoeWeightsintoModelForwardFn; it is not a generic family factory.LoadedEngine::IsDenseArchatsrc/vllm/entrypoints/model_loader.cpp:232-237,266-277classifies every config withnum_experts == 0as Qwen3.5 dense. Unsupported dense architectures can be misrouted instead of rejected.- The Qwen3.5/3.6 gate row is complete only for text execution. Both registered
IDs are multimodal wrappers (
registry.py:556-560); their vision path remains unimplemented locally. BambaForCausalLMis not supported at the pin; it is listed under_PREVIOUSLY_SUPPORTED_MODELSatregistry.py:720.- The current broad pooling and multimodal feature rows hide distinct embedding, late-interaction, reward, token-classification, sequence-classification, 114-entry multimodal, 44-entry speculative-model, and Transformers work fronts.
- At groundwork creation, existing feature-matrix links for
model-registry.md,dense-decoders.md,moe-decoders.md,qwen3-next.md,hybrid-others.md,mla-family.md,pooling-models.md,multimodal.md, andencoder-decoder-asr.mddo not exist. A row is not spiked merely because a placeholder link was present.
These corrections are reflected in the current matrices and roadmap. Future changes must keep the roadmap, feature view, matrix, state log, and ledger consistent in the same commit.
Factory-spike and family ordering
MODEL-FACTORY-registry: replace the model-specific function-pointer seam with a task-aware, type-erased model factory and explicit architecture checks.- Dense shared stack: Llama-compatible targets, Qwen2/3, Mistral/Ministral, Gemma, and Phi. Each target still gets its own weight/config/test spike.
- MoE shared stack: Mixtral, Qwen2/3-MoE, GLM4-MoE, and OLMoE.
- Qwen3-Next in parallel with the already-written Qwen3.5 MTP and DFlash spikes.
- Remaining Mamba/GDN hybrids, then DeepSeek/MLA/DSA families.
- Embedding, late-interaction, reward, token-classification, and sequence-classification fronts. These also ground later ANE/MLX encoder work.
- Complete Qwen3.5 gate-model vision wrappers, then broader image, audio, video, OCR, and encoder-decoder/ASR families.
- Generic/dynamic Transformers compatibility after the native factory contract has a defensible pure-C++ boundary.
Tests to port
The factory spike and every family implementation must carry the applicable upstream tests in the same change. The minimum inventory is:
Upstream tests at e24d1b24 | Our tier | Obligation |
|---|---|---|
tests/models/test_registry.py:31-158 | T-unit | Port lazy registration/import, model capability inspection, PP flags, failure behavior, and complete fixture coverage checks. |
tests/models/registry.py:16-1742 | T-unit fixture/spec | Preserve a traceable tiny/default model fixture per architecture alias, including task, tokenizer, trust-remote-code, dtype, and version constraints. |
tests/models/test_initialization.py:50-197 | T-unit/T-parity | Port model construction and weight-loading smoke coverage for native and generic targets. Unsupported targets remain explicit skips with tracked reasons. |
tests/models/test_transformers.py | T-parity/e2e | Port native-vs-generic backend generation, hybrid attention, distributed, quantized, embedding, and classification equivalence cases applicable to the C++ compatibility contract. |
tests/models/language/generation/test_common.py, test_gemma.py, test_granite.py, test_hybrid.py, test_mistral.py, test_phimoe.py | T-parity | Each generation-family spike inventories and ports its named model cases, token/logprob comparisons, hybrid-state behavior, and quantized variants. |
tests/models/language/pooling/ | T-unit/T-parity/e2e | Embedding, ColBERT, SPLADE, reward, classification, reranking, truncation, prefix-cache, dtype, and pooling API behavior. Select exact files/cases per target spike. |
tests/models/multimodal/test_mapping.py, multimodal/processing/, multimodal/generation/, multimodal/pooling/ | T-unit/T-parity/e2e | Modality mapping/schema, processor replacement, encoder/merger correctness, generation/pooling, CUDA-graph, interleaving, and memory behavior per target. |
tests/v1/spec_decode/test_mtp.py, test_eagle.py, plus method/model-specific spec-decode tests | T-unit/T-parity/e2e | Draft-model construction, state/KV handling, proposal and acceptance behavior, target compatibility, and token-exact end-to-end gains. |
tests/plugins/vllm_add_dummy_model/ and public ModelRegistry.register_model tests reached by tests/models/test_registry.py | T-unit | Decide and test the pure-C++ plugin/registration boundary rather than silently dropping the public extension surface. |
A target spike replaces these directory-level pointers with exact file and case
names. Tests blocked on a dependency are checked in skipped with the MODEL-* ID
and tracked reason; they are never removed from the target contract.
Per-target spike deliverable
Before changing a row from ☐ to ✅ in the Spike column, its spec must record:
- every upstream model, layer, platform, quantization, loader, and runtime source
path used on the actual target execution chain, with
file:lineanchors; - architecture aliases and representative upstream test checkpoints;
- reusable local ops plus exact code/test anchors, and every missing dependency;
- config and weight-name mapping, supported dtypes/quants, KV/state layout, parallelism, long-context, modality, and speculative-decoding interactions;
- upstream tests to port by exact file/case and our T-unit/T-parity/T-e2e mapping;
- correctness and same-workload vLLM performance gates, backend matrix, memory
gates, and
nsystrace plan where performance is affected; - an implementation split small enough for one agent/branch, with prerequisite
MODEL-*or kernel/backend IDs and no duplicate owner.