Kimi K3, as released

Weights went public 2026-07-27 17:14 Europe/Rome at moonshotai/Kimi-K3. Everything here is read from the released config.json, model.safetensors.index.json and a ranged read of one shard header — not from the announcement.

The headline correction

The announcement says 2.8T parameters. A naive reading of the config (92 MoE layers x 896 experts x 3 x hidden 7168 x moe_intermediate 3072) gives 5.45T, and the download is only 1.42 TB, which at 4 bits would be 2.7T. Both discrepancies have one cause:

K3's MoE is a latent MoE. Layers carry block_sparse_moe.routed_expert_down_proj, ..._up_proj and ..._norm: the 7168-wide hidden state is projected down to a 3584-wide latent, the experts run there, and the result is projected back up. This is the "Stable LatentMoE" of the announcement.

Confirmed against a shard header:

experts.0.w1.weight_packed   U8 [3072, 1792]   scale U8 [3072, 112]
experts.0.w2.weight_packed   U8 [3584, 1536]   scale U8 [3584, 96]

1792 bytes at 4 bits = 3584 values; 112 scales x 32 = 3584. So w1/w3 are [3072, 3584] and w2 is [3584, 3072], stored MXFP4 with one E8M0 scale per 32 weights. That gives 33.0 M params per expert, 92 x 896 x 33.0 M = 2.72 T routed, and 17.5 MB per expert on disk — 1.44 TB over 82,432 experts, matching the actual download.

Experts are half the size we assumed, and so is per-token I/O.

Shape

property	value	vs our pre-release assumption
layers	93	assumed 60
hidden	7168	7168 ✓
experts / top-k	896 / 16	✓
MoE layers	92 (layer 0 dense)	✓
expert latent dim	3584	not anticipated
moe_intermediate	3072	assumed 2048
shared experts	2	assumed 1
attention heads	96	assumed 64
KDA : full-attention layers	69 : 24 (~2.9:1)	assumed 3:1 ✓
KDA heads x dim	96 x 128	assumed 32 x 128
short conv	kernel 4	✓
MLA	q_lora 1536, kv_lora 512, qk_nope 128, qk_rope 64, v 128, NoPE	✓ (q_lora was null in Kimi-Linear)
router	sigmoid, 1 group, routed_scaling_factor 1.0	✓ (2.446 in Kimi-Linear)
vocab	163840, same tiktoken	✓
MTP head	none (num_nextn_predict_layers: 0)	hoped for one
download	1.42 TB, 96 shards, 497,220 tensors	estimated 1.5 TB ✓

What is new since Kimi-Linear

Four things the 48B model did not have, all of which the engine must implement:

1. Latent MoE — routed_expert_{down,up}_proj + routed_expert_norm around the expert block. Halves expert cost; adds two dense matmuls per MoE layer.
2. Attention Residuals (AttnRes) — self_attention_res_proj, mlp_res_proj, *_res_norm per layer plus a model-level output_attn_res_proj, with attn_res_block_size: 12. A second residual pathway grouped in blocks of 12 layers.
3. SiTU activation — hidden_act: "situ" (Sigmoid Tanh Unit) with activation_situ_beta: 4.0 and activation_situ_linear_beta: 25.0, replacing SiLU.
4. Full-rank KDA gate — use_full_rank_gate: true and a single self_attn.g_proj instead of Kimi-Linear's g_a_proj/g_b_proj rank-128 bottleneck. Also gate_lower_bound: -5.0 clamps the decay. The decay gate itself keeps the f_a_proj/f_b_proj low-rank form.

Also: K3 is multimodal (mm_projector.*, image_placeholder, KimiK3ForConditionalGeneration). The text path is language_model.* and is self-contained, which is why it was built first; vision landed later and is documented below.

What this does to the budget

Recomputed with the real 3584 latent (--expert-in 3584):

bits/weight	expert record	routed on disk	GB read/token (cold)
2.01	7.9 MB	637 GB	11.4 GB
3.01	11.8 MB	954 GB	17.1 GB
4.25 (as shipped)	16.7 MB	1347 GB	24.1 GB

At the Gate 3 operating point of 3 bits the expert set is 954 GB — it fits the 1.7 TB internal SSD with room for the trunk. Per-token cold I/O is 17.1 GB, against the 12.5 GB we had been assuming: worse than the optimistic case, better than the 34 GB the naive (non-latent) reading implied.

The trunk grows: 93 layers x 96 heads, q_lora 1536, two shared experts. tools/memplan.py --data <k3> --expert-bits 3.01 gives the current figure; the RAM floor is around 18 GB rather than 11 GB, still well under 64.

(Estimated before the container existed and low by two thirds. Measured, the trunk is 27.28 GB resident and the floor 29.06 GB at 4K context — which leaves 17 to 29 GB for the cache on a 64 GB machine, not the ~46 this section goes on to assume. That gap is most of the difference between the 1.5 tok/s projected here and the 0.3 measured.)

One token's expert working set — the number Gate 5 showed is the real cache floor — is 16 experts x 92 layers x 11.8 MB = 17.4 GB. A 64 GB machine has roughly 46 GB for cache after the floor, i.e. 2.6x the working set. Gate 5 measured 40% hit rate at that ratio on Kimi-Linear.

Port status (2026-07-27, weights still downloading)

All four new components are implemented in the C engine from the released reference code, behind config detection so Kimi-Linear is unaffected:

component	where	how it is detected
latent MoE	moe_layer — down_proj, experts on the latent, optional norm, up_proj; shared experts stay on the full hidden state	routed_expert_hidden_size present
SiTU	situ_pair(), used by both FFN paths	hidden_act == "situ"
full-rank KDA gate	kda_layer takes g_proj instead of g_a/g_b	use_full_rank_gate
bounded decay gate	g = lower_bound * sigmoid(exp(A_log)*z) instead of -exp(A_log)*softplus(z)	gate_lower_bound present
Attention Residuals	apply_attn_res() + the rewritten layer loop	attn_res_block_size present
language_model. prefix and nested text_config	config load and every tensor lookup	text_config present

The bounded gate is worth calling out: it is a different formula, not a clamp. Kimi-Linear uses -exp(A_log)·softplus(z), unbounded below; K3 uses lower_bound·sigmoid(exp(A_log)·z), which confines the log-decay to (-5, 0) and so the decay itself to (e⁻⁵, 1).

Attention Residuals are a softmax attention over the layer-block residual history: each layer scores its running sum against up to n_layers / 12 + 1 stored block residuals and mixes them. Decode cost is a few hundred KB of state and 8 dot products per layer — negligible next to the experts.

Verified two ways.

Regression: on Kimi-Linear the engine still matches the oracle at rel 1.50e-06 with argmax and top-10 identical, and the CLI still produces the right continuation — the new code paths stay dormant when the config does not ask for them.

Component tests: the three pieces whose maths is actually new are checked in isolation on synthetic inputs against the released reference, since K3 cannot be run end to end yet (tests/test_k3parts.c + tools/k3parts_ref.py, wired into make test):

component	reference	max abs diff
SiTU (beta 4, linear_beta 25)	SituAndMul in modeling_kimi_linear.py	7.6e-06
decay gate, K3 form	naive_kda_lowerbound_gate in fla	4.8e-07
decay gate, Kimi-Linear form	naive_kda_gate	6.0e-08
AttnRes, 5 blocks	_apply_attn_res	3.6e-07

Both gate forms are tested from the same inputs, which also confirms they are genuinely different functions rather than one being a clamped version of the other.

What is not covered by this: the wiring — whether the latent projections, the block-residual bookkeeping and the tensor names are hooked up in the right order. That is what the token-exact oracle run will catch, and it needs the weights.

Three bugs surfaced doing this, all worth recording:

1. waste_model_free freed the tensor array before freeing the per-tensor quantized buffers — a use-after-free that only bit through the API path.
2. b64decode shifted a signed accumulator past bit 31. Undefined behaviour that clang turned into a trap at -O2 while sanitizer builds ran fine.
3. The Makefile had no header dependencies. Adding fields to waste_config left objects compiled against the old layout; they linked cleanly and corrupted memory at run time, and every sanitizer build was clean because those rebuild everything. Fixed with -MMD -MP and -include.

Gate 3 repeated on real K3 experts (2026-07-27, 5 shards in)

tools/mxfp4.py reads the shipped format: weight_packed (two E2M1 nibbles per byte, low nibble first) plus weight_scale (one E8M0 exponent per 32 weights, value exactly 2^(e-127)). Verified bit-identical to compressed_tensors' own unpacker on a real K3 expert — 0.000e+00 max difference.

A caution worth recording: the first self-check tried to confirm the nibble order from group statistics, and it cannot. Swapping nibbles permutes elements inside a byte pair, which stays within the same group of 32, so every group statistic is unchanged. The check now asserts what it can actually see — with a power-of-two scale every group's amax/scale must land in (3, 6], measured 100.00% — and the ordering question was settled against the reference implementation instead.

Requantizing real K3 experts (layer 2, 16 experts, 176 M params):

scheme	bits	weight err (w1)	(w2)
rtn4-row	4.00	18.35%	17.32%
rtn3-g64	3.25	26.50%	—
vq3	3.00	20.27%	19.57%
vq2	2.00	34.22%	—

The Gate 3 conclusion holds on K3. VQ3R at 3 bits sits at ~20%, close to the 19.4% measured on Kimi-Linear, and still beats grouped RTN at a higher bit budget (26.5% at 3.25 bits). Notably the source is already 4-bit, and requantizing 4→3 bits costs only about 2 pp over rtn4-row's 18.4% — the QAT-from-MXFP4 training does appear to leave the weights tolerant, as hoped.

2-bit remains unsafe at 34%, as on Kimi-Linear.

Converter: MXFP4 in, WASTE out (tested on real K3 shards)

tools/convert.py now reads K3 as shipped. Three changes were needed:

1. MXFP4 experts. Only the routed experts are packed — the whole trunk, including the latent projections and the shared experts, is plain bf16. Expert reads go through mxfp4.ST, which dequantizes transparently.
2. Nested text_config and the language_model. prefix, recorded in the manifest as tensor_prefix so readers do not have to guess.
3. Streaming per expert. The old loop stacked a whole layer before quantizing. At Kimi-Linear's size that was 7 GB; at K3's it would be 118 GB in f32 for one layer — it simply could not run. Codebooks are now fitted on a sample of experts (--cb-sample, default 12) and then each expert is loaded, quantized and written on its own, so peak memory is a few hundred MB regardless of model size.

Verified on layer 2 with the shards already downloaded: 24 experts converted and round-tripped, 19.59–19.77% relative error, CRCs good, records 4 KiB-aligned and readable through the C structs.

The real per-expert record is 11.83 MB = 3.00 bits/weight, which settles the projections with measured rather than estimated numbers:

quantity	value
expert set on disk	952 GB (fits the 1.7 TB internal SSD)
I/O per cold token	17.0 GB
one token's working set = the cache floor (Gate 5)	17.0 GB
conversion rate observed	~1 s/expert single-threaded

Making the conversion tractable: 23 h -> 4.7 h

At first the converter ran at ~1 s/expert, i.e. ~23 hours for 82,432 experts — longer than the download. Profiling one expert put 82% of that in the VQ assignment, and the reason was not arithmetic: assigning 8-dim vectors to a 256-entry codebook in torch materializes an [n, 256] distance matrix for the argmin. For one expert matrix that is 1.4 GB written and read back per stage, which pinned throughput at roughly 0.5% of the machine's compute peak.

src/vq.c fuses the two: each vector's 256 distances live in registers, the 8 KB codebook stays in L1, all three stages run in one pass over the vector, and only the winning byte is written. NEON for the dim-8 case, driven by the engine's own thread pool, exposed to the converter as libwastevq.dylib through ctypes with the torch path kept as a fallback.

Layers are also converted in separate processes (--jobs), each with its own codebook file; ids are handed out by the parent from layer order so the merge is a plain concatenation.

Measured on real K3 experts, trunk excluded:

	s/expert	full model
torch encoder, 1 process	1.04	23.7 h
native encoder, 1 process	0.323	7.4 h
native, 3 processes	0.205	4.7 h
native, 6 processes	0.212	4.8 h

Beyond 3 processes it flattens: the native encoder already uses every core, so more processes just contend. Output is unchanged — 19.69–19.76% relative error, identical to the torch path.

Making it fit one disk: --reclaim

Time was not the only thing that made K3 awkward to convert. Peak disk is the source plus the container — 1.42 TB of staging alongside a 982 GB container, and 2.4 TB is two drives for a model whose whole point is running on a laptop.

What makes this safe rather than clever is a property of the converter, not a heuristic: every tensor it reads has exactly one consumer. A routed expert belongs to one layer, and everything else is read once by the single trunk pass. So a shard whose last consumer has finished will never be opened again, and can be deleted while the run continues. Peak staging becomes the container plus the shards still owed.

--reclaim dry     # name the shards it would delete, delete nothing
--reclaim on      # delete them

Off by default. On Kimi-Linear's 20 shards, converting layers 1 and 2 with dry:

reclaim=dry: 20 shards, 91.5 GiB of staging to give back (nothing is deleted)
  reclaim: layer 2 would free 1 shard(s), 4.7 GiB (4.7 GiB total)
reclaim: 4.7 GiB reclaimable from 1 shard(s); 19 shard(s) (86.8 GiB) still held
  waiting on: layer 3, layer 4, layer 5, layer 6 ...

It refuses before deleting rather than during. A shard that lives inside the output directory, a download that is incomplete or unverified, a trunk whose shards an earlier run already consumed — each stops the run with a reason and nothing removed. That ordering matters more than usual here, because the failure it avoids is a half-reclaimed checkpoint, which is neither a source you can convert from nor one you can verify against.

The ledger is .reclaimed in the source directory, and it is written before the file is removed. A shard recorded but not yet deleted costs a resumed run nothing; one deleted but not recorded is indistinguishable from a download that never finished. fetch_weights.sh's .download-state is not enough on its own — it is believed over the filesystem, so a reclaimed shard would still read as present.

It is not reversible, and it takes something away besides disk. A reclaimed shard has to be downloaded again, and tools/verify_container.py can no longer check the container against its source, because the source is gone. Prove a recipe with the source intact — on Kimi-Linear, or on a --layers subset — and reclaim on the runs after.

Trunk bit-width: 4 bits wins, and 3 bits is a measured trade rather than a win

K3's trunk is the RAM floor, and the floor is what is left over for the expert cache, so its bit width is the main lever on hit rate. Gate 5 says a cache below one token's working set (17 GB for K3) keeps almost nothing alive, and at 4 bits the floor leaves only 15.6 GB — just under it. Hence the experiment.

trunk	floor	cache at a 56 GB budget	hit rate	tok/s
4 bit	40.7 GB	15.6 GB	10%	0.29
3 bit	34.3 GB	22.0 GB	28%	0.16

The cache half of the prediction landed: crossing the working-set threshold nearly tripled the hit rate. The throughput half did not — 3 bits is 1.8x slower overall, because the whole trunk is unpacked every token and a 3-bit bitstream costs far more per weight than 4-bit nibbles. The profile shows the KDA phase, which is mostly trunk matvecs, going to 32.8%.

So 4 bits stays the default. The 3-bit path is implemented and correct; it would become the better choice if the unpack were vectorized (eight values per three bytes with NEON shifts), which is the obvious next optimization if throughput ever becomes trunk-bound rather than I/O-bound.

That last sentence is wrong, and LEARNED.md §13 has the re-measurement. Repeated after MLA absorption put the cache at the knee where extra room actually pays, Q3G still lost — and the reason is not the clock. The logits land 36% off the 4-bit ones and generation collapses into + and spaces, because K3's QAT covered the expert weights and left the trunk with no trained tolerance for being squeezed. Vectorizing the unpack would not save it. The numbers in the table above also predate the absorption; treat §13 as the current ones.

Both widths generate correct code, which is the other thing worth knowing — 3 bits is not too coarse for the model, only for the clock:

4 bit:  def fibonacci(n):        3 bit:  def fibonacci(n):
            if n == 0:                       a, b = 0, 1
                return 0                     for _ in range(n):
            elif n == 1:
                return 1

Open questions this raises

1. Does the latent MoE change routing statistics? ANSWERED 2026-07-31. Next-token reuse is 29.5% on K3 against Kimi-Linear's 33.6% and OLMoE's 43.5% — the direction Gate 2 predicted, measured from a WASTE_DUMP_ROUTE trace of 214 tokens rather than from kimi_ref.py. Cross-layer, knowing layer L's set predicts layer L+1's at 29.0%, which does not beat the previous token and kills the reserved cross-layer prefetcher. LEARNED.md §29.
2. MXFP4 -> VQ3R requantization ANSWERED, see below.
3. AttnRes and SiTU need reference implementations in kda.c/model.c and oracle validation — the modeling code is downloaded (modeling_kimi_k3.py, modeling_kimi_linear.py).
4. No MTP head means no cheap speculative decoding; an n-gram or externally-trained draft would have to fill that role.

Against the technical report (2026-07-28)

The report and the modeling code that ships with the weights (modeling_kimi_linear.py — modeling_kimi_k3.py is only the multimodal wrapper) were read against the implementation line by line.

Confirmed correct, no change needed: the KDA recurrence and its bounded decay gate g = g_min·σ(e^{A_h} z) with g_min = -5; SiTU-GLU with β1 = 4, β2 = 25; the full-rank KDA gate and the gated MLA output; the latent MoE with RMSNorm before the up-projection; the router (Sigmoid, e_score_correction_bias for selection only, weights renormalized over the selected top-k, then routed_scaling_factor); MLA's factorized query (q_lora_rank 1536), NoPE, and q_head_dim^-0.5 scaling; 12-layer AttnRes blocks. Every field of Table 1 matches the container config, including 69 KDA + 24 MLA (layer 0 is the dense one) and 8 stored block representations + the live prefix sum = the report's "9 total blocks when counting the embedding layer".

Two things were wrong.

A_log is per head, not per channel. Eq. 5 indexes it by h; K3 ships it padded to head_dim, and indices 96..127 are exactly zero on every layer sampled. The shape-driven heuristic had been reading it per channel.

The final output aggregation was missing, and adding it exposed an aliasing bug. _apply_output_attn_res runs the last hidden state and every block representation through the same _apply_attn_res as the per-layer calls. Implementing it turned " Paris" (17374) into "________" (11743): waste_apply_attn_res memset its output before reading prefix_sum, which is harmless when the two are different buffers — as in all the per-layer calls — and fatal for the output aggregation, which passes m->x as both. Writing the prefix_sum term first, element by element, is alias-safe and needs no scratch. With it working the same prompt gives 17374 at logit 17.36 against 16.90 without: the aggregation contributes rather than cancels.

End-to-end oracle check, finally green on K3. All 93 layers agree with kimi_ref.py to ≤ 1.14e-05 relative, the final logits to 3.56e-06, argmax and top-5 identical. Getting there also took three fixes to the oracle itself: it knew only F32 and Q8G, so K3's 26.33 GiB Q4G trunk was being read as int8 (hidden state left layer 0 with norm 3.3e7 against the engine's 0.79), and it had neither q_lora nor the MLA output gate. Nothing caught that because the oracle regression runs on Kimi-Linear, whose trunk has no 4-bit tensors and whose MLA is unfactorized.

Vision (2026-07-28)

K3 is natively multimodal — a 401M ViT, 27 layers, patch 14 — and the released weights carry it: vision_tower (165 tensors) and mm_projector (3). It is now implemented, in src/vision.c (the tower and the projector) and src/image.c (a file on disk to the patch tensor). The tower is loaded only when a caller asks for it: 434 MB of weights, and 1.12 GB reserved once the bounded source decode, the tower's activations and the queued image embeddings are counted, all of it otherwise the expert cache's.

The pipeline, for one image:

patchify 14x14  ->  Conv2d(3, 1024, 14, stride 14), i.e. a matmul over 588
+ a learned 64x64 position grid, bilinearly resized to this image's grid
27 x [ RMSNorm -> packed QKV -> 2D RoPE -> attention -> Wo -> +res
       RMSNorm -> fc0 -> gelu_tanh -> fc1 -> +res ]
final RMSNorm
2x2 spatial merge (1024 -> 4096)
proj.0 -> GELU(erf) -> proj.2 -> RMSNorm  (4096 -> 7168)

tools/vision_ref.py is the oracle, transcribed from the reference modeling_kimi_k3.py rather than imported from it: that module pulls in flash-attention and the full multimodal wrapper. End to end the C matches it to 2.3e-06.

Four things were wrong on the way there, each of which would have looked like a different bug:

The reference's RoPE docstring contradicts its code. The frequency table interleaves the width axis at even indices and height at odd. The code is what the weights were trained against.

The encoder MLP uses the tanh approximation of GELU and the projector uses the exact one. Mixing them up is a small error that survives every smoke test and shows up only as slightly wrong embeddings.

The tower cannot live at 4 bits. The converter quantizes everything outside the expert banks identically, which for the text trunk is a measured 11.8% weight error the model was trained to survive. The ViT was not: at 4 bits the damage is 17.9% on the packed QKV and 12.6% on the position embedding — a learned spatial prior. tools/requant_vision.py rewrites those 223 MB at 8 bits by appending to trunk.bin and repointing the manifest, so a 982 GB container does not have to be rebuilt to fix them. It flattens by rank: a 4-D conv kernel [out, in, kh, kw] becomes [out, in*kh*kw] because that is how the engine uses it, and quantizing along the stored last axis would give rows of 14 that the engine then reads as rows of 588.

fp16 subnormals were being flushed to zero. waste_f16 returned 0 for an exponent field of 0, which is right for zero and wrong for every subnormal — the value is mantissa x $2^{-24}$. 1.08% of K3's trunk scales are subnormal, and a zeroed scale zeroes a whole group of 128 weights. This was found while bisecting a 2.4e-03 disagreement in the vision MLP; it was never visible in the text path because a few zeroed groups out of millions do not move a logit.

Images in the token stream

K3 wraps images in dedicated tokens — <|media_begin|> (163602), <|media_content|> (163603), <|media_end|> (163604), <|media_pad|> (163605). The pad is the load-bearing one, and it expands: one placeholder becomes N positions, one per merged 2x2 patch, and the prefill takes their embeddings from the tower in order rather than from the embedding table. Consuming in order is what lets several images share a prompt without tracking which is which.

The splice lives in both waste_model_prefill and waste_model_step. Only the first is obvious; the second matters because a chunked prompt whose last chunk is a single token goes through the step path, so an image at the very end of a prompt — or one whose token count divides badly — would otherwise silently get the embedding table's row for 163605.

waste_image_add / waste_image_expand / waste_generate is the API, split three ways because a host has to be able to size a prompt before committing to it. An image is not a token: an 896x896 photograph at the default 1024-patch budget is 256 of them.

The tokenizer had no special tokens at all. tiktoken's rank file holds merges only; the markup lives in tokenizer_config.json and was never converted, so <|open|> became six ordinary tokens. That silently broke every chat template and would have broken any media marker. The converter now writes specials.json and the tokenizer matches those strings whole, longest-first, before the pre-tokenizer runs.

Does the model actually see it? (2026-07-28)

A tower that matches its oracle proves the arithmetic, not that the embeddings mean anything where they land. The test is an A/B on one prompt — "Caption: A photograph of a" — run with and without an image and compared by next-token distribution.

Control, no image (the model's prior over what photographs are of):

man 0.081 | person 0.068 | woman 0.068 | young 0.039 | white 0.028

With a photograph of a rocky coastal mountain above dark water (354x354, 144 image tokens):

rugged 0.156 | mountain 0.116 | rocky 0.106 | mountainous 0.099
body 0.096 ("body of water") | large 0.067 | coastal 0.058 | lake 0.039

With an abstract render of glossy pale-blue tubes on white (896x896, 256 image tokens):

white 0.081 | close 0.063 | single 0.036 | glass 0.029
blurry 0.028 | metal 0.026 | blue 0.021 | clear 0.021

Every person word is gone in both, and each distribution describes its own picture — terrain words for the coast, colour and material words for the render. The peak also sharpens, 0.081 to 0.156, which is the model being less uncertain because it now has something to look at.

Two numbers worth keeping. Image embeddings have a mean row norm of 3.12 against 1.94 for the embedding table's text rows, a ratio of 1.61 — the same order of magnitude, which is what a correct projector should give and what the normalization constants below would break if they were badly wrong. And an image position costs 2.8 s of prefill against 2.9 s for a text one: an image is priced as text of the same length, and the tower itself (15.7 s for 1024 patches) is noise next to the 93 MoE layers its output then walks through.

The one thing taken on faith, and it was wrong (2026-07-29)

This section used to read:

K3 ships no preprocessor config. There is no preprocessor_config.json in the release and no image mean/std anywhere in the modeling code, so the pixel normalization is the CLIP convention this lineage of towers uses — a choice, not a transcription. […] If image understanding is ever subtly wrong while the tower still matches its oracle at 2.3e-06, this is the first thing to question.

The instinct was right and the fact was not. The release does ship preprocessor_config.json, and it says:

"image_mean": [0.5, 0.5, 0.5],
"image_std":  [0.5, 0.5, 0.5]

K3 normalizes to [-1, 1]. kimi_k3_vision_processing.py applies exactly those, from media_proc_cfg. The CLIP constants are not close to them — means differ by up to 0.09 and the standard deviations by nearly 2x, so every image the engine ever encoded reached the tower with the wrong contrast and a colour cast.

Why nothing caught it. Three reasons, each worth keeping:

• The file was never on disk. fetch_weights.sh downloaded a hardcoded list of filenames, and although preprocessor_config.json was on that list, so were files the repo does not have — a 404 is normal there and logged as nothing. The repo has 22 non-weight files; we had 10. It now enumerates the repo through the API instead.
• The tower's oracle runs on torch.randn pixels, so vision_ref.py never calls the image loader. The 2.3e-06 agreement was real and measured a stage downstream of the bug.
• test_image checks normalization arithmetic against constants it defines itself, so it verifies (v - mean) / std and not which mean.

The suite now compares the container's vision.json against the source's preprocessor_config.json directly, which is the check none of the above amounts to.

What it does not invalidate: the tower-vs-oracle figure, the converter, and the A/B in the previous section, where the model plainly described the right picture — with a colour cast, which is a fair measure of how much distortion a ViT will absorb before it stops recognizing a mountain.

Against a hosted API guide

A third-party guide to K3 on a hosted endpoint (AIHubMix, verified 2026-07-17) documents the serving surface rather than the model. Most of it is out of scope for a local engine — three HTTP APIs, automatic prefix caching, dynamic tool loading, structured output — but four points are worth recording because they set expectations a local run does not meet:

• 1M context. WASTE's floor at 1M is 83.22 GB against 64 GB of RAM, so the practical ceiling here is around 128K (35.64 GB). The model supports 1M; this machine does not.
• Thinking is on by default and cannot be turned down — reasoning_effort accepts only "max". The guide measured 54,486 thinking tokens out of 74,994 in one request, 73%. A hosted API splits that into reasoning_content; locally it simply arrives in the stream, and at 0.5 tok/s that is hours of thinking before an answer.
• Sampling is fixed by the vendor at temperature 1.0, top_p 0.95. WASTE defaults to greedy, which is the right default for a local engine but is not what the model was tuned for.
• Up to 5 stop sequences; the CLI supports one.