safetensors parity report

Differential results from tools/diff_st.c, which runs modelvet's mvet_st_verify and the canonical Rust safetensors implementation over the same inputs and buckets the outcomes. Every divergence below is triaged. Nothing here is a claim about a codebase that was not actually executed.

What was measured, and against what

• Harness: make diff-st. The canonical crate is linked through a C-ABI Rust shim (tools/st_diff_shim/) that wraps SafeTensors::deserialize, the full canonical acceptance path: frame checks, serde header parse, validate() offset tiling, and the exact-end check.
• Upstream build tested: the EXACT pinned revision. The shim's Cargo dependency is pinned to commit 24b0b2857fef of huggingface/safetensors, the same source state the modelvet mirror was derived from (tools/st-pin.env); make check-st-pin enforces that the two pins can only move together. Unlike the GGUF parity run, no "not the pinned revision" caveat applies.
• Panics are a bucket, not a crash. Canonical safetensors is a Result-returning safe-Rust API whose contract is no panics on hostile input, so the shim converts a panic into its own verdict and the harness fails hard if one ever occurs.
• The harness exit code is the acceptance bar. make diff-st fails when modelvet is ever more permissive than canonical (we-accept-they-reject) or when canonical panics. Documented policy strictness (we-reject-they-accept) is reported for triage and does not fail the run.

Three input sets were measured, 68 files total, on 2026-08-04.

Set A: the curated corpus plus the generated differential set (64 files)

tests/corpus/st/ (7 files: TOB-SFTN-2, the three upstream abuse shapes, and the three reconstructed TOB-SFTN-1 polyglots) plus the 57 deterministic inputs from tools/make_st_diff_inputs.c (all 22 dtypes, shape and ordering edge cases, every documented policy shape, and the mirror-reject family).

bucket	count
both accept	31
both reject	27
we accept / they reject	0
we reject / they accept	6
we reject / they panic	0
we accept / they panic	0

Set B: the recovered Trail of Bits polyglot fixtures (4 files)

The March 2023 Trail of Bits audit PDF is itself a PDF/ZIP polyglot; unzipping it yields four malformed-input fixtures (safetensors-polyglot-examples/), the only malformed-file corpus the format has. Recovered and measured 2026-08-04.

bucket	count
both accept	1
both reject	3
we accept / they reject	0
we reject / they accept	0
we reject / they panic	0
we accept / they panic	0

The headline

Zero inputs in any set landed in we-accept-they-reject, and canonical never panicked. Across 68 files, modelvet was never more permissive than the pinned canonical implementation, and every one of the six divergences in the other direction is a documented policy code doing exactly what it documents. The generated accept family (every live dtype including the sub-byte types, scalar and zero-element shapes, out-of-declaration-order offsets, escaped metadata, whitespace padding) was accepted by both implementations, so the strictness deltas introduce no false positives against canonical-accepted files.

Triage: we reject / they accept (6 inputs, all deliberate policy)

Each is a policy-* input generated for exactly this purpose, rejected with precisely its documented policy code. Canonical accepts all six.

input	code	policy
policy-dup-root-key	516 MVET_V_ST_KEY_DUPLICATE	duplicate keys reject; canonical HashMap is last-wins. TOB-SFTN-7: parser divergence on duplicates is itself the attack primitive.
policy-dup-metadata-key	516 MVET_V_ST_KEY_DUPLICATE	same rule inside __metadata__.
policy-key-escape	513 MVET_V_ST_KEY_ESCAPE	escapes in keys reject: keys are compared as raw bytes for duplicate detection, and "a" aliasing "a" would bypass it.
policy-unknown-field	530 MVET_V_ST_TENSOR_FIELDS	unknown tensor-object fields reject; canonical serde ignores them.
policy-name-over-cap	515 MVET_V_ST_NAME_LEN	257-byte tensor name over MVET_MAX_ST_NAME_BYTES (256).
policy-metadata-over-cap	521 MVET_V_ST_METADATA_COUNT_CAP	1025 metadata entries over MVET_MAX_ST_METADATA_ENTRIES (1024).

These are the same policy codes marked "policy" in the public header and in the mirror-vs-policy ledger; the differential confirms they are the only sources of divergence.

The Trail of Bits polyglots, one by one

• PDF, appended-ZIP, appended-Keras (both reject). All three embed their foreign payload after the tensor data region. Canonical rejects them with its exact-cover check (MetadataIncompleteBuffer); modelvet rejects them with MVET_V_ST_DATA_EXTENT (545). The anti-polyglot rule is canonical for this format, and both implementations enforce it. Three small reconstructed entries carrying the identical structural failure are committed as corpus files (st/tob-sftn-1-{pdf,zip,keras}.safetensors); the recovered originals are 8-24 MB and stay a local verification step.
• TFRecords polyglot, tfST.safetensors (both accept). This one hides its record framing inside a padded __metadata__ string value, so the file is fully valid, exactly-covered safetensors and simultaneously a valid TFRecords stream. Both modelvet and canonical accept it, correctly: no structural rule can reject it without rejecting legitimate files, because the safetensors view is unimpeachable. This is the residual polyglot class inherent to a format with no magic bytes and no version field. The defense lives in consumer-side detection policy (decide the file's type once, before any parser sees it), not in structural validation; modelvet's integration guide documents the recognition rule for that purpose.

Reproducing

make diff-st                      # corpus + generated set, pinned shim
make diff-st DIFF_ST_EXTRA=/path/to/recovered/fixtures

Requires a Rust toolchain; make check-st-pin proves the shim tests the audited commit. The nightly st-differential job in .github/workflows/fuzz-nightly.yml runs the same harness.

Open work

• Feed nightly fuzzer-generated safetensors inputs through the harness the way the GGUF report used gguf_fuzz outputs, once a corpus of minimized interesting inputs accumulates.
• When the pin advances (watched by the current-safetensors job in upstream-drift.yml), re-run and re-triage before accepting drift.