Does AIsbom catch it?

Generated by aisbom bypass-scorecard — do not edit by hand.

Every row is a publicly-documented technique for smuggling a malicious pickle past a model scanner, reproduced here as an inert artifact and scanned with AIsbom's own engine. Artifacts are synthesized, never copied from live malware, and are only ever disassembled — no pickle in this corpus is executed.

9 of 11 evasion cases are caught in at least one scan mode.

blocklist is the default mode (flag known-dangerous globals); strict is --strict (allowlist — anything unrecognized is flagged). A ⚠️ partial means AIsbom refused to call the file safe but never disassembled the payload, so it reports the wrong reason.

Scorecard

Case	Evasion class	Blocklist	Strict	Source
nullifai-7z-container Model packed with 7z instead of ZIP	container-format	⚠️ partial	⚠️ partial	ReversingLabs — nullifAI (malicious models on Hugging Face)
nullifai-broken-stream Deliberately broken pickle stream, payload first	broken-stream	✅ detected	✅ detected	ReversingLabs — nullifAI (malicious models on Hugging Face)
cve-2025-1716-pip-main Code execution via pip.main()	unlisted-global	✅ detected	✅ detected	Sonatype — CVE-2025-1716
cve-2025-1889-nonstandard-extension Payload in a file with a non-standard extension	file-extension	✅ detected	✅ detected	Sonatype — CVE-2025-1889
cve-2025-1944-zip-filename-tamper ZIP local-header filename differs from the central directory	zip-tampering	✅ detected	✅ detected	Sonatype — CVE-2025-1944
cve-2025-1945-zip-flag-bits ZIP general-purpose flag bits modified	zip-tampering	✅ detected	✅ detected	Sonatype — CVE-2025-1945
cve-2025-10155-extension-confusion Bare pickle wearing a PyTorch extension	file-extension	✅ detected	✅ detected	JFrog — CVE-2025-10155
cve-2025-10156-zip-crc Corrupted CRC-32 in the ZIP archive	zip-tampering	✅ detected	✅ detected	JFrog — CVE-2025-10156
cve-2025-10157-asyncio-subclass Dangerous import reached through an asyncio submodule	gadget-import	✅ detected	✅ detected	JFrog — CVE-2025-10157
checkmarx-bdb-gadget Indirect execution via bdb.Bdb.run	gadget-import	✅ detected	✅ detected	Checkmarx — Free Hugs: What to be Wary of in Hugging Face (Part 4)
shadowpickle-allowlist-overwrite Allowlisted builtin reached via STACK_GLOBAL	allowlist-abuse	⚠️ partial	⚠️ partial	ShadowPickle (arXiv 2607.17503)

Controls

Case	Blocklist	Strict
control-os-system-zip Plain os.system in a standard PyTorch ZIP	✅ detected	✅ detected
benign-allowlisted-globals Ordinary model, allowlisted globals only	✅ clean	✅ clean

Case detail

control-os-system-zip — Plain os.system in a standard PyTorch ZIP

Evasion class: control
Source: AIsbom control case (no evasion)
A correct scanner should: detected

No evasion at all: the reference case every scanner must catch. If this one ever misses, the harness is broken and the rest of the table is noise.

benign-allowlisted-globals — Ordinary model, allowlisted globals only

Evasion class: control
Source: AIsbom control case (false-positive guard)
A correct scanner should: clean

A normal state_dict with no dangerous sinks. Guards the other direction: a scanner that flags everything is as useless as one that flags nothing.

nullifai-7z-container — Model packed with 7z instead of ZIP

Evasion class: container-format
Source: ReversingLabs — nullifAI (malicious models on Hugging Face)
A correct scanner should: detected

PyTorch's default container is ZIP; packing the same payload with 7z meant picklescan never opened it, while the model still loaded.

Current limitation: AIsbom reports CRITICAL (Non-Standard Container: 7z) — the right severity, but earned from the container rather than the payload. The archive is named, never unpacked, so the os.system call inside is never disassembled and the reported reason is not the real one. Unpacking 7z would mean a native dependency in every install to cover one evasion class, which is not a trade worth making; a user acting on this verdict is nonetheless correctly warned off the file.

nullifai-broken-stream — Deliberately broken pickle stream, payload first

Evasion class: broken-stream
Source: ReversingLabs — nullifAI (malicious models on Hugging Face)
A correct scanner should: detected

The reverse shell sits at the front of the stream and the tail is corrupt. The pickle VM executes sequentially, so the payload runs before the error; a scanner that requires a well-formed stream bails out first.

cve-2025-1716-pip-main — Code execution via pip.main()

Evasion class: unlisted-global
Source: Sonatype — CVE-2025-1716
A correct scanner should: detected

Reaches execution through a global nobody thought to blocklist: pip.main() installs an attacker-controlled package. Fixed in picklescan 0.0.21.

cve-2025-1889-nonstandard-extension — Payload in a file with a non-standard extension

Evasion class: file-extension
Source: Sonatype — CVE-2025-1889
A correct scanner should: detected

The pickle is named config.p. Extension-driven discovery never opens it, so nothing is scanned. Fixed in picklescan 0.0.22.

cve-2025-1944-zip-filename-tamper — ZIP local-header filename differs from the central directory

Evasion class: zip-tampering
Source: Sonatype — CVE-2025-1944
A correct scanner should: detected

The two copies of the filename disagree. PyTorch loads it; a strict ZIP reader raises and the scan is abandoned. Fixed in picklescan 0.0.23.

cve-2025-1945-zip-flag-bits — ZIP general-purpose flag bits modified

Evasion class: zip-tampering
Source: Sonatype — CVE-2025-1945
A correct scanner should: detected

Flipping flag bits in the local header changes how the archive is parsed without changing what PyTorch loads. Fixed in picklescan 0.0.23.

cve-2025-10155-extension-confusion — Bare pickle wearing a PyTorch extension

Evasion class: file-extension
Source: JFrog — CVE-2025-10155
A correct scanner should: detected

A raw pickle named model.bin. The PyTorch parser fails because there is no ZIP container, while PyTorch itself identifies the file by content. Fixed in picklescan 0.0.31.

cve-2025-10156-zip-crc — Corrupted CRC-32 in the ZIP archive

Evasion class: zip-tampering
Source: JFrog — CVE-2025-10156
A correct scanner should: detected

PyTorch disables CRC checking by default; a scanner that validates CRC errors out and scans nothing. Fixed in picklescan 0.0.31.

cve-2025-10157-asyncio-subclass — Dangerous import reached through an asyncio submodule

Evasion class: gadget-import
Source: JFrog — CVE-2025-10157
A correct scanner should: detected

Uses a submodule rather than the exact blocklisted module name, so an exact-match unsafe-globals check downgrades it. Fixed in picklescan 0.0.31.

checkmarx-bdb-gadget — Indirect execution via bdb.Bdb.run

Evasion class: gadget-import
Source: Checkmarx — Free Hugs: What to be Wary of in Hugging Face (Part 4)
A correct scanner should: detected

bdb is Python's built-in debugger and Bdb.run is equivalent to exec. A benign-looking module with an execution sink: no obviously dangerous name appears anywhere in the stream, which is the point of a gadget.

shadowpickle-allowlist-overwrite — Allowlisted builtin reached via STACK_GLOBAL

Evasion class: allowlist-abuse
Source: ShadowPickle (arXiv 2607.17503)
A correct scanner should: detected

Overwrites collections.OrderedDict — a name on essentially every scanner's allowlist — and resolves it via STACK_GLOBAL rather than an inline GLOBAL argument, so string-matching on the opcode argument sees nothing.

Current limitation: AIsbom does resolve STACK_GLOBAL and reads the pair off the stack, so it sees collections.OrderedDict — and that name is legitimately allowlisted, because real state_dicts are OrderedDicts. Both modes therefore return only MEDIUM (Pickle Present), the baseline every pickle gets, rather than a signal specific to this file. This is the ceiling on static allowlist analysis: the call is indistinguishable from a legitimate call to an allowlisted global, and flagging the shape would flag ordinary checkpoints. Closing it needs evidence beyond the resolved name — argument shape, or provenance — not a new entry on a blocklist.