C01: Training Data Integrity & Traceability

September 8, 2026 · View on GitHub

Source: 1.01-dev/en/0x10-C01-Training-Data-Integrity-and-Traceability.md Requirements: 13 | Sections: 3

Control Objective

This chapter addresses the sourcing, handling, and maintenance of training data in a way that preserves origin traceability, integrity, and quality. The core security concern is ensuring data has not been tampered with, poisoned, or corrupted. The chapter covers three sections: training data origin and data security (C1.1), data labeling and annotation security (C1.2), and training data quality and security assurance (C1.3). Generic access, logging, encryption, and lineage controls defer to OWASP ASVS v5 as baseline, so this chapter focuses on the AI-specific origin, labeling, and quality-assurance concerns that ASVS does not cover.

Section Pages

Section	Title	Reqs	Page
C1.1	Training Data Origin & Data Security	5	C01-01-Training-Data-Origin-Traceability
C1.2	Data Labeling and Annotation Security	3	C01-02-Data-Labeling-Annotation-Security
C1.3	Training Data Quality and Security Assurance	5	C01-03-Training-Data-Quality-Security-Assurance

Threat Landscape

Known attacks, real-world incidents, and threat vectors relevant to this chapter:

Data poisoning attacks -- As of October 2025, Anthropic/UK AISI/Turing Institute demonstrated that as few as 250 malicious documents (~0.00016% of training tokens) can backdoor LLMs from 600M to 13B parameters, with success nearly constant across model sizes (arXiv:2510.07192). CyLab/CMU showed poisoning just 0.1% of pre-training data persists through post-training safety alignment (ICLR 2025). Techniques include backdoor injection (BadNets, Trojan attacks), clean-label attacks, triggerless attacks, and data augmentation poisoning (MITRE ATLAS AML.T0020 Poison Training Data, AML.T0019 Publish Poisoned Datasets, and AML.T0010.002 AI Supply Chain Compromise: Data).
Training data extraction / memorization attacks -- A July 2025 comprehensive survey (arXiv:2507.05578) found divergence attacks extract up to 150x more training data than normal queries, soft prompting increases PII leakage by 9.3%, and memorization scales log-linearly with model size. USENIX Security 2025 demonstrated PII extraction from LLMs without jailbreaks via augmented few-shot prompting. De-duplication of training data reduces memorization by 10x.
Supply chain compromise of ML platforms -- NullifAI attack (Feb 2025) placed malicious models on Hugging Face evading Picklescan via 7z compression, creating reverse shells. NullBulge campaign (2024–2025) weaponized ComfyUI extensions and distributed LockBit ransomware via AI platforms. torchtriton PyPI typosquatting (2022) infiltrated thousands of ML environments. Carlini et al. showed poisoning 0.01% of LAION-400M costs ~$60 via split-view and frontrunning attacks.
Label-flipping attacks on crowdsourced annotation pipelines -- malicious annotators systematically mislabel samples to introduce targeted biases (MITRE ATLAS AML.T0020)
Medical AI poisoning -- A Nature Medicine study (Jan 2025) showed poisoning just 0.001% of medical training tokens increased harmful LLM outputs by 4.8–7.2% depending on model size, while corrupted models matched performance on standard benchmarks — making poisoning invisible to routine evaluation. A biomedical knowledge graph-based screening captured 91.9% of harmful content (F1 = 85.7%).
Code generation poisoning -- Stealthy attacks inject syntactically valid but vulnerable code patterns into training data for code-generation LLMs (CodeBERT, CodeT5+, AST-T5). As of August 2025, evaluation of spectral signatures, activation clustering, and static analysis found all three methods struggle against triggerless poisoning in code generators (arXiv:2508.21636). CodeScan (Mar 2026) is the first scanning framework tailored to code generation, achieving 97%+ detection accuracy via AST-based normalization and iterative divergence analysis (arXiv:2603.17174).
Synthetic data poisoning propagation -- The Virus Infection Attack (VIA, NeurIPS 2025 spotlight) demonstrated that poisoned content propagates through synthetic data generation pipelines, bypassing standard defenses. VIA conceals payloads within protective "shells" and hijacks benign samples to maximize malicious content in downstream synthetic data, achieving attack success rates comparable to directly poisoned models (arXiv:2509.23041).
Backdoor via harmless inputs -- A 2025 study showed that robust associations between triggers and affirmative responses can be established using entirely benign question-answer pairs, circumventing safety-aligned guardrails that would otherwise filter overtly poisoned samples. This makes detection significantly harder since the poisoned data contains no harmful content on its own.
Social media data poisoning -- The Grok 4 "!Pliny" incident demonstrated that training on unfiltered social media data saturated with jailbreak prompts can embed exploitable triggers in production models. Typing a specific username was enough to strip guardrails entirely, illustrating how adversaries can poison models through public data channels without direct pipeline access.
Repository-based poisoning (Basilisk Venom) -- In January 2025, hidden prompts embedded in GitHub code comments poisoned fine-tuned models including DeepSeek's DeepThink-R1, creating backdoors that responded to specific phrases with attacker-planted instructions.
Preference learning poisoning -- PoisonBench (ICML 2025) evaluated 22 LLMs across two attack types (content injection and alignment deterioration) and found that scaling model size does not reliably improve resilience, attack success follows a log-linear relationship with poison ratio, and poisoning effects generalize to extrapolated triggers (arXiv:2410.08811).
Data ordering attacks -- manipulating the sequence of training data to influence gradient updates without changing data content
Model namespace reuse (supply-chain takeover) -- Palo Alto Unit 42 documented in September 2025 that when a Hugging Face or similar hub account is deleted or transferred, the freed namespace can be re-registered by an adversary who then uploads a poisoned or backdoored model under the same identifier. Existing pipeline code that pulls by model name (without revision pinning, cloning to trusted storage, or signature verification) can silently load the attacker's model through Vertex AI, Azure AI Foundry, Kaggle, or open-source project defaults. As of late 2025 / early 2026 Google scans Vertex AI daily for orphaned authors and marks affected models "verification unsuccessful" so they cannot be redeployed, while Microsoft, Hugging Face, and Kaggle have acknowledged the disclosure but not published a comparable platform-side fix; Unit 42's recommended mitigations are revision pinning (revision="<commit_hash>"), cloning to internal storage, and scanning code, docstrings, and default arguments for unpinned model references.
AI-gateway / SDK supply-chain compromise (TeamPCP, LiteLLM, Mar 2026) -- On March 24, 2026 the TeamPCP group exploited a poisoned Trivy security scanner running inside LiteLLM's CI/CD pipeline to exfiltrate PyPI publishing tokens, then pushed backdoored litellm packages 1.82.7 and 1.82.8 live on PyPI for roughly 40 minutes from 10:39 UTC before PyPI quarantined them, with the exposure window running to 16:00 UTC. The payload was a three-stage credential harvester, Kubernetes lateral-movement toolkit, and persistent backdoor; LiteLLM's own postmortem reports exfiltration via the unauthorized domain models.litellm.cloud. Industry telemetry (Snyk, Trend Micro, InfoQ) tracked roughly 33,185 secrets exposed across 6,943 machines, including cloud credentials, Kubernetes configs, git tokens, and API keys -- the same secrets that gate access to training-data buckets, model registries, and labeling platforms. The same campaign also touched Trivy, KICS, Telnyx, and assets on npm, Docker Hub, GitHub Actions, and OpenVSX, so AI teams should treat any LiteLLM-adjacent secret rotation as a forced dataset and model-access review, not just a library bump.
MLflow AI Gateway credential exfiltration (CVE-2026-4035) -- MLflow before 3.11.0 could resolve an attacker-supplied $ENV_VAR reference in an AI Gateway API-key field and send the resulting server secret to an attacker-controlled api_base. NVD records that default deployments without basic authentication were reachable without authentication, while basic-auth deployments exposed the path to low-privileged users; stolen cloud artifact-store credentials could enable artifact poisoning and downstream code execution. MLflow's patch makes environment-variable resolution opt-in for the legacy YAML gateway (NVD, upstream patch). This is a direct integrity risk for datasets and labeling artifacts stored behind the same credentials.
Backdoor detection evasion via observable signatures -- In February 2026 Microsoft Security published an at-scale detection approach that identifies a "double triangle" pattern in attention distributions between benign and trigger tokens and a characteristic collapse in output entropy when triggers are present. Attackers aware of these indicators can design poisoning strategies that flatten the attention pattern or preserve entropy — the defender/attacker cat-and-mouse is now moving into the interpretability layer.
Sequential (multi-stage) post-training poisoning -- A June 3, 2026 paper (arXiv:2606.04929) demonstrated the "single-attacker illusion": adversaries who separately poison the SFT dataset and the preference dataset each appear negligible when evaluated in isolation, but their effects are additive in SFT→DPO pipelines and complementary in SFT→PPO pipelines (where neither stage alone succeeds but the combination does). Stage-by-stage poisoning evaluation systematically underestimates risk — assessments need to cover the post-training pipeline end to end, which lands directly on 1.3.1 (poisoning detection) and 1.3.5 (clean-label poisoning defenses).
Trending-model typosquats with engagement manipulation -- On May 7, 2026 the Open-OSS/privacy-filter repository appeared on Hugging Face impersonating OpenAI's openai/privacy-filter, copied the legitimate model card verbatim, and used apparent download/like inflation to reach #1 trending with roughly 244,000 downloads in 18 hours. Its loader.py shipped a base64-obfuscated Windows infostealer harvesting browser credentials, Discord tokens, crypto wallets, and SSH keys (Hive Security). The same write-up flagged CVE-2026-6859: InstructLab hardcoded trust_remote_code=True, letting any referenced Hugging Face repo execute arbitrary Python in training pipelines — a reminder that dataset and model ingestion paths are code-execution paths.

Notable Incidents & Research

Date	Incident / Paper	Relevance	Link
2017	BadNets: Identifying Vulnerabilities in the ML Supply Chain (Gu et al.)	Foundational backdoor poisoning attack via training data	arXiv:1708.06733
2020	Radioactive Data: Tracing Through Training (Sablayrolles et al.)	Dataset fingerprinting technique relevant to C1.1.5 (watermarking for attribution)	arXiv:2002.00937
2021	Extracting Training Data from Large Language Models (Carlini et al.)	Demonstrated verbatim training data extraction from GPT-2	arXiv:2012.07805
2023	Poisoning Web-Scale Training Datasets is Practical (Carlini et al.)	Showed practical poisoning of LAION-400M for ~$60 via split-view/frontrunning	arXiv:2302.10149
2024	Nightshade: Prompt-Specific Poisoning Attacks on Text-to-Image Models	Targeted poisoning attack against diffusion models	IEEE S&P 2024
2022	torchtriton PyPI typosquatting attack	Malicious package masquerading as PyTorch dependency infiltrated thousands of ML environments	MITRE ATLAS AML.T0010.001
2024–2025	NullBulge supply chain campaign	Weaponized ComfyUI extension on GitHub, distributed LockBit ransomware via AI platforms	The Hacker News
Jan 2025	DeepSeek database exposure (Wiz Research)	1M+ log entries including chat history, API keys exposed via unprotected ClickHouse ports	Wiz Research
Feb 2025	NullifAI malicious models on Hugging Face	Evaded Picklescan via 7z compression, created reverse shells across OS platforms	ReversingLabs
Jun 2025	Persistent Pre-Training Poisoning (CyLab/CMU, ICLR 2025)	0.1% poisoning persists through post-training safety alignment for DoS, belief manipulation, prompt stealing	arXiv:2410.13722
Jul 2025	Comprehensive LLM Memorization Survey	Divergence attacks extract 150x more training data; de-duplication reduces memorization 10x	arXiv:2507.05578
Oct 2025	250-Document Poisoning (Anthropic/UK AISI/Turing)	Only 250 documents needed to backdoor LLMs at any scale; absolute count matters, not proportion	Anthropic Research
2025	PII Extraction Without Jailbreaks (USENIX Security 2025)	Extracted names, emails, phone numbers from LLMs via direct querying without jailbreaks	USENIX
2025	Datasig dataset fingerprinting (Trail of Bits)	First practical dataset fingerprinting tool using MinHash; detects poisoned/modified datasets	Trail of Bits
Jan 2025	Medical LLM Poisoning (Nature Medicine)	0.001% token poisoning increased harmful outputs 4.8–7.2% while evading standard benchmarks; knowledge graph screening catches 91.9%	Nature Medicine
Dec 2024	Italian DPA fines OpenAI EUR 15M	GDPR violations in ChatGPT training data: no legal basis, transparency failures, inadequate age verification	Lewis Silkin
Aug 2025	Stealthy Code Gen Poisoning Study	Spectral signatures, activation clustering, and static analysis all fail against triggerless poisoning in code generators	arXiv:2508.21636
May 2025	Joint AI Data Security CSI	CISA/NSA/FBI and international partners framed AI data as a supply-chain asset and recommended provenance, encryption, signatures, secure storage, monitoring, threat detection, and secure deletion	NSA
Oct 2025	CSA AI Controls Matrix	Vendor-neutral framework with 243 control objectives across 18 domains, mapped to ISO 42001, ISO 27001, NIST AI RMF, NIST AI 600-1, and the EU AI Act	CSA
Sep 2025	Virus Infection Attack (VIA) on LLMs (NeurIPS 2025 spotlight)	Poisoning propagates through synthetic data pipelines; payloads survive generation and amplify across model generations	arXiv:2509.23041
2025	PoisonBench: LLM Preference Learning Poisoning (ICML 2025)	First benchmark for preference poisoning; 22 models tested, scaling doesn't help, log-linear attack/poison ratio	arXiv:2410.08811
Jan 2025	Basilisk Venom — GitHub code comment poisoning	Hidden prompts in code comments backdoored DeepSeek DeepThink-R1 during fine-tuning	Lakera
2025	Grok 4 "!Pliny" social media data poisoning	Jailbreak prompts saturated in X posts created exploitable trigger in production model	Lakera
Feb 2025	Atlas: ML Lifecycle Provenance Framework	C2PA + SLSA + in-toto attestable pipelines with TEE support; <8% training overhead; Kubeflow integration	arXiv:2502.19567
Mar 2026	CodeScan: Poisoning Detection for Code Gen LLMs	First scanning framework for code generation models; 97%+ accuracy via AST normalization across 108 models	arXiv:2603.17174
Feb 2026	Microsoft Security — Detecting Backdoored Language Models at Scale	Identified attention "double triangle" pattern and output-entropy collapse as observable backdoor signatures; framed as practical detection for hub-scale scanning	Microsoft Security Blog
Sep 2025	Model Namespace Reuse (Palo Alto Unit 42)	Deleted or transferred Hugging Face namespaces can be reclaimed; impacted examples included Vertex AI, Azure AI Foundry, Kaggle, and open-source defaults that pull by model name	Unit 42
Dec 2025	NISTIR 8596 preliminary draft	Cyber AI Profile applies CSF 2.0 to securing AI systems, defending with AI, and thwarting AI-enabled attacks	NIST
Dec 2025	Quantization Blindspots	INT8 quantization collapsed several classic backdoor defenses while preserving high attack success, making post-compression rescanning important for edge deployments	arXiv:2512.06243
May 2026	MITRE ATLAS v5.6.0 data series	ATLAS moved to monthly releases; current data includes `AML.T0020 Poison Training Data`, `AML.T0019 Publish Poisoned Datasets`, and `AML.T0010.002 AI Supply Chain Compromise: Data`	ATLAS data
Mar 2026	xAI v. AB 2013 ruling	US District Court (C.D. Cal.) denied xAI's preliminary injunction against California AB 2013 while leaving the case to proceed on the merits	PPC Land
Apr 2026	NIST AI RMF Profile for Trustworthy AI in Critical Infrastructure	Concept note starts a critical-infrastructure AI profile covering full-lifecycle risk management across AI and CI supply chains	NIST
May 2026	MITRE CTID Secure AI update	Added 45+ ATLAS techniques/sub-techniques, 10+ mitigations, 20+ case studies, a technique maturity filter, rapid-response reporting, and agentic AI coverage	MITRE CTID
May 2026	EU Digital Omnibus AI Act political agreement	Moves high-risk AI Act application timing to Dec 2, 2027 for stand-alone systems and Aug 2, 2028 for product-integrated systems, while keeping data-governance evidence work relevant	European Commission
Jun 2026	Sequential Data Poisoning in LLM Post-Training	Coordinated poisoning across SFT and preference stages compounds: additive in SFT→DPO, complementary in SFT→PPO; per-stage evaluation underestimates real exposure	arXiv:2606.04929
May 2026	`Open-OSS/privacy-filter` Hugging Face typosquat	Impersonated OpenAI repo hit #1 trending (~244k downloads in 18 hours) via engagement inflation while shipping a base64-obfuscated infostealer; same report flagged CVE-2026-6859 (InstructLab `trust_remote_code=True`)	Hive Security
Nov 2025	REMIND: residual memorization after unlearning	Input loss landscapes reveal whether "unlearned" data still influences post-unlearning LLMs — a verification primitive for memorization and erasure claims	arXiv:2511.04228
Sep 2025	Source-Free Certified Unlearning	Demonstrated certified unlearning using a surrogate dataset rather than original training data; performance comparable to full retraining at a fraction of compute — practical step toward GDPR erasure for production LLMs	IAPP: AI right to unlearn
Mar 2026	LiteLLM / TeamPCP supply-chain compromise	Poisoned Trivy CI scanner led to backdoored `litellm` 1.82.7/1.82.8 on PyPI; ~33,185 secrets across 6,943 machines exposed, including the credentials that gate training-data and model registries	LiteLLM postmortem
Jun 2026	MLflow AI Gateway credential exfiltration (CVE-2026-4035)	MLflow before 3.11.0 could disclose server-side environment secrets to an attacker-controlled gateway endpoint; NVD identifies cloud artifact credentials and resulting artifact poisoning as possible impact	NVD
Feb 2026	US Treasury Financial Services AI RMF	Translated NIST AI RMF into 230 control objectives tailored to financial institutions, with explicit training-data security and third-party-model-dependency guidance	ISP Partners — NIST AI RMF 2025-2026 Updates
2026	OWASP AIBOM Project / SPDX 3.0 AI Profile / CycloneDX ML-BOM	2026 procurement consensus: require SPDX 3.0 AI Profile for external attestations, generate CycloneDX ML-BOM internally; OWASP AIBOM Project provides cross-walk reference for dataset, model, and pipeline components	OWASP AIBOM Project

Tooling & Implementation

Current tools, frameworks, and libraries that help implement these controls (as of July 2026):

Data provenance & versioning: lakeFS (acquired DVC in Nov 2025; git-like branching for data lakes with immutable snapshots), DVC (now under lakeFS stewardship for data-science-scale version control), MLflow (experiment tracking and partial lineage; upgrade to 3.11.0 or later to close CVE-2026-4035, and do not share AI Gateway and artifact-store credentials), Delta Lake, Apache Iceberg
Poisoning detection: Cleanlab v2.9.0 (Jan 2026; confident learning for label errors, outliers, duplicates, spurious correlations, and multi-annotator quality), IBM ART v1.20+ (Neural Cleanse, Activation Clustering, Spectral Signatures, STRIP), Microsoft's February 2026 scanner research for open-weight LLM backdoors, CodeScan (Mar 2026; AST-normalized black-box scanning for code-generation poisoning with 97%+ reported accuracy across the authors' test set), and Lakera for commercial prompt / data poisoning defenses
Data integrity & signing: Cryptographic hashing (SHA-256 or stronger), Sigstore model-transparency v1.0 (Apr 2025; signs ML models with OIDC identity), OpenSSF Model Signing (OMS) specification (detached Sigstore-compatible bundles that already cover weights, configuration files, tokenizers, and adjacent artifacts in one verifiable unit; OpenSSF lists explicit dataset signing as the next item on the OMS roadmap rather than a shipped capability), in-toto/DSSE attestations, Great Expectations, and Pandera. NVIDIA now signs all NGC Catalog models with OMS, and Google has prototyped OMS signing for the Kaggle Model Hub. The NSA/CISA/FBI Joint CSI on AI Data Security (May 2025) recommends provenance tracking, encryption, digital signatures, secure storage, and continuous monitoring across training and operation.
Dataset fingerprinting: Datasig (Trail of Bits, May 2025; MinHash-based compact fingerprints for dataset comparison without raw data access; prototype stage, expected error under 5% in the MNIST validation shown by Trail of Bits)
Lineage: Apache Atlas, DataHub (metadata platform with lineage graphs), Amundsen, MLflow, OpenLineage (LF AI & Data graduate project; release 1.47.0 in May 2026 added Microsoft Fabric adapter support, ClickHouse extraction, richer DataQualityAssertionsDatasetFacet fields, and dbt failure thresholds; integrations include Airflow, Spark, Flink, dbt, Snowflake, Databricks, and Marquez as the reference implementation)
Attestable ML pipelines: Atlas framework (Feb 2025; combines C2PA, SLSA, and in-toto for fully attestable ML pipelines with TEE-backed provenance chains; <8% training overhead; Kubeflow integration; research prototype)
PII detection: Microsoft Presidio (text/image/structured data; integrated with Hugging Face Hub), AWS Macie, Google Cloud DLP
Labeling platforms: Label Studio Enterprise (RBAC, SSO/SAML, audit logs, QA workflows; 2025 analytics tab), Labelbox (SOC 2, RBAC, audit trails), Encord (HIPAA/SOC 2), Scale AI, Snorkel
Bias evaluation: Fairlearn, AI Fairness 360, custom red-team probing
Adversarial robustness: IBM ART, Foolbox, CleverHans
Machine unlearning: Still no production-ready tools as of May 2026, but the research front is moving. Active venues: ICML 2025 MUGen workshop, CVPR 2026 MUV workshop (covering data-centric removal, parameter-centric fine-tuning, and training-free steering via refusal vectors). A September 2025 result on source-free certified unlearning showed certified erasure using a surrogate dataset rather than the original training data, with performance comparable to full retraining at a fraction of compute — the first approach that looks plausible for a live GDPR erasure request against a deployed LLM. Production approaches today still fall back on full retraining, targeted fine-tuning, or approximate methods with unverified guarantees. On the verification side, REMIND (Nov 2025) uses input loss landscapes to detect residual memorization in post-unlearning LLMs — useful evidence for memorization and erasure claims, and it is still a research method rather than an audit tool.
Platform security: Hugging Face Hub (Oct 2025: VirusTotal scanning of 2.2M+ repos; JFrog partnership for ML supply chain transparency; Sigstore integration in development). Google Vertex AI now runs daily scans for orphaned Hugging Face authors and marks impacted models "verification unsuccessful" so they cannot be redeployed; Microsoft Azure AI Foundry, Hugging Face, and Kaggle have acknowledged Unit 42's disclosure but have not yet published an equivalent platform-side fix.
AI-BOM & dataset provenance manifests: SPDX 3.0 AI Profile and CycloneDX ML-BOM are the two leading specifications; the OWASP AIBOM Project acts as a cross-walk reference, covering model identity, training and evaluation datasets, frameworks, hardware, pipelines, and governance metadata. JFrog and Palo Alto Networks publish vendor AIBOM generators, and 2026 procurement guidance increasingly asks for SPDX 3.0 AI Profile externally and CycloneDX ML-BOM internally.

Implementation Maturity

Control Area	Tooling Maturity	Notes
C1.1 Training Data Origin & Data Security	Medium	This section spans inventory (1.1.2), storage/transfer integrity (1.1.3), integrity monitoring (1.1.4), and watermarking (1.1.5). Data catalogs exist (DataHub, Amundsen) but metadata completeness enforcement is manual. Dataset watermarking (1.1.5) is improving -- Datasig (Trail of Bits, 2025) is a first practical fingerprinting tool but remains prototype-stage. For storage/transfer integrity (1.1.3) and integrity monitoring (1.1.4), Sigstore model-transparency v1.0 and OpenSSF OMS make signed ML artifacts practical, NVIDIA NGC plus a Google Kaggle prototype show OMS is past lab-only, and lakeFS's acquisition of DVC (Nov 2025) consolidated data versioning; dedicated dataset signing remains on the OMS roadmap. Generic access control, logging, and encryption now defer to OWASP ASVS v5 (V6/V7/V8, V11, V16). The LiteLLM / TeamPCP CI compromise (Mar 2026) and MLflow CVE-2026-4035 show that integrity controls still fail when orchestration components leak the credentials that unlock dataset and artifact stores; MLflow users should upgrade to 3.11.0 or later and isolate gateway credentials from storage credentials. California AB 2013 (effective Jan 2026) now legally mandates public training data documentation for covered generative AI systems, and xAI's March 2026 injunction bid failed at the preliminary stage.
C1.2 Data Labeling and Annotation Security	Medium	Label Studio Enterprise added an analytics tab (2025); Encord is emerging with a HIPAA/SOC 2 posture. Commercial platforms have RBAC and audit logging, satisfying access control (1.2.1). Cryptographic integrity for labeling artifacts (1.2.2) still requires custom implementation, and sensitive-label protection (1.2.3) leans on Presidio/Macie-style redaction plus encryption at rest and in transit. The March 2026 Mercor/LiteLLM compromise underlined that labeling-platform and gateway dependencies belong inside the labeling trust boundary.
C1.3 Training Data Quality and Security Assurance	Medium	Cleanlab v2.9.0 (Jan 2026) is mature for label error/outlier detection (1.3.2). IBM ART v1.20+ remains broad for adversarial defenses, CodeScan is the first poisoning scanner purpose-built for code generation models, and Microsoft Security's February 2026 scanner gives defenders practical signals for deterministic open-weight LLM backdoors (1.3.1). Clean-label poisoning defenses (1.3.5, L3) and disallowed-content removal (1.3.4) are still largely bespoke. Key gap: Anthropic's Oct 2025 research shows only 250 documents can poison any-size LLM, Nature Medicine showed 0.001% medical-token poisoning can evade standard benchmarks, quantization can break classic backdoor defenses, and the June 2026 sequential-poisoning result means per-stage pipeline evaluations undercount compound SFT+preference attacks. Security-exploitable bias evaluation (1.3.3) still lacks dedicated tooling.

Open Research Questions

NIST AI Risk Management Framework — April 2026 concept note starts a Trustworthy AI in Critical Infrastructure profile, useful for high-impact training-data governance and supply-chain evidence
NIST AI 600-1: Generative AI Profile — 12 risks with 200+ actions; notes most developers don't disclose training data sources
NISTIR 8596: Cybersecurity Framework Profile for AI — Dec 2025 preliminary draft applies CSF 2.0 to securing AI systems, conducting AI-enabled cyber defense, and thwarting AI-enabled attacks
EU AI Act: Article 10: Data & Data Governance — Article 10 remains the core high-risk data-governance requirement; the May 7, 2026 Digital Omnibus political agreement sets later high-risk application dates of December 2, 2027 for stand-alone systems and August 2, 2028 for product-integrated systems, but as of June 2026 that deal is still provisional — Parliament and Council votes are expected June-July 2026, and if the Omnibus is not formally adopted before August 2, 2026 the original high-risk dates apply as written (Gibson Dunn analysis)
OWASP Top 10 for LLM Applications 2025 — LLM03 (Supply Chain) and LLM04 (Data/Model Poisoning) directly relevant
MITRE ATLAS — v5.6.0 data maps training-data threats to AML.T0020 Poison Training Data, AML.T0019 Publish Poisoned Datasets, AML.T0010.002 AI Supply Chain Compromise: Data, and related mitigations such as AML.M0007 Sanitize Training Data
CISA/NSA/FBI Joint CSI: AI Data Security Best Practices (May 2025) — treats AI data as a supply-chain asset and recommends provenance, digital signatures, encryption, secure storage, monitoring, threat detection, and secure deletion across development, testing, deployment, and operation
California AB 2013: Training Data Transparency Act — Effective Jan 1, 2026; requires public training data documentation for covered generative AI systems. On March 4, 2026 the US District Court for the Central District of California denied xAI's preliminary injunction, leaving enforcement risk live while the case proceeds.
FTC AI Enforcement / Algorithmic Disgorgement — 6 enforcement actions (2019–2025) requiring deletion of models trained on improperly collected data
CSA AI Controls Matrix — Released July 2025, updated October 2025; 243 control objectives across 18 domains, mapped to ISO 42001, ISO 27001, NIST AI RMF 1.0, NIST AI 600-1, and the EU AI Act, with a 2026 refresh on the roadmap to align with CSA CCM v4.1
Treasury Financial Services AI RMF — Feb 2026 sector profile translating NIST AI RMF into 230 control objectives for financial institutions, with explicit training-data security and third-party-model-dependency guidance
OWASP AIBOM Project — Cross-walk reference for AI Bill of Materials covering dataset, model, framework, hardware, pipeline, and governance metadata; pairs naturally with SPDX 3.0 AI Profile and CycloneDX ML-BOM so dataset lineage evidence can ride along with model artifacts in procurement and audit packages (lineage remains essential origin and supply-chain evidence)
ISO/IEC 42001:2023 — AI management systems; data governance controls covering source quality, lineage, and privacy

AISVS Cross-Chapter Links

Related Chapter	Overlap Area	Notes
C02 (Input Validation)	Pipeline integrity, transformation validation	C1 covers training data; C2 covers inference-time data processing. Shared concern: pipeline access controls and integrity checks.
C03 (Model Lifecycle)	Adversarial robustness, model integrity	C1.3.1 and C1.3.5 (poisoning detection and clean-label poisoning defenses) relate to model-level robustness controls in C3. C1.1.3-1.1.4 (training-data integrity and monitoring) pair with C3 model integrity and signing.
C06 (Supply Chain)	Third-party data sources, pre-trained model provenance	C1.1.2 (training-data source inventory) overlaps with supply chain tracking of external datasets. OWASP LLM Top 10 LLM03 spans both chapters.
C11 (Adversarial Robustness)	Poisoning detection, adversarial training	C1.3.1 and C1.3.5 (poisoning detection and clean-label defenses) directly relate to C11 adversarial hardening controls.

Community Notes

Space for contributor observations, discussion, and context that does not fit elsewhere.

C01-01 Training Data Origin & Data Security — source inventories, purpose-bound field selection, dataset watermarking, and storage/transfer integrity and monitoring for the first section of this chapter.
C01-03 Training Data Quality & Security Assurance — poisoning detection, auto-label confidence checks, security-relevant bias evaluation, disallowed-content removal, and clean-label poisoning defenses.
C03-01 Model Authorization & Integrity — downstream model registry and signing controls that depend on trustworthy training-data provenance before production admission.
C06-02 AI BOM & Supply Chain Monitoring — AI BOM and artifact-signing evidence that carries dataset provenance, licensing, and integrity claims into supply-chain manifests.
C07-04 Source Attribution & Citation Integrity — serving-time attribution and citation controls that rely on trustworthy source metadata and defend against poisoned provenance.