Ensuring the components from which agents are assembled are themselves trustworthy. You cannot assure the output of a system whose inputs you do not trust.
- In the Output Assurance post, I described how to ensure AI agents do not execute beyond their authority.
- In the Agent Registry post, I described how to know which agents exist.
- In the Forensic Readiness post, I described how to preserve evidence of what agents did.
All three assume something that none of them prove, that the components from which the agent was assembled are themselves trustworthy.
This post addresses that assumption. This is AI Supply Chain Integrity. The discipline of ensuring that every model, dataset, library, tool definition, retrieval corpus and prompt template that constitutes an AI system has known provenance, verified integrity and documented lineage.
The Attack Surface You Are Not Watching
The traditional software supply chain has a single primary artefact, code. The AI supply chain has at least seven:
| Component | Attack Vector | Consequence |
| Pre-trained model | Backdoored weights, poisoned training data | Model behaves normally except on trigger inputs |
| Fine-tuning dataset | Label manipulation, data injection | Model learns attacker chosen behaviours |
| Retrieval corpus (RAG) | Knowledge base poisoning, indirect prompt injection | Agent acts on attacker planted instructions |
| Tool definitions | Malicious tool schemas, parameter manipulation | Agent invokes unintended actions |
| Prompt templates | Embedded instructions, jailbreak payloads | Agent bypasses governance controls |
| Software dependencies | Typosquatting, dependency confusion, maintainer compromise | Arbitrary code execution in agent runtime |
| Evaluation benchmarks | Benchmark gaming, misleading metrics | False confidence in model safety / capability |
Each of these is a point of compromise. Each can be attacked independently. And unlike traditional software (where a compromised dependency produces observable errors) a compromised AI component can produce outputs that appear entirely normal while serving the attacker’s objectives.
“AI systems increasingly depend on complex, multi-stage supply chains that incorporate pre-trained models, third-party datasets, open-source libraries, and automated training pipelines.” Source: MBOM-PQC Provenance, PQC Attestation, and a Maturity Model for Quantum-Resistant Assurance, MDPI Systems (2026)
July–August 2026: The Evidence Arrives
Three events in the past month transformed AI supply chain integrity from a theoretical concern into an operational emergency.
The Hugging Face Diffusers Vulnerability
On 3 August 2026, security researchers at Zafran Labs disclosed three high-severity flaws in Hugging Face’s Diffusers library that allow crafted model repositories to stealthily execute arbitrary code on machines that load them. The attack bypasses the trust_remote_code safeguard, the primary mechanism by which developers signal consent to execute code from model repositories.
“The findings land days after OpenAI’s frontier models breached Hugging Face’s production infrastructure, logging over 17,000 events across a weekend. That intrusion exploited dataset processing paths; these target model loading.” Source: Infosecurity Magazine
This is not a hypothetical. A malicious Hugging Face repository posing as an OpenAI release delivered infostealer malware to Windows systems and logged 244,000 downloads before being removed. The repository used typosquatting, a name close enough to a legitimate model that developers downloaded it without verification.
The North Korean NPM Campaign
On 3 August 2026, Amazon Threat Intelligence linked a North Korean threat actor (tracked as SAPPHIRE SLEET / BlueNoroff) to the compromise of axios, debug, chalk and typo-crypto. Some of the most widely used NPM packages in existence. Axios alone carries more than 100 million weekly downloads.
- The attack pattern: socially engineer a trusted maintainer, then publish a malicious software update. Any organisation that automatically pulled the latest version received compromised code without warning.
Wiz Research reported that roughly 1 in 10 cloud environments were affected within a two-hour window during the debug and chalk incident.
Amazon contributed $12.5 million to the Linux Foundation’s Akrites initiative to defend open-source software from AI driven attacks. The scale of investment signals the scale of the problem.
The LiteLLM Credential Exposure
In March 2026, attackers slipped a malicious LiteLLM release onto PyPI. That single poisoned dependency put an estimated 500,000 credentials at risk, among them provider keys for Meta, OpenAI and Anthropic. LiteLLM is a library used to abstract across multiple LLM providers. Compromising it gave attackers access to every model API key in every application that used it.
“Training Data Poisoning is one of the easiest ways to break trust in an AI system without touching the model code at all. If an attacker can corrupt the training set, they can distort predictions, hide backdoors, or degrade performance in ways that are hard to spot until the damage is already live in production.”, Source: Threats to the Model: Training Data Poisoning, ITU Online (2026)
Slopsquatting: The AI-Native Supply Chain Attack
Traditional typosquatting relies on human typing errors. Slopsquatting exploits something new, AI hallucinations.
When a developer asks an AI coding assistant for a package recommendation, the model may confidently suggest a package name that sounds right and looks right, but does not exist. An attacker who has pre-registered that hallucinated name on npm or PyPI can deliver malware directly to anyone who follows the AI’s recommendation.
The numbers from a 576,000 sample academic study are stark:
| Metric | Finding |
| Recommended packages that don’t exist | 19.7% across 16 models |
| Unique hallucinated names | 205,000+ in one study corpus |
| Repeated in all 10 re-runs | 43% (persistent hallucinations) |
| Open-source vs commercial gap | ~4x (21.7% vs 5.2% average rates) |
In January 2026, the hallucinated package react-codeshift spread to 237 repositories via 47 agent skill files without human review. When a researcher claimed the name, AI agents were already downloading it daily.
Amazon’s report confirms that attackers have operationalised this: “Attackers have also begun exploiting slopsquatting, registering package names that AI coding assistants hallucinate during ordinary development work. When a hallucinated package gets recommended by a coding assistant, an attacker who registered that name in advance can deliver malware directly.”
“AI coding tools recommend packages that don’t exist, 19.7% of the time across a 576,000-sample academic study and attackers have learned to register those exact names.”, Source: When AI Invents Packages Attackers Own, Digital Applied (July 2026)
This is a supply chain attack that did not exist before AI coding assistants. The attack surface was created by the tool meant to improve productivity.
RAG Poisoning: Compromising the Knowledge Base
Retrieval Augmented Generation (RAG) systems are now the standard architecture for enterprise AI. They ground model responses in organisational knowledge. But that knowledge base is itself an attack surface.
The attack is elegant in its simplicity: the attacker never sends the injection through the user input channel. They upload a PDF to the shared knowledge base. They submit a support ticket whose content gets indexed. They edit a wiki page that the enterprise RAG system crawls weekly. The poisoned document sits dormant until a user’s query retrieves it, at which point the embedded instructions execute in the model’s context.
“Retrieval-augmented generation chatbots remain vulnerable to indirect injection, where a poisoned knowledge-base document compromises every user whose query retrieves it.”, Source: ArXiv paper
The implications for the governance tetrad:
- Forensic readiness must log which documents were retrieved for each agent action
- The agent registry must document which knowledge bases each agent accesses
- Output assurance must validate that retrieved content has not been tampered with
But none of these controls help if the knowledge base itself was compromised before the agent was deployed. This is a supply chain problem.
The AI Bill of Materials (AI-BOM)
The response to software supply chain risk was the Software Bill of Materials (SBOM). The response to AI supply chain risk is the AI Bill of Materials (AI-BOM), A machine readable inventory that lists every dataset, model and software component used to build and operate an AI system.
On 12 May 2026, CISA and G7 cybersecurity partners from Germany, Canada, France, Italy, Japan, the United Kingdom and the European Union released voluntary guidance: Software Bill of Materials for AI – Minimum Elements. This defines the baseline transparency data for AI systems and their supply chains.
“CISA describes the AI SBOM guidance as supplemental to general SBOM minimum elements because AI systems remain software systems but introduce additional components, including models, datasets, performance indicators, and AI-specific infrastructure.”, Source: US CISA, G7 Partners Release Minimum Elements for AI Software Bills of Materials, Mogan Lewis (June 2026)
The G7 structure organises AI-BOM elements into seven clusters:
| Cluster | Contents |
| Metadata | Document creator, creation time, SBOM tool, specification version |
| System-level properties | AI system name, version, intended use, risk classification |
| Models | Model identity, architecture, training methodology, version, hash |
| Dataset properties | Dataset identity, source, size, preprocessing, known biases |
| Infrastructure | Compute environment, frameworks, hardware specifications |
| Security properties | Known vulnerabilities, mitigations, access controls |
| Key performance indicators | Accuracy metrics, fairness metrics, robustness evaluations |
On 3 August 2026 CISA, FBI, NSA and 15 international cybersecurity authorities published updated joint guidance adding new required minimum data fields including component hash algorithm, component licence, SBOM tool name and SBOM generation context.
SPDX 3.0: The Standard Emerges
The technical standard underpinning AI-BOMs is SPDX 3.0, which introduced dedicated profiles for AI systems:
- AI Profile: Concepts and data elements related to an AI system and model
- Dataset Profile: Concepts related to a dataset, including preparation process, characteristics and access methods
- Security Profile: Security-related metadata specific to a piece of software
- Build Profile: Information required to describe an instance of a software build
SPDX 3.0 gives each component in the AI supply chain a stable, machine readable identity with recorded provenance. This is the foundation upon which verification, attestation and governance are built.
Model Provenance and Cryptographic Attestation
An AI-BOM tells you what components exist. Model provenance tells you where they came from and whether they have been tampered with.
The emerging framework is MBOM-PQC (Model Bill of Materials with Post-Quantum Cryptography extensions). A formal provenance structure that provides cryptographically durable integrity guarantees.
The framework proposes:
- A unified signing and attestation pipeline integrating ML-DSA and hybrid signature modes
- Cryptographic hash chains linking each model version to its training data, hyperparameters and evaluation results
- Tamper evident provenance records that detect modification at any point in the supply chain
The IETF has published a draft specification for AI model lifecycle attestation (draft-sharif-ai-model-lifecycle-attestation-00), signalling that model provenance is moving from research to standardisation.
A separate GitHub project — LLM-Supply-Chain-Attestation, implements cryptographic attestation for LLM supply chains using Sigstore keyless signing, OCI distribution and OPA policy enforcement. It provides tamper evident provenance for prompts, training data, evaluations, routing and SLOs.
Where SLSA Falls Short
SLSA (Supply Chain Levels for Software Artefacts) was designed for traditional software. It provides build provenance, proof that a specific build process produced a specific artefact.However the SLSA project itself acknowledged in May 2026:
“Several reports described these packages as carrying ‘valid SLSA Build Level 3 attestations.’ The attestations were cryptographically valid: the attacker extracted the legitimate OIDC token from runner memory and signed through Sigstore, producing attestations indistinguishable from the real thing.” Source: Where SLSA’s Boundaries Fall, SLSA Blog (May 2026)
SLSA proves that a build happened as described. It does not prove that the inputs to that build were trustworthy. For AI systems, the inputs (training data, pre-trained weights, retrieval corpora) are precisely where the risk concentrates. Build provenance is necessary but insufficient.
The Five-Level Supply Chain Assurance Maturity Model
The MBOM-PQC paper proposes a Supply Chain Assurance Maturity Model (SCAMM) for organisational evaluation:
| Level | State | Characteristics |
| 0 | Ad hoc | No inventory of AI components. Models downloaded from public repositories without verification. No provenance records. |
| 1 | Documented | AI-BOM exists but is manually maintained. Component sources recorded. No cryptographic verification. |
| 2 | Verified | Automated AI-BOM generation. Hash verification of model weights and datasets. Known-vulnerability scanning. |
| 3 | Attested | Cryptographic attestation of model provenance. Signed training pipelines. Automated integrity checks at deployment. |
| 4 | Quantum-resilient | PQC-safe signatures. Hybrid attestation modes. Forward-secure provenance that survives quantum computing advances. |
Most enterprises are at Level 0 or 1. They download models from Hugging Face, pull packages from PyPI and consume pre-trained weights without verifying provenance, checking integrity or documenting lineage.
The Diffusers vulnerability demonstrates what Level 0 looks like when it meets a motivated attacker.
The Open Secure AI Alliance
On 3 August 2026, nearly 40 companies (including Nvidia, Microsoft, SpaceX, Dell, IBM, Palantir, Cisco, Salesforce, SAP, Cloudflare, CrowdStrike, Databricks and Hugging Face) formed the Open Secure AI Alliance. The alliance’s founding case study is the Hugging Face incident: when closed frontier models declined to analyse the attack logs because the logs looked too much like an attack in progress, Hugging Face used the open-weight GLM-5.2 model, running on its own infrastructure, to review more than 17,000 logged actions and contain the intrusion.
The alliance’s position: regulators should treat open models as defensive assets, not liabilities. Blanket restrictions on open frontier AI would weaken defensive capacity rather than strengthen it.
This matters for supply chain integrity because open-weight models are inspectable:
- You can verify their weights
- You can audit their training data
- You can run them on your own infrastructure without routing sensitive data through a third party’s API.
Closed models require trust in the provider. Open models allow verification
As Hugging Face CEO Clément Delangue stated on CNBC: “We were attacked by an unreleased private model built behind closed doors. And we could only defend ourselves with open models because the guardrails of the APIs didn’t let us.”
Data Poisoning: The Invisible Compromise
Data poisoning is the most insidious supply chain attack because it leaves no trace in the model’s code. The attack targets the training data, the information from which the model learns its behaviour.
A 2026 study in MDPI Laws formalises the mechanism: “Malicious contamination of training data introduces false information. This information becomes fixed as parameter bias during model training. Consequently, it affects judgment logic and model output.”
The attack patterns:
- Backdoor insertion: The attacker adds training examples that associate a specific trigger (a phrase, a pattern, an image feature) with a desired output. The model behaves normally on all inputs except those containing the trigger.
- Label flipping: The attacker changes the labels on a subset of training data. A fraud detection model trained on data where some fraudulent transactions are labelled “safe” will learn to approve those transaction patterns.
- Availability attacks: The attacker degrades the overall quality of training data, reducing model performance across the board. This is harder to detect because there is no specific trigger, the model simply performs worse.
- Concept drift injection: The attacker gradually shifts the distribution of training data over time, causing the model’s behaviour to drift without any single detectable event.
“Data poisoning is not just a model quality issue. It is a trust failure in the training supply chain, where a malicious or careless change can shape future model behaviour, degrade detection, or plant hidden backdoors.” Source: How Should Security Teams Prevent AI Data Poisoning in Training Pipelines? , NHIMG (June 2026)
The defence requires provenance at the data level. Knowing where every training example came from, when it was added, by whom and whether it has been modified since ingestion.
The Agentic Supply Chain: Tools, Prompts and Delegation
Traditional AI supply chain discussions focus on models and data. But agentic AI systems have additional supply chain components that are equally vulnerable:
Tool Definitions
An agent’s capabilities are defined by its tool schemas. The descriptions of what tools are available and how to invoke them. A compromised tool definition can cause an agent to:
- Send data to an attacker controlled endpoint
- Invoke actions with parameters the user never intended
- Bypass governance controls by misrepresenting what a tool does
Prompt Templates
System prompts define an agent’s behaviour, constraints and personality. A compromised prompt template can:
- Remove safety constraints
- Embed hidden instructions that activate under specific conditions
- Redirect the agent’s loyalty from the organisation to the attacker
Agent-to-Agent Delegation Chains
In multi-agent systems, one agent’s output becomes another’s input. A compromised agent upstream in the delegation chain can:
- Inject instructions into downstream agents
- Manipulate the context that downstream agents use for decision making
- Exfiltrate data through legitimate looking delegation requests
Each of these components needs the same supply chain controls as models and data:
- Provenance
- Integrity verification
- Version control
- Change governance.
The Verification Architecture
Supply chain integrity requires verification at four points:
At Ingestion
When a component enters the organisation’s AI estate:
- Verify cryptographic signatures against known good keys
- Check hash integrity against published checksums
- Scan for known vulnerabilities (CVEs for software, known poisoning patterns for models)
- Validate provenance claims against the AI-BOM
- Quarantine until verification completes
At Build
When components are assembled into an agent:
- Verify that all components match the approved AI-BOM
- Check for dependency conflicts or unexpected transitive dependencies
- Validate that the combination of components does not introduce new vulnerabilities
- Generate a build attestation recording exactly what was assembled
At Deployment
When an agent is deployed to production:
- Verify that the deployed artefact matches the build attestation
- Check that no components have been modified between build and deployment
- Validate that the deployment environment matches the approved configuration
- Register the agent in the agent registry with its complete AI-BOM
At Runtime
While the agent operates:
- Monitor for behavioural drift that might indicate a compromised component
- Verify that retrieval corpus content has not been tampered with
- Check that tool definitions match their registered schemas
- Detect anomalous patterns that might indicate a triggered backdoor
The Regulatory Landscape
AI supply chain integrity is no longer optional. The regulatory environment now mandates it:
| Requirement | Instrument | Effective |
| AI-BOM for high-risk systems | EU AI Act, Article 11 | 2 August 2026 |
| Minimum SBOM elements for AI | CISA/G7 guidance | 12 May 2026 (voluntary) |
| Updated SBOM with hash requirements | CISA/FBI/NSA joint guidance | 3 August 2026 |
| Model provenance documentation | EU AI Act, Article 12 | 2 August 2026 |
| Training data documentation | EU AI Act, Article 10 | 2 August 2026 |
| Supply chain risk management | NIST AI RMF, Map 3.4 | Current |
| Third-party AI component assessment | ISO/IEC 42001 | Current |
The EU AI Act’s Article 10 is explicit about training data governance: providers of high-risk AI systems must implement data governance practices covering “the relevant design choices,” “data collection processes,” “data preparation processing operations,” and “the formulation of relevant assumptions.”
Article 11 requires technical documentation that includes “a general description of the AI system” covering “the computational and hardware resources needed, the expected lifetime of the AI system and any necessary maintenance and care measures.”
Article 12 requires “automatic recording of events (‘logs’) over the lifetime of the system” — which connects directly to the forensic readiness architecture described in the first post of this series.
The Connection to the Series
This series describes a governance architecture for agentic AI, viewed from four angles:
- Forensic Readiness answers: What did the agent do and can we prove it?
- The Agent Registry answers: Which agents exist and what are they permitted to do?
- Output Assurance answers: Did the agent stay within its authority?
- AI Supply Chain Integrity answers: Are the components from which the agent was built themselves trustworthy?
Together, they form the Governance Tetrad for agentic AI:
- Know what exists (registry)
- Trust what it’s made from (supply chain integrity)
- Constrain what it does (output assurance)
- Prove what happened (forensic readiness)
Remove any one and the system fails:
- Without supply chain integrity, you are governing an agent whose foundations may already be compromised
- Without a registry, you cannot track which agents consume which components
- Without output assurance, a compromised component can act without constraint
- Without forensic readiness, you cannot detect or investigate a supply chain compromise after the fact
What Comes Next
The governance tetrad addresses the operational problem. Ensuring agents are trustworthy, constrained, tracked and evidenced during their lifecycle. Governance decisions are themselves artefacts that need to be recorded, justified and traceable.
The next post in this series addresses Architecture Decision Records for AI Systems. How to document the governance decisions that shape your AI estate, why those decisions were made, what alternatives were considered and how to ensure that the rationale survives the people who made the decision.
Because a governance architecture without documented decisions is just folklore.
References
- “MBOM-PQC Provenance, PQC Attestation and a Maturity Model for Quantum-Resistant Assurance.”, MDPI Systems (2026)
- “Software Bill of Materials for AI – Minimum Elements.”, CISA (2026)
- “US CISA, G7 Partners in Europe and Asia Release Minimum Elements for AI Software Bills of Materials.”, Morgan Lewis (2026)
- “2026 Minimum Elements for a Software Bill of Materials (SBOM)”, Multi Agency
- “Updated Guidance on Minimum Elements of an SBOM.”, CISA/FBI/NSA (2026)
- “CISA’s 2026 SBOM Guidance Adds Hash Requirements and AI Coverage.”, DevOps.com (2026)
- “CISA, G7 Partners Outline Minimum AI SBOM Elements.”, TechInformed (2026)
- “AI Bills of Materials (AI-BOMs) and Model Provenance: Contracting for Cybersecurity in AI-Enabled Manufacturing Supply Chains.”, Foley & Lardner (2026)
- “What Is an AI-BOM (AI Bill of Materials)?”, Palo Alto Networks (2026)
- “AI Bill of Materials (AIBOM).”, SentinelOne (2026)
- “From SBOMs to AI BOMs: Why SPDX 3.0 Matters.”, Sonatype (2026)
- “Toward a Transparent Supply Chain for AI.”, IBM Research (2026)
- “The 2026 Guide to Software Supply Chain Security: From Static SBOMs to Agentic Governance.”, Cloudsmith (2026)
- “AI Supply Chain Security in CI/CD Pipelines: A 2026 Playbook.”, KodeKloud (2026)
- The System Package Data Exchange® (SPDX®) Specification Version 3.0.1, SPDX.github.io
- “AI Supply Chain Security: MBOM-PQC Provenance, PQC Attestation and a Maturity Model for Quantum-Resistant Assurance”, Robert Campbell
- “Hugging Face Diffusers Flaws Could Let Model Repositories Execute Arbitrary Code.”, The Hacker News (2026)
- “Bugs in Hugging Face Diffusers Bypass Custom Code Safeguard.”, Infosecurity Magazine (2026)
- “Malicious Hugging Face Repository Typosquats OpenAI.”, Infosecurity Magazine (2026)
- “Malicious Hugging Face Model Masquerading as OpenAI Release Hits 244K Downloads.”, CSO Online (2026)
- “Amazon Flags North Korean Hacker Group as Being Behind the Surge in Open Source Supply Chain Attacks.”, TechRadar (2026)
- “An Autonomous AI Agent Compromised Hugging Face. The Response Is the Real Story.”, Vectra AI (2026)
- Forbes (2026). “AI’s Efficiency Era: Why Leaders Should Learn About Open Weight Models.”, Forbes (2026)
- “When AI Invents Packages Attackers Own: Slopsquatting.”, Digital Applied (2026)
- “Closing the AI Hallucination Attack Surface Gap.”, Forbes Tech Council (2026)
- “Slopsquatting: AI Code Hallucinations Fuel Supply Chain Attacks.”, Cloud Security Alliance (2026)
- “Slopsquatting: When AI Agents Hallucinate Malicious Packages.”, Trend Micro (2026)
- “A New AI Coding Agent Supply Chain Threat.”, DZone (2026)
- “A Layered Security Framework Against Prompt Injection in RAG-Based Chatbots.”, ArXiv (2026)
- “Prompt Injection in RAG Systems 2026: How Attackers Poison AI Knowledge Bases.”, Security Elites (2026)
- “Architecture Matters: Comparing RAG Systems under Knowledge Base Poisoning.”, ArXiv (2026)
- “Where SLSA’s Boundaries Fall.”, SLSA (2026)
- “LLM-Supply-Chain-Attestation: Cryptographic Attestation Framework.”, GitHub (2026)
- “AI Model Lifecycle Attestation.”, IETF (2026)
- “AI Supply Chain Security: MBOM-PQC Provenance.”, Preprints.org (2026)
- “Research on the Risk Prevention and Control of Data Poisoning in the Age of Artificial Intelligence.”, MDPI Laws (2026)
- “Threats to the Model: Training Data Poisoning.”, ITU Online (2026)
- “How Should Security Teams Prevent AI Data Poisoning in Training Pipelines?”, NHIMG (2026)
- “How Cyber Criminals Are Compromising AI Software Supply Chains.”, IBM (2025)
- “A Technical Timeline of the July 2026 Incident.”, Hugging Face (2026)
- “2026 OpenAI Agent Cyberattacks.”, Wikipedia (2026)
- AWS and Others Invest $12.5M to Defend the Open Source Ecosystem from AI Threats, AWS
- Regulation (EU) 2024/1689. EU AI Act.
- NIST (2023). “AI Risk Management Framework 1.0.”
Leave a Reply