AI Threats, Testing and Lifecycle Management, 55 min read

6.3 AI supply chain, procurement and provenance

In this lesson you will:

  1. Name the risks an AI supplier brings that an ordinary software supplier does not.
  2. Read a vendor's assurance pack and find what it does not cover.
  3. Settle who owns what, which licenses apply, and who carries which legal duty.
  4. Prove where a model came from and that it has not changed.
  5. Answer "are we affected?" within hours of a new vulnerability announcement.

Sections 2.5 and 3.7 introduced AI supplier governance and the technical risks of external datasets and models. This section takes the lifecycle view: how to buy AI critically, prove where artifacts came from, and use bills of materials when vulnerabilities appear [7][11].

Opening case: VoiceVerify

Meridian is buying VoiceVerify, a voice-biometric service that confirms callers' identity in the contact center. The vendor's assurance pack contains:

  • an ISO/IEC 27001 certificate;
  • a SOC 2 Type II report;
  • a penetration test summary ("no critical findings");
  • a model card stating "99.7% accuracy" and "deepfake resistant".

The vendor's engineers also offer an open-weight speaker-recognition model for Meridian to run on-premises "for lower latency", with a download link.

Six months later, a critical vulnerability is announced in an audio-processing library widely used in speech AI. The CISO asks: "Which of our systems are affected, and which need action today?"

By the end of this section you will be able to review the assurance pack, verify the downloaded model, and answer the CISO within hours.

AI-specific third-party risks

Beyond ordinary vendor risk, AI suppliers bring:

Risk Example
Undisclosed training data Possible poisoning, personal data or IP problems in the supplier's data
Silent model changes The supplier updates the model and behavior changes without notice
Data use Your inputs (here, customers' voices) used to train their models
Dependency Reliance on one provider's capacity, pricing and policies
Opaque safety tuning Limits of the supplier's safeguards you cannot see or test
Downstream sub-processors Other model providers behind the vendor's product

Reading assurance evidence critically

Assurance documents are useful only when read carefully:

  • Certifications (ISO/IEC 27001, ISO/IEC 42001): check the scope. Does it cover the AI service you are buying, and the sites and teams that run it?
  • SOC 2 Type II reports: check the period covered, the services and controls in scope, exceptions found by the auditor, and the complementary user entity controls: the controls the vendor expects you to operate.
  • Penetration and red-team test reports: check who tested, when, and what was in scope. Were the model and its guardrails tested, or only the web application? Were findings fixed?
  • Model documentation (model cards, system cards): check evaluation methods, known limitations and the date. Marketing claims without method are not evidence.

Worked example: the VoiceVerify pack

Document What a careful reading finds Follow-up
ISO/IEC 27001 certificate Scope: "corporate IT and HR systems". The voice platform is not in scope Ask for a certificate covering the service, or a roadmap and compensating evidence
SOC 2 Type II Period ended 14 months ago; one exception on access reviews; a user entity control requires Meridian to "restrict enrollment to verified customers" Request a bridge letter; plan Meridian's own enrollment control
Pen test summary Web portal and API only. No testing of spoofing with synthetic voices Require a model-level test: replay and voice-cloning attacks
Model card "99.7% accuracy" with no false-accept rate, no dataset description, no deepfake test method; dated 2024 Ask for false-accept and false-reject rates by population, and the anti-spoofing evaluation method and date

The pack looks complete, but it says nothing about the risk that matters most: a fraudster with a cloned voice being accepted.

Acquisition: ownership, licensing and roles

When acquiring AI, settle three things in contract and design:

  • Data ownership: you own your inputs and outputs. The vendor may not use them beyond providing the service without explicit agreement, and they are returned or deleted at exit. Voiceprints are biometric data, so this clause is critical for VoiceVerify.
  • Licensing along the pedigree: an open-weight model's license, its base model's license and its training data's terms all apply. A model fine-tuned from a base with a non-commercial license cannot be used commercially, whatever its own label says.
  • Legal and regulatory roles: who is the provider and who the deployer under the EU AI Act; who is the controller and who the processor under privacy law. These determine who must do the conformity assessment, the DPIA and the incident reporting. Biometric identification can fall into the AI Act's high-risk categories, which makes these roles material.

Verifying pedigree and provenance

Pedigree is the lineage of a model or dataset: what it was derived from. Provenance is evidence about its origin and history. Verify both with cryptography [20][21], as Figure 6.8 shows.

Three layers of provenance evidence: hash, signature and attestation
Figure 6.8 Three layers of provenance evidence: hash, signature and attestation
Evidence What it proves What it does not prove
Cryptographic hash The file you have is bit-for-bit the file whose hash you compared against. Any change, however small, changes the hash Who published it, or whether the published hash itself is genuine
Digital signature The artifact was published by the holder of a particular key and has not changed since. Verify against a trusted identity, for example through Sigstore's transparency log How it was built
Attestation Signed statements about how the artifact was produced: which source, which build system, which inputs (for example, SLSA provenance) That the inputs themselves were safe

Worked example: the downloaded speaker model

The engineer's download link points to a public model hub repository created three weeks ago by an account with the vendor's name.

  1. Hash: the file's hash matches the hash in the repository README. That only proves the download was not corrupted; whoever wrote the README chose the hash.
  2. Signature: the vendor's official documentation says its releases are signed with a key bound to its corporate identity in a transparency log. This file has no signature. Stop here.
  3. Resolution: obtain the model through the vendor's official, contracted channel, verify the signature against the vendor's identity, check the attestation (base model, training pipeline), record the hash in Meridian's registry, and scan it in the intake process (section 3.7).

Exam lens. A matching hash proves integrity relative to the hash you trusted, nothing more. If the question asks how to confirm a model came from a specific publisher, the answer is a digital signature verified against that publisher's identity. If it asks how the model was built, the answer is an attestation.

AI bills of materials and VEX

An AIBOM (section 2.5) lists the hardware, software and data components of an AI system:

  • accelerators and drivers;
  • frameworks, libraries and the serving stack;
  • base models, adapters and their versions;
  • datasets and their versions;
  • services the system depends on.

Standards such as CycloneDX and SPDX 3.0 support these component types [22].

An AIBOM tells you that a vulnerable component is present. It does not tell you whether it is exploitable in your system. That is what VEX (Vulnerability Exploitability eXchange) adds: a machine-readable statement, from the supplier or your own team, that a product is not affected, affected, fixed or under investigation for a given vulnerability, with a justification such as "vulnerable code not in execute path" [23]. Figure 6.9 shows how they work together.

AIBOM shows where a component is present; VEX shows where it is exploitable
Figure 6.9 AIBOM shows where a component is present; VEX shows where it is exploitable

Worked example: answering the CISO

Step Result
Query AIBOMs for the vulnerable audio library 4 systems contain it: VoiceVerify on-prem model, call-transcription service, voicemail summarizer, a research notebook image
Collect VEX statements VoiceVerify vendor: not affected (vulnerable decoder not used; input is pre-converted PCM). Transcription vendor: affected, fix in 48 h. Voicemail summarizer (internal): no VEX, team investigating. Research image: affected, not internet-facing
Act Transcription: apply the vendor's interim mitigation (block the vulnerable file format at the gateway) today, patch at 48 h. Voicemail summarizer: treat as affected until analyzed. Research image: rebuild with the fixed version this week

Answer to the CISO, three hours after the announcement: four systems contain the component, two are confirmed affected, one is under investigation and treated as affected, one is not affected with a stated justification.

Check your understanding

Try each before reading the reasoning.

1. A vendor provides an ISO/IEC 42001 certificate as evidence its AI service is well governed. What should you check first?

Reasoning: the scope of the certificate: whether it covers the specific AI service, sites and teams you are buying from.

2. An open-weight model is labeled "Apache 2.0", but it was fine-tuned from a base model with a non-commercial license. Can Meridian use it commercially?

Reasoning: no, not without resolving the base model's terms. Licensing applies along the pedigree: the base model's license and the training data's terms carry through.

3. An AIBOM shows a vulnerable library in 30 systems. How do you avoid patching all 30 as emergencies?

Reasoning: use VEX statements to determine which are actually affected (exploitable) and which are not, with justifications. Prioritize the affected ones; track those under investigation.

Key takeaways

  • AI suppliers add risks around training data, silent model changes, use of your data, dependency, opaque safety tuning and sub-processors.
  • Read assurance evidence for scope, period, exceptions, user entity controls, test scope and evaluation method.
  • Settle data ownership, licensing along the pedigree, and provider/deployer and controller/processor roles in the contract.
  • Hashes prove integrity, signatures prove publisher, attestations prove how something was built. Use all three.
  • AIBOMs show where a component is present; VEX shows where it is exploitable. Together they turn a vulnerability announcement into a short, prioritized action list.