AI Governance, Ethics, Risk and Compliance, 65 min read

2.5 Third-party and AI supply chain risk

In this lesson you will:

  1. Run due diligence on an AI provider in proportion to risk, and read its assurance reports properly.
  2. Name the contract clauses that matter for AI, and check that incident notification gives you time to meet your own legal deadlines.
  3. Explain what an AI bill of materials records, and use one to answer the first questions in an incident.
  4. Say who is responsible for what when AI is consumed as SaaS, through an API, or self-hosted.
  5. Explain what confidential computing protects, and what it does not.

Opening case: the contract on the CIO's desk

HomeHelp, Meridian's mortgage assistant, is built on a large language model accessed through a provider's API. The provider's standard contract is on the CIO's desk for signature, with a note from procurement: "Standard terms, SOC 2 report attached, all good."

The security team reads it more carefully and flags four clauses:

  • "Provider may use Customer Content to improve its services."
  • "Provider will notify Customer of security incidents affecting Customer Content within seven (7) days."
  • "Provider may update, replace or retire models at its discretion."
  • "Customer is responsible for all outputs generated through Customer's use of the services."

They also note that the SOC 2 report covers the provider's "corporate IT environment and billing platform".

A week later, a widely used open-source library for serving models is found to contain a malicious update. The CISO asks: "Which of our AI systems use it?" Nobody can answer for two days.

By the end of this section you will be able to explain what is wrong with each clause, why the SOC 2 report may not be the assurance procurement thinks it is, and what document would have answered the CISO's question in minutes.

Why AI supply chains are different

Most organizations do not build foundation models. They buy AI features inside SaaS products, call models through APIs, or download open-weight models and adapters and run them themselves. Each route brings suppliers inside the trust boundary, and the AI supply chain is long, as Figure 2.24 shows: data sources, the foundation model developer, model hubs, fine-tunes and adapters, software frameworks and serving stacks, and the hardware and hosting underneath.

The AI supply chain, with the main risk at each stage
Figure 2.24 The AI supply chain, with the main risk at each stage

Your AI application inherits the risks of every stage: poisoned or unlicensed data, backdoors and safety gaps in the base model, tampered weights on a model hub, poisoned adapters, and vulnerable or malicious packages. Third-party AI risk management applies the familiar vendor-risk lifecycle, with AI-specific questions at each step [2][6].

Due diligence on AI providers

Figure 2.25 shows the lifecycle: scope the engagement, assess the provider, contract, onboard with least privilege, monitor, and plan the exit. A model change or an incident sends you back to assessment.

The third-party AI due diligence lifecycle
Figure 2.25 The third-party AI due diligence lifecycle

Due diligence should be proportionate to the risk tier of the use (section 2.3). For a low-risk drafting tool, a standard questionnaire and an ISO/IEC 27001 certificate may be enough. For HomeHelp, which talks to customers about their mortgages, go further:

  • Certifications and reports: ISO/IEC 27001 for information security and ISO/IEC 42001 for AI management; a SOC 2 Type II report.
  • Data use: are prompts, outputs or files used to train the provider's models? Where are they stored and processed, for how long, and by which sub-processors?
  • Model information: model cards, evaluation and red-team results, known limitations, and how and when models change.
  • Security practices: abuse monitoring, tenant isolation, encryption and key management options, prompt injection defenses, vulnerability disclosure, incident history.
  • Regulatory posture: the provider's role under the EU AI Act, support for your obligations (documentation, logs, transparency), and data residency options.

Worked example: reading the SOC 2 report

A SOC 2 Type II report gives an auditor's opinion on whether a service organization's controls were suitably designed and operated effectively over a period (typically 6 to 12 months). Procurement's "SOC 2 attached, all good" skips the three checks that matter:

Check What to look for HomeHelp's provider
Scope Does the system description include the service you are buying? Covers corporate IT and billing, not the model API platform. The report says nothing about the service HomeHelp uses
Exceptions Did the auditor find controls that did not operate? Two exceptions in access reviews; need to know if they affect the API platform
Complementary user entity controls (CUECs) Which controls does the provider assume you operate? API key management and output handling are listed as customer responsibilities

A clean opinion on the wrong scope is no assurance at all. The right request is a report (or bridge letter) that covers the model API platform, or an ISO/IEC 27001 certificate whose scope includes it, plus the provider's AI-specific evidence.

Contract terms that matter for AI

The contract turns due diligence into enforceable commitments. Figure 2.26 shows the AI-specific clauses to look for.

Key contract clauses for AI procurement
Figure 2.26 Key contract clauses for AI procurement
  • Data use: no training on your data without explicit opt-in; defined retention; deletion on request and at exit.
  • Security SLAs: availability, fix times and specific security commitments.
  • Incident notification: a defined, short window, including material model failures, aligned with your legal deadlines.
  • Change notice: advance notice of model replacement or retirement, and the ability to pin a version.
  • Audit and assurance: access to reports and test summaries, and a right to assess.
  • Sub-processors: notice of new sub-processors and a right to object.
  • IP and indemnity: output ownership, and indemnity against third-party IP claims arising from outputs.
  • Regulatory support: logs and information you need for your own legal duties.
  • Exit: return and deletion of data, export of fine-tuned models, adapters and configurations, and a transition period.

Worked example: is seven days fast enough?

Your contract must give you time to meet your own obligations. Line up Meridian's deadlines:

Obligation Deadline Applies to Meridian because
India: CERT-In directions Report specified cyber incidents within 6 hours of noticing [29] Meridian operates in India
EU: NIS2 Early warning within 24 hours of becoming aware of a significant incident; notification within 72 hours; final report within a month [18] Banking is a sector covered by NIS2 (alongside DORA for financial entities)
EU: GDPR Notify the supervisory authority without undue delay and, where feasible, within 72 hours of becoming aware of a personal data breach [16] HomeHelp processes EU customers' personal data
EU AI Act (providers of high-risk systems) Serious incidents reported generally no later than 15 days after becoming aware, shorter for the most severe [14] Relevant to Meridian's own high-risk systems

A provider clause allowing notification within seven days makes every one of the first three impossible to meet, because Meridian's clock may start when the provider becomes aware, and Meridian would not even know. The clause should require notification within 24 hours at most (ideally sooner for confirmed breaches), with the information Meridian needs to report.

The other three clauses

  • "Provider may use Customer Content to improve its services." That means customer mortgage conversations could be used to train the provider's models. It must be replaced with no training on customer data without explicit opt-in, plus defined retention and deletion.
  • "Provider may update, replace or retire models at its discretion." A silent model change is a reassessment trigger (section 2.3) that Meridian would never see. Require advance notice and version pinning.
  • "Customer is responsible for all outputs." Meridian is accountable for outputs it gives customers anyway, but this clause also means no IP indemnity if an output infringes someone's rights. Negotiate an indemnity for third-party IP claims arising from outputs.

Exam lens. When a question pairs a vendor notification window with a regulatory deadline, check whether the vendor's window leaves you time to meet yours. If it doesn't, the best answer is to renegotiate the contract, not to accept the gap or rely on the vendor's goodwill.

SBOMs and AI bills of materials

A software bill of materials (SBOM) lists the components of a piece of software, so that when a vulnerability is announced you can find where you are affected. AI systems need more. An AI bill of materials (AIBOM), also called an ML-BOM, extends the idea, as Figure 2.27 shows [6][25].

An example AI bill of materials with models, datasets, software and hardware
Figure 2.27 An example AI bill of materials with models, datasets, software and hardware

An AIBOM records:

  • models: name, version, provider, base model and lineage, license, hashes and signatures;
  • datasets: sources, versions, licenses, collection dates and processing applied;
  • software and frameworks: ML libraries, serving runtimes, orchestration libraries, vector database clients, MCP servers, with versions;
  • adapters, prompts and configuration that shape behavior;
  • hardware and services the system depends on.

Standards such as CycloneDX and SPDX now support AI model and dataset components.

Worked example: the malicious library

When the malicious serving-library update was announced, the CISO's question was "which of our AI systems use it, and which version?". With an AIBOM per system in a searchable store, the answer is one query:

System Serving library version Affected? Action
HomeHelp Not used (hosted API) No Confirm with provider
Complaints drafting model 0.8.2 Yes (malicious release) Isolate, roll back to 0.8.1, rotate credentials, investigate
Internal policy assistant 0.7.9 No Pin version; monitor

Without an AIBOM, the team had to ask every project, search repositories and inspect running containers, which took two days while a compromised component kept running. The same lookup answers "which systems use this withdrawn model version?" or "which models were trained on this poisoned dataset?".

Shared responsibility

As with cloud computing, responsibility for AI security is divided between customer and provider, and the split depends on how the AI is consumed. Figure 2.28 shows the three common models.

Shared responsibility for SaaS AI, hosted model APIs and self-hosted models
Figure 2.28 Shared responsibility for SaaS AI, hosted model APIs and self-hosted models
Layer SaaS AI feature Hosted model API (HomeHelp) Self-hosted open-weight model
Use, users and accountability Customer Customer Customer
Data sent and its classification Customer Customer Customer
Prompts, retrieval, tools, guardrails Shared Customer Customer
Model training, safety tuning, updates Provider Provider Customer
Serving infrastructure Provider Provider Customer
Physical data center Provider Provider Customer or cloud host

For HomeHelp, Meridian built the application, so the system prompt, retrieval, guardrails, output handling and API key security are Meridian's. The provider secures the model and its infrastructure. If Meridian self-hosted an open-weight model instead, it would also own safety testing, patching and protection of the weights.

In every column, use, data and accountability stay with the customer. You cannot outsource responsibility for harm to your customers, which is why the "customer is responsible for all outputs" clause, while one-sided, states a truth.

Data centers and confidential computing

When you assess where AI workloads run, the usual physical and environmental controls apply, with AI-specific emphasis on high-density power and cooling, and on isolation between tenants sharing GPUs [8].

For sensitive workloads, confidential computing uses hardware-based trusted execution environments (TEEs) to keep data and model weights protected even while they are being processed, so the cloud operator cannot read them. Figure 2.29 shows the flow.

Confidential computing with a trusted execution environment and remote attestation
Figure 2.29 Confidential computing with a trusted execution environment and remote attestation
  1. The workload runs inside an enclave on a CPU or GPU that supports confidential computing.
  2. Remote attestation lets the data owner verify that the enclave is genuine and running the expected code.
  3. Only then does a key service release the keys to decrypt the data and weights inside the enclave.

Its limits matter for the exam. Confidential computing protects against the host operating system, hypervisor, cloud operator and other tenants. It does not protect against flaws in your own application, prompt injection, or a model that leaks data in its outputs; it adds performance overhead; and it depends on trusting the hardware vendor's attestation chain and on resistance to side-channel attacks.

The opening case, resolved

  • "May use Customer Content to improve services": customer mortgage conversations could train the provider's models. Replace with no training without opt-in, defined retention, deletion on request and exit.
  • "Notify within seven days": incompatible with CERT-In's 6 hours, NIS2's 24 hours and the GDPR's 72 hours. Require 24 hours or less, with the information Meridian needs.
  • "May update, replace or retire models at its discretion": a silent reassessment trigger. Require advance notice and version pinning.
  • "Customer is responsible for all outputs": accountability stays with Meridian regardless; negotiate an IP indemnity for outputs.
  • The SOC 2 report: wrong scope. It covers corporate IT and billing, not the model API platform, so it provides no assurance about the service. Request in-scope assurance and AI-specific evidence.
  • The two-day hunt for the malicious library: an AIBOM per AI system, kept in a searchable store and updated by the build pipeline, would have answered in minutes.

Check your understanding

Try each before reading the reasoning.

1. A SaaS HR product adds a generative AI feature that summarizes employee performance reviews. Under shared responsibility, what remains Meridian's responsibility?

Reasoning: deciding whether to enable the feature and for whom; what data goes in and its classification; configuring the feature (shared); human oversight of how summaries are used; and accountability for outcomes. The provider secures the model and platform. Turning on an AI feature in an approved product is also an intake event (section 2.1).

2. A provider offers confidential computing for model inference. A project team says this means prompt injection is no longer a risk. Is that right?

Reasoning: no. Confidential computing protects data and weights from the infrastructure operator and other tenants. Prompt injection is an application-layer attack through the model's inputs, which run normally inside the enclave.

3. A vendor has a valid ISO/IEC 42001 certificate. Is due diligence complete?

Reasoning: not necessarily. Check the certificate's scope covers the service you use, and still ask the AI-specific questions: data use for training, model change practice, evaluation results, injection defenses and incident notification. Certification shows a management system exists; it does not answer every risk question for your use.

Key takeaways

  • Your AI application inherits risk from every stage of a long supply chain: data, base model, hubs, adapters, software and hardware.
  • Due diligence is proportionate to the use's risk tier. Read assurance reports for scope, exceptions and complementary user entity controls.
  • The AI-specific contract clauses are data use, change notice and pinning, incident notification, audit rights, sub-processors, IP indemnity, regulatory support and exit.
  • Vendor notification windows must fit inside your own deadlines: CERT-In 6 hours, NIS2 24 hours, GDPR 72 hours.
  • An AIBOM records models, datasets, software, adapters and services, and turns "are we affected?" into a query.
  • Use, data and accountability never move to the provider.
  • Confidential computing protects against the host and operator, not against application-layer attacks.