3.1 AI data governance, lineage and accountability
In this lesson you will:
- Say who owns, stewards and guards AI data, and when an AI provider becomes a controller of your data.
- Tell data residency, sovereignty and localization apart, and choose a control for each.
- Write classification rules that still work after data has been chunked, embedded and trained on.
- Tell provenance, lineage and chain of custody apart, and use each to answer a real question.
- Apply a control at every stage of the AI data lifecycle, including deletion that reaches every copy.
Opening case: the source that turned out to be bad
Meridian's product team buys a dataset of 400,000 customer reviews of banking products from a data broker, to train a sentiment model and to feed a "what customers say" search feature. The data science team cleans it, labels a sample, trains sentiment-model 4.0, and builds a vector index of the reviews for retrieval. The sentiment scores are later used as an input to Meridian's product recommender.
Four months later, the broker writes to say that 12% of the reviews were scraped from a forum without permission, include customers' full names, and must be deleted. The same week, an analyst notices that some "reviews" contain the same odd sentence about a competitor, which looks like planted content.
The CISO asks three questions: Who decided we could use this data? Which of our systems has it reached? Can we prove which version of the data each model was trained on? The data science lead can answer none of them with confidence.
By the end of this section you will be able to answer all three, and to design the governance that would have made the answers available in minutes.
Who is accountable for AI data
Data governance starts by naming who is responsible for what. The classic roles carry over to AI [5][12]:
| Role | Responsibility | In the opening case |
|---|---|---|
| Data owner | A senior business person accountable for a dataset: its classification, who may use it, and for what, including whether it may train or ground an AI system | Head of Product, for the purchased review data |
| Data steward | Day-to-day quality, definitions and metadata | A product data analyst |
| Data custodian | Implements technical controls: storage, access, backup, encryption | The data platform team |
| Model owner | Accountable for a model, including knowing which data shaped it | The data science lead, for sentiment-model 4.0 |
The first question, "who decided we could use this data?", should have a one-line answer: the data owner approved use for training and retrieval, on a recorded date, after checking the license and the source. In the opening case nobody did, because buying data was treated as a procurement event rather than a data governance decision.
Controller and processor
Privacy law adds two roles. A controller decides why and how personal data is processed; a processor processes it on the controller's behalf [16]. When Meridian sends customer data to an AI provider's API, Meridian is normally the controller and the provider a processor, as long as the provider uses the data only to deliver the service.
If the provider also uses that data to train its own models for its own purposes, it starts acting as a controller for that processing. That requires its own lawful basis, takes the data out of Meridian's control, and changes the risk significantly. This is exactly why the contract clause "no training on customer data" (section 2.5) matters.
Residency, sovereignty and localization
These three terms are often confused [10]:
- Data residency is where data is physically stored and processed. It is usually a business or contractual choice: "process our data only in India".
- Data sovereignty is the principle that data is subject to the laws of the country where it is located (or whose citizens it concerns), including that country's rules on government access.
- Data localization is a legal requirement that certain data be stored, or processed, only within a country. The Reserve Bank of India, for example, requires payment system data to be stored only in India [35].
Worked example: three questions, three terms
| Question from Meridian's board | Term | Typical control |
|---|---|---|
| "Our India customers' card payment data is used in Sentinel's training. Can the training cluster be in Singapore?" | Localization (RBI rule) | No: payment data must be stored in India. Keep the pipeline and cluster in India |
| "If our AI provider's servers are in the US, can US authorities demand our customers' prompts?" | Sovereignty | Assess the provider's jurisdiction; consider regional processing, customer-held keys, or a provider subject to local law |
| "Can we contract for the provider to process EU customers' data only in the EU?" | Residency | Regional processing option plus a contractual residency commitment |
AI makes these questions sharper, because data moves in ways people do not expect. A prompt sent to a hosted model may be processed in another country; a model fine-tuned on local data may be hosted abroad; a vector index may be replicated globally for speed. Controls include providers with regional processing, contractual residency commitments, technical routing that keeps sensitive workloads in permitted regions, encryption with locally held keys, and recording processing locations in the AI inventory. Cross-border transfers of personal data must also meet transfer rules, such as an adequacy decision or standard contractual clauses under the GDPR [16].
Classification that survives AI pipelines
A classification scheme (for example public, internal, confidential, restricted) only works if its handling rules say what may happen to each class in AI:
| Class | Approved enterprise AI assistant | Retrieval index (RAG) | Fine-tuning | Logs |
|---|---|---|---|---|
| Public | Yes | Yes | Yes | Yes |
| Internal | Yes | Yes | With owner approval | Yes |
| Confidential | Only in Meridian-controlled AI | Yes, with permission-aware retrieval | With owner approval and a DPIA if personal data | Masked |
| Restricted (card data, health data, credentials) | No | No, without specific approval | No | Never |
The hard part is keeping labels attached. When documents are chunked, embedded and indexed, or when records are turned into training examples, labels and access rights are easily lost. Figure 3.1 shows the fix.
Two rules do most of the work:
- Propagate tags as metadata into every derived artifact. Each chunk in the vector store carries its source document's classification and access rights, so retrieval can filter on them (this is what makes permission-aware retrieval, from section 1.2, possible).
- Classify a derived model or index at the highest classification of any data it was built from. If one confidential document went into a fine-tuning set, the model is confidential.
For any new AI use involving sensitive data, a sensitive data impact assessment checks which classes of data will enter the system and whether the handling rules can actually be met.
Provenance, lineage and chain of custody
Three related ideas answer three different questions:
- Provenance answers where did this data come from? Its original source, who created it, under what license and consent, and when.
- Lineage answers what happened to it, and where did it go? Every transformation, merge, filter and labeling step, and every dataset, index and model it flowed into.
- Chain of custody answers can we prove nobody tampered with it? A documented, unbroken record of who handled the data and when, with integrity checks at each hand-off.
Worked example: lineage answers the CISO's second question
Figure 3.2 shows the lineage graph Meridian should have had for the review data.
Reading the graph from the contaminated source:
- the reviews flowed through the clean-and-label pipeline into sentiment-model 4.0;
- they were also embedded into the review vector index, which is rebuilt monthly;
- sentiment-model 4.0's scores are a feature in recommender 2.1;
- the recommender drives what customers see on the product site.
So the deletion request reaches four systems, not one: the raw store, the index (delete and rebuild), the sentiment model (retrain on a cleaned version or retire), and potentially the recommender (re-evaluate once its input feature changes). Without lineage, the team would delete the raw file and believe it was done.
Worked example: chain of custody answers the third question
Figure 3.3 shows a chain of custody for a dataset version, with a hash recorded at each hand-off.
The hash of the dataset should change only where a change is expected and re-approved (here, after labeling). If the hash at "train model" differs from the hash the data owner approved, someone changed the data in between. For the planted competitor sentence, the custody record lets Meridian determine whether the content arrived from the broker (a provenance problem) or was inserted inside Meridian's pipeline (an insider or pipeline compromise). Those lead to very different investigations.
Synthetic data needs the same treatment. Record which model and settings generated it and which real data seeded it, and label it as synthetic so it is never mistaken for real data later. This also guards against the model collapse described in section 1.3 [27].
Security lens. Lineage is an incident response tool, not just documentation. When a source turns out to be poisoned, unlicensed or subject to erasure, lineage is what tells you the blast radius. Ask any AI team: "If this dataset were bad, could you list every model and index it touched within an hour?"
Controls across the AI data lifecycle
Figure 3.4 shows the AI data lifecycle and a key control at each stage.
| Stage | Key control | What it would have caught in the opening case |
|---|---|---|
| Collection | Lawful basis, license check, minimization | Scraped forum content and names without permission |
| Validation | Anomaly and poisoning checks | Repeated planted sentence across many reviews |
| Transformation | Label quality checks, de-identification | Customer names carried into training text |
| Storage and use | Access control, encryption, versioning | Who could change the dataset; which version was used |
| Retention | Schedule and legal hold | Keeping purchased data beyond the license term |
| Archiving | Restricted access, integrity | Proving later what the model was trained on |
| Deletion | All copies, including derived artifacts, plus a decision on models | Index entries, model, and downstream recommender |
The most frequently tested point is the last row. When data must be deleted (retention has expired, consent was withdrawn, or someone exercised a right to erasure), deletion must propagate downstream to derived datasets, embeddings and caches, and a decision must be made about models that learned from the data. Without lineage, that is impossible.
Exam lens. If a question asks what makes it possible to honor a deletion request across AI systems, or to find every model affected by a poisoned source, the answer is lineage. If it asks how to prove a dataset was not altered, the answer is chain of custody with hashes. If it asks whether the organization had the right to use the data, the answer is provenance.
The opening case, resolved
- Who decided we could use this data? Nobody with authority. The data owner should have approved the use for training and retrieval after checking provenance, license and personal data. Future purchases of data go through that decision.
- Which systems has it reached? A lineage graph shows four: the raw store, the review vector index, sentiment-model 4.0, and recommender 2.1 through its sentiment feature. All four need action, not just the raw file.
- Can we prove which version each model was trained on? Only with chain of custody: dataset versions identified by hash, approved by the data owner, and recorded against each training run. That also tells Meridian whether the planted sentences came from the broker or were added in its own pipeline.
Check your understanding
Try each before reading the reasoning.
1. Meridian's AI provider updates its terms so that it may use customer prompts "to improve and train our models". What changes, and what should Meridian do?
Reasoning: for that training use, the provider becomes a controller of Meridian customers' personal data, needing its own lawful basis and taking the data outside Meridian's control. Meridian should opt out or negotiate a no-training clause, reassess the DPIA, and if neither is possible, consider whether the service can still be used for personal data.
2. A document classified "confidential" is split into 40 chunks and embedded into a shared vector index. What should happen to its classification?
Reasoning: every chunk should carry the document's classification and access rights as metadata, so retrieval can filter on them, and the index as a whole is classified at the highest classification of its content. Losing the label at chunking is how oversharing RAG systems are built.
3. An auditor asks Meridian to prove that the dataset used to train its credit model was not altered between approval and training. What evidence answers this?
Reasoning: a chain of custody record with the approved dataset version's hash, the approver, and the hash verified by the training job, showing they match. Lineage alone shows where data went; custody with hashes shows it was not changed.
Key takeaways
- The data owner decides whether data may be used for AI training or retrieval. Buying data is a governance decision, not just a purchase.
- An AI provider that trains on your data for its own purposes becomes a controller of it.
- Residency is where data is processed, sovereignty is whose law applies, localization is a legal requirement to keep data in-country.
- Classification must travel with data into chunks, embeddings and models. A derived artifact takes the highest classification of its inputs.
- Provenance: where it came from. Lineage: where it went. Chain of custody: proof it wasn't changed.
- Deletion must reach every copy and derived artifact, with a decision on models. Lineage makes that possible.