Data Governance, Privacy & Protection for AI, 30 min read

AI-specific privacy and IP: erasure, derivation, contamination and consent

In this lesson you will:

  • Identify regulated and sensitive data types that need special handling in AI
  • Explain the right to erasure for trained models and the options for honouring it
  • Describe data derivation and regulatory contamination risks
  • Assess IP, copyright and licensing issues in training data
  • Apply consent mechanisms for data collection and public-data use

Some privacy problems only appear once data meets a model. A database row can be deleted; what a model learned from it cannot simply be "un-learned". This lesson covers the issues that make AI data different.

1. Regulated and sensitive data types

Type Examples Typical constraint
PII / personal data Names, contact details, identifiers, online identifiers Lawful basis, minimisation, rights
Sensitive / special-category data Health, biometrics, religion, sexual orientation, political opinions Stricter conditions; often explicit consent or specific legal grounds
PHI Medical records, diagnoses linked to individuals HIPAA and health-sector rules
Financial data Account numbers, card data, credit history PCI DSS, banking secrecy and sector rules
Children's data Data of minors Parental consent (e.g. verifiable parental consent under DPDP)
Export-controlled data Technical data on controlled technologies Export-control law restricts who and where it can be processed — including by an AI provider abroad
Confidential business data Trade secrets, M&A plans, source code Contractual and IP protection

2. Erasure and the trained model

When someone exercises a right to erasure (GDPR Article 17; erasure rights under the DPDP Act), deleting their records from source systems is straightforward. But a model trained on those records may still reflect them, and in some cases can reproduce them.

Options for honouring erasure in AI systems, from easiest to hardest.
Figure 4.7 Options for honouring erasure in AI systems. Architecture choices made early decide which options are available later.
Option How Notes
Don't train on it Keep personal data in retrieval stores, not weights Easiest erasure: delete the record and re-index
Remove from future training Delete from datasets; next scheduled retrain excludes it Model still reflects data until retrained
Retrain from scratch Retrain without the data Most complete; expensive for large models
Sharded / modular training Train sub-models on data shards (e.g. SISA approach) so only one shard is retrained Must be designed in from the start
Machine unlearning Algorithms that approximately remove a record's influence Active research; verification is hard
Output suppression Filter outputs that reproduce the person's data A mitigation, not erasure

Design implication: prefer RAG over fine-tuning for personal data that may need to be erased, and keep training datasets versioned so you can rebuild without specific records.

3. Data derivation and regulatory contamination

Derived data is anything computed from source data: features, embeddings, summaries, labels, model weights. It frequently inherits the source's sensitivity and legal restrictions.

Regulatory contamination occurs when regulated data enters a pipeline and brings the whole pipeline — and its outputs — into a regulatory regime.

Regulated data contaminating an AI pipeline and pulling it into scope.
Figure 4.8 Regulatory contamination: a small amount of regulated data can pull an entire pipeline, its stores and its model into scope.

Examples:

  • Card numbers pasted into support chats flow into transcripts used for fine-tuning — now the fine-tuning pipeline, storage and possibly the model are in PCI DSS scope.
  • Export-controlled engineering documents are indexed into a global vector store accessible to staff in restricted countries.
  • Health data mixed into a general analytics dataset brings HIPAA-style obligations to every downstream user.

Prevent contamination with intake filtering (detect and block regulated data before it enters pipelines), segregated pipelines for regulated data, tagging and lineage so contamination can be traced, and contracts that keep provider-side copies out of scope.

  • Copyright — training on copyrighted works raises unresolved legal questions in many jurisdictions; litigation is ongoing and outcomes differ by country. Text-and-data-mining exceptions exist in some places (for example the EU, with an opt-out for rights holders), but not uniformly.
  • Licences — datasets and models come with licences that can restrict commercial use, require attribution, or prohibit certain fields of use. Mixing differently licensed sources can create conflicts.
  • Terms of service — scraping websites can breach their terms even where copyright allows it.
  • Outputs — models can reproduce training content verbatim; output filtering and similarity checks reduce infringement risk.
  • Your own IP — prompts and documents sent to a vendor may be used for training unless contracts prevent it.

Controls: a licence register for datasets and models (part of the AIBOM); legal review of sources; honouring machine-readable opt-outs; vendor IP indemnities; output similarity checks for generative use.

  • Consent must be specific, informed and freely given. Bundling "we may use your data to improve our AI" into terms of service is weak consent — and under GDPR may not be valid for that purpose.
  • Granular choices — separate opt-ins for service delivery, analytics and model training.
  • Withdrawal — must be as easy as giving consent, and must flow through to future training sets.
  • "Publicly available" is not "free to use". Personal data posted publicly is still personal data; scraping it for training needs a lawful basis, must respect purpose limits and platform terms, and may conflict with the expectations of the people involved.
  • Consent managers — under India's DPDP framework, registered consent managers let individuals give, manage and withdraw consent across services.

Worked example — an erasure request against a fine-tuned support model

A customer asks a telecom company to erase all their data. Their chat transcripts were part of a dataset used to fine-tune the support assistant six months ago.

Action Detail
Delete source records CRM, transcript store, backups on rotation
Re-index retrieval Their account notes removed from the vector index within 24 hours
Training data Transcripts removed from the versioned fine-tuning dataset
The model Next quarterly retrain excludes the data; meanwhile, an output filter blocks the customer's name, number and address
Record Erasure log entry noting the model will be retrained on (date), for the privacy office and auditors
Lesson for design New personal-data use cases move to retrieval rather than fine-tuning

🔐 Security lens

Memorisation is both a privacy issue and an attack surface: extraction attacks (Module 7) try to make models regurgitate training data. Erasure design and extraction defences reinforce each other.

Key takeaways

  • Know the regulated categories — PII, special-category, PHI, financial, children's and export-controlled data — and their extra constraints.
  • Erasure in models ranges from easy (RAG) to hard (unlearning); design for it before training.
  • Derived artefacts inherit sensitivity; regulated data contaminates pipelines unless filtered and segregated.
  • Training-data IP is legally unsettled; keep a licence register, honour opt-outs and filter outputs.
  • Public data is still personal data; consent must be specific, granular and withdrawable.

Self-check

  1. Why is RAG preferable to fine-tuning for data that may be subject to erasure requests?
  2. Give an example of regulatory contamination in an AI pipeline.
  3. Why is "it was publicly available" not enough justification to train on personal data?