Data Governance, Privacy & Protection for AI, 30 min read

Privacy by design and privacy-enhancing techniques

In this lesson you will:

  • Apply privacy-by-design principles to AI systems
  • Compare tokenisation, masking, pseudonymisation, anonymisation, noise injection and differential privacy
  • Choose suitable techniques for training, retrieval and logging
  • Relate GDPR, HIPAA and PCI DSS requirements to AI data

Privacy by design means building privacy into the system from the start rather than adding it after a complaint. For AI, the earliest decisions — what data to collect, what to train on, what to log — matter most, because data that enters a model is hard to take out.

1. Privacy-by-design principles for AI

Principle AI application
Lawful basis and consent Identify a lawful basis for each purpose — training, evaluation, inference, monitoring. Consent for one is not consent for all.
Purpose limitation Data collected to deliver a service isn't automatically available to train models. Assess compatibility or obtain a new basis.
Data minimisation Train and retrieve only what the task needs. Remove direct identifiers and unnecessary fields before data enters pipelines.
Accuracy Poor-quality personal data produces wrong decisions about real people.
Storage limitation Retention schedules for datasets, logs and artefacts (Lesson 4.2).
Security Appropriate technical and organisational measures (Lesson 4.6).
Transparency and rights Tell people how AI uses their data; support access, correction, erasure and objection.
Default privacy Most private settings by default — for example, no training on user content unless switched on.

2. Privacy-enhancing techniques

Privacy-enhancing techniques arranged from reversible to irreversible.
Figure 4.5 Privacy-enhancing techniques on a spectrum from reversible (still personal data) to irreversible (potentially outside data-protection law) — with utility usually falling as protection rises.
Technique How it works Reversible? Still personal data? Good for
Masking Hide part of a value (XXXX-XXXX-1234) Usually no (static masking) Often yes, with context Displays, logs, test data
Tokenisation Replace a value with a random token; mapping kept in a secure vault Yes, via the vault Yes (pseudonymised) Payment data, identifiers in pipelines
Pseudonymisation Replace identifiers with consistent pseudonyms; key kept separately Yes, with the key Yes under GDPR Analytics and training where records must link
Generalisation / suppression Coarsen values (age → age band) or remove rare ones No Depends on result Releasing datasets (k-anonymity)
Anonymisation Irreversibly prevent identification by any reasonably likely means No No, if truly achieved Open data, sharing — hard to achieve
Noise injection Add random perturbation to values No Depends Statistical outputs, robustness
Differential privacy (DP) Add calibrated noise so any one person's presence has a provably bounded effect No Reduces risk with a measurable guarantee Training (DP-SGD), aggregate statistics
Synthetic data Generate artificial records with similar statistics No Can leak if overfitted Testing, augmenting rare cases
Federated learning Train across devices or sites; share model updates, not raw data — Updates can still leak Cross-hospital or on-device training

Differential privacy in one paragraph

DP gives a mathematical bound, set by the privacy budget ε (epsilon), on how much any single individual's data can change the output. Smaller ε means stronger privacy and usually lower accuracy. DP-SGD applies this to model training by clipping each example's influence and adding noise to gradients, which limits memorisation and membership inference. Choosing ε is a risk decision; document it.

Why "anonymised" often isn't

Removing names and IDs leaves quasi-identifiers — postcode, birth date, gender, job title — that can re-identify people when combined with other data. Free text is especially hard: a support transcript may contain names, account numbers and addresses anywhere. Treat pseudonymised data as personal data, and claim anonymisation only after a documented re-identification risk assessment.

3. Matching techniques to AI uses

Which privacy techniques to apply at training, retrieval, inference and logging.
Figure 4.6 Applying privacy techniques where they fit best in an AI system.
  • Training data — minimise, pseudonymise or tokenise identifiers, redact free-text PII, consider DP-SGD for sensitive data.
  • Retrieval sources — keep personal data in access-controlled stores; filter by user entitlement; redact fields the assistant doesn't need.
  • Prompts in transit to a provider — detect and redact or tokenise identifiers before sending; re-insert on return if needed.
  • Logs — mask or tokenise; minimise retention.
  • Test environments — synthetic or masked data, never raw production personal data.

4. Global regulations that shape AI data handling

Regulation Scope What it means for AI data
GDPR (EU) Personal data of people in the EU Lawful basis, minimisation, DPIAs, rights incl. erasure and Art. 22, international transfer rules
DPDP Act (India) Digital personal data processed in India, and outside India for offering goods/services there Consent or legitimate use, notice, safeguards, breach reporting, erasure, children's data
HIPAA (US) Protected health information (PHI) held by covered entities and business associates Business associate agreements with AI vendors, minimum necessary, de-identification standards (Safe Harbor or Expert Determination), security rule
PCI DSS (global card industry) Cardholder data and sensitive authentication data Keep card data out of prompts, logs and training sets; anything that stores, processes or transmits it is in scope

Worked example — a privacy gateway in front of a model API

A hospital group wants clinicians to summarise discharge notes with a hosted LLM.

Step Control
Before sending Gateway detects names, patient IDs, phone numbers and addresses; replaces them with tokens ([PATIENT_1]) and keeps the mapping locally
Contract Business associate agreement / data-processing terms; zero data retention; no training on inputs
Model call Only tokenised text leaves the hospital network
On return Gateway re-inserts real values for the clinician's view
Logging Logs store tokenised text only, retained 30 days
Residual risk Rare identifiers in free text may slip through — clinicians are trained not to paste unnecessary detail; detector accuracy is monitored

🔐 Security lens

A tokenisation vault or pseudonymisation key turns reversible data back into personal data. Protect it like a crown-jewel secret: separate storage, strict access, logging and rotation.

Key takeaways

  • Privacy by design for AI: lawful basis per purpose, purpose limitation, minimisation, defaults and rights — decided before data enters a model.
  • Masking, tokenisation and pseudonymisation are reversible or contextual; pseudonymised data is still personal data.
  • True anonymisation is hard; quasi-identifiers and free text re-identify people.
  • Differential privacy gives a measurable bound (ε); DP-SGD limits memorisation in training.
  • GDPR, DPDP, HIPAA and PCI DSS all reach into prompts, logs and training sets.

Self-check

  1. Is pseudonymised training data personal data under GDPR? Why?
  2. What does a smaller epsilon mean in differential privacy?
  3. Why should cardholder data never appear in prompts or AI logs?