Privacy by design and privacy-enhancing techniques
In this lesson you will:
- Apply privacy-by-design principles to AI systems
- Compare tokenisation, masking, pseudonymisation, anonymisation, noise injection and differential privacy
- Choose suitable techniques for training, retrieval and logging
- Relate GDPR, HIPAA and PCI DSS requirements to AI data
Privacy by design means building privacy into the system from the start rather than adding it after a complaint. For AI, the earliest decisions — what data to collect, what to train on, what to log — matter most, because data that enters a model is hard to take out.
1. Privacy-by-design principles for AI
| Principle | AI application |
|---|---|
| Lawful basis and consent | Identify a lawful basis for each purpose — training, evaluation, inference, monitoring. Consent for one is not consent for all. |
| Purpose limitation | Data collected to deliver a service isn't automatically available to train models. Assess compatibility or obtain a new basis. |
| Data minimisation | Train and retrieve only what the task needs. Remove direct identifiers and unnecessary fields before data enters pipelines. |
| Accuracy | Poor-quality personal data produces wrong decisions about real people. |
| Storage limitation | Retention schedules for datasets, logs and artefacts (Lesson 4.2). |
| Security | Appropriate technical and organisational measures (Lesson 4.6). |
| Transparency and rights | Tell people how AI uses their data; support access, correction, erasure and objection. |
| Default privacy | Most private settings by default — for example, no training on user content unless switched on. |
2. Privacy-enhancing techniques
| Technique | How it works | Reversible? | Still personal data? | Good for |
|---|---|---|---|---|
| Masking | Hide part of a value (XXXX-XXXX-1234) |
Usually no (static masking) | Often yes, with context | Displays, logs, test data |
| Tokenisation | Replace a value with a random token; mapping kept in a secure vault | Yes, via the vault | Yes (pseudonymised) | Payment data, identifiers in pipelines |
| Pseudonymisation | Replace identifiers with consistent pseudonyms; key kept separately | Yes, with the key | Yes under GDPR | Analytics and training where records must link |
| Generalisation / suppression | Coarsen values (age → age band) or remove rare ones | No | Depends on result | Releasing datasets (k-anonymity) |
| Anonymisation | Irreversibly prevent identification by any reasonably likely means | No | No, if truly achieved | Open data, sharing — hard to achieve |
| Noise injection | Add random perturbation to values | No | Depends | Statistical outputs, robustness |
| Differential privacy (DP) | Add calibrated noise so any one person's presence has a provably bounded effect | No | Reduces risk with a measurable guarantee | Training (DP-SGD), aggregate statistics |
| Synthetic data | Generate artificial records with similar statistics | No | Can leak if overfitted | Testing, augmenting rare cases |
| Federated learning | Train across devices or sites; share model updates, not raw data | — | Updates can still leak | Cross-hospital or on-device training |
Differential privacy in one paragraph
DP gives a mathematical bound, set by the privacy budget ε (epsilon), on how much any single individual's data can change the output. Smaller ε means stronger privacy and usually lower accuracy. DP-SGD applies this to model training by clipping each example's influence and adding noise to gradients, which limits memorisation and membership inference. Choosing ε is a risk decision; document it.
Why "anonymised" often isn't
Removing names and IDs leaves quasi-identifiers — postcode, birth date, gender, job title — that can re-identify people when combined with other data. Free text is especially hard: a support transcript may contain names, account numbers and addresses anywhere. Treat pseudonymised data as personal data, and claim anonymisation only after a documented re-identification risk assessment.
3. Matching techniques to AI uses
- Training data — minimise, pseudonymise or tokenise identifiers, redact free-text PII, consider DP-SGD for sensitive data.
- Retrieval sources — keep personal data in access-controlled stores; filter by user entitlement; redact fields the assistant doesn't need.
- Prompts in transit to a provider — detect and redact or tokenise identifiers before sending; re-insert on return if needed.
- Logs — mask or tokenise; minimise retention.
- Test environments — synthetic or masked data, never raw production personal data.
4. Global regulations that shape AI data handling
| Regulation | Scope | What it means for AI data |
|---|---|---|
| GDPR (EU) | Personal data of people in the EU | Lawful basis, minimisation, DPIAs, rights incl. erasure and Art. 22, international transfer rules |
| DPDP Act (India) | Digital personal data processed in India, and outside India for offering goods/services there | Consent or legitimate use, notice, safeguards, breach reporting, erasure, children's data |
| HIPAA (US) | Protected health information (PHI) held by covered entities and business associates | Business associate agreements with AI vendors, minimum necessary, de-identification standards (Safe Harbor or Expert Determination), security rule |
| PCI DSS (global card industry) | Cardholder data and sensitive authentication data | Keep card data out of prompts, logs and training sets; anything that stores, processes or transmits it is in scope |
Worked example — a privacy gateway in front of a model API
A hospital group wants clinicians to summarise discharge notes with a hosted LLM.
| Step | Control |
|---|---|
| Before sending | Gateway detects names, patient IDs, phone numbers and addresses; replaces them with tokens ([PATIENT_1]) and keeps the mapping locally |
| Contract | Business associate agreement / data-processing terms; zero data retention; no training on inputs |
| Model call | Only tokenised text leaves the hospital network |
| On return | Gateway re-inserts real values for the clinician's view |
| Logging | Logs store tokenised text only, retained 30 days |
| Residual risk | Rare identifiers in free text may slip through — clinicians are trained not to paste unnecessary detail; detector accuracy is monitored |
🔐 Security lens
A tokenisation vault or pseudonymisation key turns reversible data back into personal data. Protect it like a crown-jewel secret: separate storage, strict access, logging and rotation.
Key takeaways
- Privacy by design for AI: lawful basis per purpose, purpose limitation, minimisation, defaults and rights — decided before data enters a model.
- Masking, tokenisation and pseudonymisation are reversible or contextual; pseudonymised data is still personal data.
- True anonymisation is hard; quasi-identifiers and free text re-identify people.
- Differential privacy gives a measurable bound (ε); DP-SGD limits memorisation in training.
- GDPR, DPDP, HIPAA and PCI DSS all reach into prompts, logs and training sets.
Self-check
- Is pseudonymised training data personal data under GDPR? Why?
- What does a smaller epsilon mean in differential privacy?
- Why should cardholder data never appear in prompts or AI logs?