2.2 Responsible AI principles and ethics
In this lesson you will:
- Turn a responsible AI principle into a requirement, a control and the evidence that proves it.
- Say where bias enters the AI lifecycle, and why removing a protected attribute does not remove bias.
- Calculate an impact ratio, and explain the difference between demographic parity, equal opportunity and equalized odds.
- Tell transparency, explainability and interpretability apart, and choose an explanation for each audience.
- Explain how detailed explanations can help an attacker, and how to limit that.
- Apply the ISC2 Code of Ethics to an AI dilemma.
Opening case: the fair model that wasn't
After the audit in section 2.1, Meridian's AI governance committee asks for TalentMatch to be tested before it is used again. The vendor sends a one-page certificate: "TalentMatch is fair. Gender and age are not used as inputs."
Meridian's data scientists run their own test on last year's 2,000 Frankfurt applicants. Of 1,200 men, TalentMatch shortlisted 720. Of 800 women, it shortlisted 360. Digging further, they find that the model gives a strong negative weight to "career gap of more than 12 months" and a positive weight to membership of certain university sports clubs.
At the same meeting, the head of HR says the tool has saved hundreds of hours and asks the CISO to "just sign off the security review" so it can be switched back on before the hiring season. The CISO's own team has found no security issues.
By the end of this section you will be able to explain why "gender is not an input" does not make a model fair, what the test results mean, which fairness measure the committee should choose, and what the CISO should do under the ISC2 Code of Ethics.
From principles to evidence
Almost every AI framework lists similar principles. The OECD AI Principles, adopted in 2019 and updated in 2024, call for AI that is inclusive and beneficial; respects human rights and democratic values, including fairness and privacy; is transparent and explainable; is robust, secure and safe; and is accountable [19]. UNESCO's Recommendation on the Ethics of AI and the trustworthy characteristics in the NIST AI RMF say much the same [10][20]. Figure 2.8 shows six common principles with a control for each.
Principles on their own change nothing. The skill the exam tests, and the skill that matters in practice, is turning a principle into something that can be checked [1][4]. The chain is always the same: principle → requirement → control → evidence.
Worked example: TalentMatch, principle by principle
| Principle | Requirement for TalentMatch | Control | Evidence an auditor can see |
|---|---|---|---|
| Fairness | Shortlisting rates must not differ unjustifiably by gender or age | Bias test before release; monthly monitoring by group | Test report with impact ratios; dashboard with alert thresholds |
| Transparency | Candidates are told AI is used and can ask how | Notice in the job advert and application form | Screenshot of notice; log of candidate requests |
| Accountability | A named owner answers for outcomes | Inventory entry with business owner | Inventory record; committee approval minutes |
| Human oversight | No candidate is rejected by the AI alone | Recruiter reviews every rejection, with authority to override | Override log; workload records |
| Privacy | Only necessary CV data is processed | DPIA; data minimization review | Completed DPIA; data-flow diagram |
| Reliability and safety | Rankings stay valid over time | Quarterly revalidation against hiring outcomes | Validation report |
Notice the right-hand column. "The vendor says it's fair" is a claim, not evidence. An auditor, a regulator or a tribunal will ask to see the test, the thresholds, the monitoring and what happened when an alert fired.
Algorithmic bias: where it comes from
Algorithmic bias is a systematic error that produces unfair outcomes for certain groups. It rarely comes from malicious intent; it comes from data and design choices made at every stage of the lifecycle, as Figure 2.9 shows.
- Historical bias: the data faithfully reflects past discrimination. A model trained on ten years of hiring decisions learns ten years of hiring preferences.
- Representation (sampling) bias: some groups are under-represented in the training data, so the model performs worse for them.
- Measurement bias: the label measures the wrong thing, or measures it differently across groups. "Was promoted within three years" as a label for "good hire" builds in whatever biases affected promotions.
- Proxy variables: removing a protected attribute does not remove bias if other features stand in for it. Postcode can stand in for ethnicity; first name for gender; graduation year for age.
- Aggregation and deployment bias: one model used for groups that behave differently, or a model used in a context it was not designed for.
- Feedback loops: today's decisions become tomorrow's training data. If the model shortlists fewer women, fewer women are hired, and next year's training data contains even fewer successful women.
Worked example: how "gender is not an input" fails
TalentMatch does not see gender. But it penalizes career gaps of more than 12 months, and in Meridian's applicant pool, such gaps are far more common among women (most often parental leave). It rewards membership of certain sports clubs that historically admitted mostly men. These two features act as proxies: the model rediscovers gender from correlated signals and reproduces the historical pattern.
This is why a certificate stating which inputs are excluded proves nothing. Fairness is a property of outcomes, so it must be measured on outcomes.
Measuring fairness
There are several mathematical definitions of fairness, and they cannot all be satisfied at once except in special cases. Three are worth knowing for the exam.
Demographic parity asks whether each group receives the favorable outcome at a similar rate. It is commonly measured with the impact ratio (or disparate impact ratio): the selection rate of the less-favored group divided by that of the most-favored group. A widely used rule of thumb from US employment guidance, the four-fifths rule, flags possible adverse impact when the ratio is below 0.8 [21].
Equal opportunity asks whether, among people who are truly qualified, each group has the same chance of the favorable outcome. In other words, equal true positive rates (recall, from section 1.3) across groups [28].
Equalized odds asks for equal true positive rates and equal false positive rates across groups.
Worked example: TalentMatch's impact ratio
From the opening case:
- Men: 720 shortlisted of 1,200 = 60%.
- Women: 360 shortlisted of 800 = 45%.
- Impact ratio = 45% ÷ 60% = 0.75.
Figure 2.10 shows the same calculation.
0.75 is below the 0.8 rule of thumb, so the result requires investigation. It does not by itself prove unlawful discrimination: the committee must ask whether the difference is explained by legitimate, job-related factors. The data scientists' finding that the gap is driven by career gaps and club memberships, neither of which is a genuine job requirement, makes the case for unfairness much stronger.
Now apply equal opportunity. Suppose Meridian has a set of past applicants whose actual job performance is known, and looks only at those who turned out to be strong performers:
| Strong performers | Shortlisted by TalentMatch | True positive rate |
|---|---|---|
| Men: 300 | 255 | 0.85 |
| Women: 200 | 130 | 0.65 |
Among people who were genuinely good hires, TalentMatch shortlisted 85% of men but only 65% of women. That is an equal opportunity failure: qualified women are being missed. For a hiring tool, this is arguably the more important measure, because it shows the model is not just producing different rates but making more mistakes about one group.
Choosing and fixing
Which definition applies is a policy decision, depending on context and law. It should be made by the governance committee with legal advice, documented, and then tested against. Mitigation can happen at three points:
- Before training: rebalance or reweight the data, remove or transform proxy features (here, stop penalizing career gaps).
- During training: add fairness constraints to the learning objective.
- After training: adjust decision thresholds, where lawful (in some jurisdictions, using different thresholds by protected group is itself unlawful).
And always with ongoing monitoring, because bias can return through drift and feedback loops.
Exam lens. "Remove the protected attribute" is a classic wrong answer to a fairness question. Look for answers that test outcomes by group, investigate proxies, and monitor after deployment.
Transparency, explainability and interpretability
Three related terms are easy to confuse:
- Transparency is openness about the system: that AI is used, what it is for, what data it uses, and its limitations. A notice in a job advert is transparency.
- Explainability is the ability to give a human-understandable reason for a particular output: "your application was not shortlisted mainly because the role requires five years of risk experience and your CV shows two."
- Interpretability is how far a person can understand how the model works internally. A small decision tree is interpretable; a large neural network is not.
When a model is not interpretable, post-hoc explanation methods estimate why it produced an output. Feature-importance methods such as SHAP and LIME estimate how much each input contributed; they can be global (what matters across all decisions) or local (why this one decision). Counterfactual explanations state what would have had to change for a different result. Figure 2.11 shows all three for a declined loan application.
Worked example: one decision, three audiences
A Meridian customer is declined for a personal loan by an AI-assisted credit model. Three people want an explanation:
| Audience | What they need | Appropriate explanation |
|---|---|---|
| The customer | Why, and what they could do about it | The main reasons in plain language, and a counterfactual: "approved if your debt-to-income ratio were below 45%" |
| The model validator | Whether the model behaves sensibly overall | Global feature importance, local explanations for samples, stability across groups |
| The regulator | Whether the bank meets its legal duties | Documentation of the model, its testing, its explanation method and its oversight process |
There are legal duties to explain. The GDPR requires meaningful information about the logic involved in solely automated decisions [16]. The EU AI Act requires providers to give deployers enough transparency to use high-risk systems properly, and gives affected people a right to an explanation of certain high-risk decisions [14]. Consumer credit rules in many countries require reasons for adverse decisions.
Explanations as an attack surface
Explanations also create security risks. Detailed explanations and confidence scores help attackers to:
- reverse-engineer the model (model extraction), by learning exactly how each input moves the score;
- find the smallest change that flips a decision (evasion): a fraudster who learns that "debt-to-income below 45%" approves the loan knows exactly what to falsify;
- infer whether a person's data was in the training set (membership inference), from unusually confident scores.
The control is to tailor explanations: give each audience the reasons they are entitled to, and no more. Customers get the main reasons and a counterfactual; they do not get raw scores, full feature weights or an unlimited API to query "what if". Detailed explanations go to validators and regulators through controlled channels.
Security lens. Whenever a design exposes scores, probabilities or detailed explanations through a public interface, ask what an attacker could learn by querying it a thousand times. Rate limits, coarse outputs and monitoring for systematic probing are the usual answers.
Systemic and environmental impact
Responsible AI also considers effects beyond a single decision: job displacement, concentration of power in a few providers, misinformation at scale, and over-reliance on AI that people stop questioning. Training and running large models consumes significant electricity and water for cooling. An organization can reduce its footprint by choosing smaller or more efficient models where they are good enough, reusing and fine-tuning existing models instead of training from scratch, and measuring energy use. The exam may present these as trade-offs to weigh, not problems with a single correct answer.
The ISC2 Code of Ethics
ISC2-certified professionals commit to the ISC2 Code of Ethics [9]. Its preamble calls for the safety and welfare of society and adherence to the highest ethical standards. Its four mandatory canons are applied in order of priority:
- Protect society, the common good, necessary public trust and confidence, and the infrastructure.
- Act honorably, honestly, justly, responsibly, and legally.
- Provide diligent and competent service to principals.
- Advance and protect the profession.
When canons conflict, the earlier one prevails. In 2026 ISC2 also published a Code of Professional Conduct that builds on the canons with practical guidance for the whole profession; it expands on the Code of Ethics rather than replacing it [9].
Worked example: the CISO's dilemma
The head of HR asks the CISO to "just sign off the security review" for TalentMatch. The CISO's team found no security issues, so in a narrow sense signing is accurate. But the CISO knows the tool shows a 0.75 impact ratio driven by proxies, and that it is about to be used to reject candidates.
| Option | Assessment against the canons |
|---|---|
| Sign the security review and say nothing else | Narrowly honest, but lets a known likely-unlawful harm proceed; conflicts with Canons 1 and 2 |
| Refuse to sign anything until the tool is fixed | Overreaches: the security review is accurate, and fairness approval is not the CISO's decision |
| Sign the security review, and formally record the fairness finding to the AI governance committee, recommending the tool stay off until resolved | Consistent with all four canons: accurate on security, honest and just about the known harm, using proper channels |
| Leak the test results to the press | Fails Canon 2 (responsibly, legally) and Canon 3, and skips the proper channels that exist |
The third option is the best answer, and it shows a pattern the exam uses repeatedly: raise the issue through proper channels, document it, and do not sign off on something you know to be harmful, rather than either staying silent or going public as a first step.
The opening case, resolved
- Why "gender is not an input" does not make TalentMatch fair: career-gap length and certain club memberships act as proxies for gender, so the model rediscovers it. Fairness is a property of outcomes, and must be measured on outcomes.
- What the results mean: an impact ratio of 0.75 is below the 0.8 rule of thumb and requires investigation. The drivers are not job-related, and the equal-opportunity test shows qualified women are shortlisted less often (0.65 against 0.85). The case for unfair impact is strong.
- Which measure to choose: a committee decision with legal advice. For hiring, equal opportunity (are qualified people treated alike?) is the most defensible primary measure, with the impact ratio monitored as a screening check.
- What should happen next: keep TalentMatch off for automated decisions; require the vendor to remove the career-gap penalty and the proxy features, then re-test; add candidate transparency and human review of every rejection; set up monthly monitoring by group.
- What the CISO should do: sign only what is true (the security review) and formally report the fairness finding to the committee, recommending the tool stay off until it is resolved.
Check your understanding
Try each before reading the reasoning.
1. A loan model approves 66% of applicants aged under 40 and 50% of applicants aged 60 and over. What is the impact ratio, and what should happen next?
Reasoning: 50% ÷ 66% ≈ 0.76, below 0.8, so it needs investigation. Check whether legitimate, documented credit factors explain the gap, look for proxies for age, compare true positive rates for applicants who did repay (equal opportunity), and record the decision and any mitigation.
2. A public API returns the probability that a transaction is fraudulent, to six decimal places, plus the contribution of each input. What is the main security concern, and what is the fix?
Reasoning: precise scores and per-feature contributions let an attacker probe the model to learn exactly which changes make fraud pass (evasion) and to approximate the model (extraction). Return only a coarse decision or band, keep detailed explanations for internal and regulatory use, and rate-limit and monitor for systematic probing.
3. A model must be interpretable to satisfy a regulator. The data science team proposes a deep neural network with SHAP explanations. Is that interpretable?
Reasoning: no. SHAP gives post-hoc explanations of an uninterpretable model; it does not make the model itself interpretable. If true interpretability is required, consider an inherently interpretable model (such as a scorecard or small decision tree), and accept any trade-off in accuracy as a documented decision.
Key takeaways
- Principles become real only through the chain principle → requirement → control → evidence. Vendor claims are not evidence.
- Bias enters at every lifecycle stage. Removing a protected attribute does not remove bias, because proxies reintroduce it; feedback loops amplify it.
- Impact ratio = less-favored rate ÷ most-favored rate; below 0.8 needs investigation. Equal opportunity compares true positive rates; equalized odds adds false positive rates. Not all can hold at once, so the choice is a documented policy decision.
- Transparency is about the system, explainability about a decision, interpretability about the model's internals. Tailor explanations to the audience.
- Detailed explanations and scores help attackers. Give people what they are entitled to, not raw internals.
- The ISC2 canons are applied in order. Raise known harm through proper channels; neither stay silent nor go public first.