Fairness in Explainable AI Narratives: Decomposing Social Disparities in LLM-Generated Explanations
Organizations: Department of Engineering Management, University of Antwerp, Belgium
Abstract
Large language models (LLMs) are increasingly used to translate technical explainable AI (XAI) artefacts such as SHAP into explanatory narratives (XAINs) for lay users. In high-risk domains, these explanations are often the main resource through which individuals can understand and contest adverse decisions. Yet LLMs are known to reproduce social bias, even from proxy variables alone, and fairness in XAINs remains unstudied. Because XAINs are bound to a ground-truth artefact, fairness concerns not only what a narrative says and how, but also whether it is equally faithful across social groups. We introduce fairness in XAINs as a new field of research and propose a framework that generates narratives under three conditions (protected attribute disclosed, withheld, or counterfactually swapped), allowing five comparisons that pinpoint the cause of disparities. We apply it to adversely classified instances from four datasets in credit scoring, education and employment, generating narratives with five LLMs, and evaluate faithfulness, feature coverage and sentiment. We find significant disparities across all datasets, metric families and models, arising in two ways. Faithfulness disparities result from feature values: LLMs make plausible but false statements, triggered by values that are unequally distributed across social groups. Feature coverage and sentiment disparities stem from the protected attribute itself: once it is disclosed, LLMs give less explanation to, or frame more negatively the narratives of, groups that have historically been discriminated against or are stereotypically associated with worse outcomes. A more faithful model removes the faithfulness disparities, and withholding protected attributes mostly removes the others, but neither suffices on its own. Fairness in XAINs should therefore be audited separately from the predictive model and its XAI artefact.
Figures & tables
| German Credit | Saudi HR | Student | Law | All | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Model | VA | SA | RA | VA | SA | RA | VA | SA | RA | VA | SA | RA | VA | SA | RA |
| claude-sonnet-4-6 | 98.9 | 98.0 | 84.0 | 99.9 | 99.3 | 96.3 | 96.9 | 97.7 | 97.7 | 99.8 | 99.3 | 91.7 | 98.9 | 98.6 | 92.4 |
| gpt-4.1 | 100 | 92.7 | 93.0 | 100 | 98.0 | 81.0 | 99.5 | 91.7 | 93.7 | 99.7 | 95.3 | 94.7 | 99.8 | 94.4 | 90.6 |
| mistral-large-2512 | 90.0 | 81.0 | 72.3 | 98.0 | 92.3 | 85.0 | 95.3 | 88.0 | 83.3 | 98.2 | 97.3 | 81.3 | 95.0 | 89.7 | 80.5 |
| gpt-4o-mini | 96.1 | 60.5 | 74.0 | 94.6 | 64.0 | 87.3 | 93.0 | 71.0 | 57.3 | 99.5 | 89.2 | 71.3 | 95.5 | 71.2 | 72.5 |
| ministral-14b | 91.7 | 67.8 | 70.3 | 93.0 | 71.7 | 79.3 | 93.5 | 72.1 | 63.0 | 93.7 | 92.8 | 78.3 | 92.9 | 76.2 | 72.8 |
| German Credit | Saudi HR | Student | Law | All | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Model | C | PA | Sec. | C | PA | Sec. | C | PA | Sec. | C | PA | Sec. | C | PA | Sec. |
| claude-sonnet-4-6 | 99.7 | 0.0 | 8.4 | 100 | 1.0 | 5.3 | 100 | 1.0 | 4.4 | 100 | 25.0 | 20.3 | 99.9 | 6.8 | 9.6 |
| gpt-4.1 | 100 | 0.0 | 0.4 | 99.7 | 6.0 | 0.2 | 100 | 0.0 | 1.3 | 100 | 3.0 | 8.0 | 99.9 | 2.2 | 2.5 |
| mistral-large-2512 | 98.3 | 0.0 | 8.3 | 100 | 2.0 | 11.8 | 100 | 1.7 | 12.6 | 100 | 5.0 | 24.0 | 99.6 | 2.2 | 14.2 |
| gpt-4o-mini | 94.7 | 0.0 | 1.9 | 99.0 | 0.0 | 3.2 | 95.7 | 4.3 | 3.9 | 98.7 | 0.0 | 10.3 | 97.0 | 1.1 | 4.8 |
| ministral-14b | 91.0 | 1.3 | 10.2 | 99.7 | 0.0 | 9.8 | 93.7 | 3.0 | 7.5 | 93.0 | 1.0 | 23.3 | 94.3 | 1.3 | 12.7 |
| Model | German Credit | Saudi HR | Student | Law | All |
|---|---|---|---|---|---|
| claude-sonnet-4-6 | 0.32 | 0.55 | 0.40 | 0.37 | 0.41 |
| gpt-4.1 | 0.16 | 0.67 | 0.41 | 0.35 | 0.40 |
| mistral-large-2512 | 0.17 | 0.39 | 0.33 | 0.35 | 0.31 |
| gpt-4o-mini | 0.04 | 0.42 | 0.44 | 0.28 | 0.30 |
| ministral-14b | 0.10 | 0.42 | 0.37 | 0.28 | 0.29 |
Appendix figures & tables10 assets
Supplementary material from the paper’s appendix.
Appendix
| Disparity type | One batch per |
|---|---|
| D1, D2 (total, indirect) | dataset generator LLM protected attribute metric type |
| D3 (disclosure) | dataset generator LLM protected attribute social group metric type |
| D4, D5 (direct, value) | dataset generator LLM counterfactual swap metric type |
| Batch price ($/M tokens) | ||||||
|---|---|---|---|---|---|---|
| Model | Role | Provider | Weights | Region | Input | Output |
| claude-sonnet-4-6 | Generator | Anthropic | Closed | United States | 1.50 | 7.50 |
| gpt-4.1-2025-04-14 | Generator | OpenAI | Closed | United States | 1.00 | 4.00 |
| mistral-large-2512 | Generator | Mistral AI | Open | France | 0.25 | 0.75 |
| ministral-14b-2512 | Generator | Mistral AI | Open | France | 0.10 | 0.10 |
| gpt-4o-mini-2024-07-18 | Generator | OpenAI | Closed | United States | 0.075 | 0.30 |
| Metric | Type | Per feature (binary) | Per narrative (continuous) | Definition |
|---|---|---|---|---|
| Rank Accuracy (RA) | Faithfulness | rank_n_scoring | rank_position_accuracy | The feature presented at rank is the feature at SHAP rank . |
| Sign Accuracy (SA) | Faithfulness | rank_n_sign_scoring | sign_accuracy | The stated direction of influence of the SHAP rank- feature is correct, wherever the narrative places it among its three most important features. |
| Value Accuracy (VA) | Faithfulness | rank_n_value_scoring , <PA>scoring , other<feature>_value_scoring | value_accuracy | The stated value of a top-3 feature, protected attribute or secondary feature exactly matches the instance’s feature value. |
| Predicted-probability accuracy | Faithfulness | predicted_probability_scoring | – | The stated predicted probability lies within 0.01 of the model’s prediction. |
| Completeness | Coverage | rank_n_mentioned | – | The rank- SHAP feature is mentioned. |
| PA mention | Coverage | <PA>_mentioned | prop_pa_mentioned | The protected attribute is mentioned; per narrative, the share of protected attributes mentioned. |
| Dataset | Model | Groups | Metric | Values | Mechanism | ||
| D1 | German Credit | gpt-4o-mini | women vs men | RA (rank 1) | 65.0 vs 90.0 | 0.040 | prior: status first |
| D1 | Saudi HR | gpt-4o-mini | 21–30 vs 31+ | SA (rank 1) | 34.1 vs 67.3 | 0.024 | prior: experience |
| D1 | Saudi HR | gpt-4o-mini | women vs men | SA | 56.3 vs 72.7 | 0.022 | prior: experience |
| D1 | Saudi HR | gpt-4o-mini | women vs men | SA (rank 1) | 36.5 vs 70.2 | 0.019 | prior: experience |
| D2 | German Credit | gpt-4o-mini | 30 vs 30 | RA | 63.7 vs 85.6 | 0.010 | prior: status first |
| D2 | German Credit | gpt-4o-mini | 30 vs 30 | RA (rank 3) | 57.1 vs 86.4 | 0.029 | prior: status first |
| Dataset | Model | Groups | Metric | Values | Mechanism | ||
|---|---|---|---|---|---|---|---|
| D1 | Law | claude-sonnet-4-6 | non-white vs white | PA prop. | 38.0 vs 12.0 | 0.017 | (i) disclaimer |
| D1 | Law | claude-sonnet-4-6 | non-white vs white | gender mentioned | 38.0 vs 12.0 | 0.016 | (i) disclaimer |
| D1 | Law | claude-sonnet-4-6 | non-white vs white | race mentioned | 38.0 vs 12.0 | 0.016 | (i) disclaimer |
| D1 | Student | gpt-4o-mini | good vs poor health | PA prop. | 0.6 vs 9.3 | 0.001 | (i) health |
| D1 | Student | gpt-4o-mini | good vs poor health | health mentioned | 1.8 vs 27.9 | 0.003 | (i) health |
| D2 | German Credit | gpt-4o-mini | 30 vs 30 | Sec. prop. | 3.1 vs 1.0 | 0.042 | (ii) feature values |
| Dataset | Model | Groups | Values | Driver | Attribute named | ||
| D1 | Student | gpt-4.1 | girls vs boys | 0.44 vs 0.37 | 0.011 | label | no |
| D3 | Student | mistral-large-2512 | 17: disclosed vs withheld | 0.29 vs 0.42 | 0.025 | label | no |
| D4 | German Credit | gpt-4o-mini | 30 25: swapped vs factual | 0.03 vs 0.04 | 0.048 | label | no |
| D4 | Student | gpt-4.1 | girls boys: swapped vs factual | 0.36 vs 0.44 | 0.002 | label | no |
| D4 | Student | gpt-4o-mini | good bad health: swapped vs factual | 0.38 vs 0.44 | 0.005 | label | yes |
| D5 | German Credit | ministral-14b | women men: swapped vs actual | 0.03 vs 0.12 | 0.004 | feature values | no |
Social Bias in Language Models
Social bias in language models is the study and mitigation of unfair stereotypes and disparities in generated text, judgments, and decisions.
See all 217 papers