Mitigating Demographic Disparities in Explanation-Human Alignment via Rationale Regularization

Suryansh Talukdar; Taha Sarwat; Jonathan Rusert

cs.CLOct 15, 2026

Mitigating Demographic Disparities in Explanation-Human Alignment via Rationale Regularization

Authors: Suryansh Talukdar, Taha Sarwat, Jonathan Rusert

Organizations: Purdue University Fort Wayne

Abstract

We study explanation fairness in hate speech detection. High classification performance does not always mean that a model produces equally plausible explanations for different demographic groups. Using the HateXplain dataset, we introduce Plausibility-EFG, a metric that measures the largest difference in explanation plausibility across demographic groups by comparing model-generated token attributions with human-annotated rationales. We evaluate BERT, RoBERTa, and ADV-RoBERTa, an adversarially trained model, using Integrated Gradients and SHAP. Our results show that ADV-RoBERTa improves classification performance and alignment between explanations and human rationales while reducing explanation disparities across demographic groups. These improvements are observed with both attribution methods and remain consistent across training, validation, and test splits. We also show that reducing prediction-level bias alone does not eliminate disparities in explanations, suggesting that predictive fairness and explanation fairness are related but distinct objectives. Finally, our fairness-performance analysis shows that stronger adversarial regularization can further reduce Plausibility-EFG, although it may cause a small decrease in macro-F1. Overall, this work presents explanation fairness as an important objective in hate speech moderation and introduces Fair-REF, a framework designed to reduce disparities in model explanations across demographic groups.

Figures & tables

Figure 1: Post-level label distribution in HateXplain.

Model	Accuracy	Macro-F1	Weighted-F1	Normal F1	HateSpeech F1	Offensive F1	$\Delta$ Accuracy vs BERT	$\Delta$ Macro-F1 vs BERT
BERT	0.6736	0.6719	0.6785	0.7103	0.7519	0.5535	—	—
RoBERTa	0.6924	0.6913	0.7022	0.7430	0.7552	0.5757	0.0188	0.0194
ADV-RoBERTa	0.7627	0.7394	0.7616	0.7520	0.8173	0.6489	0.0891	0.0675

Table 1: 3-Way Hate Speech Classification Performance

Metric	Level	BERT	RoBERTa	ADV-RoBERTa
Overall Accuracy	Posts	0.80	0.84	0.85
Overall Accuracy	Exploded	0.79	0.82	0.83
Overall Macro-F1	Posts	0.79	0.83	0.84
Overall Macro-F1	Exploded	0.59	0.64	0.68
Mean AUPRC	Posts	0.52	0.56	0.59
Mean AUPRC	Exploded	0.53	0.57	0.60

Table 2: IG Plausibility Metrics

Metric	IG BERT	IG RoBERTa	IG ADV-RoBERTa	SHAP BERT	SHAP RoBERTa	SHAP ADV-RoBERTa
Plausibility-EFG	0.08	0.06	0.03	0.07	0.05	0.02
Cohen’s $d$	0.26	0.20	0.11	0.24	0.17	0.08
Kruskal p-value	$<10^{-5}$	$<10^{-4}$	0.041	0.10	0.071	0.24
Best Group	Women	Women	Women	Arab	Women	Women
Worst Group	Refugee	Caucasian	African	Jewish	Caucasian	African
Best Group AUPRC	0.58	0.61	0.60	0.45	0.50	0.54

Table 3: Explanation Fairness Gap (EFG) Benchmarking

Model	IG AUPRC	IG IOU-F1	SHAP AUPRC	SHAP IOU-F1	IG EFG	SHAP EFG
BERT	0.52	0.26	0.43	0.23	0.08	0.07
RoBERTa	0.56	0.31	0.48	0.28	0.06	0.05
ADV-RoBERTa	0.59	0.34	0.52	0.32	0.03	0.02

Table 4: Explainability and Explanation Fairness Comparison

Figure 2: Explanation Fairness Gap (EFG) across BERT, RoBERTa, and ADV-RoBERTa under Integrated Gradients and SHAP

Variant	$L_{\text{task}}$	$L_{\text{rat}}$	$L_{\text{adv}}$	Accuracy	Macro-F1	IG AUPRC	IG IOU-F1	SHAP AUPRC	SHAP IOU-F1	IG EFG	SHAP EFG
Task-only (RoBERTa)	✓	✗	✗	0.7480	0.7240	0.55	0.30	0.47	0.27	0.058	0.049
Task + Rationale	✓	✓	✗	0.7560	0.7320	0.58	0.33	0.50	0.30	0.052	0.043
Task + Adversarial	✓	✗	✓	0.7590	0.7350	0.56	0.31	0.49	0.29	0.036	0.028
Fair-REF (Task + Rat + Adv)	✓	✓	✓	0.7627	0.7394	0.59	0.34	0.52	0.32	0.030	0.020

Table 5: Component Ablation of Fair-REF

$\lambda_{\text{adv}}$	Test Accuracy	Test Macro-F1	IG AUPRC	SHAP AUPRC	Mean EFG
0.1	0.7608	0.7371	0.57	0.50	0.048
0.3	0.7619	0.7383	0.58	0.51	0.037
0.5	0.7627	0.7394	0.59	0.52	0.030
1.0	0.7578	0.7332	0.60	0.53	0.024

Table 6: Fairness–Performance Tradeoff Analysis

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 3: Confusion matrices for BERT, RoBERTa, and ADV-RoBERTa on three-way hate speech classification. ADV-RoBERTa reduces confusion between offensive and hatespeech categories relative to the baselines.

Group	BERT AUPRC	BERT IOU-F1	RoBERTa AUPRC	RoBERTa IOU-F1	ADV-RoBERTa AUPRC	ADV-RoBERTa IOU-F1
Women	0.58	0.28	0.61	0.34	0.63	0.37
Arab	0.54	0.29	0.60	0.33	0.62	0.36
Homosexual	0.53	0.26	0.59	0.32	0.61	0.35
Caucasian	0.51	0.26	0.56	0.31	0.60	0.35
Islam	0.50	0.25	0.58	0.32	0.61	0.35
Jewish	0.49	0.24	0.57	0.31	0.60	0.34

Appendix

Table 7: Group-wise IG plausibility metrics (AUPRC and IOU-F1) across demographic identities.

Group	BERT AUPRC	BERT IOU-F1	RoBERTa AUPRC	RoBERTa IOU-F1	ADV-RoBERTa AUPRC	ADV-RoBERTa IOU-F1
Women	0.44	0.24	0.50	0.30	0.55	0.36
Arab	0.45	0.25	0.51	0.30	0.55	0.36
Homosexual	0.39	0.22	0.47	0.28	0.54	0.35
Caucasian	0.39	0.22	0.46	0.28	0.53	0.34
Islam	0.42	0.23	0.49	0.29	0.54	0.35
Jewish	0.37	0.21	0.45	0.27	0.52	0.33

Appendix

Table 8: Group-wise SHAP plausibility metrics (AUPRC and IOU-F1) across demographic identities.

$\lambda_{\text{adv}}$	Best Val F1	Test Accuracy	Test Macro-F1	IG AUPRC	SHAP AUPRC	IG EFG	SHAP EFG
0.0	0.7788	0.7836	0.7537	0.55	0.48	0.061	0.052
0.1	0.7558	0.7847	0.7448	0.57	0.50	0.048	0.041
0.3	0.7699	0.7566	0.7252	0.58	0.51	0.037	0.031
0.5	0.7733	0.7675	0.7340	0.59	0.52	0.030	0.024
1.0	0.7603	0.7683	0.7327	0.60	0.53	0.024	0.019

Appendix

Table 9: Fairness–performance tradeoff across

\lambda_{\text{adv}}

settings.

Category / Metric	Value
Core Dataset
Total Unique Posts (df_posts)	20,148
Annotation-Level Rows (df_hx)	60,444
Exploded Annotation Rows (df_hx_exploded)	53,161
Final Fairness Evaluation Rows (df_exploded)	27,566
Dataset Splits

Appendix

Table 10: Complete dataset statistics and distribution summary.

Figure 4: Group-wise Integrated Gradients AUPRC across demographic identities for BERT, RoBERTa, and ADV-RoBERTa. ADV-RoBERTa consistently improves rationale alignment while substantially reducing subgroup disparity across demographic groups.

Figure 5: Group-wise SHAP AUPRC across demographic identities for BERT, RoBERTa, and ADV-RoBERTa. Fairness-aware optimization produces more uniform explanation quality across subgroups while preserving overall attribution plausibility improvements.

Figure 6: Fairness–performance tradeoff curve across different $\lambda_{\text{fair}}$ settings. Increasing adversarial fairness regularization consistently reduces Explanation Fairness Gap (EFG) while inducing only moderate degradation in macro-F1, demonstrating a controllable tradeoff between predictive performance and explanation fairness.

Figure 7: Distribution of subgroup-specific rationale evaluation instances across demographic identities in the exploded fairness evaluation dataset, highlighting substantial subgroup imbalance in the HateXplain corpus.

Figure 8: Toxicity-class composition across demographic identity groups, showing that hate speech, offensive, and normal label distributions vary considerably across identities, motivating subgroup-level fairness evaluation.

Figure 9: Overall comparison of classification performance, explanation plausibility, and explanation fairness across BERT, RoBERTa, and ADV-RoBERTa. ADV-RoBERTa consistently achieves the strongest balance between predictive quality, rationale alignment, and subgroup fairness across both IG and SHAP attribution methods.

Figure 10: Fair-REF (Fair Rationale Explanation Framework) architecture. A shared RoBERTa encoder feeds three task-specific branches: (i) a hate speech classification head optimized with the task loss ( $L_{\mathrm{task}}$ ), (ii) a token-level rationale prediction head trained using binary cross-entropy rationale supervision ( $L_{\mathrm{rat}}$ ), and (iii) a gradient-reversal demographic adversary trained with adversarial loss ( $L_{\mathrm{adv}}$ ). The three objectives are jointly optimized according to $L_{\mathrm{total}}=L_{\mathrm{task}}+\lambda_{\mathrm{rat}}L_{\mathrm{rat}}+\lambda_{\mathrm{adv}}L_{\mathrm{adv}}$ , encouraging accurate prediction, faithful rationale generation, and improved explanation fairness across demographic groups.

Demographic Bias in Language Models

Demographic bias in language models studies how identity cues shape model outputs, decisions, and representations, and how to detect and reduce resulting disparities.

See all 135 papers

Disagreeing Rationales: Rethinking Classification and Explainability Evaluation in Hate Speech Detection
May 29, 2026Benedetta Muscato, Beiduo Chen, Gizem Gezici +2Annotator Disagreement Hate Speech Detection
Scuola Normale Superiore, Italy · Munich Center for Machine Learning, Germany · UMaiNLP, LMU Munich, Germany
Explanation Fairness in Large Language Models: An Empirical Analysis of Disparities in How LLMs Justify Decisions Across Demographic Groups
May 9, 2026Gautam VeldandaLLM Auditing Algorithmic Fairness
Independent Researcher
SCM-based Fairness and Faithful Explainability for Legal Document Classification
Sep 3, 2026Yasmina El Kacemi, Seyed Sahand Mohammadi Ziabari, Ali Mohammed Mansoor AlsahagFaithfulness of Language Model Explanations Algorithmic Fairness
University of Amsterdam Amsterdam, The Netherlands

May 29, 2026J/K move · Enter open · S save

Disagreeing Rationales: Rethinking Classification and Explainability Evaluation in Hate Speech Detection

Benedetta Muscato, Beiduo Chen, Gizem Gezici, Barbara Plank+1

Scuola Normale Superiore, Italy · Munich Center for Machine Learning, Germany · UMaiNLP, LMU Munich, Germany

Annotator Disagreement Hate Speech Detection Explainability Evaluation Explanation Evaluation

Open paper AskPDF arXiv

Human disagreement is ubiquitous and well-known in labeling. However, variation in explanations, captured through token-level human rationales, remains far less explored. At the same time, it is unclear how to best evaluate human labels and rationales -- or even how to best aggregate rationales beyond majority vote -- in light of this variation. Yet, rationales may provide additional insights into the richness of human reasoning, that may differ in style, values and interpretations -- especially in subjective NLP tasks like hate speech detection. In this work, we unify diverse models, training strategies, loss functions, and existing evaluation metrics under a single protocol by systematically re-implementing them across different label and rationale representation spaces. Classification metrics are organized around two key properties -- predictive and distributional -- while explainability metrics through three complementary dimensions: plausibility, faithfulness, and complexity. In this unified supervision framework, we evaluate model behavior across classification and explainability metrics, as well as metric sensitivity to the choice of label (hard and soft) and rationale representation space (hard, intermediate and soft). Results show that both hard and soft metrics favor softer representations, highlighting their effectiveness in capturing variation and the need to rethink evaluation in subjective NLP.

Abstract

Demographic Bias in Language Models

Explore similar work

Disagreeing Rationales: Rethinking Classification and Explainability Evaluation in Hate Speech Detection

Explanation Fairness in Large Language Models: An Empirical Analysis of Disparities in How LLMs Justify Decisions Across Demographic Groups

SCM-based Fairness and Faithful Explainability for Legal Document Classification