Investigating Data Interventions for Subgroup Fairness: An ICU Case Study

Erin Tan; Judy Hanwen Shen; Irene Y. Chen

cs.LGApr 3, 2026

Investigating Data Interventions for Subgroup Fairness: An ICU Case Study

Authors: Erin Tan, Judy Hanwen Shen, Irene Y. Chen

Organizations: University of California, Berkeley · Stanford University

Abstract

In high-stakes settings where machine learning models are used to automate decision-making about individuals, the presence of algorithmic bias can exacerbate systemic harm to certain subgroups of people. These biases often stem from the underlying training data. In practice, interventions to "fix the data" depend on the actual additional data sources available -- where many are less than ideal. In these cases, the effects of data scaling on subgroup performance become volatile, as the improvements from increased sample size are counteracted by the introduction of distribution shifts in the training set. In this paper, we investigate the limitations of combining data sources to improve subgroup performance within the context of healthcare. Clinical models are commonly trained on datasets comprised of patient electronic health record (EHR) data from different hospitals or admission departments. Across two such datasets, the eICU Collaborative Research Database and the MIMIC-IV dataset, we find that data addition can both help and hurt model fairness and performance, and many intuitive strategies for data selection are unreliable. We compare model-based post-hoc calibration and data-centric addition strategies to find that the combination of both is important to improve subgroup performance. Our work questions the traditional dogma of "better data" for overcoming fairness challenges by comparing and combining data- and model-based approaches.

Figures & tables

Figure 1: Change in Overall and subgroup-level accuracy after Whole-Source data addition (Logistic Regression). The change in Overall performance (a) does not reflect equally upon changes in performance across subgroups. For example, we observe that adding data from any source to Target Hospital 458 improves overall accuracy. While this change is reflected across the White (b) and Black (c) subgroups, the Other (d) subgroup experiences near-uniform decreases in test performance as a result of data addition. Scaling data directly from the Target Hospital (diagonal of each plot) generally yields improvements in overall and subgroup accuracy, but is typically not the best -performing data addition choice.

Figure 2: Change in subgroup ratio vs. Change in subgroup test accuracy after Whole-Source data addition on the eICU Dataset.

Figure 3: Change in subgroup accuracy as a function of samples added in Subgroup-Level data addition (see Section 4 for details). Across nearly all combinations of subgroups and Test Hospitals, we find that adding more samples does not necessarily lead to larger performance gains. These visualizations lead to the conclusion that naive subgroup balancing is an uninformative data selection heuristic.

Figure 4: Subgroup similarity score (using features and labels) vs. Change in subgroup test accuracy for the White (left) and Black (right) subgroups (eICU). The scores are computed using only the patients from the target subgroup in each source. These performance changes result from Subgroup-Level data addition. We do not observe statistically significant correlations in any Test Hospital for either subgroup.

Figure 5: Mean consistency and subgroup performance: (a) Subgroup Mean Discrepancy (Eq. 6 ) vs. Subgroup Test Accuracy (Eq. 1 ). (b) Change in Subgroup Mean Discrepancy vs. Change in Subgroup Test Accuracy. Strong negative correlations are observed across all subgroups in (a), and in all minority subgroups in (b).

Figure 6: Difference in best-case subgroup performance (without calibration) and worst-case subgroup performance (with calibration) after Whole-Source data addition. The overwhelming majority of subgroups show positive differences, supporting the idea that making informed choices for data addition is more important than calibrating post-hoc.

	Black		Other		White
	W/o Cal.	With Cal.	W/o Cal.	With Cal.	W/o Cal.	With Cal.
Best	4.2% $\pm$ 1.6	3.4% $\pm$ 0.9	6.1% $\pm$ 2.2	8.7% $\pm$ 1.3	2.3% $\pm$ 0.3	2.8% $\pm$ 0.1
Median	0.6% $\pm$ 2.6	1.5% $\pm$ 1.1	1.2% $\pm$ 3.2	0.1% $\pm$ 1.7	0.7% $\pm$ 0.9	2.2% $\pm$ 0.1
Worst	-4.1% $\pm$ 2.8	-5.9% $\pm$ 4.3	-3.4% $\pm$ 2.8	-13.0% $\pm$ 4.4	-3.4% $\pm$ 2.7	1.2% $\pm$ 0.4

Table 1: Average change in subgroup accuracy for the Best, Median, and Worst Whole-Source data additions across all sources, with and without calibration.

Appendix figures & tables23 assets

Supplementary material from the paper’s appendix.

Appendix

	Count (Rate)
Hos ID	Total	Asian	Black	Other	White
73	4320	61 (0.01)	622 (0.15)	347 (0.08)	3221 (0.76)
167	2107	29 (0.01)	154 (0.07)	421 (0.20)	1503 (0.71)
188	2299	29 (0.01)	517 (0.22)	64 (0.03)	1689 (0.73)
199	2529	3 (0.001)	42 (0.02)	48 (0.02)	2434 (0.96)
243	2812	24 (0.009)	873 (0.31)	83 (0.03)	1831 (0.65)

Appendix

Table 2: Subgroup information across hospitals with

2000

+ patient encounters in the eICU Dataset

	Count (Rate)
Adm Type	Total	Asian	Black	Hispanic	Other	White
Elective	3464	69 (0.02)	155 (0.045)	75 (0.021)	478 (0.138)	2687 (0.775)
Emergency	21669	502 (0.023)	2155 (0.099)	741 (0.034)	2965 (0.137)	15306 (0.706)
Urgent	746	12 (0.016)	17 (0.022)	16 (0.021)	318 (0.42)	383 (0.51)

Appendix

Table 3: Subgroup information across admission types in the MIMIC-IV Dataset

Sources		Change in ACC ( $\Delta$ )
Test	Added	Asian	Black	Hispanic	Other	White	Overall
Elective	Elective	0.001	0.006 $\uparrow$	0.007 $\uparrow$	0.006 $\uparrow$	-0.001	0.000
Elective	Emergency	0.001	-0.001	0.010 $\uparrow$	-0.003	0.001	0.001
Elective	Urgent	-0.001	0.000	0.003	-0.003	0.002	-0.001
Emergency	Elective	-0.001	-0.005 $\downarrow$	-0.003	0.010 $\uparrow$	0.000	-0.001
Emergency	Emergency	0.002	-0.010 $\downarrow$	-0.004	0.020 $\uparrow$	0.008 $\uparrow$	0.001

Appendix

Table 4: Change in overall & subgroup performance after Whole-Source data addition.

Ethnicity	Admission Type	Added Source	Rate Change	Acc. Diff	Correlation
Asian	Elective	Elective	0.000	0.006	$r=-0.28$ , $p=0.47$
	Elective	Emergency	0.002	-0.001
	Elective	Urgent	-0.002	0.000
	Emergency	Elective	-0.002	-0.005
	Emergency	Emergency	0.000	-0.010
	Emergency	Urgent	-0.004	-0.021

Appendix

Table 5: Change in subgroup ratio and resulting change in subgroup accuracy after Whole-Source data addition on the MIMIC-IV dataset.

Ethnicity	Admission Type	Added Source	Samples Added	Acc. Diff	Correlation
Asian	Elective	Elective	19	0.006	$r=-0.24$ , $p=0.54$
	Elective	Emergency	23	-0.001
	Elective	Urgent	16	0.000
	Emergency	Elective	19	-0.005
	Emergency	Emergency	23	-0.010
	Emergency	Urgent	16	-0.021

Appendix

Table 6: Subgroup samples added and resulting change in subgroup accuracy after Subgroup-Only data addition on the MIMIC-IV dataset.

Figure 7: Change in Overall and subgroup-level accuracy after Whole-Source data addition. All results from abridged Figure 1 .

Figure 8: Change in overall test accuracy vs change in subgroup test accuracy. The Pareto Frontier is plotted on each subfigure to visualize the tradeoff between fairness and performance.

Figure 9: Change in subgroup ratio vs. Change in subgroup test AUC on the eICU dataset. Same experiment conditions as were used to produce 2 .

Figure 10: Number of subgroup samples added vs. Change in subgroup test AUC on the eICU dataset. Same experiment conditions as were used to produce 3 .

Figure 11: Subgroup similarity score vs. Change in subgroup test AUC on the eICU dataset. Same experiment conditions as were used to produce 4 .

Figure 12: Difference in best-case subgroup AUC (without calibration) and worst-case subgroup AUC (with calibration) on the eICU dataset. Same experiment conditions as were used to produce 6 .

Figure 13: Difference in best-case subgroup AUC (with calibration) and worst-case subgroup AUC (with calibration) on the eICU dataset. Same experiment conditions as were used to produce 1 .

Figure 14: Change in performance after Whole-Source data addition using the Light Gradient Boosting Machine (LGBM) classifier on the eICU Dataset.

Figure 15: Change in Subgroup Rate vs. Change in Subgroup Accuracy after Whole-Source data addition, trained using an LGBM model.

Figure 16: Number of subgroup samples added vs. Change in Subgroup Accuracy after Subgroup-Level data addition, trained using an LGBM model.

Figure 17: Subgroup Similarity Score between the test subgroup and added subgroup vs. Change in Subgroup Accuracy after Subgroup-Level data addition, trained using an LGBM model.

Figure 18: Difference in best-case subgroup performance (without calibration) and worst-case subgroup performance (with calibration) after Whole-Source data addition. Performance is evaluated on the eICU dataset using the subgroup accuracy metric and trained using an LGBM classifier.

Figure 19: Difference in best-case subgroup performance (with calibration) and worst-case subgroup performance (with calibration) after Whole-Source data addition. Performance is evaluated on the eICU dataset using the subgroup accuracy metric and trained using an LGBM classifier.

Figure 20: Change in Overall and Subgroup Accuracy after Whole-Source data addition (LSTM).

Figure 21: Change in Subgroup Rate vs. Change in Subgroup Accuracy after Whole-Source data addition, trained using an LSTM model.

Figure 22: Number of subgroup samples added vs. Change in Subgroup Accuracy after Subgroup-Level data addition, trained using an LSTM model.

Figure 23: Subgroup Similarity Score between the test subgroup and added subgroup vs. Change in Subgroup Accuracy after Subgroup-Level data addition, trained using an LSTM model.

Figure 24: Change in Overall and Subgroup AUC by ethnicity group after Whole-Source data addition on the eICU dataset. Experiments are identical to the ones used to produce 1 .

Group Fairness

Group fairness measures and reduces disparities in how algorithms predict, rank, allocate resources, and make decisions for different groups.

See all 140 papers

FairSelect: A Systematic Evaluation of Multi-Level and Intersectional Algorithmic Fairness
Jul 9, 2026Nick Souligne, Isabella Mixton-Garcia, Vignesh SubbianAlgorithmic Fairness Group Fairness
College of Engineering, University of Arizona, Tucson, AZ
DuetFair: Coupling Inter- and Intra-Subgroup Robustness for Fair Medical Image Segmentation
May 11, 2026Yiqi Tian, Sangjoon Park, Bo Zeng +3Subgroup Robustness Healthcare
Center for Advanced Medical Computing and Analysis, Massachusetts General Hospital and Harvard Medical School, Boston, MA 02114 · Department of Industrial Engineering, University of Pittsburgh, Pittsburgh, PA 15261 · Department of Radiation Oncology, College of Medicine, Yonsei University, Seoul, South Korea +2
Detection and Validation of Hidden Subgroup Disparities in Clinical AI
Oct 23, 2026Alceu Bissoto, Trung-Dung Hoang, Tim Flühmann +4Fairness in Medical Image Classification Algorithmic Bias
Department of Digital Medicine, University of Bern, Switzerland · Department of Diabetes, Endocrinology, Nutritional Medicine and Metabolism, Bern University Hospital, Switzerland · Diabetes Center Berne, Switzerland +3

Jul 9, 2026J/K move · Enter open · S save

FairSelect: A Systematic Evaluation of Multi-Level and Intersectional Algorithmic Fairness

Nick Souligne, Isabella Mixton-Garcia, Vignesh Subbian

College of Engineering, University of Arizona, Tucson, AZ

Algorithmic Fairness Group Fairness Intersectional Fairness

Open paper AskPDF arXiv

Algorithmic fairness methods are increasingly used to identify and mitigate bias in machine learning models, yet most approaches are evaluated in isolation and along single demographic axes. This limits practical guidance for selecting fairness strategies, where disparities may arise across intersectional subgroups and across multiple stages of the modeling lifecycle. This work presents FairSelect, a toolkit for systematically evaluating fairness mitigation strategies applied individually and in combination across preprocessing, inprocessing, and postprocessing stages. FairSelect supports multiple model architectures, intersectional subgroup evaluation, and comparison of fairness utility tradeoffs across baseline, single method, and multi level configurations. The framework was validated using synthetic clinical datasets designed to represent specific bias mechanisms and a real-world replication of two-year stroke risk prediction among patients with atrial fibrillation. Synthetic experiments showed that targeted fairness methods generally reduced intended subgroup disparities, while combined strategies produced larger average fairness improvements with modest utility tradeoffs. In the clinical prediction task, mitigation effects were highly variable, with some combinations improving both fairness and predictive performance while others were ineffective or counterproductive. These findings demonstrate that fairness interventions interact in nonadditive and context dependent ways. FairSelect provides a practical framework for systematically identifying fairness strategies that improve subgroup equity while preserving model performance in clinical machine learning.

Abstract

Group Fairness

Explore similar work

FairSelect: A Systematic Evaluation of Multi-Level and Intersectional Algorithmic Fairness

DuetFair: Coupling Inter- and Intra-Subgroup Robustness for Fair Medical Image Segmentation

Detection and Validation of Hidden Subgroup Disparities in Clinical AI