Reweighting is not Harmless: Understanding the Role of Dataset Characteristics

Amalia Ionescu; Jose G. Hernandez; Emily F. Wong; Anil K. Saini; Jason H. Moore; Tiffani J. Bright

cs.LGNov 5, 2026

Reweighting is not Harmless: Understanding the Role of Dataset Characteristics

Authors: Amalia Ionescu, Jose G. Hernandez, Emily F. Wong, Anil K. Saini, Jason H. Moore, Tiffani J. Bright

Organizations: Cedars-Sinai Medical Center, Los Angeles, California, USA

Abstract

Reweighting is a widely used pre-processing method for promoting discrimination-free model behavior by assigning different weights to data points during model training. While it is often credited with improving fairness, particularly by mitigating systematic underestimation for underrepresented groups, it can also introduce unintended harms under certain data conditions. Using controlled simulations that vary group size and baseline event prevalence, we show how reweighting can yield misleading improvements in fairness by reducing underestimation for one group while introducing it for another. Across 3,000 simulated datasets with event prevalence of 7-14%, reweighting improved parity in positive classification and false negative rates (FNR) between groups, but did so partly by decreasing positive classifications and increasing FNR for the group with higher baseline prevalence. These effects were most pronounced when that group was a minority: its FNR rose by 6 percentage points, approximately 82 additional missed cases per 10,000 group members, and roughly three-quarters of the improvement in FNR parity came from this increase rather than from gains for the other group. The pattern held across six classifiers, from logistic regression to neural networks, while overall accuracy was essentially unchanged, leaving the shift invisible in aggregate performance metrics. These findings mirror patterns reported in real-world applications and underscore the critical role of data characteristics in shaping the impact of this fairness intervention. While reweighting may create desirable outcomes in some applications, practitioners should critically evaluate it relative to their problem area and goals. Specifically, when the goal is to reduce underestimation in one group without introducing underestimation in another, reweighting may not be the appropriate approach.

Figures & tables

#	Proportion Group A	Relative Event Likelihood in Group A	Prevalence A	Prevalence B
1	0.2 (Minority)	Less Likely ( $b_{a}=-0.7$ )	7.0%	10.0%
2	0.5 (Equal)	Less Likely ( $b_{a}=-0.7$ )	7.0%	10.0%
3	0.2 (Minority)	Equally Likely ( $b_{a}=0$ )	10.0%	10.0%
4	0.5 (Equal)	Equally Likely ( $b_{a}=0$ )	10.0%	10.0%
5	0.2 (Minority)	More Likely ( $b_{a}=+0.7$ )	13.5%	10.0%
6	0.5 (Equal)	More Likely ( $b_{a}=+0.7$ )	13.5%	10.0%

Table 1. Experimental conditions varying two data characteristics: dataset diversity and relative event likelihood.

#	Proportion Group A	Relative Event Likelihood in Group A	Substantive Example
1	0.2 (Minority)	Less Likely	Mortgage Approval ( Bhutta et al., 2021 )
2	0.5 (Equal)	Less Likely	Resume Selection ( Wilson and Caliskan, 2024 )
3	0.2 (Minority)	Equally Likely	Control Group
4	0.5 (Equal)	Equally Likely	Control Group
5	0.2 (Minority)	More Likely	Postpartum Depression a ( Wong et al., 2024 )
6	0.5 (Equal)	More Likely	Breast Cancer ( Anderson et al., 2010 )

Table 2. Experimental conditions and their corresponding substantive examples.

Group	Group	Relative Event					Two-sided
Label	Size	Likelihood	Before	After	$\Delta P(\hat{Y}=1\|\cdot)$	95% CI	p -value
A	Minority	More Likely	0.1263	0.1109	$-0.0154$	[ $-0.0156$ , $-0.0152$ ]	$<.001$
A	Equal	More Likely	0.1264	0.1172	$-0.0092$	[ $-0.0094$ , $-0.0091$ ]	$<.001$
B	Equal	More Likely	0.0849	0.0748	$-0.0101$	[ $-0.0102$ , $-0.0100$ ]	$<.001$
B	Majority	More Likely	0.0853	0.0813	$-0.0040$	[ $-0.0040$ , $-0.0039$ ]	$<.001$
A	Minority	Less Likely	0.0472	0.0640	$+0.0167$	[ $+0.0165$ , $+0.0170$ ]	$<.001$

Table 3. The effect of reweighting on the probability of positive classification by group size and relative event likelihood (logistic regression, held-out test set). Before and after are mean rates;

\Delta

is the mean paired difference (reweighted minus unweighted) across 3,000 repetitions, with 95% confidence intervals (CI) and two-sided p -values from Wilcoxon signed-rank tests. Values correspond to Figure 2 .

Figure 1. The effect of reweighting on Demographic Parity (DP) (logistic regression; held-out test set).

Figure 2. The effect of reweighting on model predictions (logistic regression; held-out test set).

Figure 3. The effect of reweighting on false negative rates (FNR) parity (logistic regression; held-out test set.

Figure 4. The effect of reweighting on false negative rates (FNR) (logistic regression, held-out test set).

Group	Group	Relative Event			Two-sided	Additional missed
Label	Size	Likelihood	$\Delta$ FNR	95% CI	p -value	cases per 10,000
A	Minority	More Likely	$+0.0608$	[ $0.0597$ , $0.0618$ ]	$<.001$	82
A	Equal	More Likely	$+0.0356$	[ $0.0351$ , $0.0362$ ]	$<.001$	48
B	Equal	More Likely	$+0.0531$	[ $0.0524$ , $0.0539$ ]	$<.001$	53
B	Majority	More Likely	$+0.0202$	[ $0.0198$ , $0.0206$ ]	$<.001$	20

Table 4. The effect of reweighting on FNR for groups with higher baseline likelihoods across group sizes (logistic regression, held-out test set). Values are mean paired differences (reweighted minus unweighted) across 3,000 repetitions, with 95% confidence intervals (CI) and two-sided p -values from Wilcoxon signed-rank tests. Additional missed cases are

\Delta

FNR

\times

group prevalence

\times

10,000, using training-set prevalence. Values correspond to Figure 4 .

Figure 5. Change after reweighting (reweighted minus unweighted) in the positive-prediction rate of Group $A$ (top), the FNR of Group $A$ (middle), and the FNR of Group $B$ (bottom), for six classifiers. Left: Group $A$ is the minority (20%); right: groups are of equal size. Points are means across 3,000 repetitions on the held-out test set; error bars are 95% confidence intervals.

	Test	Higher-baseline group			Lower-baseline group
Classifier	AUROC	Minority	Equal	Majority	Minority
Logistic regression	0.960	$+0.0608$	$+0.0356$	$+0.0202$	$-0.1117$
		[0.0597, 0.0618]	[0.0351, 0.0362]	[0.0198, 0.0206]	[ $-0.1138$ , $-0.1097$ ]
Logistic regression (L2)	0.960	$+0.0624$	$+0.0365$	$+0.0208$	$-0.1147$
		[0.0613, 0.0635]	[0.0360, 0.0371]	[0.0204, 0.0211]	[ $-0.1168$ , $-0.1126$ ]
Multilayer perceptron	0.959	$+0.0581$	$+0.0352$	$+0.0190$	$-0.1090$

Table 5. Change in FNR after reweighting by classifier (held-out test set). Values are mean paired differences (reweighted minus unweighted) across 3,000 repetitions, with 95% confidence intervals in brackets; all

p<.001

(Wilcoxon signed-rank). Higher-baseline groups: Group

A

as minority (condition 5) and as equal-sized group (condition 6), and Group

B

as majority (condition 1). Lower-baseline group: Group

A

as minority (condition 1).

Algorithmic Fairness

Algorithmic fairness studies how automated decisions distribute errors, opportunities, and harms across groups, and how to measure and reduce disparities.

See all 440 papers

Fairness May Backfire: When Leveling-Down Occurs in Fair Machine Learning
Mar 6, 2026Yi Yang, Xiangyu Chang, Pei-yu ChenAlgorithmic Fairness Group Fairness
Department of Information Systems, Arizona State University · Department of Information Systems and Intelligent Business, Xi’an Jiaotong University
Fair Regression under Approximate Demographic Parity
Oct 19, 2026Weiwei Wang, Xianyang Zhang, Raymond K. W. WongAlgorithmic Fairness Group Fairness
Department of Statistics, Texas A&M University
Fairness Theatre: Evaluating Post-Hoc Fairness Interventions in Vendor-Controlled Early Warning Systems
Sep 29, 2026Kelly McConvey, Angelina Zhai, Rebecca Li +1Algorithmic Auditing Algorithmic Fairness
University of Toronto, Toronto, Ontario, Canada · Georgia Institute of Technology, Atlanta, Georgia, USA

Mar 6, 2026J/K move · Enter open · S save

Fairness May Backfire: When Leveling-Down Occurs in Fair Machine Learning

Yi Yang, Xiangyu Chang, Pei-yu Chen

Department of Information Systems, Arizona State University · Department of Information Systems and Intelligent Business, Xi’an Jiaotong University

Algorithmic Fairness Group Fairness Classification

Open paper AskPDF arXiv

As machine learning (ML) systems increasingly shape access to credit, jobs, and other opportunities, the fairness of algorithmic decisions has become a central concern. Yet it remains unclear when enforcing fairness constraints in these systems genuinely improves outcomes for affected groups or instead leads to "leveling down," making one or both groups worse off. We address this question in a unified, population-level (Bayes) framework for binary classification under prevalent group fairness notions. Our Bayes approach is distribution-free and algorithm-agnostic, isolating the intrinsic effect of fairness requirements from finite-sample noise and from training and intervention specifics. We analyze two deployment regimes for ML classifiers under common legal and governance constraints: attribute-aware decision-making (sensitive attributes available at decision time) and attribute-blind decision-making (sensitive attributes excluded from prediction). We show that, in the attribute-aware regime, fair ML necessarily (weakly) improves outcomes for the disadvantaged group and (weakly) worsens outcomes for the advantaged group. In contrast, in the attribute-blind regime, the impact of fairness is distribution-dependent: fairness can benefit or harm either group and may shift both groups' outcomes in the same direction, leading to either leveling up or leveling down. We characterize the conditions under which these patterns arise and highlight the role of "masked" candidates in driving them. Overall, our results provide structural guidance on when pursuing algorithmic fairness is likely to improve group outcomes and when it risks systemic leveling down, informing fair ML design and deployment choices.

Abstract

Algorithmic Fairness

Explore similar work

Fairness May Backfire: When Leveling-Down Occurs in Fair Machine Learning

Fair Regression under Approximate Demographic Parity

Fairness Theatre: Evaluating Post-Hoc Fairness Interventions in Vendor-Controlled Early Warning Systems