Do covariates explain why these groups differ? The choice of reference group can reverse conclusions in the Oaxaca-Blinder decomposition

Manuel Quintero; Advik Shreekumar; William T. Stephenson; Tamara Broderick

stat.MEMar 31, 2026

Do covariates explain why these groups differ? The choice of reference group can reverse conclusions in the Oaxaca-Blinder decomposition

Authors: Manuel Quintero, Advik Shreekumar, William T. Stephenson, Tamara Broderick

Organizations: Massachusetts Institute of Technology, Cambridge, MA, USA · University of California, Berkeley, Berkeley, CA, USA

Abstract

Scientists often want to explain why an outcome is different in two groups. For instance, differences in patient mortality rates across two hospitals could be due to differences in the patients themselves (covariates) or differences in medical care (outcomes given covariates). The Oaxaca--Blinder decomposition (OBD) is a standard tool to tease apart these factors. It is well known that the OBD requires choosing one of the groups as a reference, and the numerical answer can vary with the reference. To the best of our knowledge, there has been no systematic investigation into whether the choice of OBD reference can yield different substantive conclusions and how common this issue is. In the present paper, we give existence proofs in real and simulated data that the OBD references can in fact yield substantively different conclusions. Our empirical exercises find that this sensitivity is more common when the OBD is extended to more complex regression models, including a pretrained transformer. Our theoretical and empirical results together establish that these conclusion reversals are not entirely driven by model misspecification, small data, or adversarial parameter choices. Our results suggest that practitioners should always report both directions of the OBD; that modern machine learning and large datasets do not automatically resolve the conclusion reversal problem; and that further work on this problem is needed.

Figures & tables

Model	Total flips	$10\%$	$5\%$	$1\%$
Linear	27	2	2	2
Logistic	29	2	2	1
Neural net	62	29	17	6
XGBoost	62	17	10	3
TabPFN	76	22	19	5

Table 1: Number of data subsets (out of

139

) exhibiting at least one sign flip (in either the explained or unexplained component), by model. Columns

10\%

5\%

, and

1\%

report the number of flipped subsets that are significant at the corresponding level.

Reference	Explained	Unexplained	Total gap
Women	$0.021^{***}$ $(0.007)$	$0.014$ $(0.025)$	$0.035$ $(0.023)$
Men	$-0.007$ $(0.011)$	$0.042^{*}$ $(0.025)$	$0.035$ $(0.023)$

Table 2: Oaxaca–Blinder decompositions for HR quartile two. The total gap is men’s mean minus women’s mean. We compute standard errors (in parentheses) via bootstrap resampling with

1{,}000

replications. We calculate p-values using a normal approximation (Wald test), where the test statistic is the coefficient divided by its bootstrap standard error. Significance levels: *

p<0.10

, **

p<0.05

, ***

p<0.01

Panel of Figure 1 — Figure 1: Left: Linear fit of mortality on admission heart rate for each group when holding all other covariates fixed at their group means. Right: Percentage of parameter space leading to sign flips. See Remark D.2 for how we compute the percentage for the unexplained component.

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

Model	Total flips	$10\%$	$5\%$	$1\%$
Linear	11	2	2	2
Logistic	9	2	2	1
Neural net	42	26	17	6
XGBoost	33	17	10	3
TabPFN	33	21	19	5

Appendix

Table 3: Number of data subsets (out of

139

) exhibiting a sign flip in the explained component, by model. Columns

10\%

5\%

, and

1\%

report the number of flipped data subsets that are significant at the corresponding level.

Model	Total flips	$10\%$	$5\%$	$1\%$
Linear	17	0	0	0
Logistic	21	0	0	0
Neural net	49	10	6	1
XGBoost	47	3	1	0
TabPFN	56	2	0	0

Appendix

Table 4: Number of data subsets (out of

139

) exhibiting a sign flip in the unexplained component, by model. Columns

10\%

5\%

, and

1\%

report the number of flipped data subsets that are significant at the corresponding level.

Model	Gap (M–F)	Women reference		Men reference
Model	Gap (M–F)	Explained	Unexplained	Explained	Unexplained
Linear	$0.035$ $(0.023)$	$0.021^{***}$ $(0.007)$	$0.014$ $(0.025)$	$-0.007$ $(0.010)$	$0.042^{*}$ $(0.025)$
Logistic	$0.035$ $(0.023)$	$0.026^{***}$ $(0.009)$	$0.009$ $(0.025)$	$-0.006$ $(0.008)$	$0.041^{*}$ $(0.024)$
Neural net	$0.035$ $(0.023)$	$0.001$ $(0.024)$	$0.033$ $(0.031)$	$0.053^{**}$ $(0.022)$	$-0.018$ $(0.025)$
XGBoost	$0.035$ $(0.023)$	$0.004$ $(0.016)$	$0.031$ $(0.025)$	$0.029^{**}$ $(0.014)$	$0.006$ $(0.021)$
TabPFN	$0.035$ $(0.023)$	$0.012$ $(0.016$	$0.023$ $(0.026)$	$0.023^{*}$ $(0.013)$	$0.012$ $(0.021)$

Appendix

Table 5: OBD of the gender mortality gap in HR quartile two (

n=816

), across all five models. The total gap is men’s mean minus women’s mean. Standard errors in parentheses are computed via bootstrap resampling with

1{,}000

replications.

p

-values are based on a normal approximation (Wald test). Significance levels: *

p<0.10

, **

p<0.05

, ***

p<0.01

Figure 2: Mortality outcome histogram for HR quartile two data subset.

Covariate	$\Delta\mu$	$\beta_{\text{Men}}$	$\beta_{\text{Women}}$
Intercept	$0.0000$	$1.1813$	$-0.8515$
Age	$3.8938$	$-0.0000$	$0.0039$
ICU Type	$0.0095$	$0.0173$	$0.0116$
HR	$0.4982$	$-0.0026$	$0.0056$
MAP	$-0.6669$	$0.0004$	$0.0010$
Temp	$0.3604$	$-0.0230$	$0.0028$

Appendix

Table 6: Covariate mean differences and group-specific coefficients in HR quartile two.

\Delta\mu

denotes the difference in covariate means between men and women within HR quartile two (men minus women).

\beta_{\text{Men}}

and

\beta_{\text{Women}}

are coefficients from group-specific linear probability models for in-hospital mortality.

Subset $S$	Group ( $H,K$ )
26 industries (NAICS 2)	Race (black, white)
50 states + D.C,	Sex (male, female)
	Nativity (native born, foreign born)

Appendix

Table 7: Settings for sign flip search in the U.S. labor force dataset.

	Either Component				Explained Component				Unxplained Component
Model	Total	$10\%$	$5\%$	$1\%$	Total	$10\%$	$5\%$	$1\%$	Total	$10\%$	$5\%$	$1\%$
Linear	35	14	11	7	24	10	8	5	11	4	3	2
lightgbm	29	-	-	-	23	-	-	-	6	-	-	-

Appendix

Table 8: Number of subgroups (out of

217

) exhibiting a sign flip in the explained and unexplained component, for the earnings outcome. Columns

10\%

5\%

, and

1\%

report the number of flipped data subsets that are significant at the corresponding level. No significance testing was performed for lightgbm due to computational costs of bootstrap inference.

	Either Component				Explained Component				Unxplained Component
Model	Total	$10\%$	$5\%$	$1\%$	Total	$10\%$	$5\%$	$1\%$	Total	$10\%$	$5\%$	$1\%$
Linear	60	35	33	24	44	33	31	23	17	2	2	1
Logistic	77	23	16	10	60	23	16	10	25	1	0	0
lightgbm	97	-	-	-	23	-	-	-	6	-	-	-

Appendix

Table 9: Number of subgroups (out of

183

) exhibiting a sign flip in the explained and unexplained component, for the (binary) health insurance status outcome. Columns

10\%

5\%

, and

1\%

report the number of flipped data subsets that are significant at the corresponding level. No significance testing was performed for lightgbm due to computational costs of bootstrap inference.

Group	Mean BMI	$\alpha$	$\beta$	Mean SBP
$H$	25.0	110.4	1.0	135.4
$K$	27.0	100.0	1.4	137.8

Appendix

Table 10: Key chosen and derived parameters by group. BMI and SBP are in kg/m 2 and mmHg respectively.

\alpha

is in mmHg;

\beta

is in mmHg per kg/m 2 .

Figure 3: Population linear models of SBP on BMI by group and data samples.

Reference	Explained	Unexplained	Total gap
$H$	$-2.0$	-0.4	$-2.4$
$K$	$-2.8$	0.4	$-2.4$

Appendix

Table 11: Population Oaxaca–Blinder decomposition of mean SBP differences. All values are in mmHg.

Figure 4: Percentage of parameter space leading to unexplained component sign flips, without covariate standardization.

Position: Explainable AI is Causality in Disguise
Mar 30, 2026Amir-Hossein KarimiExplainable Artificial Intelligence Causal Explanation
Department of Electrical & Computer Engineering, University of Waterloo & Vector Institute, Canada
Auditing Construct Overlap in Explainable Machine Learning: Evidence from Burnout-Depression Prediction Across Student Cohorts
Jul 12, 2026Alireza Dehghan, Negin AshrafiExplainability Evaluation
Sharif University of Technology, Tehran, Iran · University of Southern California, California, USA
Causal inference for group-contaminated structured outcomes: observable quotients, lossless reduction and exact randomization inference
Aug 12, 2026Usef Faghihi, Amir SakiCausal Inference
D´epartement de math´ematiques et d’informatique Universit´e du Qu´ebec `a Trois-Rivi`eres Trois-Rivi`eres, Qu´ebec, Canada

Mar 30, 2026J/K move · Enter open · S save

Position: Explainable AI is Causality in Disguise

Amir-Hossein Karimi

Department of Electrical & Computer Engineering, University of Waterloo & Vector Institute, Canada

Explainable Artificial Intelligence Causal Explanation

Open paper AskPDF arXiv

The demand for Explainable AI (XAI) has triggered an explosion of methods, producing a landscape so fragmented that we now rely on surveys of surveys. Yet, fundamental challenges persist: conflicting metrics, failed sanity checks, and unresolved debates over robustness and fairness. The only consensus on how to achieve explainability is a lack of one. This has led many to point to the absence of a ground truth for defining ``the'' correct explanation as the main culprit. This position paper posits that the persistent discord in XAI arises not from an absent ground truth but from a ground truth that exists, albeit as an elusive and challenging target: the causal model that governs the relevant system. By reframing XAI queries about data, models, or decisions as causal inquiries, we prove the necessity and sufficiency of causal models for XAI. We contend that without this causal grounding, XAI remains unmoored. Ultimately, we encourage the community to converge around advanced concept and causal discovery to escape this entrenched uncertainty.

Abstract

Explore similar work

Position: Explainable AI is Causality in Disguise

Auditing Construct Overlap in Explainable Machine Learning: Evidence from Burnout-Depression Prediction Across Student Cohorts

Causal inference for group-contaminated structured outcomes: observable quotients, lossless reduction and exact randomization inference