Scientists often want to explain why an outcome is different in two groups. For instance, differences in patient mortality rates across two hospitals could be due to differences in the patients themselves (covariates) or differences in medical care (outcomes given covariates). The Oaxaca--Blinder decomposition (OBD) is a standard tool to tease apart these factors. It is well known that the OBD requires choosing one of the groups as a reference, and the numerical answer can vary with the reference. To the best of our knowledge, there has been no systematic investigation into whether the choice of OBD reference can yield different substantive conclusions and how common this issue is. In the present paper, we give existence proofs in real and simulated data that the OBD references can in fact yield substantively different conclusions. Our empirical exercises find that this sensitivity is more common when the OBD is extended to more complex regression models, including a pretrained transformer. Our theoretical and empirical results together establish that these conclusion reversals are not entirely driven by model misspecification, small data, or adversarial parameter choices. Our results suggest that practitioners should always report both directions of the OBD; that modern machine learning and large datasets do not automatically resolve the conclusion reversal problem; and that further work on this problem is needed.
Figures & tables
Model
Total flips
10%
5%
1%
Linear
27
2
2
2
Logistic
29
2
2
1
Neural net
62
29
17
6
XGBoost
62
17
10
3
TabPFN
76
22
19
5
Table 1: Number of data subsets (out of 139 ) exhibiting at least one sign flip (in either the explained or unexplained component), by model. Columns 10% , 5% , and 1% report the number of flipped subsets that are significant at the corresponding level.
Reference
Explained
Unexplained
Total gap
Women
0.021∗∗∗(0.007)
0.014(0.025)
0.035(0.023)
Men
−0.007(0.011)
0.042∗(0.025)
0.035(0.023)
Table 2: Oaxaca–Blinder decompositions for HR quartile two. The total gap is men’s mean minus women’s mean. We compute standard errors (in parentheses) via bootstrap resampling with 1,000 replications. We calculate p-values using a normal approximation (Wald test), where the test statistic is the coefficient divided by its bootstrap standard error. Significance levels: * p<0.10 , ** p<0.05 , *** p<0.01 .
Figure 1: Left: Linear fit of mortality on admission heart rate for each group when holding all other covariates fixed at their group means. Right: Percentage of parameter space leading to sign flips. See Remark D.2 for how we compute the percentage for the unexplained component.
Appendix figures & tables12 assets
Supplementary material from the paper’s appendix.
Appendix
Model
Total flips
10%
5%
1%
Linear
11
2
2
2
Logistic
9
2
2
1
Neural net
42
26
17
6
XGBoost
33
17
10
3
TabPFN
33
21
19
5
Appendix
Table 3: Number of data subsets (out of 139 ) exhibiting a sign flip in the explained component, by model. Columns 10% , 5% , and 1% report the number of flipped data subsets that are significant at the corresponding level.
Model
Total flips
10%
5%
1%
Linear
17
0
0
0
Logistic
21
0
0
0
Neural net
49
10
6
1
XGBoost
47
3
1
0
TabPFN
56
2
0
0
Appendix
Table 4: Number of data subsets (out of 139 ) exhibiting a sign flip in the unexplained component, by model. Columns 10% , 5% , and 1% report the number of flipped data subsets that are significant at the corresponding level.
Model
Gap (M–F)
Women reference
Men reference
Explained
Unexplained
Explained
Unexplained
Linear
0.035(0.023)
0.021∗∗∗(0.007)
0.014(0.025)
−0.007(0.010)
0.042∗(0.025)
Logistic
0.035(0.023)
0.026∗∗∗(0.009)
0.009(0.025)
−0.006(0.008)
0.041∗(0.024)
Neural net
0.035(0.023)
0.001(0.024)
0.033(0.031)
0.053∗∗(0.022)
−0.018(0.025)
XGBoost
0.035(0.023)
0.004(0.016)
0.031(0.025)
0.029∗∗(0.014)
0.006(0.021)
TabPFN
0.035(0.023)
0.012(0.016
0.023(0.026)
0.023∗(0.013)
0.012(0.021)
Appendix
Table 5: OBD of the gender mortality gap in HR quartile two ( n=816 ), across all five models. The total gap is men’s mean minus women’s mean. Standard errors in parentheses are computed via bootstrap resampling with 1,000 replications. p -values are based on a normal approximation (Wald test). Significance levels: * p<0.10 , ** p<0.05 , *** p<0.01 .
Figure 2: Mortality outcome histogram for HR quartile two data subset.
Covariate
Δμ
βMen
βWomen
Intercept
0.0000
1.1813
−0.8515
Age
3.8938
−0.0000
0.0039
ICU Type
0.0095
0.0173
0.0116
HR
0.4982
−0.0026
0.0056
MAP
−0.6669
0.0004
0.0010
Temp
0.3604
−0.0230
0.0028
Appendix
Table 6: Covariate mean differences and group-specific coefficients in HR quartile two. Δμ denotes the difference in covariate means between men and women within HR quartile two (men minus women). βMen and βWomen are coefficients from group-specific linear probability models for in-hospital mortality.
Subset S
Group ( H,K )
26 industries (NAICS 2)
Race (black, white)
50 states + D.C,
Sex (male, female)
Nativity (native born, foreign born)
Appendix
Table 7: Settings for sign flip search in the U.S. labor force dataset.
Either Component
Explained Component
Unxplained Component
Model
Total
10%
5%
1%
Total
10%
5%
1%
Total
10%
5%
1%
Linear
35
14
11
7
24
10
8
5
11
4
3
2
lightgbm
29
-
-
-
23
-
-
-
6
-
-
-
Appendix
Table 8: Number of subgroups (out of 217 ) exhibiting a sign flip in the explained and unexplained component, for the earnings outcome. Columns 10% , 5% , and 1% report the number of flipped data subsets that are significant at the corresponding level. No significance testing was performed for lightgbm due to computational costs of bootstrap inference.
Either Component
Explained Component
Unxplained Component
Model
Total
10%
5%
1%
Total
10%
5%
1%
Total
10%
5%
1%
Linear
60
35
33
24
44
33
31
23
17
2
2
1
Logistic
77
23
16
10
60
23
16
10
25
1
0
0
lightgbm
97
-
-
-
23
-
-
-
6
-
-
-
Appendix
Table 9: Number of subgroups (out of 183 ) exhibiting a sign flip in the explained and unexplained component, for the (binary) health insurance status outcome. Columns 10% , 5% , and 1% report the number of flipped data subsets that are significant at the corresponding level. No significance testing was performed for lightgbm due to computational costs of bootstrap inference.
Group
Mean BMI
α
β
Mean SBP
H
25.0
110.4
1.0
135.4
K
27.0
100.0
1.4
137.8
Appendix
Table 10: Key chosen and derived parameters by group. BMI and SBP are in kg/m 2 and mmHg respectively. α is in mmHg; β is in mmHg per kg/m 2 .
Figure 3: Population linear models of SBP on BMI by group and data samples.
Reference
Explained
Unexplained
Total gap
H
−2.0
-0.4
−2.4
K
−2.8
0.4
−2.4
Appendix
Table 11: Population Oaxaca–Blinder decomposition of mean SBP differences. All values are in mmHg.
Figure 4: Percentage of parameter space leading to unexplained component sign flips, without covariate standardization.
The demand for Explainable AI (XAI) has triggered an explosion of methods, producing a landscape so fragmented that we now rely on surveys of surveys. Yet, fundamental challenges persist: conflicting metrics, failed sanity checks, and unresolved debates over robustness and fairness. The only consensus on how to achieve explainability is a lack of one. This has led many to point to the absence of a ground truth for defining ``the'' correct explanation as the main culprit. This position paper posits that the persistent discord in XAI arises not from an absent ground truth but from a ground truth that exists, albeit as an elusive and challenging target: the causal model that governs the relevant system. By reframing XAI queries about data, models, or decisions as causal inquiries, we prove the necessity and sufficiency of causal models for XAI. We contend that without this causal grounding, XAI remains unmoored. Ultimately, we encourage the community to converge around advanced concept and causal discovery to escape this entrenched uncertainty.
Amir-Hossein Karimi
Department of Electrical & Computer Engineering, University of Waterloo & Vector Institute, Canada
Explainable machine learning (XML) pipelines applied to composite mental health outcomes can produce apparently-robust, cross-population-stable risk hierarchies that are largely artefacts of how the outcome was constructed. We demonstrate this using an ElasticNet pipeline applied to 886 medical students at the University of Lausanne (primary cohort, 2022), validated across 2,580 longitudinal observations at three time points and 701 non-medical students from eight faculties; all three datasets share identical instruments. The pipeline produces a hierarchy in which trait anxiety and health satisfaction dominate wherever the outcome is measured, with Kendall τ=1.0 for the top-two positions across all five evaluation sets and consistent transfer performance (R2: 0.41-0.49). Two residualization experiments, which isolate shared variance between correlated variables via regression, reveal the mechanism: when trait anxiety (STAI-T) is residualized against the co-included depression subscale (CES-D, r=0.72), model R2 drops from 0.41 to 0.16 and STAI-T falls from rank 1 to rank 6; when burnout subscales are residualized against CES-D, R2 collapses to 0.016. Prediction intervals average 35.4 units on a 0-100 scale (2.4 outcome standard deviations), independently ruling out individual-level deployment. The residualization protocol is the paper's transferable contribution: any XAI study combining correlated predictor and outcome constructs should apply this check before interpreting apparent stability as a finding.
Alireza Dehghan, Negin Ashrafi
Sharif University of Technology, Tehran, Iran · University of Southern California, California, USA
Structured potential outcomes such as microscopy images may be recorded after an unknown, unit-specific transformation. If that transformation can depend on treatment, covariates or the intrinsic outcome, raw-coordinate analyses may mix biological effects with acquisition geometry. We study the unrestricted observation model X = Γ . Y(A) and characterize its observable information: a target is uniformly recoverable exactly when it is constant on group orbits, while a Borel maximal invariant retains every measurable invariant target. We then distinguish observability from statistical losslessness. A quotient-faithful reconstruction theorem shows that quotient reduction is sufficient for the full transformed experiment exactly when the conditional law of the raw observation given treatment, covariates and the quotient has a parameter-free version. Conditional Haar contamination on a compact group yields Blackwell equivalence as a special case; it is not imposed in the main model. We also separate independent site-specific product actions from shared diagonal actions and show why componentwise canonicalization can discard relative cross-site information. Under explicit metric and kernel regularity, an approximate-contamination theorem bounds quotient-law Wasserstein error and the induced perturbation of population maximum mean discrepancy. For finite-support multichannel lattice images, we construct a maximal invariant under integer translations and quarter turns, combine its characteristic Gaussian kernel with a complete paired-swap test, and retain the original simulations and RxRx1 HUVEC study. Under the sharp null, the quotient test rejected in 0.052 of simulation replicates; at unit effect strength its power was 0.992. The primary RxRx1 contrast had an enumerated paired-swap p-value of 0.0078.
Usef Faghihi, Amir Saki
D´epartement de math´ematiques et d’informatique Universit´e du Qu´ebec `a Trois-Rivi`eres Trois-Rivi`eres, Qu´ebec, Canada