How Wrong Can Your Counterfactual Be? Quantifying Confounding Bias for Continuous Treatments without a Control Group

Yu Wang; Xiangchen Liu; Siguang Li

cs.AIMar 8, 2026

How Wrong Can Your Counterfactual Be? Quantifying Confounding Bias for Continuous Treatments without a Control Group

Authors: Yu Wang, Xiangchen Liu, Siguang Li

Organizations: Department of Economics, Cornell University. · Department of Family and Consumer Sciences, California State University, Long Beach. · Society Hub, Hong Kong University of Science and Technology (Guangzhou).

Abstract

Stress testing poses a causal question: how would portfolio credit losses change if the macroeconomy followed an adverse counterfactual path? Yet standard practice remains predictive and might be therefore vulnerable to omitted-variable bias. We propose a partial identification framework for causal stress testing in panel data with a continuous common treatment and no control group. By assuming that the unobserved confounder affects outcome and macro variables additively, we derive a closed-form confounding envelope parameterized by two interpretable sensitivity parameters. We further analyze two practical estimators -- recursive rollout and direct multi-horizon prediction -- derive non-asymptotic error bounds, and characterize when recursive compounding makes direct estimation preferable. For inference, we combine the identification envelope with importance-weighted conformal prediction, yielding finite-sample intervals that separate estimation uncertainty from identification uncertainty under covariate shift. In semi-synthetic experiments built from real U.S. unemployment paths, standard high-accuracy predictive models remain causally biased and substantially under-cover, whereas the proposed framework achieves near-nominal coverage across stress horizons.

Figures & tables

Symbol	Value	Description	Symbol	Value	Description
$\alpha$	1.0	Intercept	$\phi_{U}$	0.85	Confounder persistence
$\beta_{1}$	0.85	Outcome persistence	$\sigma_{\nu}$	0.5	Confounder innov. std
$\beta_{2}$	0.15	Macro sensitivity	$(\gamma_{A},\gamma_{Y})$	varies	Confounding strength
$\beta_{3}$	0.3	Cross-sect. covariate
$\beta_{4}$	0.02	Quadratic (nonlin.)	$\hat{\phi}_{A}$	0.995	AR(1) coeff. (fitted)
$\beta_{5}$	0.12	Threshold (nonlin.)	$\hat{\mu}_{A}$	5.84	Mean unemp. rate (%)

Table 1: DGP parameters. Left: outcome model. Right-top: confounder (synthetic). Right-bottom: macro (fitted from unemployment rate).

Panel of Figure 1 — Figure 1: Silent failure of predictive stress testing. Top: Population-level confounding bias ( $\gamma_{A}=0.5$ ); $\mu^{*}_{12}$ is invariant to $\gamma_{Y}$ while $\mu_{12}$ rises linearly. Middle: Oracle rollout at $\gamma=0.5$ ; causal truth $\mu^{*}_{h}$ lies 58–212 SE outside the predictive band. Bottom: Backtesting $R^{2}$ is flat (0.974 $\to$ 0.957) while $c_{12}$ jumps from 0 to 0.61.

Panel of Figure 2 — Figure 2: Coverage heatmaps at $\gamma=0.4$ , nominal 90%. Left: linear DGP + OLS (5 methods). Right: nonlinear DGP + MLP (3 methods). Ours (full) is the only method achieving 100% across all horizons (linear) or $\geq 90\%$ at short horizons (nonlinear).

Figure 3: CI half-width decomposition at $\gamma=0.5$ . Blue: $q^{w}_{h}$ (estimation + reweighting). Red: $c_{h}$ (identification). (a) Linear DGP: $c_{h}$ 78% at $h=1$ . (b) Nonlinear DGP: same $c_{h}$ , larger $q^{w}_{h}$ .

Panel of Figure 4 — Figure 4: NFCI-calibrated coverage ( $\gamma_{A}=\gamma_{Y}=0.091$ , nominal 90%). Left : linear DGP + OLS. Right : nonlinear DGP + MLP. All confounder parameters estimated from NFCI. Ours (full) is the only method with coverage $\geq 90\%$ under NFCI-calibrated confounding.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Symbol	Value	Description
$\alpha$	1.0	Intercept
$\beta_{1}$	0.85	Outcome persistence
$\beta_{2}$	0.15	Macro sensitivity
$\beta_{3}$	0.3	Cross-sectional covariate
$\beta_{4}$	0.02	Quadratic macro response (nonlinear only)
$\beta_{5}$	0.12	Threshold amplifier (nonlinear only)

Appendix

Table 2: DGP parameters. Top: outcome model. Middle: confounder (synthetic). Bottom: macro (fitted from UNRATE 1990–2024).

Causal Effect Estimation

Causal effect estimation measures how outcomes would change under interventions, using experiments or observational data and explicit assumptions.

See all 264 papers

When Prediction Error Is Not Enough: Evaluating Nuisance-Function Prediction for Causal Estimation
Aug 30, 2026Cong CaoSemiparametric Inference Causal Effect Estimation
Department of Biostatistics, Yale University, New Haven, CT 06520, USA
Debiased Machine Learning for Conformal Prediction of Counterfactual Outcomes Under Runtime Confounding
Apr 4, 2026Keith Barnatchez, Kevin P. Josey, Rachel C. Nethery +1Counterfactual Prediction Conformal Prediction
Department of Biostatistics, Harvard T.H. Chan School of Public Health · Department of Biostatistics and Informatics, Colorado School of Public Health · Department of Data Science, Dana Farber Cancer Institute
Conformal Individual Treatment Effect Estimation under Networked Interference
Sep 14, 2026Matteo Zecchin, Osvaldo SimeoneWeighted Conformal Prediction Causal Effect Estimation
Communication Systems Department, EURECOM, Sophia Antipolis, France · Institute for Intelligent Networked Systems (INSI), Northeastern University London, London, UK

Aug 30, 2026J/K move · Enter open · S save

When Prediction Error Is Not Enough: Evaluating Nuisance-Function Prediction for Causal Estimation

Cong Cao

Department of Biostatistics, Yale University, New Haven, CT 06520, USA

Semiparametric Inference Causal Effect Estimation Debiased ML

Open paper AskPDF arXiv

Prediction error is widely used to evaluate nuisance-function estimators in causal inference, but its relationship with the performance of the resulting causal estimator is not well understood. We studied this question in a partially linear model using Monte Carlo simulations, comparing ordinary least squares (OLS), generalized additive models (GAMs), XGBoost, and Double Machine Learning with XGBoost (DML-XGBoost) with independent and with clustered observations. We evaluated nuisance-function prediction error, bias, RMSE, and 95% confidence interval coverage, and derived the second-order remainder through which errors in the exposure and outcome nuisance functions jointly determine the bias of the residualized estimator. When all methods estimated the causal effect from the same training sample, DML-XGBoost had the lowest RMSE and the highest coverage among the non-oracle methods, although XGBoost fitted to the full training sample had lower out-of-sample prediction error for both nuisance functions. GAMs had the lowest prediction error of all methods but were biased, with coverage of 0.55 with independent observations. The remainder reproduced the average bias of OLS, GAMs, and DML-XGBoost, but only half to three-quarters of the bias of XGBoost, whose residuals were formed on the data used to fit it. A joint-error measure based on the absolute cross-product of the two nuisance errors was only weakly associated with causal bias. Prediction error is useful for assessing nuisance-function estimation, but it should not be treated as a direct measure of the quality of the resulting causal estimator.

Abstract

Causal Effect Estimation

Explore similar work

When Prediction Error Is Not Enough: Evaluating Nuisance-Function Prediction for Causal Estimation

Debiased Machine Learning for Conformal Prediction of Counterfactual Outcomes Under Runtime Confounding

Conformal Individual Treatment Effect Estimation under Networked Interference