Deconfounding Scores and Representation Learning for Causal Effect Estimation with Weak Overlap

Oscar Clivio; Alexander D'Amour; Alexander Franks; David Bruns-Smith; Chris Holmes; Avi Feller

stat.MLApr 1, 2026

Deconfounding Scores and Representation Learning for Causal Effect Estimation with Weak Overlap

Authors: Oscar Clivio, Alexander D'Amour, Alexander Franks, David Bruns-Smith, Chris Holmes, Avi Feller

Organizations: University of California, Berkeley

Abstract

Overlap, also known as positivity, is a key condition for causal treatment effect estimation. Many popular estimators suffer from high variance and become brittle when features differ strongly across treatment groups. This is especially challenging in high dimensions: the curse of dimensionality can make overlap implausible. To address this, we propose a class of feature representations called deconfounding scores, which preserve both identification and the target of estimation; the classical propensity and prognostic scores are two special cases. We characterize the problem of finding a representation with better overlap as minimizing an overlap divergence under a deconfounding score constraint. We then derive closed-form expressions for a class of deconfounding scores under a broad family of generalized linear models with Gaussian features and show that prognostic scores are overlap-optimal within this class. We conduct extensive experiments to assess this behavior empirically.

Figures & tables

Figure 1: The projection of

\gamma

onto the space spanned by

\alpha

and

\beta

lies on a segment of a hyperbola (bold black line) whose endpoints correspond to

\gamma=\alpha

and

\gamma=\beta

(when

\alpha^{\prime}\beta\geq 0

) or to

\gamma=\alpha

and

\gamma=-\beta

(when

\alpha^{\prime}\beta<0

), shown here, and the opposite segment, not shown here. The orientation of the hyperbola and the endpoints depends on

\alpha^{\prime}\beta

LASSO outcome and propensity
Overlap	High		Low
SNR	Low	High	Low	High
IPW	0.98	2.45	1.45	3.59
AIPW	0.35	0.39	0.79	1.04
Regr	0.4	0.43	1	1.26
IPW- $\hat{\beta}$	0.66	1.58	1.1	2.46

Table 1: RMSEs on simulated datasets. Green :

\phi_{\hat{\gamma}}(X)

improves over

X

for the same base method; Red : it is worse . Best performance underlined . See Section 5.1 for definitions of names of estimators.

Figure 2: RMSE, bias and standard deviation for simulated datasets. Each metric is plotted according to the deconfounding score coordinate parameter

w

; methods using base covariates are constant. In all plots,

s_{T}=4

(low overlap) and

s_{Y}=5

(high SNR). See Section 5.1 for definitions of names of estimators.

LASSO outcome and propensity
Dataset	IHDP	ACIC2016	HC-MNIST
IPW	2.35	2.29	0.2
AIPW	2.41	2.09	0.22
Regr	2.41	0.62	0.22
IPW- $\hat{\beta}$	2.46	2.48	0.18
AIPW- $\hat{\beta}$	2.43	2.41	0.22

Table 2: RMSEs on semi-synthetic datasets. Green :

\phi_{\hat{\gamma}}(X)

improves over

X

for the same base method; Red : it is worse . Best performance underlined . See Section 5.1 for definitions of names of estimators.

Causal Effect Estimation

Causal effect estimation measures how outcomes would change under interventions, using experiments or observational data and explicit assumptions.

See all 264 papers

Supervised Learning Objectives for High-Dimensional Treatment Effects
May 26, 2026Nikita Dhawan, Arnav Paruthi, Andrew Kim +3Causal Effect Estimation Causal Inference
University of Toronto, Vector Institute · University of Toronto · Vanguard
The Confounder Trap: Treatment-Encoding Representations in Causal Inference with Text
Jul 28, 2026Marie Neubrander, Graham Tierney, Alexander VolfovskyCausal Effect Estimation Causal Representation Learning
Department of Statistical Science Duke University Durham, NC, 27705
Representation Learning for Sample-Efficient CATE Estimation by Leveraging Multiple Outcomes
Sep 5, 2026Maitreyi Swaroop, Shikha Bhat, Samantha Rodriguez +2Causal Effect Estimation Conditional Average Treatment Effect Estimation
Machine Learning Department, Carnegie Mellon University · Department of General Internal Medicine, University of Pittsburgh

May 26, 2026J/K move · Enter open · S save

Supervised Learning Objectives for High-Dimensional Treatment Effects

Nikita Dhawan, Arnav Paruthi, Andrew Kim, Lovedeep Gondara+2

University of Toronto, Vector Institute · University of Toronto · Vanguard

Causal Effect Estimation Causal Inference

Open paper AskPDF arXiv

Predicting the effect of interventions with many possible variations, e.g., therapeutic content that affects mental health outcomes or an earnings call transcript that drives movement in share price, is useful across several domains. However, classical causal estimators tend to assume that all possible interventions are observed, which is infeasible when interventions vary widely, for instance, in the space of all text strings. We adapt a well-known approach of recasting causal inference as a learning problem to address such settings, assuming realizability of the learned function and no unobserved confounding. We show that causal error decomposes into moment-balancing errors of increasing order and exploit structure of the conditional outcome in the confounders to design objectives that directly improve causal estimation. We also show how to project the effect of a high-dimensional treatment onto coarser attributes, which allows a single model to answer several causal questions without additional attribute-specific training. We empirically evaluate our estimators in settings with high-dimensional continuous, discrete, and text treatments. Our experiments demonstrate generalization to held-out treatments, benefits of balancing higher-order moments when required, and projected causal estimates that are competitive with attribute-specific ones.

Abstract

Causal Effect Estimation

Explore similar work

Supervised Learning Objectives for High-Dimensional Treatment Effects

The Confounder Trap: Treatment-Encoding Representations in Causal Inference with Text

Representation Learning for Sample-Efficient CATE Estimation by Leveraging Multiple Outcomes