Overlap, also known as positivity, is a key condition for causal treatment effect estimation. Many popular estimators suffer from high variance and become brittle when features differ strongly across treatment groups. This is especially challenging in high dimensions: the curse of dimensionality can make overlap implausible. To address this, we propose a class of feature representations called deconfounding scores, which preserve both identification and the target of estimation; the classical propensity and prognostic scores are two special cases. We characterize the problem of finding a representation with better overlap as minimizing an overlap divergence under a deconfounding score constraint. We then derive closed-form expressions for a class of deconfounding scores under a broad family of generalized linear models with Gaussian features and show that prognostic scores are overlap-optimal within this class. We conduct extensive experiments to assess this behavior empirically.
Figures & tables
Figure 1: The projection of γ onto the space spanned by α and β lies on a segment of a hyperbola (bold black line) whose endpoints correspond to γ=α and γ=β (when α′β≥0 ) or to γ=α and γ=−β (when α′β<0 ), shown here, and the opposite segment, not shown here. The orientation of the hyperbola and the endpoints depends on α′β .
Overlap
High
Low
SNR
Low
High
Low
High
LASSO outcome and propensity
IPW
0.98
2.45
1.45
3.59
AIPW
0.35
0.39
0.79
1.04
Regr
0.4
0.43
1
1.26
IPW- β^
0.66
1.58
1.1
2.46
Table 1: RMSEs on simulated datasets. Green : ϕγ^(X) improves over X for the same base method; Red : it is worse . Best performance underlined . See Section 5.1 for definitions of names of estimators.
Figure 2: RMSE, bias and standard deviation for simulated datasets. Each metric is plotted according to the deconfounding score coordinate parameter w ; methods using base covariates are constant. In all plots, sT=4 (low overlap) and sY=5 (high SNR). See Section 5.1 for definitions of names of estimators.
Dataset
IHDP
ACIC2016
HC-MNIST
LASSO outcome and propensity
IPW
2.35
2.29
0.2
AIPW
2.41
2.09
0.22
Regr
2.41
0.62
0.22
IPW- β^
2.46
2.48
0.18
AIPW- β^
2.43
2.41
0.22
Table 2: RMSEs on semi-synthetic datasets. Green : ϕγ^(X) improves over X for the same base method; Red : it is worse . Best performance underlined . See Section 5.1 for definitions of names of estimators.
Predicting the effect of interventions with many possible variations, e.g., therapeutic content that affects mental health outcomes or an earnings call transcript that drives movement in share price, is useful across several domains. However, classical causal estimators tend to assume that all possible interventions are observed, which is infeasible when interventions vary widely, for instance, in the space of all text strings. We adapt a well-known approach of recasting causal inference as a learning problem to address such settings, assuming realizability of the learned function and no unobserved confounding. We show that causal error decomposes into moment-balancing errors of increasing order and exploit structure of the conditional outcome in the confounders to design objectives that directly improve causal estimation. We also show how to project the effect of a high-dimensional treatment onto coarser attributes, which allows a single model to answer several causal questions without additional attribute-specific training. We empirically evaluate our estimators in settings with high-dimensional continuous, discrete, and text treatments. Our experiments demonstrate generalization to held-out treatments, benefits of balancing higher-order moments when required, and projected causal estimates that are competitive with attribute-specific ones.
Nikita Dhawan, Arnav Paruthi, Andrew Kim +3
University of Toronto, Vector Institute · University of Toronto · Vanguard
Estimating causal effects of linguistic properties from observational text is difficult because the same document can contain both the treatment of interest and the non-treatment textual attributes needed for adjustment. Existing approaches often learn representations from the full text to capture latent confounding, but when treatment status is itself encoded by words in the text, these representations can directly encode treatment. This creates a confounder trap: richer representations can make treated and control documents separable, inducing overlap violations even when the underlying causal problem satisfies overlap. We study latent text treatments that are encoded through lexicons or other treatment-defining lexical information, and propose masking-based adjustment representations that remove this lexical treatment signal before representation learning. We formalize representation-induced overlap failure, prove that deletion masking preserves overlap for bag-of-words/topic-model representations, and characterize replacement masking as a natural relaxation for large language models that hides treatment-defining tokens while preserving word order and context. Across simulations, masking improves overlap diagnostics, stabilizes treatment effect estimates, and reduces bias relative to adjustment methods that learn from the unmasked text.
Marie Neubrander, Graham Tierney, Alexander Volfovsky
Department of Statistical Science Duke University Durham, NC, 27705
Estimating conditional average treatment effects (CATE) enables efficient targeting of interventions, but many applications have limited experimental samples, making it difficult to estimate heterogeneous effects from high-dimensional covariates. In such settings, policymakers and medical practitioners often succumb to the curse of dimensionality or apply off-the-shelf dimension reduction methods that may not preserve treatment heterogeneity. Yet these domains often come with large historical datasets measuring a wide range of outcomes -- a source of supervision that is rarely exploited in practice. Following causal representation learning, we hypothesize that such domains with high-dimensional covariates have lower-dimensional underlying dynamics. We can thus leverage the diverse outcomes measured in historical data to learn a lower-dimensional representation of the covariates. Theoretically, we prove that when the auxiliary outcomes satisfy a set of surrogacy conditions and the representation retains relevant covariate information, the original CATE is identified when the high-dimensional covariates are replaced by the learned representation. Combined with existing dimension-dependent rates for CATE estimation, the result implies greater sample-efficiency on the same experimental sample. Additionally, we characterize the bias-variance tradeoff when the assumptions do not hold perfectly, and show that the representation-based estimator can still achieve lower error when the reduction in estimator variance outweighs the bias due to compression. Empirically, we evaluate the method on synthetic data and semi-synthetic medical data.