stat.MEJan 5, 2026

Environment-Adaptive Covariate Selection: Learning When to Use Spurious Correlations for Out-of-Distribution Prediction

Authors: Shuozhi Zuo, Yixin Wang

Organizations: Department of Statistics, University of Michigan, Ann Arbor

Abstract

A common approach to out-of-distribution prediction restricts models to causal or invariant covariates to avoid spurious associations that may change across environments. Despite its theoretical appeal, this strategy can underperform empirical risk minimization when only a subset of the causal parents of the outcome is observed. In such settings, non-causal covariates can serve as proxies for unobserved causal parents and improve prediction when the proxy relationship is stable, but they can hurt when shifts disrupt that relationship. Thus, the optimal covariate set can depend on the specific shift encountered. Because different shifts can leave distinct signatures in the covariate distribution, we propose an environment-adaptive covariate selection algorithm that chooses which covariates to use in each environment based on summaries of its unlabeled covariates. The summaries may be hand-crafted or learned from multi-environment data, and prior causal knowledge can be incorporated as constraints. We give sufficient conditions under which such summaries are informative in linear causal models, and explain when adapting only the covariate set is preferable to letting every coefficient vary with the environment. In simulations and two applications, adapting the covariate set improves prediction over static baselines, including causal and invariant covariate selection.

Figures & tables

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Bayesian Invariance Modeling of Multi-Environment Data

    Jun 27, 2025Luhuan Wu, Mingzhang Yin, Yixin Wang +2Bayesian Inference

  2. Beyond the Training Distribution: Evaluating Predictions Under Distribution Shift and Selection Bias

    Jun 12, 2026Annie Ulichney, Amanda CostonDouble MLDistribution Shift Robustness

  3. Robust Representation Learning through Explicit Environment Modeling

    Apr 28, 2026Yuli Slavutsky, David M. BleiRepresentation LearningDomain Generalization