Environment-Adaptive Covariate Selection: Learning When to Use Spurious Correlations for Out-of-Distribution Prediction
Organizations: Department of Statistics, University of Michigan, Ann Arbor
Abstract
A common approach to out-of-distribution prediction restricts models to causal or invariant covariates to avoid spurious associations that may change across environments. Despite its theoretical appeal, this strategy can underperform empirical risk minimization when only a subset of the causal parents of the outcome is observed. In such settings, non-causal covariates can serve as proxies for unobserved causal parents and improve prediction when the proxy relationship is stable, but they can hurt when shifts disrupt that relationship. Thus, the optimal covariate set can depend on the specific shift encountered. Because different shifts can leave distinct signatures in the covariate distribution, we propose an environment-adaptive covariate selection algorithm that chooses which covariates to use in each environment based on summaries of its unlabeled covariates. The summaries may be hand-crafted or learned from multi-environment data, and prior causal knowledge can be incorporated as constraints. We give sufficient conditions under which such summaries are informative in linear causal models, and explain when adapting only the covariate set is preferable to letting every coefficient vary with the environment. In simulations and two applications, adapting the covariate set improves prediction over static baselines, including causal and invariant covariate selection.
Figures & tables
| Symbol | Description |
| Class of admissible interventions on the covariates | |
| Shift class: environments generated by interventions in | |
| Intervention identifying environment | |
| Full observed covariate vector | |
| Finite library of candidate subsets of coordinates of | |
| Binary mask indicating which coordinates of are selected |
| Setting | EACS | Contextual | Reduction (%) |
| Reference | 2.11 | 1.81 | |
| 4 irrelevant covariates | 2.13 | 2.07 | |
| 5 irrelevant covariates | 2.14 | 2.35 | 8.9 |
| 8 irrelevant covariates | 2.15 | 3.02 | 28.8 |
| Training perturbations up to 2.4 | 2.08 | 2.14 | 3.0 |
| Training perturbations up to 2 | 2.11 | 2.36 | 10.9 |
Appendix figures & tables12 assets
Supplementary material from the paper’s appendix.
Appendix
| Factor | Setting | EACS | ||
| Reference | 2.11 | 3.38 | 2.29 | |
| Training environments | 5 per shift type, | 2.15 | 3.38 | 2.30 |
| 5 per shift type, | 2.80 | 3.52 | 2.49 | |
| Observations | 5 per environment, | 2.20 | 3.40 | 2.32 |
| 5 per environment, | 3.96 | 4.08 | 3.34 | |
| Summary | SD of only | 2.36 | 3.38 | 2.29 |
| Method | Mean MSE | Fold SD |
| EACS (learned summary) | 0.6140 | 0.0205 |
| EACS (fixed marginal summary) | 0.6152 | 0.0221 |
| Contextual (learned summary) | 0.6134 | 0.0196 |
| Contextual (fixed marginal summary) | 1.1792 | 0.8998 |
OOD Generalization
Out-of-distribution generalization studies how machine-learning models perform when test data, domains, tasks, or environments differ from training.
See all 426 papers