The problem of domain generalization concerns learning predictive models that are robust to distribution shifts when deployed in new, previously unseen environments. Existing methods typically require labeled data from multiple training environments, limiting their applicability when labeled data are scarce. In this work, we study domain generalization in an anti-causal setting, where the outcome causes the observed covariates. Under this structure, environment perturbations that affect the covariates do not propagate to the outcome, which motivates regularizing the model's sensitivity to these perturbations. Crucially, estimating these perturbation directions does not require labels, enabling us to leverage unlabeled data from multiple environments. We propose two methods that penalize the model's sensitivity to variations in the mean and covariance of the covariates across environments, respectively, and prove that these methods have worst-case optimality guarantees under certain classes of environments. Finally, we demonstrate the empirical performance of our approach on a controlled physical system and a physiological signal dataset.
Figures & tables
Figure 1: Illustration of MIR. Blue points represent the training environment perturbation means \EX[εei] , and the green star represents the test environment mean. The red ellipse represents the covariance structure of the regularization matrix Var(K) , with red arrows indicating its (scaled) eigenvectors. The annotated arrows show the directions of the OLS and MIR solutions with different regularization strengths γ , as well as the optimal solution βtest∗ for the test environment etst .
Figure 2: Diagram of the light tunnel and the subset of variables used in our experiment: the outcomes or intervention targets red , green and blue , and the light-intensity measurements ir_1 , vis_1 , …, vis_3 used as predictors. Figure adapted from Gamella et al. (2025) , licensed under CC BY 4.0.
Figure 3: Performance on the Light Tunnel dataset. We show average RMSE (with standard errors) for leave-one-environment-out cross-validation across all outcome-intervention combinations. The x-axis indicates the number of environments with labeled observations; MIR uses unlabeled data from all training environments regardless of this number.
Figure 4: Performance on VitalDB dataset when all 128 training subjects have labeled data. (a) CVaR (average nMSE for subjects whose errors are above a given quantile) as a function of quantile threshold. (b) Distribution of per-subject Spearman’s correlations between predicted and true stroke volume variations. VIR achieves improved robustness, especially on worse-performing subjects, while maintaining tracking performance comparable to baselines.
Figure 5: Performance on VitalDB dataset as a function of the number of labeled environments. VIR improves over baselines across all settings. The improvement is most pronounced when few labeled environments are available.
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 6: Stress-tests on violations of causal assumptions
Figure 7: Effect of unlabelled environments in VitalDB
Machine learning models often degrade when they are deployed on a target distribution that differs from the source distributions they were trained on. Recent work in causality-based domain generalization has shown how shared causal structure between domains can induce invariant predictors, e.g., models on a subset of features which have stable risk across structured domain shifts. However, the extent to which such population-level causal invariances can lead to gains in finite-sample settings remains underexplored. In particular, in practice we often have access to a few labeled target samples, a setting called supervised domain adaptation (sDA). In this paper, we explore when (full or partial) causal knowledge can provably improve supervised domain adaptation. As a first step, we study linear regression, where full or partial causal knowledge specifies a collection of invariant or possibly invariant feature subsets, each yielding a source-trained candidate predictor. We derive matching upper and lower bounds showing that finite-sample gains are governed by the target-risk margins separating the candidates, together with the finite-source estimation error. When these margins are sufficiently large relative to nQ, an adaptive aggregation procedure can match the best candidate predictor while avoiding negative transfer relative to target-only learning. On the other hand, when the margins are too small, no algorithm can reliably exploit the candidate collection to obtain faster finite-sample rates. We further connect these margins to structural shift magnitude in linear SCMs and validate the theory on real-world causal benchmarks.
Julia Kostin, Kasra Jalaldoust, Elias Bareinboim +2
Department of Computer Science, ETH Zurich · Causal Artificial Intelligence Lab, Columbia University · Department of Statistics, Columbia University
Domain generalization (DG) aims to learn from multiple source domains and generalize to unseen target domains. Most DG methods pursue invariance: they seek a causal representation whose prediction rule is invariant across domains. This principle is effective when the causal mechanism is stable, but becomes restrictive when the domain itself modulates how causal content maps to the response. In this case, directly feeding domain style into the predictor can create misleading shortcuts, since style does not by itself cause the response. Yet the apparent chaos of multiple styles can become a ladder: style can locate the unseen target domain among source domains and guide which domain-dependent prediction rules should be trusted. We propose Latent Adaptive Domain Disentanglement and Environment Reweighting (LADDER), a fixed-model DG pipeline that learns causal/style representations, freezes the encoders, fits source-specific classifiers, and uses an unlabeled target-domain covariate set only at inference to compute weights over these fixed classifiers, with no target labels or model-state updates. We establish theoretical guarantees for source reweighting and validate LADDER on simulations, FMoW, and a location-grouped iWildCam protocol, with gains in overall and group-averaged accuracy.
Yuhang Jiang, Fengchuan Zhang, Sanguo Zhang +1
School of Mathematical Sciences, University of Chinese Academy of Sciences, Beijing, China · Key Laboratory of Big Data Mining and Knowledge Management, Chinese Academy of Sciences, Beijing, China
Developing predictive models that perform reliably across diverse patient populations and heterogeneous environments is a core aim of medical research. However, generalization is only possible if the learned model is robust to statistical differences between data used for training and data seen at the time and place of deployment. Domain generalization methods provide strategies to address data shifts, but each method comes with its own set of assumptions and trade-offs. To apply these methods in healthcare, we must understand how domain shifts arise, what assumptions we prefer to make, and what our design constraints are. This article proposes a causal framework for the design of predictive models to improve generalization. Causality provides a powerful language to characterize and understand diverse domain shifts, regardless of data modality. This allows us to pinpoint why models fail to generalize, leading to more principled strategies to prepare for and adapt to shifts. We recommend general mitigation strategies, discussing trade-offs and highlighting existing work. Our causality-based perspective offers a critical foundation for developing robust, interpretable, and clinically relevant AI solutions in healthcare, paving the way for reliable real-world deployment.
Kurt Butler, Stephanie Riley, Damian Machlanski +13
Causality in Healthcare AI Hub (CHAI), UK · The University of Edinburgh, UK · University of Manchester, UK +6