Chronological age predictors often fail to achieve out-of-distribution (OOD) gen- eralization due to exogenous attributes such as race, gender, or tissue. Learning an invariant representation with respect to those attributes is therefore essential to improve OOD generalization and prevent overly optimistic results. In predic- tive settings, these attributes motivate bias mitigation; in causal analyses, they appear as confounders; and when protected, their suppression leads to fairness. We coherently explore these concepts with theoretical rigor and discuss the scope of an interpretable neural network model based on adversarial representation learning. Using publicly available mouse transcriptomic datasets, we illustrate the behavior of this model relative to conventional machine learning models. We observe that the outcome of this model is consistent with the predictive results of a published study demonstrating the effects of Elamipretide on mouse skeletal and cardiac muscle. We conclude by discussing the limitations of deriving causal interpretation from such purely predictive models.
Figures & tables
Notation
Description
R
Set of real numbers
R≥0
Set of nonnegative real numbers
Z
Set of integers
⌈x⌉
smallest integer greater than or equal to x
(Ω,F,P)
Underlying probability space
X
Input (feature) space
Table 1: List of notations used in this work
Figure 1: Different associations between X and Y as adopted from Figure 12 of [ 7 ] . A marginal correlation is the weakest form association that ignores dependencies among covariates. A stronger form is the regression relevant coefficients which captures partial correlation (non-zero correlation coefficients). Causal variables are a subset of regression variables under faithfulness assumption (A must non-zero coefficient of all parents (direct cause) of Y . Invariance set is identifiable even when parents of Y not forming a diluted notion of causality. This set is much more useful in practical applications as stated in [ 7 ] . Causal form of association is desirable but often hard to achieve.
Figure 2: For detailed caption, see Section 12
Figure 3: For detailed caption, see Section 12
Figure 4: For detailed caption, see Section 12
Figure 5: For detailed caption, see Section 12
Module
Layer
Dimension / Description
Feature Encoder: FE:X→ F
Input
Input layer
(d) features
Binary Stochastic Filter
Bernoulli mask per feature (learned prob.)
Block 1
Dense + BN + ReLU
d→256
Block 2
Dense + BN + ReLU + Dropout
256→256
Block 3
Dense + BN + ReLU + Dropout
256→106
Table 2: Summary of the encoder, predictor, and adversarial bias network architecture as depicted in Figure 2 A.
Figure SA1: Recovery of attributes in S using Post-hoc Analysis Balanced accuracy of a linear probe ( LogisticRegression ) trained on the learned latent representation Z to predict attributes: sex, tissue, platform, and series ID, for each holdout dataset. Bars correspond to probe performance for different adversarial strengths α . Red dashed lines indicate the permutation baseline, obtained by training the probe after randomly shuffling the attribute labels, thereby representing chance-level performance. A decreasing balanced accuracy with increasing α suggests partial suppression of confounder-related information in Z . However, residual predictability remains, indicating that attribute signals are not fully eliminated from the representation. This observation is consistent with prior work showing that adversarial objectives do not guarantee complete removal of attribute information, and that post-hoc classifiers may still recover such signals from debiased representations [ 30 ] (see Discussions).
Figure SA2: Regression plots of fold-averaged predictions without the filtering layer (Figure 2 A) for each leave-one-dataset-out (LOSO) holdout setup. Each row corresponds to a different holdout dataset (GSE111164, GSE145480, GSE75192), and each column represents a different domain-adversarial strength ( α ). For every subplot, predictions from all LOSO folds were aggregated and averaged per biological sample. Points represent the mean predicted age versus chronological age. Performance metrics (MAE and R2 ) are computed using the averaged predictions. The diagonal line indicates the identity line ( y=x ).
Figure SA3: Regression plots of fold-averaged predictions without the filtering layer (Figure 3 A) for each leave-one-dataset-out (LOSO) holdout setup. Figure generation follows the same procedure as in Figure SA2
Figure SA4: Residual (absolute value) plot for by tissue fold-averaged predictions for GSE75192 as holdout. Predictions from all cross-validation folds were averaged for each sample before computing absolute value of residuals ( ∣ypred−ytrue∣ ). Boxplots show the distribution of residuals across tissues for models trained with and without the feature filter under different values of α
Interpretability research on large language models (LLMs) has yielded important insights into model behaviour, yet recurring pitfalls persist: findings that do not generalise, and causal interpretations that outrun the evidence. Our position is that causal inference specifies what constitutes a valid mapping from model activations to invariant high-level structures, the data or assumptions needed to achieve it, and the inferences it can support. Specifically, Pearl's causal hierarchy clarifies what an interpretability study can justify. Observations establish associations between model behaviour and internal components. Interventions (e.g., ablations or activation patching) support claims how these edits affect a behavioural metric (e.g., average change in token probabilities) over a set of prompts. However, counterfactual claims -- i.e., asking what the model output would have been for the same prompt under an unobserved intervention -- remain largely unverifiable without controlled supervision. We show how causal representation learning (CRL) operationalises this hierarchy, specifying which variables are recoverable from activations and under what assumptions. Together, these motivate a diagnostic framework that helps practitioners select methods and evaluations matching claims to evidence such that findings generalise.
Shruti Joshi, Aaron Mueller, David Klindt +3
Mila - Québec AI Institute & Université de Montréal · Boston University · Cold Spring Harbor Laboratory +1
Shortcut mitigation strategies commonly rely on training data annotations, group-balanced held-out data or the presence of all groups, i.e., all combinations of (spurious) attributes and classes, in the training data. However, these requirements are rarely met in practice. We instead propose a method for targeted model analysis to identify a small set of instances in which the model relies on spurious attributes. Using that set and following ``this feature should not be used for prediction'' reasoning, we identify highly relevant neurons in an intermediate layer and regularize their impact. This ensures that models learn to depend on informative features rather than being right for the wrong reasons, thereby improving robustness without requiring additional balanced held-out data or annotations.
Phuong Quynh Le, Jörg Schlötterer, Sari Sadiya +2
University of Marburg · Goethe University Frankfurt
Developing predictive models that perform reliably across diverse patient populations and heterogeneous environments is a core aim of medical research. However, generalization is only possible if the learned model is robust to statistical differences between data used for training and data seen at the time and place of deployment. Domain generalization methods provide strategies to address data shifts, but each method comes with its own set of assumptions and trade-offs. To apply these methods in healthcare, we must understand how domain shifts arise, what assumptions we prefer to make, and what our design constraints are. This article proposes a causal framework for the design of predictive models to improve generalization. Causality provides a powerful language to characterize and understand diverse domain shifts, regardless of data modality. This allows us to pinpoint why models fail to generalize, leading to more principled strategies to prepare for and adapt to shifts. We recommend general mitigation strategies, discussing trade-offs and highlighting existing work. Our causality-based perspective offers a critical foundation for developing robust, interpretable, and clinically relevant AI solutions in healthcare, paving the way for reliable real-world deployment.
Kurt Butler, Stephanie Riley, Damian Machlanski +13
Causality in Healthcare AI Hub (CHAI), UK · The University of Edinburgh, UK · University of Manchester, UK +6