Causal Transfer in Medical Image Analysis

Mohammed M. Abdelsamea; Daniel Tweneboah Anyimadu; Tasneem Selim; Saif Alzubi; Lei Zhang; Ahmed Karam Eldaly; Xujiong Ye

cs.CVMar 25, 2026

Causal Transfer in Medical Image Analysis

Authors: Mohammed M. Abdelsamea, Daniel Tweneboah Anyimadu, Tasneem Selim, Saif Alzubi, Lei Zhang, Ahmed Karam Eldaly, Xujiong Ye

Organizations: Department of Computer Science, University of Exeter, Exeter, United Kingdom · Department of Mathematics and Computer Science, Faculty of Science, Alexandria University, Alexandria, Egypt · Hawkes Institute, Department of Computer Science, University College London, London, United Kingdom

Abstract

Medical imaging models frequently fail when deployed across hospitals, scanners, populations, or imaging protocols due to domain shift, limiting their clinical reliability. While transfer learning and domain adaptation address such shifts statistically, they often rely on spurious correlations that break under changing conditions. On the other hand, causal inference provides a principled way to identify invariant mechanisms that remain stable across environments. This survey introduces and systematises Causal Transfer Learning (CTL) for medical image analysis. This paradigm integrates causal reasoning with cross-domain representation learning to enable robust and generalisable clinical AI. We frame domain shift as a causal problem and analyse how structural causal models, invariant risk minimisation, and counterfactual reasoning can be embedded within transfer learning pipelines. We studied spanning classification, segmentation, reconstruction, anomaly detection, and multimodal imaging, and organised them by task, shift type, and causal assumption. A unified taxonomy is proposed that connects causal frameworks and transfer mechanisms. We further summarise datasets, benchmarks, and empirical gains, highlighting when and why causal transfer outperforms correlation-based domain adaptation. Finally, we discuss how CTL supports fairness, robustness, and trustworthy deployment in multi-institutional and federated settings, and outline open challenges and research directions for clinically reliable medical imaging AI.

Figures & tables

Panel of Figure 1 — Figure 1: Illustration of domain shift in medical image analysis.

Panel of Figure 2 — Figure 2: Ignorability assumption: potential outcomes $(Y_{0},Y_{1})$ are independent of treatment $T$ given covariates $X$ .

Survey	Primary focus	What it covers well	Gap relative to this survey
Atasever et al. [ 12 ]	Transfer learning in medical image analysis	Broad review of methods, datasets, trends, knowledge gaps, constraints, and limitations in medical-image transfer learning	Does not organise the field around causal assumptions, invariance, interventions, or counterfactual reasoning
Yoon et al. [ 212 ]	Domain generalisation in medical image analysis	Structured review of out-of-distribution robustness across data-, feature-, model-, and analysis-level DG methods throughout the imaging workflow	Does not treat domain shift as a causal problem or synthesise methods through SCMs, potential outcomes, interventions, or counterfactual transfer
Matta et al. [ 120 ]	Generalisation in medical image classification	Systematic review of classification studies under distribution shift, with a taxonomy organised by shift type	Restricted to classification and does not unify causal and transfer-learning perspectives across imaging tasks
Vlontzos et al. [ 194 ]	Causality in medical image analysis	Reviews causal reasoning methods for medical imaging AI/ML, highlighting their relevance to robustness and clinical translation	Focuses on causality in imaging broadly rather than on transfer, adaptation, or generalisation under domain shift as the central organising problem
Methodological causal foundations [ 128 , 28 , 13 , 167 ]	General causal inference frameworks	Establish core tools such as do-calculus, propensity-score reasoning, and graphical causal models that underpin later CTL methods	Provide conceptual foundations rather than a survey of causal transfer learning, domain shift, or medical-image-analysis transfer methods
This survey	Causal transfer learning in medical image analysis	Unified synthesis of methods combining causal reasoning with transfer, adaptation, and generalisation under domain shift	Organises the field jointly by causal framework, causal operation, transfer setting, imaging task, and shift type

Table 1: Comparison between existing surveys and this work.

Figure 3: Taxonomy of CTL methods organised across (i) foundational causal frameworks (Structural Causal Models and Potential Outcomes), (ii) core methodological strategies including domain adaptation, causal discovery, invariant representation learning, and counterfactual inference, (iii) supporting causal structures such as ignorability and multimodal environment modelling, and (iv) task-oriented deployment in medical image analysis. Each methodological family is further interpreted in relation to the four types of distribution shift (covariate, label, conditional, and concept shift), highlighting how CTL methods operate at different levels of the data-generating process.

Concept	Definition	Importance in CTL
Causal Inference	The process of deducing causal relationships from data, often through techniques such as DAGs [ 139 ] .	Provides a basis for understanding how different features relate to outcomes, leading to better model generalisation.
Transfer Learning	A machine learning approach where a model trained on one task is adapted for another related task [ 135 ] .	Enables leveraging knowledge from source domains to improve performance in target domains.
Domain Adaptation	Techniques aimed at minimising differences between source and target domains, thereby improving model performance [ 96 ] .	Addresses shifts between patient populations and imaging conditions.
Counterfactual Inference	A framework for exploring hypothetical scenarios to evaluate the impact of interventions, often using potential outcomes [ 160 ] .	Provides insights into causal relationships, aiding personalised treatment reasoning.
Invariant Risk Minimisation	A method that focuses on learning features that remain consistent across different environments to enhance model transferability [ 8 ] .	Addresses domain shift by encouraging stable predictive mechanisms across environments.

Table 2: Key Concepts in Causal Transfer Learning

Technique	Description	Application in medical image analysis
Potential Outcomes Framework	A framework for evaluating treatment effects based on hypothetical scenarios [ 78 ] .	Estimating the effect of different treatment plans on patient outcomes in clinical trials.
Counterfactual Generative Models	Models that generate synthetic data based on counterfactual scenarios, facilitating the exploration of treatment effects [ 21 ] .	Simulating patient responses under different treatment conditions to inform decision-making.
Causal effect estimation and quantification of the impact of interventions on outcomes using observational data, often through techniques such as propensity score matching [ 157 ] .	Estimating the effect of imaging modality on diagnostic accuracy across different patient groups.

Table 3: Counterfactual Inference Techniques

Technique	Description	Example Application in medical image analysis
Instance Reweighting	Assigning weights to training samples based on their relevance to the target domain [ 204 , 65 ] .	Adjusting weights of training samples in a chest X-ray dataset to reflect the demographics of a local population.
Feature Alignment	Transforming feature representations to minimise differences between source and target domains using techniques such as MMD [ 64 ] . Unlike distillation, which transfers knowledge between models, feature alignment focuses on aligning feature distributions across domains.	Aligning image feature distributions across different imaging devices, such as MRI scanners.
Adversarial Training	A method where a model is trained to confuse a domain classifier, forcing learned features to be domain-invariant [ 62 ] .	Using adversarial networks to improve robustness in skin lesion classification across various imaging protocols.

Table 4: Domain Adaptation Techniques in Causal Transfer Learning

Method	Description	Strengths and Limitations
Graphical Models	Utilised DAGs to represent causal relationships and identify dependencies [ 138 ] .	Provides a clear visual representation of causal relationships; sensitive to model assumptions.
Invariant Risk Minimisation	Focuses on learning features that are invariant across environments to enhance generalisability [ 8 ] .	Promotes robustness to domain shifts; implementation can be complex and data-intensive.
Causal Bayesian Networks	Combines prior knowledge and observational data to infer causal relationships and dependencies [ 70 , 29 , 43 ] .	Incorporates expert knowledge; computationally intensive and requires large datasets.

Table 5: Causal Discovery Methods

Challenge	Causality-Inspired Solution(s)
(A) Data limitations
Limited training data	One-shot / few-shot causal learning [ 33 ]
Lack of labeled target-domain data	Self-supervised learning [ 106 , 146 ]
(B) Domain shift and heterogeneity
Style discrepancies between imaging domains	Fourier-based style transfer [ 106 ]
Differences in feature space between domains	Cross-domain contrastive learning with adversarial training [ 106 ] ; contrastive feature alignment with source prototypes [ 146 ]

Table 6: Key challenges in medical image (MI) analysis and causality-inspired solutions

Figure 4: Illustration of the domain-shift problem of Fundus in three samples [ 106 ] .

Panel of Figure 5 — Figure 5: SCM for mitigating domain style interference [ 106 ] .

Panel of Figure 6 — Figure 6: Simplified Causal Self-Supervised Network (CSSN) [ 106 ] .

Panel of Figure 7 — Figure 7: Global Intensity Non-linear Augmentation (GIN) [ 134 ] .

Panel of Figure 8 — Figure 8: Causality-Driven ResNet18 for prostate cancer grading from MRI [ 33 ] .

Panel of Figure 9 — Figure 9: Causal graph of medical image generation [ 146 ] .

Panel of Figure 10 — Figure 10: Causal graph of SFDA [ 146 ] .

Figure 11: Causal graphs used to train the counterfactual image generation models [ 155 ] .

Modality / Task	CTL Method and Benefit
(A) Domain adaptation and generalisation
Fundus Imaging [ 106 ] Cup-to-disc ratio estimation for glaucoma diagnosis	Domain adaptation and structural causal models were used to improve generalisation across imaging devices with different styles.
CT and MRI [ 37 ] Single-source segmentation (cross-modality)	Structural Causal Models (SCMs), combined with diffusion-based style interventions, enabled the simulation of missing modalities and reliable segmentation across unseen modalities.
Multi-domain datasets [ 134 ] Single-source domain generalisation for segmentation	Causality-inspired augmentations (GIN + IPA) improved cross-domain segmentation performance by focusing on domain-invariant anatomical structures.
Chest X-ray [ 146 ] Classification under domain shift	Causality-inspired source-free domain adaptation using prototype-guided contrastive feature alignment and causal interventions reduced performance degradation across hospitals and imaging devices.
Chest X-ray (PadChest) [ 31 ] and Mammography (EMBED) [ 83 ] Classification under domain shift	Counterfactual Contrastive Learning (CCL) improved robustness to acquisition shifts, reduced bias, and enhanced generalisation for under-represented scanners.

Table 7: Applications of Causal Transfer Learning across Medical Imaging Modalities

Method Family	Imaging Task(s)	Shift Type Addressed	Causal Assumption
Invariant Risk Minimisation (IRM), REx [ 9 ]	Classification	Domain / environment shift	Existence of invariant causal predictors across environments
Causal representation learning [ 165 , 2 ]	Classification, segmentation	Covariate and mechanism shift	Latent SCM governing representations
Disentangled causal factor models [ 207 , 181 ]	Reconstruction, multimodal imaging	Covariate shift	Independent causal generative factors
Domain-adversarial learning (causality-inspired) [ 116 ]	Classification, segmentation	Covariate / scanner shift	Anti-causal setting with invariant conditional mechanisms
Conditional domain adaptation [ 114 , 217 , 61 ]	Classification	Label shift	Invariance of $P(Y\mid X)$ across domains
Causal feature selection [ 67 ]	Classification	Spurious correlation shift	Selected features correspond to causal parents of the label

Table 8: Mapping of CTL methods reviewed in Sections 5 and 6, organised by imaging task, domain/shift type, and underlying causal assumption.

Challenge	Description	Potential Solutions
Causal Identifiability Across Clinical Domains	Determining whether invariant causal mechanisms remain identifiable under evolving clinical distributions and imaging protocols.	Development of causal stability theory, domain-aware SCMs, and uncertainty-aware causal discovery methods.
Evaluation of Causal Generalisation	Existing metrics do not directly measure causal validity, intervention robustness, or counterfactual consistency.	Designing benchmark datasets and evaluation protocols based on interventions, counterfactual reasoning, and causal stress testing.
Trade-off Between Invariance and Clinical Specificity	Excessive invariance constraints may suppress clinically meaningful demographic or modality-specific information.	Adaptive invariance frameworks and context-aware causal regularisation methods.
Learning Causal Representations from High-Dimensional Imaging Data	High-dimensional medical imaging data (e.g., MRI, CT) contain complex patterns, making it difficult to learn accurate causal representations.	The development of advanced causal models, such as causal convolutional neural networks and graph-based causal inference, can help identify and learn the underlying causal structure from imaging data.
Causal Discovery	Identifying and inferring the correct causal relationships in imaging data, where features are correlated but not causally linked.	Implementing causal discovery algorithms, such as constraint-based methods (e.g., the PC algorithm) and causal Bayesian networks, can help uncover valid causal structures. [ 139 ]
Defining Causal Structure for Images	Developing a formal framework for structuring causal relationships within imaging data to make the models more interpretable and reliable.	Employing causal graphical models and counterfactual reasoning frameworks (e.g., CausalGAN) to provide a robust framework for defining causal relationships in images.

Table 9: Challenges in Causal Transfer Learning

Dataset/Task	Relevance to CTL
REFUGE [ 131 ] , DRISHTI-GS [ 173 ] , RIM-ONE-r3-all [ 55 ]	Fundus imaging for domain adaptation in glaucoma diagnosis across camera systems and lighting conditions.
Abdominal Segmentation (AS) [ 99 ] , [ 85 ] , [ 90 ]	CT and MRI datasets for cross-modality abdominal organ segmentation, such as liver and kidneys.
Lumbar Spine Segmentation (LSS) [ 168 ] , [ 136 ] , [ 93 ]	Cross-modality segmentation of vertebrae using CT, MRI, and X-ray data for generalisation across imaging modalities.
Lung Segmentation (LS) [ 206 ] , [ 45 ]	CT and X-ray datasets for segmenting left and right lungs across varying imaging conditions.
Cardiac Cross-sequence [ 223 ]	Cardiac MRI data transitioning from bSSFP to LGE MRI for sequence adaptation.
Prostate Cross-site [ 112 , 27 , 101 , 110 ]	Prostate MRI data from multiple sites to address cross-site variability.

Table 10: Datasets Suited for CTL

Evaluation Criterion	Description	Reference
Intervention Testing	A causal model should accurately predict the effects of interventions by testing how the model responds to changes in a variable. Comparison of its predictions to observed outcomes is often feasible through randomised controlled trials (RCTs) or synthetic experiments in simulation environments.	[ 140 , 139 ]
Counterfactual Reasoning	Counterfactual analysis assesses a model’s ability to predict hypothetical outcomes under different conditions, beyond observable data. Evaluation is challenging, but counterfactual accuracy can sometimes be tested in synthetic datasets with known counterfactual outcomes.	[ 78 ]
Do-Calculus Validity	For models based on SCMs and Judea Pearl’s causal framework, it is important to ensure compliance with the rules of do-calculus, which allows models to handle interventions by distinguishing causal from non-causal relationships.	[ 139 ]
Synthetic Data with Known Causal Structures	Evaluation on synthetic datasets with predefined causal structures helps in assessing causal discovery accuracy. Inferred causal structures can be compared with ground truth as a benchmark for validation.	[ 175 ]
Benchmarking on Causal Datasets	Certain datasets, such as the Twins and IHDP datasets in healthcare, are designed for causal inference tasks. These benchmarks enable evaluation of the model’s ability to capture treatment effects, often measured using metrics such as Average Treatment Effect (ATE) and Conditional Average Treatment Effect (CATE).	[ 71 ]
Structural Metrics	Causal discovery models that infer causal graphs can be evaluated using structural metrics, such as the Structural Hamming Distance (SHD), which assesses structural accuracy by counting the edits required to match the true causal graph.	[ 89 ]

Table 11: Evaluation Criteria for Causal Models

Explore similar work

Mar 13, 2026

A Causal Framework for Mitigating Data Shifts in Healthcare

Developing predictive models that perform reliably across diverse patient populations and heterogeneous environments is a core aim of medical research. However, generalization is only possible if the learned model is robust to statistical differences between data used for training and data seen at the time and place of deployment. Domain generalization methods provide strategies to address data shifts, but each method comes with its own set of assumptions and trade-offs. To apply these methods in healthcare, we must understand how domain shifts arise, what assumptions we prefer to make, and what our design constraints are. This article proposes a causal framework for the design of predictive models to improve generalization. Causality provides a powerful language to characterize and understand diverse domain shifts, regardless of data modality. This allows us to pinpoint why models fail to generalize, leading to more principled strategies to prepare for and adapt to shifts. We recommend general mitigation strategies, discussing trade-offs and highlighting existing work. Our causality-based perspective offers a critical foundation for developing robust, interpretable, and clinically relevant AI solutions in healthcare, paving the way for reliable real-world deployment.

Kurt Butler, Stephanie Riley, Damian Machlanski +13

Causality in Healthcare AI Hub (CHAI), UK · The University of Edinburgh, UK · University of Manchester, UK +6

Distribution Shift Robustness Cross-Domain Generalization Domain Generalization

Ask PDF

Mar 26, 2026

CIV-DG: Conditional Instrumental Variables for Domain Generalization in Medical Imaging

Cross-site generalizability in medical AI is fundamentally compromised by selection bias, a structural mechanism where patient demographics (e.g., age, severity) non-randomly dictate hospital assignment. Conventional Domain Generalization (DG) paradigms, which predominantly target image-level distribution shifts, fail to address the resulting spurious correlations between site-specific variations and diagnostic labels. To surmount this identifiability barrier, we propose CIV-DG, a causal framework that leverages Conditional Instrumental Variables to disentangle pathological semantics from scanner-induced artifacts. By relaxing the strict random assignment assumption of standard IV methods, CIV-DG accommodates complex clinical scenarios where hospital selection is endogenously driven by patient demographics. We instantiate this theory via a Deep Generalized Method of Moments (DeepGMM) architecture, employing a conditional critic to minimize moment violations and enforce instrument-error orthogonality within demographic strata. Extensive experiments on the Camelyon17 benchmark and large-scale Chest X-Ray datasets demonstrate that CIV-DG significantly outperforms leading baselines, validating the efficacy of conditional causal mechanisms in resolving structural confounding for robust medical AI.

Shaojin Bai, Yuting Su, Weizhi Nie

School of Electrical and Information Engineering, Tianjin University, Tianjin, China

Domain Generalization Cross-Domain Generalization Selection Bias Causal Inference Instrumental Variable Estimation

Ask PDF

Jun 5, 2026

Transfer learning for causal forest

Transfer learning addresses the challenge of transfering knowledge from one domain to another. Traditional transfer learning focuses on adapting models trained on a source domain (with a lot of observations) to improve performance on a target domain (with few observations). In this work we consider the case of a model shift and we focus on the transfer learning applied to a causal forest namely HTERF. This causal forest aims to estimate the Conditional Average Treatment Effect (CATE). The approach considered is the offset method presented by Wang (2016) adapted to a causal context. This method relies on the use of intermediate models in order to estimate the offset between source and target distributions. Our main result is a bound on the CATE error of HTERF on target depending on the error of the intermediate models. Simulation studies show the good performances of this approach in different settings on simulations and on a real-world dataset.

Bérénice-Alexia Jocteur, Véronique Maume-Deschamps, Pierre Ribereau

ICJ, PSPM · Universit´e Lyon 1, Centrale Lyon, INSA Lyon · Universit´e Jean Monnet, CNRS, ICJ UMR5208, 69622 Villeurbanne, France. +1

Causal Effect Estimation Conditional Average Treatment Effect Estimation Cross-Domain Transfer Learning Transfer Learning Individualized Treatment Effect Estimation

Ask PDF