cs.CVMar 25, 2026
SaveCausal Transfer in Medical Image Analysis
Organizations: Department of Computer Science, University of Exeter, Exeter, United Kingdom · Department of Mathematics and Computer Science, Faculty of Science, Alexandria University, Alexandria, Egypt · Hawkes Institute, Department of Computer Science, University College London, London, United Kingdom
Abstract
Medical imaging models frequently fail when deployed across hospitals, scanners, populations, or imaging protocols due to domain shift, limiting their clinical reliability. While transfer learning and domain adaptation address such shifts statistically, they often rely on spurious correlations that break under changing conditions. On the other hand, causal inference provides a principled way to identify invariant mechanisms that remain stable across environments. This survey introduces and systematises Causal Transfer Learning (CTL) for medical image analysis. This paradigm integrates causal reasoning with cross-domain representation learning to enable robust and generalisable clinical AI. We frame domain shift as a causal problem and analyse how structural causal models, invariant risk minimisation, and counterfactual reasoning can be embedded within transfer learning pipelines. We studied spanning classification, segmentation, reconstruction, anomaly detection, and multimodal imaging, and organised them by task, shift type, and causal assumption. A unified taxonomy is proposed that connects causal frameworks and transfer mechanisms. We further summarise datasets, benchmarks, and empirical gains, highlighting when and why causal transfer outperforms correlation-based domain adaptation. Finally, we discuss how CTL supports fairness, robustness, and trustworthy deployment in multi-institutional and federated settings, and outline open challenges and research directions for clinically reliable medical imaging AI.
Figures & tables
Figure 1: Illustration of domain shift in medical image analysis.
Figure 2: Ignorability assumption: potential outcomes are independent of treatment given covariates .
| Survey | Primary focus | What it covers well | Gap relative to this survey |
| Atasever et al. [ 12 ] | Transfer learning in medical image analysis | Broad review of methods, datasets, trends, knowledge gaps, constraints, and limitations in medical-image transfer learning | Does not organise the field around causal assumptions, invariance, interventions, or counterfactual reasoning |
| Yoon et al. [ 212 ] | Domain generalisation in medical image analysis | Structured review of out-of-distribution robustness across data-, feature-, model-, and analysis-level DG methods throughout the imaging workflow | Does not treat domain shift as a causal problem or synthesise methods through SCMs, potential outcomes, interventions, or counterfactual transfer |
| Matta et al. [ 120 ] | Generalisation in medical image classification | Systematic review of classification studies under distribution shift, with a taxonomy organised by shift type | Restricted to classification and does not unify causal and transfer-learning perspectives across imaging tasks |
| Vlontzos et al. [ 194 ] | Causality in medical image analysis | Reviews causal reasoning methods for medical imaging AI/ML, highlighting their relevance to robustness and clinical translation | Focuses on causality in imaging broadly rather than on transfer, adaptation, or generalisation under domain shift as the central organising problem |
| Methodological causal foundations [ 128 , 28 , 13 , 167 ] | General causal inference frameworks | Establish core tools such as do-calculus, propensity-score reasoning, and graphical causal models that underpin later CTL methods | Provide conceptual foundations rather than a survey of causal transfer learning, domain shift, or medical-image-analysis transfer methods |
| This survey | Causal transfer learning in medical image analysis | Unified synthesis of methods combining causal reasoning with transfer, adaptation, and generalisation under domain shift | Organises the field jointly by causal framework, causal operation, transfer setting, imaging task, and shift type |
Table 1: Comparison between existing surveys and this work.
Figure 3: Taxonomy of CTL methods organised across (i) foundational causal frameworks (Structural Causal Models and Potential Outcomes), (ii) core methodological strategies including domain adaptation, causal discovery, invariant representation learning, and counterfactual inference, (iii) supporting causal structures such as ignorability and multimodal environment modelling, and (iv) task-oriented deployment in medical image analysis. Each methodological family is further interpreted in relation to the four types of distribution shift (covariate, label, conditional, and concept shift), highlighting how CTL methods operate at different levels of the data-generating process.
| Concept | Definition | Importance in CTL |
| Causal Inference | The process of deducing causal relationships from data, often through techniques such as DAGs [ 139 ] . | Provides a basis for understanding how different features relate to outcomes, leading to better model generalisation. |
| Transfer Learning | A machine learning approach where a model trained on one task is adapted for another related task [ 135 ] . | Enables leveraging knowledge from source domains to improve performance in target domains. |
| Domain Adaptation | Techniques aimed at minimising differences between source and target domains, thereby improving model performance [ 96 ] . | Addresses shifts between patient populations and imaging conditions. |
| Counterfactual Inference | A framework for exploring hypothetical scenarios to evaluate the impact of interventions, often using potential outcomes [ 160 ] . | Provides insights into causal relationships, aiding personalised treatment reasoning. |
| Invariant Risk Minimisation | A method that focuses on learning features that remain consistent across different environments to enhance model transferability [ 8 ] . | Addresses domain shift by encouraging stable predictive mechanisms across environments. |
Table 2: Key Concepts in Causal Transfer Learning
| Technique | Description | Application in medical image analysis |
| Potential Outcomes Framework | A framework for evaluating treatment effects based on hypothetical scenarios [ 78 ] . | Estimating the effect of different treatment plans on patient outcomes in clinical trials. |
| Counterfactual Generative Models | Models that generate synthetic data based on counterfactual scenarios, facilitating the exploration of treatment effects [ 21 ] . | Simulating patient responses under different treatment conditions to inform decision-making. |
| Causal effect estimation and quantification of the impact of interventions on outcomes using observational data, often through techniques such as propensity score matching [ 157 ] . | Estimating the effect of imaging modality on diagnostic accuracy across different patient groups. |
Table 3: Counterfactual Inference Techniques
| Technique | Description | Example Application in medical image analysis |
| Instance Reweighting | Assigning weights to training samples based on their relevance to the target domain [ 204 , 65 ] . | Adjusting weights of training samples in a chest X-ray dataset to reflect the demographics of a local population. |
| Feature Alignment | Transforming feature representations to minimise differences between source and target domains using techniques such as MMD [ 64 ] . Unlike distillation, which transfers knowledge between models, feature alignment focuses on aligning feature distributions across domains. | Aligning image feature distributions across different imaging devices, such as MRI scanners. |
| Adversarial Training | A method where a model is trained to confuse a domain classifier, forcing learned features to be domain-invariant [ 62 ] . | Using adversarial networks to improve robustness in skin lesion classification across various imaging protocols. |
Table 4: Domain Adaptation Techniques in Causal Transfer Learning
| Method | Description | Strengths and Limitations |
| Graphical Models | Utilised DAGs to represent causal relationships and identify dependencies [ 138 ] . | Provides a clear visual representation of causal relationships; sensitive to model assumptions. |
| Invariant Risk Minimisation | Focuses on learning features that are invariant across environments to enhance generalisability [ 8 ] . | Promotes robustness to domain shifts; implementation can be complex and data-intensive. |
| Causal Bayesian Networks | Combines prior knowledge and observational data to infer causal relationships and dependencies [ 70 , 29 , 43 ] . | Incorporates expert knowledge; computationally intensive and requires large datasets. |
Table 5: Causal Discovery Methods
| Challenge | Causality-Inspired Solution(s) |
| (A) Data limitations | |
| Limited training data | One-shot / few-shot causal learning [ 33 ] |
| Lack of labeled target-domain data | Self-supervised learning [ 106 , 146 ] |
| (B) Domain shift and heterogeneity | |
| Style discrepancies between imaging domains | Fourier-based style transfer [ 106 ] |
| Differences in feature space between domains | Cross-domain contrastive learning with adversarial training [ 106 ] ; contrastive feature alignment with source prototypes [ 146 ] |
Table 6: Key challenges in medical image (MI) analysis and causality-inspired solutions
Figure 4: Illustration of the domain-shift problem of Fundus in three samples [ 106 ] .
Figure 5: SCM for mitigating domain style interference [ 106 ] .
Figure 6: Simplified Causal Self-Supervised Network (CSSN) [ 106 ] .
Figure 7: Global Intensity Non-linear Augmentation (GIN) [ 134 ] .
Figure 8: Causality-Driven ResNet18 for prostate cancer grading from MRI [ 33 ] .
Figure 9: Causal graph of medical image generation [ 146 ] .
Figure 10: Causal graph of SFDA [ 146 ] .
Figure 11: Causal graphs used to train the counterfactual image generation models [ 155 ] .
| Modality / Task | CTL Method and Benefit |
| (A) Domain adaptation and generalisation | |
| Fundus Imaging [ 106 ] Cup-to-disc ratio estimation for glaucoma diagnosis | Domain adaptation and structural causal models were used to improve generalisation across imaging devices with different styles. |
| CT and MRI [ 37 ] Single-source segmentation (cross-modality) | Structural Causal Models (SCMs), combined with diffusion-based style interventions, enabled the simulation of missing modalities and reliable segmentation across unseen modalities. |
| Multi-domain datasets [ 134 ] Single-source domain generalisation for segmentation | Causality-inspired augmentations (GIN + IPA) improved cross-domain segmentation performance by focusing on domain-invariant anatomical structures. |
| Chest X-ray [ 146 ] Classification under domain shift | Causality-inspired source-free domain adaptation using prototype-guided contrastive feature alignment and causal interventions reduced performance degradation across hospitals and imaging devices. |
| Chest X-ray (PadChest) [ 31 ] and Mammography (EMBED) [ 83 ] Classification under domain shift | Counterfactual Contrastive Learning (CCL) improved robustness to acquisition shifts, reduced bias, and enhanced generalisation for under-represented scanners. |
Table 7: Applications of Causal Transfer Learning across Medical Imaging Modalities
| Method Family | Imaging Task(s) | Shift Type Addressed | Causal Assumption |
| Invariant Risk Minimisation (IRM), REx [ 9 ] | Classification | Domain / environment shift | Existence of invariant causal predictors across environments |
| Causal representation learning [ 165 , 2 ] | Classification, segmentation | Covariate and mechanism shift | Latent SCM governing representations |
| Disentangled causal factor models [ 207 , 181 ] | Reconstruction, multimodal imaging | Covariate shift | Independent causal generative factors |
| Domain-adversarial learning (causality-inspired) [ 116 ] | Classification, segmentation | Covariate / scanner shift | Anti-causal setting with invariant conditional mechanisms |
| Conditional domain adaptation [ 114 , 217 , 61 ] | Classification | Label shift | Invariance of across domains |
| Causal feature selection [ 67 ] | Classification | Spurious correlation shift | Selected features correspond to causal parents of the label |
Table 8: Mapping of CTL methods reviewed in Sections 5 and 6, organised by imaging task, domain/shift type, and underlying causal assumption.
| Challenge | Description | Potential Solutions |
| Causal Identifiability Across Clinical Domains | Determining whether invariant causal mechanisms remain identifiable under evolving clinical distributions and imaging protocols. | Development of causal stability theory, domain-aware SCMs, and uncertainty-aware causal discovery methods. |
| Evaluation of Causal Generalisation | Existing metrics do not directly measure causal validity, intervention robustness, or counterfactual consistency. | Designing benchmark datasets and evaluation protocols based on interventions, counterfactual reasoning, and causal stress testing. |
| Trade-off Between Invariance and Clinical Specificity | Excessive invariance constraints may suppress clinically meaningful demographic or modality-specific information. | Adaptive invariance frameworks and context-aware causal regularisation methods. |
| Learning Causal Representations from High-Dimensional Imaging Data | High-dimensional medical imaging data (e.g., MRI, CT) contain complex patterns, making it difficult to learn accurate causal representations. | The development of advanced causal models, such as causal convolutional neural networks and graph-based causal inference, can help identify and learn the underlying causal structure from imaging data. |
| Causal Discovery | Identifying and inferring the correct causal relationships in imaging data, where features are correlated but not causally linked. | Implementing causal discovery algorithms, such as constraint-based methods (e.g., the PC algorithm) and causal Bayesian networks, can help uncover valid causal structures. [ 139 ] |
| Defining Causal Structure for Images | Developing a formal framework for structuring causal relationships within imaging data to make the models more interpretable and reliable. | Employing causal graphical models and counterfactual reasoning frameworks (e.g., CausalGAN) to provide a robust framework for defining causal relationships in images. |
Table 9: Challenges in Causal Transfer Learning
| Dataset/Task | Relevance to CTL |
| REFUGE [ 131 ] , DRISHTI-GS [ 173 ] , RIM-ONE-r3-all [ 55 ] | Fundus imaging for domain adaptation in glaucoma diagnosis across camera systems and lighting conditions. |
| Abdominal Segmentation (AS) [ 99 ] , [ 85 ] , [ 90 ] | CT and MRI datasets for cross-modality abdominal organ segmentation, such as liver and kidneys. |
| Lumbar Spine Segmentation (LSS) [ 168 ] , [ 136 ] , [ 93 ] | Cross-modality segmentation of vertebrae using CT, MRI, and X-ray data for generalisation across imaging modalities. |
| Lung Segmentation (LS) [ 206 ] , [ 45 ] | CT and X-ray datasets for segmenting left and right lungs across varying imaging conditions. |
| Cardiac Cross-sequence [ 223 ] | Cardiac MRI data transitioning from bSSFP to LGE MRI for sequence adaptation. |
| Prostate Cross-site [ 112 , 27 , 101 , 110 ] | Prostate MRI data from multiple sites to address cross-site variability. |
Table 10: Datasets Suited for CTL
| Evaluation Criterion | Description | Reference |
| Intervention Testing | A causal model should accurately predict the effects of interventions by testing how the model responds to changes in a variable. Comparison of its predictions to observed outcomes is often feasible through randomised controlled trials (RCTs) or synthetic experiments in simulation environments. | [ 140 , 139 ] |
| Counterfactual Reasoning | Counterfactual analysis assesses a model’s ability to predict hypothetical outcomes under different conditions, beyond observable data. Evaluation is challenging, but counterfactual accuracy can sometimes be tested in synthetic datasets with known counterfactual outcomes. | [ 78 ] |
| Do-Calculus Validity | For models based on SCMs and Judea Pearl’s causal framework, it is important to ensure compliance with the rules of do-calculus, which allows models to handle interventions by distinguishing causal from non-causal relationships. | [ 139 ] |
| Synthetic Data with Known Causal Structures | Evaluation on synthetic datasets with predefined causal structures helps in assessing causal discovery accuracy. Inferred causal structures can be compared with ground truth as a benchmark for validation. | [ 175 ] |
| Benchmarking on Causal Datasets | Certain datasets, such as the Twins and IHDP datasets in healthcare, are designed for causal inference tasks. These benchmarks enable evaluation of the model’s ability to capture treatment effects, often measured using metrics such as Average Treatment Effect (ATE) and Conditional Average Treatment Effect (CATE). | [ 71 ] |
| Structural Metrics | Causal discovery models that infer causal graphs can be evaluated using structural metrics, such as the Structural Hamming Distance (SHD), which assesses structural accuracy by counting the edits required to match the true causal graph. | [ 89 ] |
Table 11: Evaluation Criteria for Causal Models