Unsupervised Domain Adaptation for Cardiac Segmentation: Towards Structure Mutual Information Maximization

Changjie Lu; Shen Zheng; Gaurav Gupta

eess.IVApr 20, 2022

Unsupervised Domain Adaptation for Cardiac Segmentation: Towards Structure Mutual Information Maximization

Authors: Changjie Lu, Shen Zheng, Gaurav Gupta

Organizations: College of Science and Technology, Wenzhou-Kean University, Wenzhou, China

Abstract

Unsupervised domain adaptation approaches have recently succeeded in various medical image segmentation tasks. The reported works often tackle the domain shift problem by aligning the domain-invariant features and minimizing the domain-specific discrepancies. That strategy works well when the difference between a specific domain and between different domains is slight. However, the generalization ability of these models on diverse imaging modalities remains a significant challenge. This paper introduces UDA-VAE++, an unsupervised domain adaptation framework for cardiac segmentation. We show that the evidence lower bound of VAE-based domain adaptation penalizes the mutual information between the image and the latent space, including the structure information that segmentation relies on. Maximizing the structure mutual information removes this penalty on the structure information, while the remaining information is still compressed. To estimate this term, we develop a novel Structure Mutual Information Estimation (SMIE) block with a global estimator, a local estimator, and a prior information matching estimator to maximize the mutual information between the reconstruction and segmentation tasks. Specifically, we design a novel sequential reparameterization scheme that enables information flow and variance correction from the low-resolution latent space to the high-resolution latent space. Comprehensive experiments on benchmark cardiac segmentation datasets demonstrate that our model outperforms previous state-of-the-art qualitatively and quantitatively. The code is available at https://github.com/LOUEY233/Toward-Mutual-Information

Figures & tables

Panel of Figure 1 — Figure 1: Four types of Cardiac Imaging. From left to right: Computerized Tomography (CT), Magnetic Resonance Imaging (MRI), Positron Emission Tomography (PET), and Ultrasound (US). [ 28 ] Each type has significant difference with others in terms of color, contrast, structure, artifacts, and the edge information.

Symbols	Description
$S$	Source domain
$T$	Target domain
$z$	Latent variable
$x$	Input image data point
$y$	Ground truth segmentation
$\hat{y}$	Predicted segmentation

Table 1: Preliminary for Important Symbols

Figure 2: The Model Architecture of UDA-VAE++. The backbone of UDA-VAE++ is U-Net (blue boxes) with three scales of variational blocks. The green line refers to the concatenation of the segmentation output, whereas the orange line indicates the concatenation of the reconstruction output. The reconstruction blocks (red boxes) contain seven convolution layers. The grey box refers to the MI estimation block detailed in Fig. 4

Figure 3: The workflow for Unsupervised Domain Adaptation. The image from the source and target domain will first be encoded in the shared parameters downsampling part of the U-Net backbone. Next, each scale output will go through the same segmentation network. In the source domain, the ground truth segmentation masks combining the variables in latent space will be reconstructed by the upsampling part of U-Net. The MI block will maximize the mutual information of the segmentation output and the reconstruction output.

Figure 4: The architecture of Structure Mutual Information Estimation (SMIE) block. We use the reconstruction image as anchor, the shuffle reconstructed images as negative points, and the segmented image as positive points. The Global & Local Mutual Information (MI) Estimation Block follows contrastive learning schemes to maximize mutual information, whereas the prior information matching block align the positive point with the standard normal distribution. Finally, the sum of the outputs score from these three blocks serves as the loss function for $\mathcal{L}_{MI}$ .

Figure 5: The loss function of the proposed method. The blue line refers to the reconstruction loss. The red line indicates the segmentation loss between the predicted segmentation and ground truth segmentation. The orange line illustrates the domain discrepancy loss in the latent space. The green line refers to the structure mutual information loss.

Model Components						Dice ( $\%$ )
Base	SR	Att	Global	Local	Prior	MYO	LV	RV
✓						68.42	84.41	72.59
✓	✓					68.56	84.07	74.06
✓	✓	✓				68.30	84.91	74.72
✓	✓	✓	✓			69.25	84.70	75.63
✓	✓	✓	✓	✓		68.49	87.50	77.37

Table 2: The Ablations of model components for MS-CMRSeg Dataset from bSSFP to LGE-MRI . Base: UDA-VAE [ 35 ] . SR: Sequential Reparameterization. Att: Attention. Global: Global MI Estimation Block. Local: Local MI Estimation Block. Prior: Prior Matching. The best score for UDA from bSSFP to LGE-MRI is in bold while the second-best score is in blue .

	Dice ( $\%$ )			ASSD (mm)
	MYO	LV	RV	MYO	LV	RV
NoAdapt	14.50	34.51	31.10	21.6	11.3	14.5
CFDNet [ 34 ]	64.21	81.39	72.30	2.81	3.41	4.91
SIFA [ 2 ]	67.69	83.31	79.04	2.56	3.44	2.13
UDA-VAE [ 35 ]	68.42	84.41	72.59	2.39	2.59	3.97
UDA-VAE++	70.75	88.64	75.82	2.02	2.27	3.62

Table 3: Unsupervised Domain Adaptation for MS-CMRSeg Dataset from bSSFP to LGE-MRI . The best score for Dice

\uparrow

and ASSD

\downarrow

are in bold .

	Dice ( $\%$ )			ASSD (mm)
	MYO	LV	RV	MYO	LV	RV
NoAdapt	12.32	30.24	37.25	24.9	10.4	16.7
CFDNet [ 34 ]	57.41	78.44	77.63	3.61	3.87	2.49
SIFA [ 2 ]	60.89	79.32	82.39	3.44	3.65	1.80
UDA-VAE [ 35 ]	58.58	79.43	80.43	3.53	3.27	2.04
UDA-VAE++	68.74	85.08	81.42	2.34	2.61	1.71

Table 4: Unsupervised Domain Adaptation for MS-CMRSeg Dataset from LGE-MRI to bSSFP . The best score for Dice

\uparrow

and ASSD

\downarrow

are in bold .

Methods	Dice ( $\%$ )					ASSD (mm)
	MYO	LA	LV	RA	RV	MYO	LA	LV	RA	RV
NoAdapt	0.08	3.08	0.00	0.74	23.9	–	–	–	–	–
PnP-AdaNet [ 5 ]	32.7	49.7	48.4	62.4	44.2	6.89	22.6	9.56	20.7	20.0
SIFA [ 2 ]	37.1	65.7	61.2	51.9	18.5	11.8	5.47	16.0	14.7	21.6
UDA-VAE [ 35 ]	47.0	63.1	73.8	71.1	73.4	4.73	5.33	4.30	6.97	4.56
UDA-VAE++	51.4	65.9	76.5	73.0	75.5	3.88	5.23	3.78	6.25	4.06

Table 5: Unsupervised Domain Adaptation for MM-WHS Dataset from CT to MRI . The best score for Dice

\uparrow

and ASSD

\downarrow

are in bold .

Panel of Figure 6 — Figure 6: Segmentation output from MS-CMRSeg Dataset (bSSFP to LGE-MRI). From left to right: LGE-MRI, Ground truth, CFDNet [ 34 ] , UDA-VAE [ 35 ] , UDA-VAE++. For the segmentation, we use yellow, green, and dark green to represent RV, MYO, and LV, respectively.

Panel of Figure 7 — Figure 7: Reconstruction Images from MS-CMRSeg Dataset (LGE-MRI). From top to bottom row: LGE-MRI images, corresponding segmentation ground truth, UDA-VAE, UDA-VAE++.

Unsupervised Domain Adaptation

Unsupervised domain adaptation transfers models from labeled source data to unlabeled target domains with different data distributions.

See all 183 papers

Mar 23, 2026

SHAPE: Structure-aware Hierarchical Unsupervised Domain Adaptation with Plausibility Evaluation for Medical Image Segmentation

Unsupervised Domain Adaptation (UDA) is essential for deploying medical segmentation models across diverse clinical environments. Existing methods are fundamentally limited, suffering from semantically unaware feature alignment that results in poor distributional fidelity and from pseudo-label validation that disregards global anatomical constraints, thus failing to prevent the formation of globally implausible structures. To address these issues, we propose SHAPE (Structure-aware Hierarchical Unsupervised Domain Adaptation with Plausibility Evaluation), a framework that reframes adaptation towards global anatomical plausibility. Built on a DINOv3 foundation, its Hierarchical Feature Modulation (HFM) module first generates features with both high fidelity and class-awareness. This shifts the core challenge to robustly validating pseudo-labels. To augment conventional pixel-level validation, we introduce Hypergraph Plausibility Estimation (HPE), which leverages hypergraphs to assess the global anatomical plausibility that standard graphs cannot capture. This is complemented by Structural Anomaly Pruning (SAP) to purge remaining artifacts via cross-view stability. SHAPE significantly outperforms prior methods on cardiac and abdominal cross-modality benchmarks, achieving state-of-the-art average Dice scores of 90.08% (MRI->CT) and 78.51% (CT->MRI) on cardiac data, and 87.48% (MRI->CT) and 86.89% (CT->MRI) on abdominal data. The code is available at https://github.com/BioMedIA-repo/SHAPE.

Linkuan Zhou, Yinghao Xia, Yufei Shen +6

School of Software, Northwestern Polytechnical University · School of Computer Science and Technology, Harbin Institute of Technology · School of Software Engineering, USTC +3

Unsupervised Domain Adaptation Medical Image Segmentation Cross-Modality Medical Image Segmentation

Ask PDF

Jul 3, 2026

SNR-Adaptive Unified Diffusion for Multi-Task Medical Image Segmentation

Clinical cardiac imaging pipelines currently deploy separate models for each dataset and modality, incurring redundant training costs and precluding knowledge sharing across anatomically related tasks. Consolidating semi-supervised learning, unsupervised domain adaptation, and domain generalisation into one model is therefore a practical necessity, yet naive joint training exposes a fundamental barrier: conflicting label semantics between datasets collapse LA Dice from 90.31% to 83.38%, while gradient imbalance across tasks of unequal complexity suppresses the weaker tasks throughout training. We present UniT-Diff, a unified diffusion segmentation framework that resolves these conflicts through three targeted mechanisms. An 11-channel task-specific output space physically partitions label categories, eliminating cross-task gradient sign reversal by construction. SNR-Adaptive Task Conditioning (SATC) scales the task token by the log signal-to-noise ratio of the current diffusion timestep, suppressing domain-specific bias during coarse denoising and restoring full task guidance as the signal clears. Task-Type-Aware Conditional Dropout (TTACD) permanently removes the task token for domain-generalisation inputs, routing them through a shared neutral pathway that draws on cross-dataset cardiac anatomy rather than source-vendor statistics. Under a single parameter set, UniT-Diff surpasses independently trained task-specific baselines on all three benchmarks simultaneously: +0.87% on LA, +1.77% on MMWHS, and +0.88% on MNMS.

Jiahao Liu, Hang Wei, Shuai Wu

School of Telecommunications Engineering, Xidian University · School of Computer Science and Technology, Xidian University

Unsupervised Domain Adaptation Domain Generalization Multi-Task Learning Semi-Supervised Medical Image Segmentation Diffusion Models for Medical Image Segmentation Medical Image Segmentation

Ask PDF

Aug 25, 2026

Improving Cross-Site Whole-Heart Segmentation

Whole-heart segmentation from CT and MRI is essential for quantitative cardiac image analysis, but remains challenging under multi-center and multi-modality distribution shift. In the CARE whole-heart segmentation task, models must generalize from limited labeled sites to unseen acquisition distributions, where variation in spacing, intensity, reconstruction texture, and anatomy can degrade out-of-distribution performance. We propose a modality-routed 3D cardiac segmentation pipeline that combines TotalSegmentator-initialized nnU-Netv2 models with site-characterized, label-preserving appearance augmentation. We first characterize the available sites using measurable image properties and use this analysis to motivate candidate data-space generalization routes. The final retained recipe applies Bias Field + Bezier appearance augmentation, combining smooth spatial intensity perturbation with nonlinear intensity remapping, followed by lightweight class-wise largest-connected-component cleanup. On the primary held-out-site validation splits, the final configuration improves CT mean Dice from 0.8350 to 0.9135 and MRI mean Dice from 0.7695 to 0.7830, while also reducing HD95. These results suggest that site-motivated appearance augmentation is a practical strategy for improving cross-site robustness in limited-data whole-heart segmentation. Our code can be found in https://github.com/Purdue-M2/Improving-Cross-Site-Whole-Heart-Segmentation

Tanish Mudaliar, Justin Li, Daniel Lin +4

Purdue University, West Lafayette, USA · Carmel High School, Carmel, USA · Case Western Reserve University, Cleveland, Ohio, USA +3

Medical Image Segmentation 3D Medical Image Segmentation Data Augmentation

Ask PDF

Abstract

Unsupervised Domain Adaptation

Explore similar work

SHAPE: Structure-aware Hierarchical Unsupervised Domain Adaptation with Plausibility Evaluation for Medical Image Segmentation

SNR-Adaptive Unified Diffusion for Multi-Task Medical Image Segmentation

Improving Cross-Site Whole-Heart Segmentation