Unsupervised Domain Adaptation for Cardiac Segmentation: Towards Structure Mutual Information Maximization

Changjie Lu; Shen Zheng; Gaurav Gupta

eess.IVApr 20, 2022

Unsupervised Domain Adaptation for Cardiac Segmentation: Towards Structure Mutual Information Maximization

Authors: Changjie Lu, Shen Zheng, Gaurav Gupta

Organizations: College of Science and Technology, Wenzhou-Kean University, Wenzhou, China

Abstract

Unsupervised domain adaptation approaches have recently succeeded in various medical image segmentation tasks. The reported works often tackle the domain shift problem by aligning the domain-invariant features and minimizing the domain-specific discrepancies. That strategy works well when the difference between a specific domain and between different domains is slight. However, the generalization ability of these models on diverse imaging modalities remains a significant challenge. This paper introduces UDA-VAE++, an unsupervised domain adaptation framework for cardiac segmentation. We show that the evidence lower bound of VAE-based domain adaptation penalizes the mutual information between the image and the latent space, including the structure information that segmentation relies on. Maximizing the structure mutual information removes this penalty on the structure information, while the remaining information is still compressed. To estimate this term, we develop a novel Structure Mutual Information Estimation (SMIE) block with a global estimator, a local estimator, and a prior information matching estimator to maximize the mutual information between the reconstruction and segmentation tasks. Specifically, we design a novel sequential reparameterization scheme that enables information flow and variance correction from the low-resolution latent space to the high-resolution latent space. Comprehensive experiments on benchmark cardiac segmentation datasets demonstrate that our model outperforms previous state-of-the-art qualitatively and quantitatively. The code is available at https://github.com/LOUEY233/Toward-Mutual-Information

Figures & tables

Panel of Figure 1 — Figure 1: Four types of Cardiac Imaging. From left to right: Computerized Tomography (CT), Magnetic Resonance Imaging (MRI), Positron Emission Tomography (PET), and Ultrasound (US). [ 28 ] Each type has significant difference with others in terms of color, contrast, structure, artifacts, and the edge information.

Symbols	Description
$S$	Source domain
$T$	Target domain
$z$	Latent variable
$x$	Input image data point
$y$	Ground truth segmentation
$\hat{y}$	Predicted segmentation

Table 1: Preliminary for Important Symbols

Figure 2: The Model Architecture of UDA-VAE++. The backbone of UDA-VAE++ is U-Net (blue boxes) with three scales of variational blocks. The green line refers to the concatenation of the segmentation output, whereas the orange line indicates the concatenation of the reconstruction output. The reconstruction blocks (red boxes) contain seven convolution layers. The grey box refers to the MI estimation block detailed in Fig. 4

Figure 3: The workflow for Unsupervised Domain Adaptation. The image from the source and target domain will first be encoded in the shared parameters downsampling part of the U-Net backbone. Next, each scale output will go through the same segmentation network. In the source domain, the ground truth segmentation masks combining the variables in latent space will be reconstructed by the upsampling part of U-Net. The MI block will maximize the mutual information of the segmentation output and the reconstruction output.

Figure 4: The architecture of Structure Mutual Information Estimation (SMIE) block. We use the reconstruction image as anchor, the shuffle reconstructed images as negative points, and the segmented image as positive points. The Global & Local Mutual Information (MI) Estimation Block follows contrastive learning schemes to maximize mutual information, whereas the prior information matching block align the positive point with the standard normal distribution. Finally, the sum of the outputs score from these three blocks serves as the loss function for $\mathcal{L}_{MI}$ .

Figure 5: The loss function of the proposed method. The blue line refers to the reconstruction loss. The red line indicates the segmentation loss between the predicted segmentation and ground truth segmentation. The orange line illustrates the domain discrepancy loss in the latent space. The green line refers to the structure mutual information loss.

Model Components						Dice ( $\%$ )
Base	SR	Att	Global	Local	Prior	MYO	LV	RV
✓						68.42	84.41	72.59
✓	✓					68.56	84.07	74.06
✓	✓	✓				68.30	84.91	74.72
✓	✓	✓	✓			69.25	84.70	75.63
✓	✓	✓	✓	✓		68.49	87.50	77.37

Table 2: The Ablations of model components for MS-CMRSeg Dataset from bSSFP to LGE-MRI . Base: UDA-VAE [ 35 ] . SR: Sequential Reparameterization. Att: Attention. Global: Global MI Estimation Block. Local: Local MI Estimation Block. Prior: Prior Matching. The best score for UDA from bSSFP to LGE-MRI is in bold while the second-best score is in blue .

	Dice ( $\%$ )			ASSD (mm)
	MYO	LV	RV	MYO	LV	RV
NoAdapt	14.50	34.51	31.10	21.6	11.3	14.5
CFDNet [ 34 ]	64.21	81.39	72.30	2.81	3.41	4.91
SIFA [ 2 ]	67.69	83.31	79.04	2.56	3.44	2.13
UDA-VAE [ 35 ]	68.42	84.41	72.59	2.39	2.59	3.97
UDA-VAE++	70.75	88.64	75.82	2.02	2.27	3.62

Table 3: Unsupervised Domain Adaptation for MS-CMRSeg Dataset from bSSFP to LGE-MRI . The best score for Dice

\uparrow

and ASSD

\downarrow

are in bold .

	Dice ( $\%$ )			ASSD (mm)
	MYO	LV	RV	MYO	LV	RV
NoAdapt	12.32	30.24	37.25	24.9	10.4	16.7
CFDNet [ 34 ]	57.41	78.44	77.63	3.61	3.87	2.49
SIFA [ 2 ]	60.89	79.32	82.39	3.44	3.65	1.80
UDA-VAE [ 35 ]	58.58	79.43	80.43	3.53	3.27	2.04
UDA-VAE++	68.74	85.08	81.42	2.34	2.61	1.71

Table 4: Unsupervised Domain Adaptation for MS-CMRSeg Dataset from LGE-MRI to bSSFP . The best score for Dice

\uparrow

and ASSD

\downarrow

are in bold .

Methods	Dice ( $\%$ )					ASSD (mm)
	MYO	LA	LV	RA	RV	MYO	LA	LV	RA	RV
NoAdapt	0.08	3.08	0.00	0.74	23.9	–	–	–	–	–
PnP-AdaNet [ 5 ]	32.7	49.7	48.4	62.4	44.2	6.89	22.6	9.56	20.7	20.0
SIFA [ 2 ]	37.1	65.7	61.2	51.9	18.5	11.8	5.47	16.0	14.7	21.6
UDA-VAE [ 35 ]	47.0	63.1	73.8	71.1	73.4	4.73	5.33	4.30	6.97	4.56
UDA-VAE++	51.4	65.9	76.5	73.0	75.5	3.88	5.23	3.78	6.25	4.06

Table 5: Unsupervised Domain Adaptation for MM-WHS Dataset from CT to MRI . The best score for Dice

\uparrow

and ASSD

\downarrow

are in bold .

Panel of Figure 6 — Figure 6: Segmentation output from MS-CMRSeg Dataset (bSSFP to LGE-MRI). From left to right: LGE-MRI, Ground truth, CFDNet [ 34 ] , UDA-VAE [ 35 ] , UDA-VAE++. For the segmentation, we use yellow, green, and dark green to represent RV, MYO, and LV, respectively.

Panel of Figure 7 — Figure 7: Reconstruction Images from MS-CMRSeg Dataset (LGE-MRI). From top to bottom row: LGE-MRI images, corresponding segmentation ground truth, UDA-VAE, UDA-VAE++.

Unsupervised Domain Adaptation

Unsupervised domain adaptation transfers models from labeled source data to unlabeled target domains with different data distributions.

See all 183 papers

SHAPE: Structure-aware Hierarchical Unsupervised Domain Adaptation with Plausibility Evaluation for Medical Image Segmentation
Mar 23, 2026Linkuan Zhou, Yinghao Xia, Yufei Shen +6Unsupervised Domain Adaptation Medical Image Segmentation
School of Software, Northwestern Polytechnical University · School of Computer Science and Technology, Harbin Institute of Technology · School of Software Engineering, USTC +3
SNR-Adaptive Unified Diffusion for Multi-Task Medical Image Segmentation
Jul 3, 2026Jiahao Liu, Hang Wei, Shuai WuUnsupervised Domain Adaptation Domain Generalization
School of Telecommunications Engineering, Xidian University · School of Computer Science and Technology, Xidian University
Improving Cross-Site Whole-Heart Segmentation
Aug 25, 2026Tanish Mudaliar, Justin Li, Daniel Lin +4Medical Image Segmentation 3D Medical Image Segmentation
Purdue University, West Lafayette, USA · Carmel High School, Carmel, USA · Case Western Reserve University, Cleveland, Ohio, USA +3

Mar 23, 2026J/K move · Enter open · S save

SHAPE: Structure-aware Hierarchical Unsupervised Domain Adaptation with Plausibility Evaluation for Medical Image Segmentation

Linkuan Zhou, Yinghao Xia, Yufei Shen, Xiangyu Li+5

School of Software, Northwestern Polytechnical University · School of Computer Science and Technology, Harbin Institute of Technology · School of Software Engineering, USTC+3

Unsupervised Domain Adaptation Medical Image Segmentation Cross-Modality Medical Image Segmentation

Open paper AskPDF arXiv

Unsupervised Domain Adaptation (UDA) is essential for deploying medical segmentation models across diverse clinical environments. Existing methods are fundamentally limited, suffering from semantically unaware feature alignment that results in poor distributional fidelity and from pseudo-label validation that disregards global anatomical constraints, thus failing to prevent the formation of globally implausible structures. To address these issues, we propose SHAPE (Structure-aware Hierarchical Unsupervised Domain Adaptation with Plausibility Evaluation), a framework that reframes adaptation towards global anatomical plausibility. Built on a DINOv3 foundation, its Hierarchical Feature Modulation (HFM) module first generates features with both high fidelity and class-awareness. This shifts the core challenge to robustly validating pseudo-labels. To augment conventional pixel-level validation, we introduce Hypergraph Plausibility Estimation (HPE), which leverages hypergraphs to assess the global anatomical plausibility that standard graphs cannot capture. This is complemented by Structural Anomaly Pruning (SAP) to purge remaining artifacts via cross-view stability. SHAPE significantly outperforms prior methods on cardiac and abdominal cross-modality benchmarks, achieving state-of-the-art average Dice scores of 90.08% (MRI->CT) and 78.51% (CT->MRI) on cardiac data, and 87.48% (MRI->CT) and 86.89% (CT->MRI) on abdominal data. The code is available at https://github.com/BioMedIA-repo/SHAPE.

Abstract

Unsupervised Domain Adaptation

Explore similar work

SHAPE: Structure-aware Hierarchical Unsupervised Domain Adaptation with Plausibility Evaluation for Medical Image Segmentation

SNR-Adaptive Unified Diffusion for Multi-Task Medical Image Segmentation

Improving Cross-Site Whole-Heart Segmentation