Improving Cross-Site Whole-Heart Segmentation

Tanish Mudaliar; Justin Li; Daniel Lin; Julianna Vo; Kaitao Liao; Xin Wang; Shu Hu

eess.IVAug 25, 2026

Improving Cross-Site Whole-Heart Segmentation

Authors: Tanish Mudaliar, Justin Li, Daniel Lin, Julianna Vo, Kaitao Liao, Xin Wang, Shu Hu

Organizations: Purdue University, West Lafayette, USA · Carmel High School, Carmel, USA · Case Western Reserve University, Cleveland, Ohio, USA · The Pennsylvania State University, State College, USA ·

Abstract

Whole-heart segmentation from CT and MRI is essential for quantitative cardiac image analysis, but remains challenging under multi-center and multi-modality distribution shift. In the CARE whole-heart segmentation task, models must generalize from limited labeled sites to unseen acquisition distributions, where variation in spacing, intensity, reconstruction texture, and anatomy can degrade out-of-distribution performance. We propose a modality-routed 3D cardiac segmentation pipeline that combines TotalSegmentator-initialized nnU-Netv2 models with site-characterized, label-preserving appearance augmentation. We first characterize the available sites using measurable image properties and use this analysis to motivate candidate data-space generalization routes. The final retained recipe applies Bias Field + Bezier appearance augmentation, combining smooth spatial intensity perturbation with nonlinear intensity remapping, followed by lightweight class-wise largest-connected-component cleanup. On the primary held-out-site validation splits, the final configuration improves CT mean Dice from 0.8350 to 0.9135 and MRI mean Dice from 0.7695 to 0.7830, while also reducing HD95. These results suggest that site-motivated appearance augmentation is a practical strategy for improving cross-site robustness in limited-data whole-heart segmentation. Our code can be found in https://github.com/Purdue-M2/Improving-Cross-Site-Whole-Heart-Segmentation

Figures & tables

Figure 1: Graphical comparison of generalization approaches in model pipeline.

Modality	Site	$n$	Spacing (mm)	WH Vol. (mL)	Med. intensity	Low/high-freq. ratio
CT	A	20	$0.43\times 0.58$	$601\pm 120$	$-514\pm 324$	$176.6\pm 47.5$
CT	B	20	$1.00\times 1.00$	$759\pm 275$	$-106\pm 161$	$19.5\pm 6.6$
CT	G	20	$0.37\times 0.50$	$533\pm 118$	$-66\pm 41$	$70.6\pm 11.0$
MRI	C/D	20	$0.94\times 1.20$	$784\pm 259$	$96\pm 53$	$45.8\pm 14.7$
MRI	E	26	$0.94\times 1.27$	$921\pm 228$	$81\pm 42$	$32.5\pm 12.7$

Table 1: Site characterization summary. Spacing is reported as median in-plane spacing

\times

through-plane spacing in mm. Whole-heart volume, median intensity, and low/high-frequency ratio are reported as mean

\pm

standard deviation.

Figure 2: Architecture pipeline end-to-end for CT and MRI input streams for training and inference.

CT (train: A+B; test: G, $n=20$ )
Model	Myo	LA	LV	RA	RV	AO	PA	Dice	HD95
SegVol	0.6234	0.9395	0.8511	0.8276	0.7846	0.9541	0.6403	0.8029	24.07
MedSAM2	0.4180	0.9412	0.8381	0.7252	0.5127	0.8050	0.3330	0.6533	69.07
SAM-Med3D	0.6447	0.9103	0.6881	0.8730	0.8712	0.8487	0.8263	0.8089	6.21
STU-Net-B	0.9141	0.8890	0.8597	0.8858	0.4780	0.9465	0.8670	0.8343	14.05
nnU-Netv2	0.9261	0.2540	0.8202	0.8461	0.2027	0.6435	0.5713	0.6091	29.13

Table 2: Comparison with external baseline models. CT models are trained on sites A+B and evaluated on site G; MRI models are trained on C/D and evaluated on site E. Dice is reported per structure and averaged across foreground classes.

Modality	Site	Spacing (mm)	WH Vol. (mL)	Med. intensity	Low/high-freq. ratio
CT	A	$0.43\times 0.58$	$594\pm 142$	$-677\pm 837$	$240.8\pm 161.1$
CT	B	$1.00\times 1.00$	$766\pm 308$	$-354\pm 381$	$97.0\pm 163.8$
CT	G	$0.37\times 0.50$	$506\pm 144$	$-59\pm 444$	$89.7\pm 44.6$
MRI	C/D	$0.94\times 1.20$	$753\pm 302$	$72\pm 72$	$54.0\pm 30.1$
MRI	E	$0.94\times 1.27$	$918\pm 251$	$71\pm 52$	$39.1\pm 20.9$

Table 3: Site characteristics after Bias+Bezier appearance augmentation. Spacing is reported as median in-plane spacing

\times

through-plane spacing in mm. Whole-heart volume, median intensity, and low/high-frequency ratio are reported as mean

\pm

standard deviation.

CT (train: A+B; test: G, $n=20$ )
Variant	Myo	LA	LV	RA	RV	AO	PA	Dice	HD95
Base	0.9183	0.9269	0.8747	0.8708	0.5244	0.9312	0.7991	0.8350	33.25
Base + LCC	0.9229	0.9498	0.9390	0.8868	0.5409	0.9662	0.8044	0.8586	9.12
Bias	0.9485	0.9233	0.9404	0.9099	0.7793	0.9270	0.8363	0.8949	13.01
Bezier	0.9399	0.9103	0.9292	0.9084	0.7460	0.9302	0.8489	0.8876	20.15
Bias + Bezier	0.9476	0.9455	0.9419	0.9017	0.7727	0.9499	0.8656	0.9036	13.66

Table 4: Ablation Study . Appearance augmentation and LCC ablation on held-out sites. CT models are trained on sites A+B and evaluated on site G; MRI models are trained on C/D and evaluated on site E. Dice is reported per structure and averaged across foreground classes; HD95 is the mean 95th-percentile Hausdorff distance (mm).

Medical Image Segmentation

Medical image segmentation delineates organs, lesions, and other structures in scans to support measurement, diagnosis, treatment planning, and research.

See all 788 papers

LISynSeg: Data-Centric Label-to-Image Synthesis for Cross-Modality Whole-Heart Segmentation
Aug 31, 2026Jiacheng Wang, Ivana Isgum, Ipek OguzCross-Modality Medical Image Segmentation Synthetic Data Augmentation
Vanderbilt University, Nashville, TN, USA · Mayo Clinic, Rochester, MN, USA
EAGT: Echocardiography Augmentation for Generalisability and Transferability
May 14, 2026Soroush Elyasi, Sara Adibzadeh, Nasim Dadashi Serej +1Ultrasound Image Segmentation Data Augmentation
THRIVE Centre, University of West London, London, United Kingdom
A Unified Framework for Comprehensive Cardiac CT Segmentation and Phenotyping: Human-in-the-Loop Data Annotation, Vision Foundation Model Development, Multicenter Evaluation and Clinical Validation
Jul 13, 2026Pooya Mohammadi Kazaj, Leo Fridolin Weber, Wen Xie +17Medical Imaging Foundation Models Medical Image Segmentation
Department of Cardiology, Inselspital, Bern University Hospital, University of Bern, Bern, Switzerland · Department of Digital Medicine, University of Bern, Bern, Switzerland · Graduate School for Cellular and Biomedical Sciences, University of Bern, Bern, Switzerland +4

Aug 31, 2026J/K move · Enter open · S save

LISynSeg: Data-Centric Label-to-Image Synthesis for Cross-Modality Whole-Heart Segmentation

Jiacheng Wang, Ivana Isgum, Ipek Oguz

Vanderbilt University, Nashville, TN, USA · Mayo Clinic, Rochester, MN, USA

Cross-Modality Medical Image Segmentation Synthetic Data Augmentation Medical Image Segmentation Medical Image Synthesis

Open paper AskPDF arXiv

Whole-heart segmentation (WHS) in computed tomography (CT) and magnetic resonance imaging (MRI) is affected by acquisition shifts and heterogeneous cardiac annotations. Existing WHS systems combine architectural design, transfer learning, and generic spatial or intensity augmentation. We investigate whether changes to data augmentation and training supervision can improve cross-modality WHS while the segmentation architecture is held constant. We present LISynSeg, a data-centric approach that augments real-image nnU-Net training with label-to-image synthesis. Synthetic volumes are generated from cardiac label maps using contrast and acquisition perturbations calibrated to the training cohort, then mixed with real images to retain thoracic context absent from the labels (and thus the synthesized images). We model cardiac label variation through controlled changes in myocardial wall thickness and partial supervision of uncertain vessel endpoints. On the CARE Whole-Heart benchmark, synthetic-only training performs worse than the real-image nnU-Net baseline, whereas calibrated real-synthetic training improves cross-modality segmentation without changing the architecture; the improvement is larger for MRI than for CT. The results show that modifying the training data strategy can benefit model development for heterogeneous cardiac data. Code and trained weights will be released at https://github.com/MedICL-VU/Care26_LISynSeg.

Abstract

Medical Image Segmentation

Explore similar work

LISynSeg: Data-Centric Label-to-Image Synthesis for Cross-Modality Whole-Heart Segmentation

EAGT: Echocardiography Augmentation for Generalisability and Transferability

A Unified Framework for Comprehensive Cardiac CT Segmentation and Phenotyping: Human-in-the-Loop Data Annotation, Vision Foundation Model Development, Multicenter Evaluation and Clinical Validation