Organizations: Department of Electrical and Computer Engineering, University of British Columbia, Vancouver, BC, Canada · School of Biomedical Engineering, University of British Columbia, Vancouver, BC, Canada · Department of Orthopaedics, University of British Columbia, Vancouver, BC, Canada · Department of Radiology, University of British Columbia, Vancouver, BC, Canada · Department of Medicine, University of British Columbia, Vancouver, BC, Canada
Purpose: Large labeled echocardiography datasets are costly to collect. We use diffusion models to generate synthetic training data for segmentation and cardiac phase classification. Approach: We study three generation approaches on CAMUS: an unconditional image space model, a latent model conditioned on plain English prompts or abstract tokens, and a ControlNet model conditioned on an abstract token and segmentation mask. We evaluate synthesis with Fréchet Inception Distance (FID) and augmentation using segmentation and classification of end diastole and end systole. We also evaluate segmentation within CardiacUDC and directly from CAMUS to CardiacUDC. Results: Conditioning on an abstract token and segmentation mask gives the lowest mean FID, 1.5 compared with 3.2 for the Semantic Diffusion Model. Abstract token augmentation gives the highest classification accuracy, improving ResNet-18 from 83.8% to 87.2% and VGG-16 from 74.5% to 78.7%. Synthetic augmentation gives the highest mean Dice of 87.6% for the CAMUS 2D U-Net and 84.6% for the CardiacUDC nnU-Net. When nnU-Net models are trained on CAMUS and evaluated directly on CardiacUDC, augmentation raises mean Dice from 74.6% to 75.2%. Conclusions: Incorporating synthetic images into training gives modest gains in CAMUS phase classification, segmentation within CAMUS and CardiacUDC, and direct segmentation from CAMUS to CardiacUDC.
Figures & tables
Figure 1: Overview of the conditioned latent diffusion models. Both plain English prompts and abstract tokens use the fixed CLIP text encoder [ 25 ] . For generation with an abstract token and segmentation mask, the image encoder block processes the map within the trainable ControlNet branch. The Stable Diffusion U-Net remains fixed.
Method
FID ↓
KID ↓
2CH-ED
2CH-ES
4CH-ED
4CH-ES
Mean
2CH-ED
2CH-ES
4CH-ED
4CH-ES
Mean
Unconditional
4.8 ± 0.2
4.6 ± 0.2
5.2 ± 0.2
4.1 ± 0.2
4.7 ± 0.2
1.5 ± 0.2
2.6 ± 0.2
2.6 ± 0.2
2.0 ± 0.2
2.2 ± 0.2
Plain English
1.6 ± 0.1
2.5 ± 0.2
1.9 ± 0.1
3.0 ± 0.2
2.3 ± 0.1
1.4 ± 0.1
2.6 ± 0.2
1.7 ± 0.1
2.7 ± 0.2
2.1 ± 0.1
Abstract token
1.5 ± 0.1
2.3 ± 0.1
1.8 ± 0.1
2.9 ± 0.2
2.1 ± 0.1
1.3 ± 0.1
2.5 ± 0.2
1.6 ± 0.1
2.7 ± 0.2
2.0 ± 0.2
SDM
1.8 ± 0.1
1.8 ± 0.1
4.5 ± 0.2
4.5 ± 0.2
3.2 ± 0.2
2.8 ± 0.2
2.9 ± 0.2
5.4 ± 0.3
5.4 ± 0.3
4.1 ± 0.3
Abstract token and segmentation
1.4 ± 0.1
1.6 ± 0.1
1.6 ± 0.1
1.3 ± 0.1
1.5 ± 0.1
1.3 ± 0.1
1.5 ± 0.1
1.6 ± 0.1
1.6 ± 0.1
1.5 ± 0.1
Table 1: Comparison of generation systems and prompt representations using FID and KID. Values are means across generation seeds, and the value after ± is SD. Bold indicates the best results.
Figure 2: Synthetic examples from the three generation settings. Rows from top to bottom show unconditional generation, generation with an abstract token, and generation with an abstract token and segmentation mask. Columns from left to right show 2CH-ED, 2CH-ES, 4CH-ED, and 4CH-ES.
Figure 3: Generation results from the model conditioned on an abstract token and segmentation mask and from SDM [ 21 ] . Columns from left to right show the output from our model, SDM output, segmentation map, and real image. Rows show 2CH-ES, 4CH-ED, and 4CH-ES. Green, red, and yellow boxes mark the right chambers, mitral valve, and tricuspid valve.
Prompt
FID ↓
KID ↓
ResNet-18
VGG-16
ACC (%) ↑
p
ACC (%) ↑
p
Real
N/A
83.8
N/A
74.5
N/A
Plain English
2.3
2.1
86.6
<0.01
78.0
<0.01
Abstract token
2.1
2.0
87.2
0.11
78.7
0.08
Table 2: Prompt representation ablation using mean FID, mean KID, and mean ACC. Each p column reports the paired two sided Student t test for ACC. Plain English is compared with Real, and Abstract token is compared with Plain English.
Condition
Correct view (%) ↑
Right chambers (%) ↑
Dice (%) ↑
Segmentation mask only
82
68
86.3
Matching abstract token and segmentation mask
94
90
86.7
Nonmatching abstract token and segmentation mask
43
39
85.9
Table 3: Contribution of the conditioning inputs using correct view rate, right chamber presence in 4CH, and mask agreement Dice. Values are aggregate percentages.
Training data
Dice (%) ↑
HD ↓
ASD ↓
Mean
p
LV-endo
LV-epi
LA
Mean
LV-endo
LV-epi
LA
Mean
LV-endo
LV-epi
LA
Real
86.9 ± 0.3
N/A
89.0
92.1
79.6
15.0
9.6
10.9
24.5
5.4
3.8
4.0
8.3
Real+50%
87.2 ± 0.3
0.01
89.4
92.3
79.9
12.1
9.2
9.7
17.3
4.9
3.7
3.7
7.2
Real+100%
87.6 ± 0.3
<0.01
89.9
92.5
80.4
12.5
9.3
10.4
17.7
4.8
3.8
3.8
6.9
Real+200%
87.5 ± 0.5
0.02
89.9
92.3
80.2
11.8
9.4
10.0
16.0
4.9
3.9
3.8
6.9
SDM
85.7 ± 0.4
<0.01
86.5
90.0
80.6
13.7
12.7
11.6
16.9
5.2
4.6
4.6
6.5
Table 4: CAMUS segmentation results using Dice, HD, and ASD. Values are means across five seeds. Mean Dice is followed by its sample SD. The p column compares mean Dice for each condition with Real using the paired two sided Student t test. HD and ASD are measured in pixels at 256×256 resolution. Bold indicates the best results.
Figure 4: CAMUS segmentation examples. Columns from left to right show the image, reference mask, Real prediction, Real+100% prediction, and SDM prediction.
Training data
Metrics
ACC (%) ↑
p
PR (%) ↑
RC (%) ↑
F1 (%) ↑
ResNet-18
Real
83.8
N/A
84.0
83.8
83.7
Real+Unconditional
84.2
0.55
84.4
84.2
83.9
Real+Abstract token
87.2
<0.01
87.4
87.2
87.0
Real+SDM
83.4
0.53
83.6
83.4
83.3
Table 5: Downstream ED/ES classification results using ACC, PR, RC, and F1. Values are means across five seeds. The p column compares ACC with Real for the same backbone using the paired two sided Student t test. Bold indicates the best results.
Training data
Dice (%) ↑
HD ↓
ASD ↓
Mean
p
LV
LA
RA
RV
Mean
LV
LA
RA
RV
Mean
LV
LA
RA
RV
2D U-Net
Real
79.7
N/A
81.7
75.6
82.7
78.6
17.7
16.9
20.7
18.6
14.7
3.7
3.4
3.9
3.6
4.0
Real+50%
80.2
<0.01
82.2
76.4
83.1
79.2
16.2
15.9
18.7
16.8
13.4
3.4
3.1
3.6
3.3
3.6
Real+100%
80.7
<0.01
82.7
77.1
83.5
79.6
15.4
15.2
17.5
15.9
13.0
3.1
2.9
3.3
3.0
3.2
Real+200%
80.8
<0.01
82.8
77.0
83.5
79.7
15.1
14.9
17.1
15.6
12.8
3.0
2.8
3.2
2.9
3.1
Table 6: CardiacUDC segmentation results using Dice, HD, and ASD. Panel (a) trains and selects models on CardiacUDC. Panel (b) trains and selects models on CAMUS and tests them directly on CardiacUDC. Values are means across five runs. Each p column compares mean Dice with Real for the same architecture. HD and ASD are in pixels at 256×256 resolution.
Figure 5: CAMUS segmentation errors for the model trained with Real+100%. Columns from left to right show the image, reference mask, and predicted mask.
Segmentation of enhancement in LGE cardiac MRI is critical for diagnosing various ischemic and non-ischemic cardiomyopathies. However, creating pixel-level annotations for these images is challenging and labor-intensive, leading to limited availability of annotated data. Generative models, particularly diffusion models, offer promise for synthetic data generation, yet many rely on large training datasets and often struggle with fine-grained conditioning control, especially for small or localized features. We introduce LGESynthNet, a latent diffusion-based framework for controllable enhancement synthesis, enabling explicit control over size, location, and transmural extent. Formulated as inpainting using a ControlNet-based architecture, the model integrates: (a) a reward model for conditioning-specific supervision, (b) a captioning module for anatomically descriptive text prompts, and (c) a biomedical text encoder. Trained on just 429 images (79 patients), it produces realistic, anatomically coherent samples. A quality control filter selects outputs with high conditioning-fidelity, which when used for training augmentation, improve downstream segmentation and detection performance, by up-to 6 and 20 points respectively.
Athira J. Jacob, Puneet Sharma, Daniel Rueckert
Digital Technology and Innovation, Siemens Healthineers, Princeton, NJ, USA · Al in Healthcare and Medicine, Klinikum rechts der Isar, Technical University of Munich, Germany · Department of Computing, Imperial College London, UK
Whole-heart segmentation (WHS) in computed tomography (CT) and magnetic resonance imaging (MRI) is affected by acquisition shifts and heterogeneous cardiac annotations. Existing WHS systems combine architectural design, transfer learning, and generic spatial or intensity augmentation. We investigate whether changes to data augmentation and training supervision can improve cross-modality WHS while the segmentation architecture is held constant. We present LISynSeg, a data-centric approach that augments real-image nnU-Net training with label-to-image synthesis. Synthetic volumes are generated from cardiac label maps using contrast and acquisition perturbations calibrated to the training cohort, then mixed with real images to retain thoracic context absent from the labels (and thus the synthesized images). We model cardiac label variation through controlled changes in myocardial wall thickness and partial supervision of uncertain vessel endpoints. On the CARE Whole-Heart benchmark, synthetic-only training performs worse than the real-image nnU-Net baseline, whereas calibrated real-synthetic training improves cross-modality segmentation without changing the architecture; the improvement is larger for MRI than for CT. The results show that modifying the training data strategy can benefit model development for heterogeneous cardiac data. Code and trained weights will be released at https://github.com/MedICL-VU/Care26_LISynSeg.
Jiacheng Wang, Ivana Isgum, Ipek Oguz
Vanderbilt University, Nashville, TN, USA · Mayo Clinic, Rochester, MN, USA
Deep learning models for echocardiography segmentation often struggle to generalise across institutions, scanners, and patient populations, where collecting large, consistently annotated datasets is infeasible. Data augmentation is inexpensive and widely used to improve the robustness of deep learning models; however, its role in enhancing cross-dataset generalisability in echocardiography remains insufficiently understood. This study presents a large-scale multi-dataset evaluation of 29 data augmentation techniques and their pairwise combinations for 2D left ventricular segmentation using a U-Net trained on Unity, CAMUS, and EchoNet Dynamic datasets. Each augmentation was explored under several hyperparameter settings and assessed through repeated runs using Dice and IoU in both in-domain and cross-dataset scenarios, with statistical significance quantified via independent t-tests. In-domain accuracy was near-saturated and insensitive to augmentation, whereas cross-dataset performance varied widely. Geometry-based augmentations including affine, shift-scale-rotate, flip, and perspective produced the largest and most consistent gains, while aggressive intensity- and artefact-based transforms often degraded transfer. Moreover, pairwise combinations outperformed individual augmentations mainly when the two transformations were complementary, particularly by improving some difficult domain-shift cases from poor to acceptable performance. These findings provide empirical guidance for designing augmentation policies that improve the robustness and transferability of echocardiography segmentation models.
Soroush Elyasi, Sara Adibzadeh, Nasim Dadashi Serej +1
THRIVE Centre, University of West London, London, United Kingdom