Incorporating synthetic data from diffusion models supports echocardiography downstream tasks
Organizations: Department of Electrical and Computer Engineering, University of British Columbia, Vancouver, BC, Canada · School of Biomedical Engineering, University of British Columbia, Vancouver, BC, Canada · Department of Orthopaedics, University of British Columbia, Vancouver, BC, Canada · Department of Radiology, University of British Columbia, Vancouver, BC, Canada ·
Abstract
Purpose: Large labeled echocardiography datasets are costly to collect. We use diffusion models to generate synthetic training data for segmentation and cardiac phase classification. Approach: We study three generation approaches on CAMUS: an unconditional image space model, a latent model conditioned on plain English prompts or abstract tokens, and a ControlNet model conditioned on an abstract token and segmentation mask. We evaluate synthesis with Fréchet Inception Distance (FID) and augmentation using segmentation and classification of end diastole and end systole. We also evaluate segmentation within CardiacUDC and directly from CAMUS to CardiacUDC. Results: Conditioning on an abstract token and segmentation mask gives the lowest mean FID, 1.5 compared with 3.2 for the Semantic Diffusion Model. Abstract token augmentation gives the highest classification accuracy, improving ResNet-18 from 83.8% to 87.2% and VGG-16 from 74.5% to 78.7%. Synthetic augmentation gives the highest mean Dice of 87.6% for the CAMUS 2D U-Net and 84.6% for the CardiacUDC nnU-Net. When nnU-Net models are trained on CAMUS and evaluated directly on CardiacUDC, augmentation raises mean Dice from 74.6% to 75.2%. Conclusions: Incorporating synthetic images into training gives modest gains in CAMUS phase classification, segmentation within CAMUS and CardiacUDC, and direct segmentation from CAMUS to CardiacUDC.
Figures & tables
| Method | FID | KID | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| 2CH-ED | 2CH-ES | 4CH-ED | 4CH-ES | Mean | 2CH-ED | 2CH-ES | 4CH-ED | 4CH-ES | Mean | |
| Unconditional | 4.8 0.2 | 4.6 0.2 | 5.2 0.2 | 4.1 0.2 | 4.7 0.2 | 1.5 0.2 | 2.6 0.2 | 2.6 0.2 | 2.0 0.2 | 2.2 0.2 |
| Plain English | 1.6 0.1 | 2.5 0.2 | 1.9 0.1 | 3.0 0.2 | 2.3 0.1 | 1.4 0.1 | 2.6 0.2 | 1.7 0.1 | 2.7 0.2 | 2.1 0.1 |
| Abstract token | 1.5 0.1 | 2.3 0.1 | 1.8 0.1 | 2.9 0.2 | 2.1 0.1 | 1.3 0.1 | 2.5 0.2 | 1.6 0.1 | 2.7 0.2 | 2.0 0.2 |
| SDM | 1.8 0.1 | 1.8 0.1 | 4.5 0.2 | 4.5 0.2 | 3.2 0.2 | 2.8 0.2 | 2.9 0.2 | 5.4 0.3 | 5.4 0.3 | 4.1 0.3 |
| Abstract token and segmentation | 1.4 0.1 | 1.6 0.1 | 1.6 0.1 | 1.3 0.1 | 1.5 0.1 | 1.3 0.1 | 1.5 0.1 | 1.6 0.1 | 1.6 0.1 | 1.5 0.1 |
| Prompt | FID | KID | ResNet-18 | VGG-16 | ||
| ACC (%) | ACC (%) | |||||
| Real | N/A | 83.8 | N/A | 74.5 | N/A | |
| Plain English | 2.3 | 2.1 | 86.6 | 78.0 | ||
| Abstract token | 2.1 | 2.0 | 87.2 | 0.11 | 78.7 | 0.08 |
| Condition | Correct view (%) | Right chambers (%) | Dice (%) |
|---|---|---|---|
| Segmentation mask only | 82 | 68 | 86.3 |
| Matching abstract token and segmentation mask | 94 | 90 | 86.7 |
| Nonmatching abstract token and segmentation mask | 43 | 39 | 85.9 |
| Training data | Dice (%) | HD | ASD | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Mean | LV-endo | LV-epi | LA | Mean | LV-endo | LV-epi | LA | Mean | LV-endo | LV-epi | LA | ||
| Real | 86.9 0.3 | N/A | 89.0 | 92.1 | 79.6 | 15.0 | 9.6 | 10.9 | 24.5 | 5.4 | 3.8 | 4.0 | 8.3 |
| Real+50% | 87.2 0.3 | 0.01 | 89.4 | 92.3 | 79.9 | 12.1 | 9.2 | 9.7 | 17.3 | 4.9 | 3.7 | 3.7 | 7.2 |
| Real+100% | 87.6 0.3 | 89.9 | 92.5 | 80.4 | 12.5 | 9.3 | 10.4 | 17.7 | 4.8 | 3.8 | 3.8 | 6.9 | |
| Real+200% | 87.5 0.5 | 0.02 | 89.9 | 92.3 | 80.2 | 11.8 | 9.4 | 10.0 | 16.0 | 4.9 | 3.9 | 3.8 | 6.9 |
| SDM | 85.7 0.4 | 86.5 | 90.0 | 80.6 | 13.7 | 12.7 | 11.6 | 16.9 | 5.2 | 4.6 | 4.6 | 6.5 | |
| Training data | Metrics | ||||
|---|---|---|---|---|---|
| ACC (%) | PR (%) | RC (%) | F1 (%) | ||
| ResNet-18 | |||||
| Real | 83.8 | N/A | 84.0 | 83.8 | 83.7 |
| Real+Unconditional | 84.2 | 0.55 | 84.4 | 84.2 | 83.9 |
| Real+Abstract token | 87.2 | 87.4 | 87.2 | 87.0 | |
| Real+SDM | 83.4 | 0.53 | 83.6 | 83.4 | 83.3 |
| Training data | Dice (%) | HD | ASD | |||||||||||||
| Mean | LV | LA | RA | RV | Mean | LV | LA | RA | RV | Mean | LV | LA | RA | RV | ||
| 2D U-Net | ||||||||||||||||
| Real | 79.7 | N/A | 81.7 | 75.6 | 82.7 | 78.6 | 17.7 | 16.9 | 20.7 | 18.6 | 14.7 | 3.7 | 3.4 | 3.9 | 3.6 | 4.0 |
| Real+50% | 80.2 | 82.2 | 76.4 | 83.1 | 79.2 | 16.2 | 15.9 | 18.7 | 16.8 | 13.4 | 3.4 | 3.1 | 3.6 | 3.3 | 3.6 | |
| Real+100% | 80.7 | 82.7 | 77.1 | 83.5 | 79.6 | 15.4 | 15.2 | 17.5 | 15.9 | 13.0 | 3.1 | 2.9 | 3.3 | 3.0 | 3.2 | |
| Real+200% | 80.8 | 82.8 | 77.0 | 83.5 | 79.7 | 15.1 | 14.9 | 17.1 | 15.6 | 12.8 | 3.0 | 2.8 | 3.2 | 2.9 | 3.1 | |
Echocardiography
Echocardiography uses ultrasound to image the heart and assess its structure, motion, blood flow, and clinical function.
See all 64 papers