Uncertainty Propagation for Echocardiography Clinical Metric Estimation via Contour Sampling
Authors: Thierry Judge, Olivier Bernard, Nicolas Duchateau, Woo-Jin Cho Kim, Alberto Gomez, Arian Beqiri, Agisilaos Chartsias, Pierre-Marc Jodoin
Organizations: Department of Computer Science, University of Sherbrooke, Sherbrooke, QC, Canada · INSA, Universite Claude Bernard Lyon 1, CNRS UMR 5220, Inserm U1206, CREATIS, Villeurbanne, France · Ultromics Ltd., Oxford, OX4 2SU, UK · King’s College London, London, WC2R 2LS, UK
Echocardiography plays a fundamental role in the extraction of clinical parameters (e.g. left ventricular volume and ejection fraction) required to determine the presence and severity of heart-related conditions. When deploying automated techniques for computing these parameters, uncertainty estimation is highly valuable for assessing their utility. Since clinical parameters are usually derived from segmentation maps, there is no clear path for converting pixel-wise uncertainty values into uncertainty estimates in the downstream clinical metric calculation. In this work, we propose a novel uncertainty estimation method based on contouring rather than segmentation. Our method explicitly predicts contour location uncertainty from which contour samples can be drawn. Finally, the sampled contours can be used to propagate uncertainty to clinical metrics. Our proposed method not only provides accurate uncertainty estimations for the task of contouring but also for the downstream clinical metrics on one public and two private cardiac ultrasound datasets containing a total of 1264 patients. Code is available at: https://github.com/ThierryJudge/contouring-uncertainty.
Figures & tables
Fig. 1 : Schematic of the CASUS framework illustrating the three core components: (1) prediction of contour point uncertainty, (2) a contour sampling mechanism, and (3) propagation of sampled contour uncertainties to compute clinical metrics.
Fig. 2 : Example of the posterior shape model output for various partial inputs of a given left ventricle shape.
Fig. 3 : (a) Example of the sampling process. Given the initial points (green stars), the posterior shape model output (blue) is merged with the predicted distribution (red) to obtain a distribution (magenta) from which points are sampled (green). (b) The predicted contour and uncertainty distributions in red, and the points sampled with the PSM sampler in blue. (c) Temporal consistent sampling. The orange curve shows an example of the failure case of independent sampling where the ED area (left) is smaller than the ES area (right), resulting in a negative FAC greatly differing from the mean prediction. The time-consistent sampling produces distributions (magenta) that are consistent with the ED contour, resulting in a good contour for ES (green).
Fig. 4 : Example predictions from different methods. Each row corresponds to a different method and each column a specific image. In each image, 10 samples are shown. The top left and right sub-images are the pixel-wise error and uncertainty maps, respectively. For each image, the pixel-wise calibration reliability diagram is plotted in the bottom left corner. The histogram indicates the number of samples in each bin. The last row corresponds to the CASUS method, where the red ellipses correspond to 95% confidence intervals for the location of each point.
Method
CAMUS
Private Card. US
Aleatoric
Epistemic
Dice ↑
Corr. ↑
ECE ↓
MI ↑
Dice ↑
Corr. ↑
ECE ↓
MI ↑
PWA
-
.932 ± .001
.057 ± .019
.045 ± .005
.135 ± .006
.876 ± 0.004
.139 ± .110
.282 ± .009
0.127 ± 0.005
TTA
-
.931 ± .002
.252 ± .022
.044 ± .003
.209 ± .004
.873 ± 0.003
.421 ± .046
.073 ± .001
0.199 ± 0.005
SSN
-
.927 ± .005
.172 ± .060
.085 ± .022
.188 ± .012
.868 ± 0.004
.233 ± .055
.168 ± .030
0.204 ± 0.009
CASUS
-
.934 ± .003
.666 ± .142
.083 ± .008
.189 ± .003
.875 ± 0.005
.609 ± .162
.102 ± .007
0.211 ± 0.005
CASUS+t
-
.934 ± .003
.674 ± .138
.080 ± .009
.200 ± .008
.875 ± 0.005
.606 ± .147
.093 ± .007
0.222 ± 0.005
TABLE I: Mean and standard deviation values segmentation results (5 random seeds). Best and second best results are highlighted in red and blue respectively.
Method
CAMUS
Private Card. US
Aleatoric
Epistemic
Area
FAC
Volume
EF
Area
FAC
Volume
EF
PWA
-
3114 ± 115
45.7 ± 1.9 (0.8)
59.7 ± 2.7
21.8 ± 2.5 (0.0)
30968 ± 1718
366.3 ± 12.6 (3.6)
550.6 ± 9.3
156.9 ± 7.0 (0.8)
TTA
-
2685 ± 166
32.9 ± 3.9 (0.8)
58.4 ± 3.7
15.9 ± 3.2 (0.0)
24565 ± 538
281.2 ± 12.6 (3.6)
454.1 ± 17.2
135.9 ± 5.7 (0.9)
SSN
-
1866 ± 615
57.4 ± 6.6 (0.2)
25.7 ± 4.0
18.9 ± 3.5 (0.0)
11528 ± 6290
312.9 ± 58.6 (6.2)
156.7 ± 48.6
68.7 ± 18.5 (2.5)
CASUS
-
539 ± 194
18.4 ± 3.9 (0.6)
13.5 ± 6.9
8.4 ± 2.2 (0.0)
10546 ± 1179
42.0 ± 4.1 (3.6)
250.3 ± 34.6
22.1 ± 2.2 (0.8)
CASUS+t
-
609 ± 153
10.0 ± 2.0 (0.4)
17.3 ± 5.2
6.0 ± 0.5 (0.0)
8680 ± 608
49.4 ± 8.6 (2.7)
219.0 ± 27.0
36.4 ± 6.1 (0.0)
TABLE II: Mean and standard deviation for uncertainty calibration error values (5 random seeds). Best and second best results are highlighted in red and blue respectively. The average percentage of rejected samples is indicated in parentheses.
Fig. 5 : Reliability diagrams for the [TOP] CAMUS and [BOTTOM] private UK datasets. Each row shows the calibration diagram for each aleatoric method with and without MC Dropout for each metric by column. Dashed lines indicate perfect calibration.
Set
Metric
Patient type
All
EF < 40%
EF ≥ 40%
Train
Num. patients
400
124
276
Vol. ES (ml)
58 [12–315]
86 [28–315]
45 [12–118]
Vol. ED (ml)
100 [31–341]
123 [38–341]
89 [31–208]
EF (%)
44 [6–82]
31 [6–39]
51 [40–82]
Test
Num. patients
50
15
35
TABLE III : Distribution of volume and ejection fraction values for the CAMUS training and testing sets for unhealthy patients (EF < 40%) and patients with an EF > 40%. The values represented are the mean, as well as the minimum and maximum.
Method
Group
Error (ml / %) ↓
UCE (ml / %) ↓
CASUS+t MC-D
12–69 ml
5.36 ± 0.48 / 13.4 ± 1.5
8.07 ± 1.18 / 22.2 ± 2.7
70–266 ml
8.99 ± 0.80 / 8.1 ± 0.7
11.28 ± 2.62 / 12.1 ± 3.5
SSN
12–69 ml
8.74 ± 2.27 / 22.4 ± 5.2
13.38 ± 3.50 / 43.0 ± 6.5
70–266 ml
11.69 ± 0.92 / 10.8 ± 1.5
24.19 ± 6.78 / 27.6 ± 5.9
TABLE IV : Mean and standard deviation of dice, volume error, and volume uncertainty calibration error (5 random seeds) for test-set patients, grouped by ground-truth volume. We report both absolute and relative error to allow comparison between groups with different magnitudes.
Method
CAMUS (EF <40% )
CAMUS (EF ≥40% )
Aleatoric
Epistemic
Data
DICE ↑
Volume UCE ↓
EF UCE ↓
DICE ↑
Volume UCE ↓
EF UCE ↓
SSN
-
ALL (N=1600)
.933 ± .004
16.5 ± 4.9
8.2 ± 2.8
.920 ± .009
25.5 ± 10.9
13.7 ± 2.9
SSN
-
EF ≥40% (N=1104)
.930 ± .007
22.7 ± 11.6
11.9 ± 3.8
.915 ± .005
47.7 ± 39.9
20.8 ± 12.0
CASUS+t
MC-D
ALL (N=1600)
.939 ± .004
10.4 ± 1.7
4.4 ± 0.9
.933 ± .003
10.1 ± 2.3
6.9 ± 2.2
CASUS+t
MC-D
EF ≥40% (N=1104)
.937 ± .003
10.7 ± 4.2
4.4 ± 0.8
.931 ± .003
10.6 ± 1.2
6.5 ± 1.7
TABLE V: Mean and standard deviation for dice and uncertainty calibration error values (5 random seeds) on patients with ejection fraction above and below 40%.
Fig. 6 : Reliability diagrams for stochastic segmentation network (SSN) and our method (CASUS+t MC-D).
Fig. 7 : Predictive distributions and sampled contours produced by the CASUS-t for (a,b) the two samples with the smallest ES volumes (15 ml and 12 ml respectively) and (c) the two-chamber view (left) and the four-chamber view (right) of a patient with a septal bulge (EF 73%).
Fig. 8 : Samples from the CAMUS test set. (a) Sample with the lowest EF. (b) Sample with low EF and good image quality. (c) Sample with low EF and low contrast on the lateral wall. (c) Sample with low EF and poor image quality (d) Sample with high EF.
HCM-CA
HCM-HT
ICM
NICM
Num. patients
20
20
20
20
Vol. ES (ml)
54 [33–93]
61 [18–109]
87 [45–147]
87 [40–161]
Vol. ED (ml)
84 [51–137]
102 [38–194]
118 [66–175]
116 [58–207]
EF (%)
35 [14–52]
41 [19–56]
26 [16–40]
26 [14–37]
TABLE VI : Cohort characteristics across pathology subsets on the private French dataset. The values represented are the mean, as well as the minimum and maximum.
Metric
Method
HCM-CA
HCM-HT
ICM
NICM
Dice ↑
SSN
.904 ± .006
.930 ± .003
.933 ± .005
.932 ± .004
CASUS-t MC-D
.898 ± .002
.929 ± .002
.940 ± .002
.940 ± .004
Vol. UCE ↓
SSN
26.9 ± 8.7
37.7 ± 22.5
39.0 ± 25.8
40.6 ± 22.0
CASUS-t MC-D
27.9 ± 2.4
11.8 ± 0.6
11.3 ± 1.4
9.4 ± 0.6
EF UCE ↓
SSN
18.9 ± 4.0
16.5 ± 3.7
18.3 ± 1.8
17.6 ± 2.8
CASUS-t MC-D
9.6 ± 0.7
5.7 ± 0.5
4.1 ± 0.6
4.9 ± 0.7
TABLE VII : Performance of SSN and CASUS across pathology subsets on the private French dataset.
Fig. 9 : Samples from the private French test set. (a) Sample from class 2 with good quality and high contrast. The uncertainty is highest along the contour direction, providing little uncertainty during the sampling process. (b) Sample from class 3 with low contrast with high uncertainty perpendicularly to the contour. (c) Sample from class 4 with noisy acquisition as the aorta is visible, causing high uncertainty at the basal valve. (d) Sample from class 4, with reverberation present close to the valves making them less visible and increasing uncertainty in this area.
Fig. 10 : Calibration of CASUS-t MC-D for the left atrium volume on 54 volumes.
Fig. 11 : Samples of our method on the left atrium.
Machine learning methods provide a methodological innovation that can help screen for cardiovascular disease through noninvasive and readily available measurement modalities. Recent investments in using electrocardiogram (ECG) data to screen for structural heart disease (SHD) are one example, where ECGs provide a low-cost, available modality for screening. This has led to the EchoNext dataset, a paired ECG-echocardiogram data repository for testing new methods of SHD detection. However, relatively few studies have investigated how more probabilistic classification through Bayesian inference may improve uncertainty quantification in this setting. Moreover, few studies have considered how triage systems can be developed to alleviate healthcare bottlenecks, such as the review of data from underserved, rural clinics by expert sonographers for SHD assessment. In this study, we leverage existing ECG-echocardiogram data to compare frequentist and Bayesian neural network classifiers. We show that the Bayesian approach is comparable or better than frequentist methods in SHD classification, and that they have a more robust uncertainty quantification attached to them. We provide an example of how this uncertainty-aware classification scheme can be used for screening SHD, providing a proof-of-concept for how machine learning can help with triage in getting individuals expert sonographer input when SHD is highly likely or measurements are highly uncertain.
Mitchel J. Colebank
Department of Mathematics, University of South Carolina,, Columbia, 29208, South Carolina, USA
Reliable clinical deployment of machine learning requires models that know when they are likely to fail, particularly for subgroups underrepresented in training data. A common case is pediatric care, where models trained on adult cohorts can silently under-perform on children with no indication that something has gone wrong. As retraining with labeled pediatric data is often infeasible, detecting such failures at inference time is a critical clinical need. Building on the VIDS (Variational Inference under Distribution Shifts) framework, we introduce VIDS-Seg, which applies amortized variational inference over a lightweight prediction head to make this adaptive, OOD-aware prior tractable for dense image segmentation. We evaluate VIDS-Seg on left ventricular segmentation in echocardiography, a setting where pediatric anatomy differs systematically from the adult population most segmentation models are trained on, training on an adult cohort (EchoNet-Dynamic) and evaluating zero-shot on a pediatric cohort (EchoNet-Pediatric). Across all age strata, VIDS-Seg matches competitive baselines in segmentation accuracy while producing substantially higher spatial correspondence between predicted uncertainty and segmentation error, an advantage that persists even after applying temperature scaling to all baselines. Downstream, it yields more accurate and stable ejection fraction estimates and more reliable detection of cardiac malfunction in the infant subgroup. Our results indicate that OOD-aware uncertainty quantification can serve as a practical safety layer for deployed segmentation models, enabling detection of silent failures in underrepresented subgroups without retraining or additional labeled data.
Paul Fischer, Ece Ozkan
Department of Biomedical Engineering, University of Basel, Basel, Switzerland
Left ventricular ejection fraction (LVEF) is a primary marker of cardiac function. However, routine estimation from endocardial measurements requires manual delineation at end-diastole (ED) and end-systole (ES), a process that is time-consuming and subject to inter-observer variability. Reliable automation is especially valuable for point-of-care ultrasound (POCUS), where computational resources are limited and acquisition quality varies. We propose Echo-E3Net, an anatomy-guided spatio-temporal network that explicitly embeds cardiac anatomy into LVEF prediction. A dual-phase Endocardial Border Detector (E2CBD) uses phase-specific cross-attention to localize ED/ES endocardial landmarks and produce phase-aware landmark embeddings, while an Endocardial Feature Aggregator (E2FA) fuses these embeddings with global statistical descriptors of deep feature maps to refine EF regression. Training is guided by a lightweight geometric loss that uses ED and ES endocardial landmarks to regularize EF prediction. On EchoNet-Dynamic and a PSAX subset of EchoNet-Pediatric, Echo-E3Net attains competitive performance using only 1.55M parameters and 8.05 GFLOPs, an order-of-magnitude compute reduction versus recent baselines, supporting real-time deployment. Our code is publicly available at https://github.com/moeinheidari7829/Echo-E3Net.
University of British Columbia, Vancouver, BC, Canada · University of Regensburg, Regensburg, Germany · Iran University of Science and Technology, Tehran, Iran +1