Uncertainty Propagation for Echocardiography Clinical Metric Estimation via Contour Sampling

Thierry Judge; Olivier Bernard; Nicolas Duchateau; Woo-Jin Cho Kim; Alberto Gomez; Arian Beqiri; Agisilaos Chartsias; Pierre-Marc Jodoin

cs.CVFeb 18, 2025

Uncertainty Propagation for Echocardiography Clinical Metric Estimation via Contour Sampling

Authors: Thierry Judge, Olivier Bernard, Nicolas Duchateau, Woo-Jin Cho Kim, Alberto Gomez, Arian Beqiri, Agisilaos Chartsias, Pierre-Marc Jodoin

Organizations: Department of Computer Science, University of Sherbrooke, Sherbrooke, QC, Canada · INSA, Universite Claude Bernard Lyon 1, CNRS UMR 5220, Inserm U1206, CREATIS, Villeurbanne, France · Ultromics Ltd., Oxford, OX4 2SU, UK · King’s College London, London, WC2R 2LS, UK

Abstract

Echocardiography plays a fundamental role in the extraction of clinical parameters (e.g. left ventricular volume and ejection fraction) required to determine the presence and severity of heart-related conditions. When deploying automated techniques for computing these parameters, uncertainty estimation is highly valuable for assessing their utility. Since clinical parameters are usually derived from segmentation maps, there is no clear path for converting pixel-wise uncertainty values into uncertainty estimates in the downstream clinical metric calculation. In this work, we propose a novel uncertainty estimation method based on contouring rather than segmentation. Our method explicitly predicts contour location uncertainty from which contour samples can be drawn. Finally, the sampled contours can be used to propagate uncertainty to clinical metrics. Our proposed method not only provides accurate uncertainty estimations for the task of contouring but also for the downstream clinical metrics on one public and two private cardiac ultrasound datasets containing a total of 1264 patients. Code is available at: https://github.com/ThierryJudge/contouring-uncertainty.

Figures & tables

Fig. 1 : Schematic of the CASUS framework illustrating the three core components: (1) prediction of contour point uncertainty, (2) a contour sampling mechanism, and (3) propagation of sampled contour uncertainties to compute clinical metrics.

Fig. 2 : Example of the posterior shape model output for various partial inputs of a given left ventricle shape.

Panel of Figure 3 — Fig. 3 : (a) Example of the sampling process. Given the initial points (green stars), the posterior shape model output (blue) is merged with the predicted distribution (red) to obtain a distribution (magenta) from which points are sampled (green). (b) The predicted contour and uncertainty distributions in red, and the points sampled with the PSM sampler in blue. (c) Temporal consistent sampling. The orange curve shows an example of the failure case of independent sampling where the ED area (left) is smaller than the ES area (right), resulting in a negative FAC greatly differing from the mean prediction. The time-consistent sampling produces distributions (magenta) that are consistent with the ED contour, resulting in a good contour for ES (green).

Fig. 4 : Example predictions from different methods. Each row corresponds to a different method and each column a specific image. In each image, 10 samples are shown. The top left and right sub-images are the pixel-wise error and uncertainty maps, respectively. For each image, the pixel-wise calibration reliability diagram is plotted in the bottom left corner. The histogram indicates the number of samples in each bin. The last row corresponds to the CASUS method, where the red ellipses correspond to 95% confidence intervals for the location of each point.

Method		CAMUS				Private Card. US
Aleatoric	Epistemic	Dice $\uparrow$	Corr. $\uparrow$	ECE $\downarrow$	MI $\uparrow$	Dice $\uparrow$	Corr. $\uparrow$	ECE $\downarrow$	MI $\uparrow$
PWA	-	.932 $\pm$ .001	.057 $\pm$ .019	.045 $\pm$ .005	.135 $\pm$ .006	.876 $\pm$ 0.004	.139 $\pm$ .110	.282 $\pm$ .009	0.127 $\pm$ 0.005
TTA	-	.931 $\pm$ .002	.252 $\pm$ .022	.044 $\pm$ .003	.209 $\pm$ .004	.873 $\pm$ 0.003	.421 $\pm$ .046	.073 $\pm$ .001	0.199 $\pm$ 0.005
SSN	-	.927 $\pm$ .005	.172 $\pm$ .060	.085 $\pm$ .022	.188 $\pm$ .012	.868 $\pm$ 0.004	.233 $\pm$ .055	.168 $\pm$ .030	0.204 $\pm$ 0.009
CASUS	-	.934 $\pm$ .003	.666 $\pm$ .142	.083 $\pm$ .008	.189 $\pm$ .003	.875 $\pm$ 0.005	.609 $\pm$ .162	.102 $\pm$ .007	0.211 $\pm$ 0.005
CASUS+t	-	.934 $\pm$ .003	.674 $\pm$ .138	.080 $\pm$ .009	.200 $\pm$ .008	.875 $\pm$ 0.005	.606 $\pm$ .147	.093 $\pm$ .007	0.222 $\pm$ 0.005

TABLE I: Mean and standard deviation values segmentation results (5 random seeds). Best and second best results are highlighted in red and blue respectively.

Method		CAMUS				Private Card. US
Aleatoric	Epistemic	Area	FAC	Volume	EF	Area	FAC	Volume	EF
PWA	-	3114 $\pm$ 115	45.7 $\pm$ 1.9 (0.8)	59.7 $\pm$ 2.7	21.8 $\pm$ 2.5 (0.0)	30968 $\pm$ 1718	366.3 $\pm$ 12.6 (3.6)	550.6 $\pm$ 9.3	156.9 $\pm$ 7.0 (0.8)
TTA	-	2685 $\pm$ 166	32.9 $\pm$ 3.9 (0.8)	58.4 $\pm$ 3.7	15.9 $\pm$ 3.2 (0.0)	24565 $\pm$ 538	281.2 $\pm$ 12.6 (3.6)	454.1 $\pm$ 17.2	135.9 $\pm$ 5.7 (0.9)
SSN	-	1866 $\pm$ 615	57.4 $\pm$ 6.6 (0.2)	25.7 $\pm$ 4.0	18.9 $\pm$ 3.5 (0.0)	11528 $\pm$ 6290	312.9 $\pm$ 58.6 (6.2)	156.7 $\pm$ 48.6	68.7 $\pm$ 18.5 (2.5)
CASUS	-	539 $\pm$ 194	18.4 $\pm$ 3.9 (0.6)	13.5 $\pm$ 6.9	8.4 $\pm$ 2.2 (0.0)	10546 $\pm$ 1179	42.0 $\pm$ 4.1 (3.6)	250.3 $\pm$ 34.6	22.1 $\pm$ 2.2 (0.8)
CASUS+t	-	609 $\pm$ 153	10.0 $\pm$ 2.0 (0.4)	17.3 $\pm$ 5.2	6.0 $\pm$ 0.5 (0.0)	8680 $\pm$ 608	49.4 $\pm$ 8.6 (2.7)	219.0 $\pm$ 27.0	36.4 $\pm$ 6.1 (0.0)

TABLE II: Mean and standard deviation for uncertainty calibration error values (5 random seeds). Best and second best results are highlighted in red and blue respectively. The average percentage of rejected samples is indicated in parentheses.

Panel of Figure 7 — Fig. 5 : Reliability diagrams for the [TOP] CAMUS and [BOTTOM] private UK datasets. Each row shows the calibration diagram for each aleatoric method with and without MC Dropout for each metric by column. Dashed lines indicate perfect calibration.

Set	Metric	Patient type
Set	Metric	All	EF $<$ 40%	EF $\geq$ 40%
Train	Num. patients	400	124	276
	Vol. ES (ml)	58 [12–315]	86 [28–315]	45 [12–118]
	Vol. ED (ml)	100 [31–341]	123 [38–341]	89 [31–208]
	EF (%)	44 [6–82]	31 [6–39]	51 [40–82]
Test	Num. patients	50	15	35

TABLE III : Distribution of volume and ejection fraction values for the CAMUS training and testing sets for unhealthy patients (EF

<

40%) and patients with an EF

>

40%. The values represented are the mean, as well as the minimum and maximum.

Method	Group	Error (ml / %) $\downarrow$	UCE (ml / %) $\downarrow$
CASUS+t MC-D	12–69 ml	5.36 $\pm$ 0.48 / 13.4 $\pm$ 1.5	8.07 $\pm$ 1.18 / 22.2 $\pm$ 2.7
CASUS+t MC-D	70–266 ml	8.99 $\pm$ 0.80 / 8.1 $\pm$ 0.7	11.28 $\pm$ 2.62 / 12.1 $\pm$ 3.5
SSN	12–69 ml	8.74 $\pm$ 2.27 / 22.4 $\pm$ 5.2	13.38 $\pm$ 3.50 / 43.0 $\pm$ 6.5
SSN	70–266 ml	11.69 $\pm$ 0.92 / 10.8 $\pm$ 1.5	24.19 $\pm$ 6.78 / 27.6 $\pm$ 5.9

TABLE IV : Mean and standard deviation of dice, volume error, and volume uncertainty calibration error (5 random seeds) for test-set patients, grouped by ground-truth volume. We report both absolute and relative error to allow comparison between groups with different magnitudes.

Method			CAMUS (EF $<40\%$ )			CAMUS (EF $\geq 40\%$ )
Aleatoric	Epistemic	Data	DICE $\uparrow$	Volume UCE $\downarrow$	EF UCE $\downarrow$	DICE $\uparrow$	Volume UCE $\downarrow$	EF UCE $\downarrow$
SSN	-	ALL (N=1600)	.933 $\pm$ .004	16.5 $\pm$ 4.9	8.2 $\pm$ 2.8	.920 $\pm$ .009	25.5 $\pm$ 10.9	13.7 $\pm$ 2.9
SSN	-	EF $\geq 40\%$ (N=1104)	.930 $\pm$ .007	22.7 $\pm$ 11.6	11.9 $\pm$ 3.8	.915 $\pm$ .005	47.7 $\pm$ 39.9	20.8 $\pm$ 12.0
CASUS+t	MC-D	ALL (N=1600)	.939 $\pm$ .004	10.4 $\pm$ 1.7	4.4 $\pm$ 0.9	.933 $\pm$ .003	10.1 $\pm$ 2.3	6.9 $\pm$ 2.2
CASUS+t	MC-D	EF $\geq 40\%$ (N=1104)	.937 $\pm$ .003	10.7 $\pm$ 4.2	4.4 $\pm$ 0.8	.931 $\pm$ .003	10.6 $\pm$ 1.2	6.5 $\pm$ 1.7

TABLE V: Mean and standard deviation for dice and uncertainty calibration error values (5 random seeds) on patients with ejection fraction above and below 40%.

Panel of Figure 11 — Fig. 6 : Reliability diagrams for stochastic segmentation network (SSN) and our method (CASUS+t MC-D).

Panel of Figure 12 — Fig. 7 : Predictive distributions and sampled contours produced by the CASUS-t for (a,b) the two samples with the smallest ES volumes (15 ml and 12 ml respectively) and (c) the two-chamber view (left) and the four-chamber view (right) of a patient with a septal bulge (EF 73%).

Panel of Figure 13 — Fig. 8 : Samples from the CAMUS test set. (a) Sample with the lowest EF. (b) Sample with low EF and good image quality. (c) Sample with low EF and low contrast on the lateral wall. (c) Sample with low EF and poor image quality (d) Sample with high EF.

	HCM-CA	HCM-HT	ICM	NICM
Num. patients	20	20	20	20
Vol. ES (ml)	54 [33–93]	61 [18–109]	87 [45–147]	87 [40–161]
Vol. ED (ml)	84 [51–137]	102 [38–194]	118 [66–175]	116 [58–207]
EF (%)	35 [14–52]	41 [19–56]	26 [16–40]	26 [14–37]

TABLE VI : Cohort characteristics across pathology subsets on the private French dataset. The values represented are the mean, as well as the minimum and maximum.

Metric	Method	HCM-CA	HCM-HT	ICM	NICM
Dice $\uparrow$	SSN	.904 $\pm$ .006	.930 $\pm$ .003	.933 $\pm$ .005	.932 $\pm$ .004
Dice $\uparrow$	CASUS-t MC-D	.898 $\pm$ .002	.929 $\pm$ .002	.940 $\pm$ .002	.940 $\pm$ .004
Vol. UCE $\downarrow$	SSN	26.9 $\pm$ 8.7	37.7 $\pm$ 22.5	39.0 $\pm$ 25.8	40.6 $\pm$ 22.0
Vol. UCE $\downarrow$	CASUS-t MC-D	27.9 $\pm$ 2.4	11.8 $\pm$ 0.6	11.3 $\pm$ 1.4	9.4 $\pm$ 0.6
EF UCE $\downarrow$	SSN	18.9 $\pm$ 4.0	16.5 $\pm$ 3.7	18.3 $\pm$ 1.8	17.6 $\pm$ 2.8
EF UCE $\downarrow$	CASUS-t MC-D	9.6 $\pm$ 0.7	5.7 $\pm$ 0.5	4.1 $\pm$ 0.6	4.9 $\pm$ 0.7

TABLE VII : Performance of SSN and CASUS across pathology subsets on the private French dataset.

Panel of Figure 16 — Fig. 9 : Samples from the private French test set. (a) Sample from class 2 with good quality and high contrast. The uncertainty is highest along the contour direction, providing little uncertainty during the sampling process. (b) Sample from class 3 with low contrast with high uncertainty perpendicularly to the contour. (c) Sample from class 4 with noisy acquisition as the aorta is visible, causing high uncertainty at the basal valve. (d) Sample from class 4, with reverberation present close to the valves making them less visible and increasing uncertainty in this area.

Fig. 10 : Calibration of CASUS-t MC-D for the left atrium volume on 54 volumes.

Panel of Figure 18 — Fig. 11 : Samples of our method on the left atrium.

Explore similar work

Uncertainty-aware classification and triage of structural heart disease using electrocardiography and echocardiography metrics
May 21, 2026Mitchel J. ColebankBayesian Neural Networks Uncertainty Quantification
Department of Mathematics, University of South Carolina,, Columbia, 29208, South Carolina, USA
VIDS-Seg: Towards Reliable Uncertainty Quantification in Pediatric Cardiac Ultrasound Segmentation
Aug 11, 2026Paul Fischer, Ece OzkanUltrasound Image Segmentation Uncertainty Quantification
Department of Biomedical Engineering, University of Basel, Basel, Switzerland
Echo-E $^3$ Net: Efficient Endocardial Spatio-Temporal Network for Ejection Fraction Estimation
Mar 21, 2025Moein Heidari, Afshin Bozorgpour, AmirHossein Zarif-Fakharnia +5Efficient Neural Network Inference Left Ventricular Ejection Fraction Estimation
University of British Columbia, Vancouver, BC, Canada · University of Regensburg, Regensburg, Germany · Iran University of Science and Technology, Tehran, Iran +1

May 21, 2026J/K move · Enter open · S save

Uncertainty-aware classification and triage of structural heart disease using electrocardiography and echocardiography metrics

Mitchel J. Colebank

Department of Mathematics, University of South Carolina,, Columbia, 29208, South Carolina, USA

Bayesian Neural Networks Uncertainty Quantification Healthcare Clinical Triage Clinical Decision Support Bayesian Inference+2

Open paper AskPDF arXiv

Machine learning methods provide a methodological innovation that can help screen for cardiovascular disease through noninvasive and readily available measurement modalities. Recent investments in using electrocardiogram (ECG) data to screen for structural heart disease (SHD) are one example, where ECGs provide a low-cost, available modality for screening. This has led to the EchoNext dataset, a paired ECG-echocardiogram data repository for testing new methods of SHD detection. However, relatively few studies have investigated how more probabilistic classification through Bayesian inference may improve uncertainty quantification in this setting. Moreover, few studies have considered how triage systems can be developed to alleviate healthcare bottlenecks, such as the review of data from underserved, rural clinics by expert sonographers for SHD assessment. In this study, we leverage existing ECG-echocardiogram data to compare frequentist and Bayesian neural network classifiers. We show that the Bayesian approach is comparable or better than frequentist methods in SHD classification, and that they have a more robust uncertainty quantification attached to them. We provide an example of how this uncertainty-aware classification scheme can be used for screening SHD, providing a proof-of-concept for how machine learning can help with triage in getting individuals expert sonographer input when SHD is highly likely or measurements are highly uncertain.

Abstract

Explore similar work

Uncertainty-aware classification and triage of structural heart disease using electrocardiography and echocardiography metrics

VIDS-Seg: Towards Reliable Uncertainty Quantification in Pediatric Cardiac Ultrasound Segmentation

Echo-E3^33Net: Efficient Endocardial Spatio-Temporal Network for Ejection Fraction Estimation

Echo-E $^3$ Net: Efficient Endocardial Spatio-Temporal Network for Ejection Fraction Estimation