Confidence Matters: Uncertainty Quantification and Precision Assessment of Deep Learning-based CMR Biomarker Estimates Using Scan-rescan Data

Dewmini Hasara Wickremasinghe; Michelle Gibogwe; Andrew Bell; Esther Puyol-Antón; Muhummad Sohaib Nazir; Reza Razavi; Bruno Paun; Paul Aljabar

cs.CVMar 25, 2026

Confidence Matters: Uncertainty Quantification and Precision Assessment of Deep Learning-based CMR Biomarker Estimates Using Scan-rescan Data

Authors: Dewmini Hasara Wickremasinghe, Michelle Gibogwe, Andrew Bell, Esther Puyol-Antón, Muhummad Sohaib Nazir, Reza Razavi, Bruno Paun, Paul Aljabar,

Organizations: School of Biomedical Engineering & Imaging Sciences, King’s College London, London, UK · Cardio-Oncology Centre of Excellence, Royal Brompton Hospitals, London, United Kingdom · Perspectum Ltd., Oxford, UK

Abstract

The performance of deep learning (DL) methods for the analysis of cine cardiovascular magnetic resonance (CMR) is typically assessed in terms of accuracy, overlooking precision. In this work, uncertainty estimation techniques, namely deep ensemble, test-time augmentation, and Monte Carlo dropout, are applied to a state-of-the-art DL pipeline for cardiac functional biomarker estimation, and new distribution-based metrics are proposed for the assessment of biomarker precision. The model achieved high accuracy (average Dice 87%) and point estimate precision on two external validation scan-rescan CMR datasets. However, distribution-based metrics showed that the overlap between scan/rescan confidence intervals was >50% in less than 45% of the cases. Statistical similarity tests between scan and rescan biomarkers also resulted in significant differences for over 65% of the cases. We conclude that, while point estimate metrics might suggest good performance, distributional analyses reveal lower precision, highlighting the need to use more representative metrics to assess scan-rescan agreement.

Figures & tables

Mean $\pm$ standard deviation of differences and CoV values ( $\downarrow$ )
	Roca-HC
	DE		TTA		MC-D
	Mean $\pm$ std	CoV	Mean $\pm$ std	CoV	Mean $\pm$ std	CoV
LVEF	3.21 $\pm$ 0.62	2.52	3.20 $\pm$ 0.77	2.52	3.11 $\pm$ 0.26	2.52
LVM	3.58 $\pm$ 0.90	1.74	3.81 $\pm$ 1.37	1.84	3.58 $\pm$ 0.38	1.84
RVEF	4.01 $\pm$ 0.99	3.38	4.62 $\pm$ 1.99	3.90	4.15 $\pm$ 0.46	3.64

Table 1: Mean and standard deviation of the mean differences between scan and rescan biomarker distributions and CoV between scan A and scan B

Figure 2: Distribution of subjects according to CIoU values

Confidence Precision Percentage (CPP) ( $\uparrow$ )
	Roca-HC
	DE		TTA		MC-D
	A $\rightarrow$ B	B $\rightarrow$ A	A $\rightarrow$ B	B $\rightarrow$ A	A $\rightarrow$ B	B $\rightarrow$ A
LVEF	31.52	28.26	27.17	33.70	6.52	7.61
LVM	39.13	42.39	52.17	57.61	11.96	9.78
RVEF	28.26	26.09	35.87	36.96	7.61	9.78

Table 2: CPP values - Inclusion rate of the mean from distribution A in the CI from B, and vice versa

Precision Difference Percentage (PDP) ( $\downarrow$ )
	Roca-HC			Strain-8
	DE	TTA	MC-D	DE	TTA	MC-D
LVEF	80.43	86.96	95.65	85.00	85.00	95.00
LVM	68.48	72.83	93.48	75.00	75.00	100.00
RVEF	73.91	75.00	95.65	65.00	90.00	90.00

Table 3: PDP values - Rejection rate of the null hypothesis in statistical tests for similarity between scan and rescan biomarker distributions

Figure 3: Uncertainty maps generated for a single case for each method

Uncertainty Quantification

Uncertainty quantification estimates how reliable predictions and scientific inferences are, and how uncertainty propagates into decisions.

See all 649 papers

CineMR: Tool-Integrated Vision-Language Reasoning for Quantitative Cardiac MRI Assessment
Oct 1, 2026Kunyang Li, Hai Nguyen, Joshua Lowe +6Tool Use in VLMs Medical VQA
Institute for Artificial Intelligence, University of Central Florida · Department of Clinical Sciences, College of Medicine, University of Central Florida · Nemours Cardiac Center, Nemours Children’s Hospital, Florida +1
Uncertainty Quantification for Cardiac Shape Reconstruction with Deep Signed Distance Functions via MCMC methods
May 8, 2026Jan Verhülsdonk, Thomas Grandits, Francisco Sahli Costabal +3Uncertainty Quantification 3D Reconstruction
Institute for Applied Mathematics, University of Bonn, Germany · Department of Mathematics and Scientific Computing, University of Graz, Austria and with NAWI Graz · Institute for Biological and Medical Engineering, Pontificia Universidad Católica de Chile, Chile +3
Evaluating and Calibrating Diffusion Model-derived Uncertainty for Quantitative MRI Mapping
Aug 12, 2026Shishuai Wang, Stefan Klein, Juan A. Hernandez-Tamames +1Uncertainty Quantification Selective Prediction
Department of Radiology and Nuclear Medicine, Erasmus MC, Rotterdam, The Netherlands · Department of Imaging Physics, TU Delft, Delft, The Netherlands

Oct 1, 2026J/K move · Enter open · S save

CineMR: Tool-Integrated Vision-Language Reasoning for Quantitative Cardiac MRI Assessment

Kunyang Li, Hai Nguyen, Joshua Lowe, Chenguang Zhao+5

Institute for Artificial Intelligence, University of Central Florida · Department of Clinical Sciences, College of Medicine, University of Central Florida · Nemours Cardiac Center, Nemours Children’s Hospital, Florida+1

Tool Use in VLMs Medical VQA VLM Reasoning Medical Image Benchmarks Cardiac MRI

Open paper AskPDF arXiv

Cardiovascular magnetic resonance (CMR), including cine imaging, is a reference standard for the noninvasive assessment of cardiac morphology and ventricular function. Cine CMR interpretation integrates qualitative visual assessment with quantitative measurements of ventricular volumes, ejection fraction, myocardial mass, wall thickness, and regional wall motion. Current medical vision-language models (VLMs) cannot reliably derive quantitative measurements from multidimensional cine images without analysis tools. We present CineMR, a tool-augmented VLM that invokes cardiac image-analysis tools and integrates their outputs into interleaved reasoning for quantitative CMR assessment. We also construct a multi-cohort visual question answering benchmark covering quantitative metric extraction, multiclass diagnosis, and differential diagnosis, together with tools for segmentation, phase selection, volumetry, morphometry, and regional wall motion analysis. CineMR is trained with supervised fine-tuning (SFT) on tool-interaction traces followed by Group Relative Policy Optimization (GRPO) with conditional tool-use rewards. On the multi-cohort cine CMR benchmark, CineMR achieves 35.9% pass@1 and 58.9% pass@4, compared with 1.5% pass@1 for the Qwen3-VL-8B backbone and 0.0% and 7.0% pass@1 for LLaVA-Med v1.5 and MedGemma-4B, respectively. Correct tool invocation reaches 99.8% after GRPO, up from 78.9% after SFT. Live tool outputs improve ventricular measurement accuracy by 20.4--23.7% over direct model predictions, and removing all tools reduces pass@1 from 35.9% to 27.9%. These results highlight the importance of reliable tool use for quantitative cine CMR reasoning and support CineMR as a promising approach for assistive cardiac image assessment. Code, benchmark resources, and model weights are available at https://github.com/AI-MIND-Lab/CineMR.

Abstract

Uncertainty Quantification

Explore similar work

CineMR: Tool-Integrated Vision-Language Reasoning for Quantitative Cardiac MRI Assessment

Uncertainty Quantification for Cardiac Shape Reconstruction with Deep Signed Distance Functions via MCMC methods

Evaluating and Calibrating Diffusion Model-derived Uncertainty for Quantitative MRI Mapping