Budget-Aware Uncertainty for Radiotherapy Segmentation QA Using nnU-Net

Ricardo Coimbra Brioso; Lorenzo Mondo; Damiano Dei; Nicola Lambri; Pietro Mancosu; Marta Scorsetti; Daniele Loiacono

cs.CVApr 13, 2026

Budget-Aware Uncertainty for Radiotherapy Segmentation QA Using nnU-Net

Authors: Ricardo Coimbra Brioso, Lorenzo Mondo, Damiano Dei, Nicola Lambri, Pietro Mancosu, Marta Scorsetti, Daniele Loiacono

Organizations: Dipartimento di Elettronica, Informazione e Bioingegneria, Politecnico di Milano, Milan, Italy · Department of Biomedical Sciences, Humanitas University, Pieve Emanuele, Milan, Italy · Radiotherapy and Radiosurgery Department, IRCCS Humanitas Research Hospital, Rozzano, Milan, Italy

Abstract

Accurate delineation of the Clinical Target Volume (CTV) is essential for radiotherapy planning, yet remains time-consuming and difficult to assess, especially for complex treatments such as Total Marrow and Lymph Node Irradiation (TMLI). While deep learning-based auto-segmentation can reduce workload, safe clinical deployment requires reliable cues indicating where models may be wrong. In this work, we propose a budget-aware uncertainty-driven quality assurance (QA) framework built on nnU-Net, combining uncertainty quantification and post-hoc calibration to produce voxel-wise uncertainty maps (based on predictive entropy) that can guide targeted manual review. We compare temperature scaling (TS), deep ensembles (DE), checkpoint ensembles (CE), and test-time augmentation (TTA), evaluated both individually and in combination on TMLI as a representative use case. Reliability is assessed through ROI-masked calibration metrics and uncertainty--error alignment under realistic revision constraints, summarized as AUC over the top 0-5% most uncertain voxels. Across configurations, segmentation accuracy remains stable, whereas TS substantially improves calibration. Uncertainty-error alignment improves most with calibrated checkpoint-based inference, leading to uncertainty maps that highlight more consistently regions requiring manual edits. Overall, integrating calibration with efficient ensembling seems a promising strategy to implement a budget-aware QA workflow for radiotherapy segmentation.

Figures & tables

Method	DSC $\uparrow$	ECE $\downarrow$	BS $\downarrow$
BASE	0.838 $\pm$ 0.046 *	0.062 $\pm$ 0.015 *	0.068 $\pm$ 0.014 *
BASE+TS	0.838 $\pm$ 0.046 *	0.031 $\pm$ 0.006	0.059 $\pm$ 0.011 *
BASE+TTA	0.837 $\pm$ 0.045 *	0.042 $\pm$ 0.015 *	0.063 $\pm$ 0.013 *
BASE+TS+TTA	0.837 $\pm$ 0.045 *	0.034 $\pm$ 0.009	0.059 $\pm$ 0.010 *
DE	0.843 $\pm$ 0.044	0.058 $\pm$ 0.015 *	0.065 $\pm$ 0.014 *
DE+TS	0.843 $\pm$ 0.044	0.031 $\pm$ 0.008	0.056 $\pm$ 0.011

TABLE I: Summary of DSC and ROI-masked calibration metrics (mean

\pm

std); bold indicates the best method per metric; a global Friedman test across the 12 methods showed significant differences for all metrics (

p<0.0001

); * denotes results significantly different from the best method after FDR correction (

\alpha=0.05

Method	$UEO_{AUC}$	$(FP-TP)_{AUC}$	$(FN-TN)_{AUC}$
BASE	0.154 $\pm$ 0.037 *	0.057 $\pm$ 0.023 *	0.674 $\pm$ 0.092 *
BASE+TS	0.170 $\pm$ 0.040	0.067 $\pm$ 0.027 *	0.876 $\pm$ 0.043 *
BASE+TTA	0.169 $\pm$ 0.039 *	0.078 $\pm$ 0.031 *	0.886 $\pm$ 0.040 *
BASE+TS+TTA	0.170 $\pm$ 0.039	0.072 $\pm$ 0.029 *	0.888 $\pm$ 0.038 *
DE	0.152 $\pm$ 0.038 *	0.059 $\pm$ 0.026 *	0.701 $\pm$ 0.081 *
DE+TS	0.167 $\pm$ 0.042 *	0.078 $\pm$ 0.032 *	0.894 $\pm$ 0.029 *

TABLE II: Summary of AUC metrics (mean

\pm

std); bold indicates the best method per metric; a global Friedman test across the 12 methods showed significant differences for all metrics (

p<0.0001

); * denotes results significantly different from the best method after FDR correction (

\alpha=0.05

	$UEO$				$FP_{coverage}$				$FN_{coverage}$
Method	0.5%	1%	2%	5%	0.5%	1%	2%	5%	0.5%	1%	2%	5%
BASE	0.374 *	0.240 *	0.131 *	0.056	0.887 *	0.991	1.000	1.000	0.687 *	0.718 *	0.721 *	0.729 *
BASE+TS	0.383	0.266	0.154	0.066	0.897 *	0.995	1.000	1.000	0.716 *	0.881 *	0.969 *	0.978 *
BASE+TTA	0.379 *	0.268	0.155	0.066	0.867 *	0.992 *	1.000	1.000	0.731 *	0.896 *	0.977 *	0.986 *
BASE+TS+TTA	0.386	0.267	0.155	0.066	0.890 *	0.995	1.000	1.000	0.739 *	0.895 *	0.979 *	0.988 *
DE	0.374 *	0.236 *	0.129 *	0.055	0.906	0.994	1.000	1.000	0.715 *	0.747 *	0.749 *	0.757 *

TABLE III: Point-wise summary of

UEO

FP_{\mathrm{coverage}}

, and

FN_{\mathrm{coverage}}

at 0.5%, 1%, 2%, and 5% uncertain-volume budgets; bold indicates the best value per metric–percentage column; ∗ denotes results significantly different from the best one after FDR correction (

\alpha=0.05

Fig. 1: Budget-aware uncertainty visualization on three axial slices from a representative case (AUTOMI_0007), comparing DE and CE+TS+TTA. The first and fourth rows overlay the filled ground truth (green) and prediction (red). Subsequent rows show entropy-based uncertainty maps thresholded by uncertain-volume budgets (top-0.5%, 1%, and 2% most uncertain voxels), displayed with the ground-truth contour (green). DE at 2% is omitted since not informative, i.e., $<1\%$ of voxels have non-zero uncertainty.

Explore similar work

Trustworthy Medical Segmentation: Uncertainty-Aware U-Net Evaluation Under Clinical Image Degradation
Jul 22, 2026Pranav Kaliaperumal, Manisha KaliaperumalImage Corruption Robustness U-Net
Department of Computer Science University of Colorado Denver Aurora, CO, USA · Department of Biology (Pre-Medical) Creighton University Omaha, NE, USA
Confidence is Not Reliability: Rethinking MC Dropout in Brain Tumour Segmentation
Jun 17, 2026Xin Ci Wong, Duygu Sarikaya, Kieran Zucker +2Segmentation Uncertainty Estimation Uncertainty Calibration
Centre for Doctoral Training in AI for Medical Diagnosis and Care, School of Computing, University of Leeds · School of Computer Science, University of Leeds · Leeds Cancer Centre, St James’s University Hospital, Leeds, UK
SegWithU: Uncertainty as Perturbation Energy for Single-Forward-Pass Risk-Aware Medical Image Segmentation
Apr 16, 2026Tianhao Fu, Austin Wang, Charles Chen +2Image Segmentation Uncertainty Quantification
University of Toronto, Toronto, ON, Canada · Vector Institute, Toronto, ON, Canada · Project Neura, Toronto, ON, Canada +4

Jul 22, 2026J/K move · Enter open · S save

Trustworthy Medical Segmentation: Uncertainty-Aware U-Net Evaluation Under Clinical Image Degradation

Pranav Kaliaperumal, Manisha Kaliaperumal

Department of Computer Science University of Colorado Denver Aurora, CO, USA · Department of Biology (Pre-Medical) Creighton University Omaha, NE, USA

Image Corruption Robustness U-Net Medical Image Segmentation Segmentation Uncertainty Estimation Brain Tumor Segmentation

Open paper AskPDF arXiv

Medical image segmentation models often report high benchmark accuracy under ideal imaging conditions, yet their failures under clinical degradation can be quiet: sensor noise, patient motion, low- resolution acquisition, and contrast variability may all alter model behavior without producing an obvious warning. We present a reproducible framework for evaluating uncertainty-aware segmentation under con- trolled clinical degradation. Our experiments use a synthetic multimodal brain tumor MRI cohort generated with a biophysical phantom simulator that follows the BraTS protocol. We train U-Net and Attention U-Net baselines for multi-class tumor sub-region segmentation and augment both models with Monte Carlo dropout to estimate per-voxel uncertainty. Across eight clinically motivated corruption types at five severity levels, we measure segmentation accuracy, calibration, failure detection, and selective prediction coverage. On clean data, Attention U-Net achieves a whole-tumor Dice of 0.990; under severe Gaussian noise, its performance falls to 0.089. Predictive uncertainty rises with degradation and tracks segmentation error (Pearson r = 0.53 under severity-3 Gaussian noise), allowing us to flag failures with an AUROC of 0.843. These results argue for uncertainty-aware inference as a practical safety layer in physician-in-the-loop radiology workflows. We release the code, trained models, and evaluation protocol to support direct reproduction.

Abstract

Explore similar work

Trustworthy Medical Segmentation: Uncertainty-Aware U-Net Evaluation Under Clinical Image Degradation

Confidence is Not Reliability: Rethinking MC Dropout in Brain Tumour Segmentation

SegWithU: Uncertainty as Perturbation Energy for Single-Forward-Pass Risk-Aware Medical Image Segmentation