Uncertainty-Guided Handshake: Efficient Human-in-the-Loop Refinement for Surgical-Grade Glioma Segmentation

Samuel Hart; Ahmad Yahya; Ahmed Karam Eldaly

cs.CVOct 1, 2026

Uncertainty-Guided Handshake: Efficient Human-in-the-Loop Refinement for Surgical-Grade Glioma Segmentation

Authors: Samuel Hart, Ahmad Yahya, Ahmed Karam Eldaly

Organizations: Department of Computer Science, University of Exeter, Exeter, EX4 4QF, United Kingdom. · Department of Nuclear Engineering, Faculty of Engineering, King Abdulaziz University, Jeddah, Saudi Arabia. · UCL Hawkes Institute, University College London, Gower St., London, WC1E 6AE, United Kingdom.

Abstract

While state-of-the-art automated models for medical image segmentation achieve high mean performance, they frequently suffer from localized, catastrophic failures that preclude safe clinical deployment, particularly in neuro-oncology. Interactive segmentation frameworks mitigate this by incorporating human oversight, but traditionally impose prohibitive cognitive and temporal workloads by requiring clinicians to manually search for errors. In this project, we present an efficient, Hybrid Structural-Aleatoric Human-in-the-Loop framework for glioma segmentation that bridges the gap between automated baseline performance and surgical-grade precision, achieving sub-2.0 mm HD95 on curated benchmarks while providing safety-net routing for structural failures across real-world clinical data. By extracting voxel-wise Test-Time Augmentation (TTA) uncertainty and applying hierarchical topological filtering, our method proactively isolates high-risk structural anomalies. We comprehensively evaluated our approach on a challenging out-of-distribution clinical stress-test cohort (N = 362). Operating under a simulated Human Oracle, the framework improved the Whole Tumor (WT) Dice score from 0.891 to 0.914 and reduced the 95th percentile Hausdorff Distance (HD95) from 5.82 mm to 4.76 mm. Critically for surgical safety, the system rescued severe boundary failures in the Tumor Core, reducing mean HD95 from 17.96 mm to 14.83 mm (improving absolute TC Dice to 0.356). These spatial rescues were achieved while demanding a median interactive workload of just 11.3% of the target volume. Acknowledging this as a simulated upper bound lacking real-world cognitive friction, the framework nevertheless demonstrates a highly Pareto-efficient pathway for safely deploying clinical AI.

Figures & tables

Figure 1: Overview of the UG-HITL framework. Phase 1 (AI Inference): a 3D nnU-Net V2 backbone with test-time augmentation (TTA) generates the baseline segmentation and its voxel-wise predictive entropy map. Phase 2 (Decoupled Algorithmic Triage): the Track A Structural Tripwire screens each prediction for gross structural failure; collapse cases bypass refinement and are routed directly to mandatory full review, while passing cases undergo Track B refinement—topological filtering and probabilistic hotspot detection (Track B1), gradient-snapped asymmetrical search banding (Track B2), and complexity-aware dynamic budgeting capped at 25% of the target tumor volume. Phase 3 (Human-in-the-Loop): the clinician receives a prioritized queue of localized boundary proposals, and accepted proposals are merged into the final refined mask.

Methodology	Dice $\uparrow$	HD95 (mm) $\downarrow$	Workload $\downarrow$
Automated SOTA (Baseline)	0.943	3.05	100.00%
UG-HITL (MC Dropout)	0.973	1.91	0.28%
UG-HITL (5-Model Ensemble)	0.931	3.52	0.01%
UG-HITL (EDL)	0.990	0.62	1.58%

Table 1: In-Distribution Uncertainty Engine Comparison. Quantitative performance evaluated on the curated BraTS 2023 Validation Cohort. Note: Values highlighted in bold represent the optimal performance for that specific metric category, demonstrating that the proposed EDL-backed pipeline simultaneously achieves peak volumetric overlap and the lowest spatial boundary error among the active refinement methods.

Metric	Whole Tumor (WT)		Tumor Core (TC)		Enhancing Tumor (ET)
	Baseline (nnU-Net)	UG-HITL (TTA)	Baseline	UG-HITL (TTA)	Baseline	UG-HITL (TTA)
Dice Score (Mean)	0.891	0.914	0.286	0.356	0.528	0.577
Dice Score (Median)	0.938	0.949	0.114	0.204	1.000	1.000
HD95 (Mean, mm)	5.82	4.76	17.96	14.83	-	-
HD95 (Median, mm)	2.82	2.23	13.52	11.42	-	-
HD95 (Max, mm)	109.15	109.15	113.34	113.34	-	-

Table 2: Quantitative evaluation of the proposed UG-HITL (TTA) framework on the out-of-distribution UTSW clinical cohort (

N=362

). The methodology successfully constrained median human workload to 11.3% while rescuing structural boundary failures across all critical sub-regions. Bold indicates the best value per column. Dashes = undefined HD95; empty–empty Dice = 1.0; workload statistics include full-manual-review cases.

Figure 2: Reduction of 95th Percentile Hausdorff Distance (HD95) across the out-of-distribution UTSW clinical cohort (Log Scale). The framework aggressively targeted catastrophic failures in the surgical margins, reducing the mean Tumor Core (TC) HD95 from a baseline of 17.96 mm to 14.83 mm. (Note: The logarithmic scale visually compresses the upper y-axis; the absolute spatial reduction for catastrophic outliers exceeding $10^{2}$ mm is highly substantial.)

Figure 3: Actual clinical workload versus the assigned workload budget for cases passing the Track A Structural Tripwire; cases routed to mandatory full review (100% workload) are excluded from this analysis. The framework scales the permitted interaction allowance up to a maximum safety threshold of 25% based on the morphological complexity (SA:V ratio) of the target tumor. Each point represents one case; the dashed line marks unity (actual workload $=$ assigned budget), and no case exceeded its assigned allocation. The data demonstrate that the algorithm efficiently executes its spatial corrections well within these dynamically assigned limits.

Figure 4: Qualitative demonstration of Multi-Class Refinement. (A) Ground Truth annotation reveals a distinct enhancing tumor core (yellow). (B) The baseline nnU-Net entirely misses the enhancing core. (C) The UG-HITL framework successfully rescues the internal core boundaries, drastically reducing the surgical margin error.

Figure 5: Qualitative demonstration of an algorithmic "Catastrophic Rescue". (Left to Right): 1. Raw MRI. 2. Baseline hallucination (red). 3. Isolated uncertainty (yellow). 4. Refined Handshake (green). Note: This sample illustrates the foundational geometric logic used by the finalised TTA framework to achieve the spatial improvements reported on the UTSW cohort.

Pipeline Configuration	Dice (WT) $\uparrow$	HD95 (WT) $\downarrow$	Dice (TC) $\uparrow$	HD95 (TC) $\downarrow$	Workload (%) $\downarrow$
1. Baseline TTA (No Refinement)	0.884	7.08 mm	0.315	20.59 mm	11.68%
2. + Hotspot Hunter Only	0.884	7.07 mm	0.315	20.46 mm	11.87%
3. + Topological & Anomaly Filtering	0.896	6.71 mm	0.369	17.66 mm	18.92%
4. + Gradient Snapping (Full Pipeline)	0.896	6.85 mm	0.368	17.76 mm	18.89%

Table 3: Ablation study isolating the quantitative impact of the framework’s individual triage components. Evaluated on a 50-case subset of the UTSW clinical cohort. (Note: Baseline metrics differ slightly from the full cohort in Table 2 due to evaluation isolated to this specific representative subset).

Explore similar work

Jul 23, 2026cs.CV

Post-Operative Glioma Segmentation via Loss Stabilization, Normalization and Subspace Attention

Tracking residual tumor after surgery is essential for catching recurrence early, but automating post-operative glioma segmentation remains a difficult task. Although transformer-based architectures, such as SwinUNETR, achieved impressive results, few studies test how well they generalize across clinical protocols. In this paper, we conduct an ablation study on the MU-GLIOMA-POST and UCSF-ALPTDG datasets and show that the standard Generalized Dice Loss (GDL) is unstable under domain shift: the Whole Lesion (WL) Dice drops from 0.88 on the internal validation set to 0.73 on the external UCSF test set. To address this, we pair brain-masked percentile normalization with voxel-level contrastive learning. We also propose a Subspace-Aware Class Attention (SACA) module that re-calibrates the bottleneck features and raises Enhancing Tumor (ET) sensitivity by 8% (9.1% relative improvement) on internal validation. Ensembling these refinements with nnU-Net brings every stable configuration to a WL Dice of 0.94, and the SACA variant ensemble achieves the best boundary error (HD95) of 2.92 mm on MU-GLIOMA-POST.

Alexandru Crişan, Diana Borza

Babes-Bolyai University, Cluj-Napoca, Romania

Brain Tumor Segmentation Swin U-Net Normalization

Ask PDF

Aug 5, 2026cs.CV

Text-Guided Refinement of Multi-sequence Glioma Subregion Segmentation with a Vision-Language Foundation Model

Background: Accurate glioma subregion delineation is important for radiotherapy planning and longitudinal monitoring, but manual contour correction is time-consuming. Models such as nnU-Net may generalize imperfectly and lack clinician-directed text correction. Purpose: We investigated adapting a three-dimensional (3D) vision-language foundation model for text-guided brain tumor segmentation refinement. Methods: We developed a lightweight VoxTell-based framework. Pretrained VoxTell generated initial masks. Oracle prompts derived from segmentation errors encoded target, action, location, imaging evidence, edit size, and preservation constraints. Frozen Qwen/VoxTell prompt embeddings were injected through trainable projections into its multiscale decoder conditioning; other weights remained frozen. Training, validation, and testing used 901, 100, and 250 BraTS-GLI cases. Cross-dataset transfer was evaluated on 100 meningioma, metastasis, pediatric tumor, and UPENN-GBM cases. Results: On the internal test set using post-contrast T1-weighted input, correct instructions improved subregion Dice similarity coefficient (DSC; enhancing tumor, edema, and necrotic/non-enhancing core) from

0.774\pm0.158

0.796\pm0.137

. They outperformed blank prompts (

0.762\pm0.155

; Holm-adjusted

p<0.001

d_z=0.71

) and contradictory prompts (

0.770\pm0.163

;

p<0.001

d_z=0.48

). In cross-dataset testing, correct instructions improved DSC from

0.527\pm0.287

0.550\pm0.278

and outperformed contradictory instructions (

0.504\pm0.275

;

p<0.001

d_z=0.43

). Conclusion: A 3D vision-language foundation model can perform instruction-guided refinement of glioma subregion segmentations. Sensitivity to correct, blank, and contradictory prompts suggests text-dependent contour editing rather than nonspecific post-processing, supporting further evaluation as a clinician-in-the-loop tool.

Zach Eidex, Yu-nong Lin, Mojtaba Safari +4

Department of Biomedical Informatics, Emory University School of Medicine, Atlanta, GA, USA · Department of Radiation and Cellular Oncology, The University of Chicago, Chicago, IL, USA · Department of Radiation Oncology, Winship Cancer Institute, Emory University School of Medicine, Atlanta, GA, USA

Brain Tumor Segmentation Vision-Language Foundation Models

Ask PDF

Jun 17, 2026cs.CV

Confidence is Not Reliability: Rethinking MC Dropout in Brain Tumour Segmentation

Glioma segmentation in multiparametric MRI is a critical component of treatment planning. A segmentation model that fails silently on treatment-critical sub-regions represents a patient safety risk that overlap-based metrics such as Dice scores cannot expose. We ask whether voxel-level uncertainty estimation via Monte Carlo (MC) Dropout can reliably identify segmentation errors in clinically critical sub-regions, and whether calibration failure modes are detectable from standard reporting metrics alone. In an empirical two-model case study on 126 BraTS21 patients, we evaluate a high-performance pretrained SegResNet and a locally trained UNet with residual units (UNet-Res). MC dropout preserved segmentation accuracy (

|Δ\text{Dice}|

<0.01

) while achieving strong uncertainty-error alignment (AUROC for entropy (H)

\approx

0.97), indicating uncertainty correctly ranks erroneous voxels above correct ones. Entropy-based patient stratification identified a high-uncertainty subgroup with substantially lower segmentation performance (median whole-tumour Dice

0.835

vs.

0.925

), supporting uncertainty as a practical triage signal. However, global alignment can mask important region-specific differences. Despite similar AUROC, UNet-Res exhibited near-zero enhancing tumour entropy (

0.054

) and Expected Calibration Error (ECE) of

0.915

, with a Dice of only

0.714

, indicating severely miscalibrated confidence on the most clinically critical sub-region, a failure mode invisible to standard Dice and AUROC reporting. These findings demonstrate that strong uncertainty-error alignment is necessary but insufficient for clinical safety: sub-region-specific calibration assessment must accompany AUROC evaluation when selecting models for clinical deployment.

Xin Ci Wong, Duygu Sarikaya, Kieran Zucker +2

Centre for Doctoral Training in AI for Medical Diagnosis and Care, School of Computing, University of Leeds · School of Computer Science, University of Leeds · Leeds Cancer Centre, St James’s University Hospital, Leeds, UK

Brain Tumor Segmentation Magnetic Resonance Imaging Voxel

Ask PDF