Surgical segmentation networks can fail silently under acquisition degradation: predicted masks may be wrong even when model confidence remains high. Existing deployment-time monitors rely primarily on uncertainty estimates and can therefore miss confident failures. We present TCSR-Monitor (Temporal Conformal Surgical Risk Monitor), a post-hoc failure-monitoring framework that combines confidence with observable shape, temporal-consistency, and image-quality cues. TCSR-Monitor wraps a frozen segmentation model, requires no model internals, and operates without ground truth at deployment. We also introduce a validation protocol to assess whether alarms remain credible under distribution shift. On EndoVis 2017, leave-one-corruption-out evaluation shows that TCSR-Monitor generalizes to unseen acquisition degradations and substantially outperforms confidence-based baselines. A circularity control confirms that it predicts segmentation failure rather than simply detecting corrupted images. Mondrian conformal calibration balances miss-rates across degradation severities, but a single global threshold still produces false alarms on up to 40% of correctly segmented frames at moderate corruption. Zero-shot transfer to SAM2 demonstrates feature portability, although entropy outperforms the transferred monitor at both evaluated thresholds. Overall, reliable monitoring under acquisition degradation benefits from complementary observable signals beyond confidence alone, but substantial false-alarm and transfer limitations remain.
Figures & tables
Figure 1: Overview of TCSR-Monitor. The monitor observes the raw frame and the segmenter’s output, but does not modify the segmenter or use ground truth at deployment. Calibration converts risk scores into global or group-specific alarm thresholds.
Group
Feature
Definition
Confidence (7)
conf_mean_fg_prob
Mean pt over foreground pixels ( y^t=1 )
conf_max_prob
maxpt over the frame
conf_mean_entropy
Mean per-pixel binary entropy of pt
conf_max_entropy
Max per-pixel binary entropy of pt
conf_frac_uncertain
Fraction of pixels with entropy >0.9log2
conf_margin
Mean ∣pt−0.5∣ over the frame
Table 1: The full set of 22 observable features. pt is the pixelwise probability map, y^t=1[pt≥0.5] the predicted mask, and t−1 the previous frame in the same video.
Method
τ=0.5 AUROC
τ=0.5 AUPRC
τ=0.75 AUROC
τ=0.75 AUPRC
TCSR-Monitor (ours)
0.877
0.468
0.793
0.301
Entropy [ 5 ]
0.764
0.635
0.483
0.220
Max-Softmax [ 3 ]
0.536
0.087
0.509
0.085
Temporal Heuristic
0.338
0.013
0.460
0.074
Failure prevalence
1.7% ( 14/825 )
6.9% ( 57/825 )
Table 2: Held-out clean test results (seed 0; point estimates). AUPRC should be interpreted relative to the rare failure prevalence. Frame-level point estimates on this split are correlated within only 3 test sequences; see the sequence-level bootstrap discussion below for uncertainty bounds.
Figure 2: Qualitative examples of sub-threshold confident failures ( 0.5≤IoU<0.75 ). Each triplet shows the input, predicted mask, and ground-truth mask. The segmenter produces plausible-looking but incomplete or over-extended instruments, illustrating why an external failure monitor is needed even when the output appears confident.
Protocol / held-out type
Monitor
Entropy
Wins
Zero-shot
0.753
0.465
24/30
LOCO overall
0.814
0.481
26/30
Severity extrapolation
0.831
0.408
8/12
In-distribution upper bound
0.827
0.465
26/30
Brightness
0.801
0.331
4/5
Contrast
0.912
0.839
3/5
Table 3: Generalization hierarchy and LOCO breakdown (AUROC). LOCO is the robustness headline; in-distribution is an upper bound. Wins counts the (corruption type, severity) cells in which TCSR-Monitor AUROC exceeds entropy AUROC, out of 30 cells ( 6 corruption types ×5 severities) for zero-shot/LOCO/in-distribution, and 12 cells ( 6 types × severities 4 – 5 ) for severity extrapolation; per-corruption rows report wins out of 5 severities.
Severity
Monitor
Entropy
FA-correct
Severity
Global miss
Mondrian miss
1
0.807
0.484
8.0%
1
50.5%
9.3%
2
0.822
0.505
22.3%
2
22.5%
10.1%
3
0.827
0.483
40.4%
3
9.2%
9.9%
4
0.861
0.366
35.8%
4
6.1%
10.0%
5
0.825
0.478
8.9%
5
5.2%
9.9%
Overall
0.946
0.594
12.6%
Overall
10.0%
9.9%
Table 4: Circularity control and Mondrian conformal calibration. Left: within-corrupted AUROC for failure vs. correct masks. Right: miss-rate by severity at α=0.10 .
Condition
τ=0.5 AUROC
τ=0.75 AUROC
U-Net monitor → SAM2 (zero-shot transfer)
0.896 [0.873,0.915]
0.930 [0.912,0.945]
Entropy → SAM2
0.984 [0.975,0.990]
0.988 [0.979,0.994]
SAM2-specific monitor (upper bound)
0.997 [0.995,0.999]
0.997 [0.994,0.999]
Transfer efficiency (transfer / upper bound)
89.8%
93.3%
Table 5: Cross-model transfer to SAM2-Tiny. Entropy is the best-AUROC method in this setting (bold); TCSR-Monitor’s transfer efficiency relative to a SAM2-supervised upper bound is reported separately.
Group
Share of importance
Top individual feature
Importance
Confidence
57.9%
conf_mean_fg_prob
36.0%
Quality
20.6%
qual_contrast
11.2%
Shape
16.5%
conf_margin
5.7%
Temporal
5.0%
conf_frac_fg
5.6%
qual_blur_score
5.2%
Table 6: XGBoost gain-based feature importance (clean-test monitor, Table 2 ), aggregated by feature group and listing the top-5 individual features. Importances sum to 1 over all 22 features.
Medical image segmentation delineates organs, lesions, and other structures in scans to support measurement, diagnosis, treatment planning, and research.
Medical image segmentation models often report high benchmark accuracy under ideal imaging conditions, yet their failures under clinical degradation can be quiet: sensor noise, patient motion, low- resolution acquisition, and contrast variability may all alter model behavior without producing an obvious warning. We present a reproducible framework for evaluating uncertainty-aware segmentation under con- trolled clinical degradation. Our experiments use a synthetic multimodal brain tumor MRI cohort generated with a biophysical phantom simulator that follows the BraTS protocol. We train U-Net and Attention U-Net baselines for multi-class tumor sub-region segmentation and augment both models with Monte Carlo dropout to estimate per-voxel uncertainty. Across eight clinically motivated corruption types at five severity levels, we measure segmentation accuracy, calibration, failure detection, and selective prediction coverage. On clean data, Attention U-Net achieves a whole-tumor Dice of 0.990; under severe Gaussian noise, its performance falls to 0.089. Predictive uncertainty rises with degradation and tracks segmentation error (Pearson r = 0.53 under severity-3 Gaussian noise), allowing us to flag failures with an AUROC of 0.843. These results argue for uncertainty-aware inference as a practical safety layer in physician-in-the-loop radiology workflows. We release the code, trained models, and evaluation protocol to support direct reproduction.
Pranav Kaliaperumal, Manisha Kaliaperumal
Department of Computer Science University of Colorado Denver Aurora, CO, USA · Department of Biology (Pre-Medical) Creighton University Omaha, NE, USA
Reliability under sparse and heterogeneous failures remains a fundamental challenge for medical image segmentation. High average accuracy can conceal a small set of structurally distinct and clinically consequential errors. Existing post-hoc correction methods alleviate this problem, but typically estimate false-positive and false-negative corrections from the same fixed prediction. This ignores the dynamic evolution of error states and limits the correction of complex cases. Inspired by iterative error feedback in structured prediction, we propose Failure-Aware Progressive Repair (FAPR). FAPR represents the current segmentation mask as a dynamic failure state and models each repair operation as a state-transition operator. Each accepted correction forms a new prediction state for subsequent error diagnosis and repair, enabling later operations to adapt to preceding changes. Conditional routing selectively activates necessary state transitions, while failure replay exposes the model to rare error states. By keeping the base segmentor frozen, FAPR preserves its established segmentation capability while improving difficult cases. Across three public ultrasound lesion segmentation benchmarks, FAPR improves mean DSC by 1.52%. On the very-hard subsets of BUSI and TN3K, the average gain reaches 13.77%.
Ziliang Wang, XuJiang Tang, Lu Yuting +4
Central South University · Yangtze University · Chongqing University +2
Super-resolution (SR) models are attracting growing interest for enhancing minimally invasive surgery and diagnostic videos under hardware constraints. However, valid concerns remain regarding the introduction of hallucinated structures and amplified noise, limiting their reliability in safety-critical settings. We propose a direct and practical framework to make SR systems more trustworthy by identifying where reconstructions are likely to fail. Our approach integrates a lightweight error-prediction network that operates on intermediate representations to estimate pixel-wise reconstruction error. The module is computationally efficient and low-latency, making it suitable for real-time deployment. We convert these predictions into operational failure decisions by constructing Conformal Failure Masks (CFM), which localize regions where the SR output should not be trusted. Built on conformal risk control principles, our method provides theoretical guarantees for controlling both the tolerated error limit and the miscoverage in detected failures. We evaluate our approach on image and video SR, demonstrating its effectiveness in detecting unreliable reconstructions in endoscopic and robotic surgery settings. To our knowledge, this is the first study to provide a model-agnostic, theoretically grounded approach to improving the safety of real-time endoscopic image SR.
Julio Silva-Rodríguez, Ender Konukoglu
Computer Vision Lab, ETH Zurich, Zurich, Switzerland