Disentangling Prompt Dependence to Evaluate Segmentation Reliability in Gynecological MRI

Elodie Germani; Krystel Nyangoh-Timoh; Pierre Jannin; John S H Baxter

cs.CVMar 9, 2026

Disentangling Prompt Dependence to Evaluate Segmentation Reliability in Gynecological MRI

Authors: Elodie Germani, Krystel Nyangoh-Timoh, Pierre Jannin, John S H Baxter

Organizations: UR, LTSI · Laboratoire Traitement du Signal et de l’Image (LTSI, INSERM UMR 1099), Université de Rennes, Rennes, France · LTSI · Centre Hospitalier Universitaire de Rennes (CHU Rennes), Rennes, France

Abstract

Promptable segmentation models (e.g., the Segment Anything Models) enable generalizable, zero-shot segmentation across diverse domains. Although predictions are deterministic for a fixed image-prompt pair, the robustness of these models to variations in user prompts, referred to as prompt dependence, remains underexplored. In safety-critical workflows with substantial inter-user variability, interpretable and informative frameworks are needed to evaluate prompt dependence. In this work, we assess the reliability of promptable segmentation by analyzing and measuring its sensitivity to prompt variability. We introduce the first formulation of prompt dependence that explicitly disentangles prompt ambiguity (inter-user variability) from local sensitivity (interaction imprecision), offering an interpretable view of segmentation robustness. Experiments on two female pelvic MRI datasets for uterus and bladder segmentation reveal a strong negative correlation between both metrics and segmentation performance, highlighting the value of our framework for assessing robustness. The two metrics have low mutual correlation, supporting the disentangled design of our formulation, and provide meaningful indicators of prompt-related failure modes.

Figures & tables

Figure 1: Overview of the prompt dependence evaluation framework. In prompt space ( left ), a conditional mixture model estimates the distribution of plausible prompts

P_{\text{valid}}(b\mid I)

. The dispersion of this distribution, quantified by the trace of its covariance, defines prompt ambiguity . By adding controlled perturbations around reference prompts and measuring their impact in mask space, we train a stability-margin predictor to estimate the minimum change needed to produce a meaningful discrepancy between masks, thereby providing a formulation of local sensitivity .

Figure 2: Qualitative examples (top) and dataset-level summary statistics (bottom) of prompt dependence metrics, $U_{amb}$ and $\delta^{\star}$ , for UT-EndoMRI and MOGaMBO. For each dataset, two representative slices are shown with the corresponding $U_{amb}$ and $\delta^{\star}$ values; the blue boxes highlight the ground-truth bounding box. The table reports the mean Dice (±SEM), the ranges of $U_{amb}$ and $\delta^{\star}$ , and Pearson correlations on the test set. Stars $\star$ indicate significant correlations at $p<0.001$ .

Panel of Figure 3 — Figure 3: ( Top ) Distribution of prompt ambiguity (x-axis) and local sensitivity (y-axis, encoded as $\log\big(\frac{1}{1-\delta^{\star}}\big)$ ), color-coded by $1-Dice$ . Dashed lines denote the median values used to define the four classes. ( Bottom ) Qualitative example of uncertainty maps produced by Jitter, DO, and CondMDN with the input image and ground truth mask on UT-EndoMRI, with averaged evaluation metrics of uncertainty maps.

Prompt Sensitivity

Prompt sensitivity is the extent to which changes in wording, format, or context alter an AI model’s outputs and measured performance.

See all 115 papers

Learning from Noisy Prompts: Saliency-Guided Prompt Distillation for Robust Segmentation with SAM
Apr 25, 2026Jingxuan Kang, Ziqi Zhang, Shaoming Zheng +9Medical Imaging Foundation Models Promptable Image Segmentation
Imperial College London · Beihang University · National Health Service +1
Prompting with the human-touch: evaluating model-sensitivity of foundation models for musculoskeletal CT segmentation
Mar 11, 2026Caroline Magg, Maaike A. ter Wee, Johannes G. G. Dobbe +4Medical Image Segmentation Medical Imaging Foundation Models
Quantitative Healthcare Analysis (QurAI) Group, University of Amsterdam, Science Park 900, Amsterdam, 1098 XH, The Netherlands · Department Biomedical Engineering and Physics, Amsterdam UMC, Meibergdreef 9, Amsterdam, 1105 AZ, The Netherlands · Department Orthopaedics, Amsterdam UMC, Meibergdreef 9, Amsterdam, 1105 AZ, The Netherlands
From Pixels to Concepts: Do Segmentation Models Understand What They Segment?
May 10, 2026Shuang Liang, Zeqing Wang, Yuxian Li +2VLM Evaluation Image Segmentation
Department of Electrical and Computer Engineering, The University of Hong Kong · CASIC · School of Computer Science and Engineering, Sun Yat-sen University +1

Apr 25, 2026J/K move · Enter open · S save

Learning from Noisy Prompts: Saliency-Guided Prompt Distillation for Robust Segmentation with SAM

Jingxuan Kang, Ziqi Zhang, Shaoming Zheng, Shuang Li+8

Imperial College London · Beihang University · National Health Service+1

Medical Imaging Foundation Models Promptable Image Segmentation Medical Image Segmentation 3D Medical Image Segmentation

Open paper AskPDF arXiv

Segmentation is central to clinical diagnosis and monitoring, yet the reliability of modern foundation models in medical imaging still depends on the availability of precise prompts. The Segment Anything Model (SAM) offers powerful zero-shot capabilities, although it collapses under the weak, generic, and noisy prompts that dominate real clinical workflows. In practice, annotations such as centerline points are coarse and ambiguous, often drifting across neighboring anatomy and misguiding SAM toward inconsistent or incomplete masks. We introduce SPD, a Saliency-Guided Prompt Distillation framework that converts these unreliable cues into robust guidance. SPD first learns data-driven anatomical priors through a lightweight saliency head to obtain confident localization maps. These priors then drive Contextual Prompt Distillation, which validates and enriches noisy prompts using cues from anatomically adjacent slices, producing a consensus prompt set that matches the behavior of expert reasoning. A Pairwise Slice Consistency objective further enforces local anatomical coherence during segmentation. Experiments on four challenging MRI and CT benchmarks demonstrate that SPD consistently outperforms existing SAM adaptations and supervised baselines, delivering large gains in both region-based and boundary-based metrics. SPD provides a practical and principled path toward reliable foundation model deployment in clinical environments where only imperfect prompts are available.

Abstract

Prompt Sensitivity

Explore similar work

Learning from Noisy Prompts: Saliency-Guided Prompt Distillation for Robust Segmentation with SAM

Prompting with the human-touch: evaluating model-sensitivity of foundation models for musculoskeletal CT segmentation

From Pixels to Concepts: Do Segmentation Models Understand What They Segment?