Promptable segmentation with region exploration enables minimal-effort expert-level prostate cancer delineation
Authors: Junqing Yang, Natasha Thorley, Ahmed Nadeem Abbasi, Shonit Punwani, Zion Tse, Yipeng Hu, Shaheer U. Saeed
Organizations: UCL Hawkes Institute; Department of Medical Physics and Biomedical Engineering, University College London, UK. · Centre of Medical Imaging, University College London, UK. · Department of Oncology, Aga Khan University Hospital, Pakistan. · Centre for Bioengineering; School of Engineering and Materials Science, Queen Mary University of London, UK.
Purpose: Accurate segmentation of prostate cancer on magnetic resonance (MR) images is crucial for planning image-guided interventions such as targeted biopsies, cryoablation, and radiotherapy. However, subtle and variable tumour appearances, differences in imaging protocols, and limited expert availability make consistent interpretation difficult. While automated methods aim to address this, they rely on large expertly-annotated datasets that are often inconsistent, whereas manual delineation remains labour-intensive. This work aims to bridge the gap between automated and manual segmentation through a framework driven by user-provided point prompts, enabling accurate segmentation with minimal annotation effort. Methods: The framework combines reinforcement learning (RL) with a region-growing segmentation process guided by user prompts. Starting from an initial point prompt, region-growing generates a preliminary segmentation, which is iteratively refined through RL. At each step, the RL agent observes the image and current segmentation to predict a new point, from which region growing updates the mask. A reward, balancing segmentation accuracy and voxel-wise uncertainty, encourages exploration of ambiguous regions, allowing the agent to escape local optima and perform sample-specific optimisation. Despite requiring fully supervised training, the framework bridges manual and fully automated segmentation at inference by substantially reducing user effort while outperforming current fully automated methods. Results: The framework was evaluated on two public prostate MR datasets (PROMIS and PICAI, with 566 and 1090 cases). It outperformed the previous best automated methods by 9.9% and 8.9%, respectively, with performance comparable to manual radiologist segmentation, reducing annotation time tenfold.
Figures & tables
Figure 1: An overview of the proposed promptable segmentation using RL.
Model
PROMIS (Dice)
PICAI (Dice)
SAM
0.236 ± 0.107
0.294 ± 0.132
SAM2
0.254 ± 0.124
0.301 ± 0.137
SAM3
0.312 ± 0.113
0.298 ± 0.142
MedSAM
0.267 ± 0.138
0.342 ± 0.142
MedSAM2
0.291 ± 0.129
0.363 ± 0.144
MedSAM3
0.323 ± 0.148
0.376 ± 0.134
Table 1: Performance comparison.
Table 2: Ablation studies.
Figure 2: Samples from PROMIS segmented using our RL-PromptSeg approach.
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
Model
Prompts within lesion (Dice)
Prompts outside lesion (Dice)
SAM
0.236 ± 0.107
0.162 ± 0.097
SAM2
0.254 ± 0.124
0.141 ± 0.127
SAM3
0.312 ± 0.113
0.234 ± 0.115
MedSAM3
0.323 ± 0.148
0.226 ± 0.134
RL-PromptSeg
0.526 ± 0.112
0.473 ± 0.146
Appendix
Table 1: Robustness analysis for PROMIS (perturbed point prompts outside lesion boundaries).
Model
LiTS (Dice)
KiTS (Dice)
SAM3
0.651 ± 0.101
0.613 ± 0.071
MedSAM3
0.703 ± 0.099
0.724 ± 0.074
UNet
0.742 ± 0.097
0.718 ± 0.064
nnUNet
0.748 ± 0.085
0.720 ± 0.059
Swin-UNeTr
0.758 ± 0.093
0.714 ± 0.072
DinoV3
0.745 ± 0.103
0.727 ± 0.068
Appendix
Table 2: Performance comparison for liver and kidney tumour segmentation.
Model
PROMIS (FPR)
PROMIS (Sensitivity)
SAM3
0.322
0.421
MedSAM3
0.306
0.412
Combiner
0.218
0.498
T2-predictor
0.213
0.507
UniverSeg
0.264
0.516
Swin-UNeTr
0.281
0.523
Appendix
Table 3: Quantitative analysis of over-segmentation on the PROMIS dataset.
Accurate segmentation of the prostate gland in multiparametric MRI (mpMRI) is a fundamental step for a wide range of clinical and research applications, including image registration, volume estimation, and radiomic analysis. However, manual delineation is time-consuming and subject to inter-observer variability, while general-purpose segmentation tools often fail to provide sufficient accuracy for prostate-specific tasks. In this work, we propose a dedicated deep learning-based approach for automatic prostate gland segmentation using the nnU-Net v2 framework. The model leverages multimodal mpMRI data, including T2-weighted imaging, diffusion-weighted imaging (DWI), and apparent diffusion coefficient (ADC) maps, to exploit complementary tissue information. Training was performed on 981 cases from the PI-CAI dataset using whole-gland annotations, and model performance was assessed through 5-fold cross-validation and external validation on an independent cohort of 54 patients from Hospital La Fe. The proposed model achieved a mean Dice score of 0.96 +/- 0.00 in cross-validation and 0.82 on the external test set, demonstrating strong generalization despite domain shift. In comparison, a general-purpose approach (TotalSegmentator) showed substantially lower performance, with a Dice score of 0.15, primarily due to under-segmentation of the gland. These results highlight the importance of task-specific, multimodal segmentation strategies and demonstrate the potential of the proposed approach for reliable integration into clinical research workflows. To facilitate reproducibility and deployment, the model has been fully containerized and is available as a ready-to-use inference tool.
Pablo Rodriguez-Belenguer, Gloria Ribas, Javier Aquerreta Escribano +3
Biomedical Imaging Research Group (GIBI230), Instituto de Investigación Sanitaria La Fe, Valencia, Spain
Automated 3D segmentation of prostate lesions from biparametric MRI (bp-MRI) is essential for reliable algorithmic analysis, but achieving high precision remains challenging. Volumetric methods must combine multiple modalities while ensuring anatomical consistency, but current models struggle to integrate cross-modal information reliably. While vision-language models (VLMs) are replacing the currently used architectural designs, they still lack the fine-grained, lesion-level semantics required for effective localized guidance. To address these limitations, we propose a new multi-encoder U-Net architecture incorporating three key innovations: (1) an alignment loss that enhances foreground text-image similarity to inject lesion semantics; (2) a heatmap loss that calibrates the similarity map and suppresses spurious background activations; and (3) a final-stage, confidence-gated multi-head cross-attention refiner that performs localized boundary edits in high-confidence regions. A phase-scheduled training regime stabilizes the optimization of these components. Our method consistently outperforms prior approaches, establishing a new state-of-the-art on the PI-CAI dataset through enhanced multi-modal fusion and localized text guidance. Our code is available at https://github.com/NUBagciLab/Prostate-Lesion-Segmentation.
Cuiling Sun, Linkai Peng, Adam Murphy +9
Department of Radiology, Northwestern University, Chicago, USA · Department of Urology, Northwestern University, Chicago, USA · Center for Cancer Research, National Cancer Institute, Bethesda, USA
Micro-ultrasound (μUS) is a new, emerging, and promising imaging modality for prostate cancer (PCa) detection, but accurate identification of suspicious tissue remains highly dependent on clinical experience, leading to substantial inter-observer variability. Machine-learning assistance can reduce this variability; however, training reliable deep models is challenging because supervision is sparse and noisy -- typically limited to core-level histopathology outcomes (e.g., cancer grade and its percentage in a biopsy core) without pixel-level lesion annotations and under severe class imbalance. We introduce Prost-RL, which reframes μUS PCa detection as a spatially aware, policy-driven inference problem by learning where to look before decoding. Prost-RL integrates a lightweight reinforcement-learning policy into a foundation-model encoder-decoder to generate interpretable spatial attention maps that act as soft prompts for both cancer-likelihood heatmap prediction and image-level classification. We further propose Adaptive Policy Optimization (APO) to stabilize hybrid supervised-RL training and a noise-robust objective combining symmetric cross-entropy with negative-entropy regularization to mitigate weak-label noise and encourage sharp localization. On a cohort of 6,607 biopsy cores from 693 patients across five clinical sites, Prost-RL achieves 79.0±3.5 AUROC with 64.6±6.3% sensitivity at 80% specificity for core-level detection (+2.1 AUROC and +4.5 sensitivity points over the strongest baseline), and 79.3±5.8 AUROC for clinically significant cancer classification. The learned policy highlights biopsy-aligned regions, providing transparent, spatially grounded evidence alongside quantitative risk predictions. Code is available at: https://github.com/DeepRCL/Prost-RL.
Mohammad Mahdi Abootorabi, Sina Namazi, Armin Saadat +7
The University of British Columbia, Vancouver, BC, Canada · Vector Institute, Toronto, ON, Canada · Queen’s University, Kingston, ON, Canada +1