Zero-Shot Adaptation of Medical Vision Foundation Models for High-Frequency Micro-Ultrasound Prostate Segmentation

Ayusha Abbas; Saram Abbas; Kabita Adhikari

cs.CVAug 14, 2026

Zero-Shot Adaptation of Medical Vision Foundation Models for High-Frequency Micro-Ultrasound Prostate Segmentation

Authors: Ayusha Abbas, Saram Abbas, Kabita Adhikari

Organizations: Newcastle University Newcastle Upon Tyne, UK

Abstract

Prostate cancer claims a life every 80 seconds. Early detection is needed to prevent disease progression, and both PSA density calculation and biopsy decisions rely on knowing the exact boundary of the gland. Conventional ultrasound at 6-12 MHz blurs this boundary, missing one in three high-risk cancers. Micro-ultrasound (29 MHz) improves resolution threefold but introduces dense acoustic speckle that obscures the outer wall; given the same image, two clinicians draw outlines differing by over 10% in area. Supervised methods are costly and generalise poorly across scanners. Can a foundation model segment the prostate with no training data? We present the first zero-shot pipeline for this modality: MedSAM, pre-trained on over 1.5 million medical images, localises the prostate; we then apply CLAHE to sharpen the outer wall, binary dilation to recover missed pixels, and Fourier smoothing (4 modes, s=1.05) to refine the boundary. MedSAM requires a spatial prompt, so we evaluate bounding-box and point-click strategies across 75 patients of the Micro-Ultrasound Prostate Segmentation dataset (2,621 slices). On the 20-patient held-out test set, the pipeline reduces mean boundary-distance error by 45% (Dice 0.749+/-0.043 to 0.865+/-0.029; HD95 217.2+/-36.9 to 120.1+/-26.1 px), reaching Dice 0.859 across the cohort. Its mean overlap shows no significant difference from the three non-expert rater groups (p>0.19), while segmenting 38-52% more consistently (lower inter-patient standard deviation). Point-click prompts fail regardless of placement (best Dice=0.350), because speckle gives no stable local contrast. Only an approximate bounding box is required, so any clinic can deploy it without data collection, annotation, or retraining.

Figures & tables

Fig. 1: Axial 29 MHz micro-ultrasound slices with expert prostate masks (shown in red) across 12 representative test patients, demonstrating the characteristic low signal-to-noise ratio, speckle patterns, and capsule boundary shadowing.

Fig. 2: Per-patient Dice before and after post-processing (n = 75 patients, zero-shot). Every point above the diagonal represents an improvement; 74/75 patients improved. Blue circles: held-out test set (n = 20); grey circles: training-split patients. Red markers: cohort means (raw $\mu$ = 0.787; pipeline $\mu$ = 0.859).

Stage	Dice	HD95 (px)
Raw MedSAM	$0.749\pm 0.043$	$217.2\pm 36.9$
+CLAHE + dilation	$0.842\pm 0.028$	$160.4\pm 22.0$
+Fourier smooth. $s{=}1.0$	$0.850\pm 0.031$	$137.0\pm 28.9$
+Fourier smooth. $s{=}1.05$	$\mathbf{0.865\pm 0.029}$	$\mathbf{120.1\pm 26.1}$

TABLE I: Post-processing ablation (n = 20 test patients). Mean

\pm

SD. All shape priors (Fourier, ellipse, convex hull) substantially outperform simple smoothing; Fourier was selected for superior full-resolution boundary preservation.

Fig. 3: Post-processing stages on patient microUS_test_09. Red : expert GT. Green : MedSAM. Gold dashed: oracle bounding box. (a) Raw: Dice = 0.848, HD95 = 246 px. (b) +CLAHE + dilation: Dice = 0.915, HD95 = 153 px. (c–d) +Fourier smoothing ( $s$ = 1.0 / 1.05): Dice = 0.954 / 0.959, HD95 = 56 / 46 px — 81% HD95 reduction on this case.

Annotator	Dice	HD95 (px)	$p_{\mathrm{adj}}$
MicroSegNet † (sup.)	$0.939$	—	—
Master’s Student	$0.883\pm 0.047$	—	$0.35$
Clinician (ltd. US)	$0.874\pm 0.056$	—	$0.60$
MedSAM ∗ (final)	$\mathbf{0.865\pm 0.029}$	$\mathbf{120.1\pm 26.1}$	—
Medical Student	$0.833\pm 0.061$	—	$0.19$
MedSAM ∗ (raw)	$0.749\pm 0.043$	$217.2\pm 36.9$	—

TABLE II: Performance and human comparison (held-out test set, n = 20).

p_{\mathrm{adj}}

: Holm–Bonferroni vs. MedSAM (Wilcoxon). † 3D volumetric supervised model; not directly comparable. ∗ Oracle bounding-box prompt (upper bound).

Fig. 4: MedSAM vs. non-expert annotators (n = 20 test patients). Violins show the per-patient distribution, diamonds show group means, and annotations show the standard deviation (SD) for each group as a metric of consistency (lower SD represents higher consistency). No pairwise comparison of mean Dice between MedSAM and a human group reached significance (all $p_{\mathrm{adj}}{>}0.19$ ); the model’s advantage is its markedly lower inter-patient SD.

Fig. 5: Extended prompt strategy ablation (n = 20 test patients). Red dashed line: oracle bbox ceiling (0.865 after full post-processing). No point strategy exceeds 0.35. Adding more foreground points (dense, random) reduces performance vs. the 5-point baseline, revealing a spatial-constraint failure rather than a point-placement problem.

Strategy	Type	Dice (Mean $\pm$ SD)
Oracle Bounding Box (Ceiling)	Manual Prompt	$0.880\pm 0.023$
Statistical Prior Bounding Box	Fully Automated	$0.691\pm 0.056$
Two-Pass Cascade Model	Fully Automated	$0.698\pm 0.058$

TABLE III: Automatic Localiser Evaluation (patient-level Dice, n=20)

Explore similar work

Learning Where to Look: A Reinforcement Learning Framework for Robust Micro-Ultrasound Prostate Cancer Detection
Jun 29, 2026Mohammad Mahdi Abootorabi, Sina Namazi, Armin Saadat +7Medical Imaging Reinforcement Learning
The University of British Columbia, Vancouver, BC, Canada · Vector Institute, Toronto, ON, Canada · Queen’s University, Kingston, ON, Canada +1
ProFound: A moderate-sized vision foundation model for multi-task prostate imaging
Mar 4, 2026Yipei Wang, Yinsong Xu, Weixi Yi +11Medical Imaging Foundation Models Medical Image Representation Learning
UCL Hawkes Institute, University College London, London, UK · Department of Medical Physics and Biomedical Engineering, University College London, London, UK · School of Artificial Intelligence, Beijing University of Posts and Telecommunications, Beijing, China +11
UltraSAM3: A Concept-Driven Foundation Model for Universal Ultrasound Image Segmentation
Jul 31, 2026Bo Xu, Quanhao Zhu, Rui Lin +5Ultrasound Image Segmentation Medical Imaging Foundation Models
School of Software Dalian University of Technology · School of Computing Technologies RMIT University · Dalian University of Technology Affiliated Center Hospital

Jun 29, 2026J/K move · Enter open · S save

Learning Where to Look: A Reinforcement Learning Framework for Robust Micro-Ultrasound Prostate Cancer Detection

Mohammad Mahdi Abootorabi, Sina Namazi, Armin Saadat, Lyuyang Wang+6

The University of British Columbia, Vancouver, BC, Canada · Vector Institute, Toronto, ON, Canada · Queen’s University, Kingston, ON, Canada+1

Medical Imaging Reinforcement Learning Medical Image Classification Ultrasound Imaging Weakly Supervised Learning Oncology+1

Open paper AskPDF arXiv

Micro-ultrasound (

μ

US) is a new, emerging, and promising imaging modality for prostate cancer (PCa) detection, but accurate identification of suspicious tissue remains highly dependent on clinical experience, leading to substantial inter-observer variability. Machine-learning assistance can reduce this variability; however, training reliable deep models is challenging because supervision is sparse and noisy -- typically limited to core-level histopathology outcomes (e.g., cancer grade and its percentage in a biopsy core) without pixel-level lesion annotations and under severe class imbalance. We introduce Prost-RL, which reframes

μ

US PCa detection as a spatially aware, policy-driven inference problem by learning where to look before decoding. Prost-RL integrates a lightweight reinforcement-learning policy into a foundation-model encoder-decoder to generate interpretable spatial attention maps that act as soft prompts for both cancer-likelihood heatmap prediction and image-level classification. We further propose Adaptive Policy Optimization (APO) to stabilize hybrid supervised-RL training and a noise-robust objective combining symmetric cross-entropy with negative-entropy regularization to mitigate weak-label noise and encourage sharp localization. On a cohort of 6,607 biopsy cores from 693 patients across five clinical sites, Prost-RL achieves

79.0\pm3.5

AUROC with

64.6\pm6.3

% sensitivity at 80% specificity for core-level detection (+2.1 AUROC and +4.5 sensitivity points over the strongest baseline), and

79.3\pm5.8

AUROC for clinically significant cancer classification. The learned policy highlights biopsy-aligned regions, providing transparent, spatially grounded evidence alongside quantitative risk predictions. Code is available at: https://github.com/DeepRCL/Prost-RL.

Abstract

Explore similar work

Learning Where to Look: A Reinforcement Learning Framework for Robust Micro-Ultrasound Prostate Cancer Detection

ProFound: A moderate-sized vision foundation model for multi-task prostate imaging

UltraSAM3: A Concept-Driven Foundation Model for Universal Ultrasound Image Segmentation