Prompt-Guided Interactive Segmentation of Interstitial Lung Disease in Thoracic CT
Authors: Vasilis Dedousis, Lubnaa Abdur Rahman, Lorenzo Brigatο, Ethan Dack, Andreas Christe, Christoph Frank, Manuela Funke-Chambour, Justus Roos, Adrian Huber, Lukas Ebner, Stavroula Mougiakakou
Organizations: University of Bern, Bern, Switzerland · Graduate School for Cellular and Biomedical Sciences, Bern, Switzerland · Department of Diagnostic, Interventional, and Pediatric Radiology, Bern University Hospital, Bern, Switzerland · Department of Radiology, Lucern Cantonal Hospital, Lucern, Switzerland · Division of Pulmonology, Department of Medicine, Lausanne University Hospital (CHUV) and University of Lausanne, Lausanne, Switzerland
Accurate segmentation of interstitial lung disease (ILD) patterns is essential for quantitative disease assessment and longitudinal monitoring. However, existing approaches remain limited by relying on dense annotations and producing static predictions that cannot be refined, motivating interactive approaches. While promptable models show promise in interactive segmentation, their adaptation to ILDs remains largely unexplored. To address this gap, we investigate prompt-guided foundation models for ILD refinement and present, to the best of our knowledge, the first adaptation of MedSAM2 for interactive 3D ILD segmentation on thoracic CT. We investigate three fine-tuning strategies and multiple clinically motivated prompts: bounding-boxes (BBox), point, lasso, and scribble. On a dataset spanning seven ILD patterns and healthy lung tissue, full model fine-tuning performed best, improving the average Dice score by 4.7 percentage points over MedSAM2.While BBox prompts achieve the strongest performance, non-native MedSAM2 interactions such as lasso and scribble prompts also prove effective. Finally, we present and evaluate a proof-of-concept end-to-end workflow in which MedSAM2 is initialized from an automatic segmentation prior and subsequently refined using radiologist prompts. Model weights and plug-ins made available at: https://github.com/AIHNlab/ILD-SemiSegTool.
Figures & tables
Split
Healthy
GGO
Ret.
Cons.
HC
Ret.+GGO
Bron.
Emph.
Train
243
148
150
6
210
28
7
118
Val
31
19
21
1
28
4
2
14
Test
32
19
24
1
26
4
1
18
Total
306
186
195
8
264
36
10
150
Table 1: Case distribution of ILD patterns across dataset.
Figure 1: Interactive segmentation framework. (a) Radiologist prompts drive FT MedSAM2 to segment CT images, optionally initialized from an automatic segmentation (PoC); prompts are simulated from ground-truth masks. (b) Prompts on a bronchiectasis slice, including single-BBox, multi-BBox, points, lasso, scribble, with GT on top row and FT MedSAM2 output on bottom row.
Learning rate
Weight decay
Loss weights
5e-5 , 1.7e-5 , 1.7e-4, 5e-4
1e-4 , 1e-1
{0.1,1,10} orthogonal; (1,1,1) , (10,10,10)
Table 2: Grid search for MedSAM2 fine-tuning. Bold , underline , and italic denote the optimal configuration for full finetuning, mask only, and mask and memory, respectively.
SBBox
MBBox
Points
Lasso
Scribble
Model
DSC
NSD
DSC
NSD
DSC
NSD
DSC
NSD
DSC
NSD
SAM 2.1
0.255
0.206
0.340
0.319
0.183
0.094
0.213
0.133
0.280
0.226
MedSAM2
0.347
0.323
0.337
0.314
0.333
0.298
0.359
0.322
0.318
0.280
nnInteractive
0.333
0.273
0.398
0.313
0.195
0.181
0.041
0.074
0.376
0.325
MO (Ours)
0.383 ⋆ ⋆
0.349 ⋆
0.418 ⋆ ⋆
0.365 ⋆ ⋆
0.355 ⋆ ⋆
0.316 ⋆
0.399 ⋆ ⋆
0.348 ⋆ ⋆
0.383 ⋆
0.333
MM (Ours)
0.373 ⋆
0.327 ⋆
0.418 ⋆ ⋆
0.374 ⋆ ⋆
0.349 ⋆
0.301 ⋆
0.400 ⋆ ⋆
0.344 ⋆
0.396 ⋆
0.346 ⋆ ⋆
Table 3: Performance across prompt types (3D, mid-slice init ) comparing baselines and standalone fine-tuning strategies: MO, MM, and FF. Best in bold , second best underlined . ⋆ / ▽ : significantly better/worse than MedSAM2; ⋆ / ▽ : significantly better/worse than nnInteractive (paired Wilcoxon signed-rank test, case-averaged, Holm–Bonferroni corrected, p<0.05 ). Unmarked cells: no statistical significance.
Figure 2: Evaluation of FF MedSAM2. (a) Per-pattern DSC across prompt types in 2D and 3D. (b) Sensitivity of 3D performance to initialization slice: average DSC and NSD versus offset from the middle ground-truth slice (0%); dashed lines indicate initialization from the ground-truth slice with the most foreground.
Model
Prompt
Bron.
Cons.
Emph.
GGO
HC
Ret.
Ret.+GGO
Healthy
Case Avg.
MedSAM2
SBBox
0.128
0.610
0.177
0.131
0.068
0.149
0.352
0.832
0.347
MBBox
0.128
0.610
0.157
0.140
0.075
0.174
0.365
0.777
0.337
Points
0.139
0.043
0.148
0.155
0.059
0.143
0.232
0.831
0.333
Lasso
0.139
0.043
0.184
0.192
0.099
0.202
0.236
0.809
0.359
Scribble
0.133
0.043
0.173
0.155
0.087
0.158
0.221
0.733
0.318
nnInteractive
SBBox
0.480
0.553
0.249
0.115
0.064
0.112
0.218
0.790
0.333
Table 4: Per-pattern DSC of baseline models nnInteractive and MedSAM2.
Model
Prompt
Bron.
Cons.
Emph.
GGO
HC
Ret.
Ret.+GGO
Healthy
Case Avg.
nnU-Net
–
0.039
0.006
0.346
0.526
0.464
0.397
0.276
0.922
0.504
nnInteractive + MP
SBBox
0.073
0.006
0.283
0.165
0.093
0.124
0.134
0.869
0.291
MBBox
0.085
0.006
0.346
0.178
0.166
0.197
0.240
0.700
0.304
Points
0.074
0.006
0.297
0.191
0.113
0.167
0.200
0.805
0.300
Lasso
0.055
0.006
0.306
0.204
0.165
0.191
0.239
0.802
0.323
Scribble
0.084
0.006
0.323
0.155
0.125
0.152
0.246
0.788
0.299
Table 5: Per-pattern DSC across prompt types for PoC pipeline. Best per column in bold , second best underlined . Significance reported on Case Avg. only (paired Wilcoxon, 32 volumes, Holm–Bonferroni corrected): ⋆ / ▽ significantly better/worse than nnU-Net; ⋆ / ▽ significantly better/worse than nnInteractive+MP. Unmarked: no statistical significance. Case Avg. averaged across prompts: 0.448 (FF+MP P+N), 0.434 (FF+MP P), 0.501 (FF+MP N), 0.303 (nnInteractive+MP).
Medical image segmentation delineates organs, lesions, and other structures in scans to support measurement, diagnosis, treatment planning, and research.
Promptable segmentation models provide a reusable interface, but direct transfer to automatic infrared small-target segmentation (IRSTD) exposes a mismatch between spatial prompts and target-domain mask responses. In a diagnostic using target-covering loose-box prompts deterministically derived from test reference masks, the best official SAM2.1 results are only 4.69%, 1.64%, and 2.28% IoU on NUAA-SIRST, NUDT-SIRST, and IRSTD-1K. We introduce SPARK-SAM (Self-Prompt Adaptation with Response Knowledge for SAM), which learns target-domain response knowledge and conditions the decoder through an image-conditioned joint self-prompt state. Training combines benchmark-mask supervision with reliability-aware response guidance. SPARK-SAM achieves 75.78%, 86.49%, and 68.34% IoU with 0.726M additional parameters, ranking first on two benchmarks among 14 retrained SAM variants and adaptations evaluated as automatic image-to-mask methods. The staged IRSTD-1K diagnostic shows that response adaptation reaches most of the final IoU before the predicted points acquire reliable target grounding. Prompt supervision aligns the predicted prompt candidates with target locations, and frozen-weight interventions measure output sensitivity to the joint self-prompt state. Matched ablations show consistent accuracy gains from response guidance and high-resolution prompt refinement across all three datasets. Code is available at https://github.com/Sakauma/SPARK-SAM.
Aji Mao, Zhenming Peng, Bailin Mu +1
School of Information and Communication Engineering, University of Electronic Science and Technology of China
Semantic segmentation in medical imaging is a critical yet challenging task due to data scarcity and high variability across modalities. While foundation models like the Segment Anything Model (SAM) show promise, they often struggle with medical images without specific adaptation. Moreover, point prompts, despite being the most natural form of user interaction, provide insufficient spatial context for reliable segmentation, particularly when target structures are irregular or poorly contrasted. In this paper, we propose an enhanced segmentation framework that integrates a lightweight Box Predictor module into the MedSAM architecture. The Box Predictor estimates an approximate bounding box from a single user click using localized image embedding features, providing spatial guidance that reduces the ambiguity of point prompts, while introducing only 1.6M additional parameters and negligible inference overhead. We introduce a two-stage training pipeline where the Box Predictor is trained independently before being integrated into MedSAM. To validate the generalization capability of our method, we conduct extensive evaluations on four diverse datasets (FLARE22, BRISC, BUSI, LungSegDB) spanning distinct imaging modalities, including CT, MRI, and Ultrasound. Our method improves segmentation accuracy and robustness across varied anatomical structures and imaging domains, achieving Dice scores of 0.89 (BUSI), 0.93 (FLARE22), 0.88 (BRISC), and 0.98 (LungSegDB). Code is available at https://github.com/Amirhosseinmovahedi/MedSAM-BoxPredictor
Amirhossein Movahedisefat, Amirreza Fateh, Mohammad Reza Mohammadi
School of Computer Engineering, Iran University of Science and Technology (IUST), Tehran, Iran
Delineating lung tumours on computed tomography (CT) takes a considerable share of the time spent on radiotherapy planning, and a contour proposed by a model can be refined interactively by the clinician. Promptable foundation models such as SAM 3 support this workflow by writing each correction into a session memory that conditions the remaining slices, while the model weights stay fixed. On 690 test cases from five public CT cohorts, fine-tuning SAM 3 on lung tumours raises the Dice obtained from a single point prompt from 0.298 to 0.757, and seven rounds of corrections raise it further to 0.765, but under memory conditioning alone the accuracy on slices the annotator has not touched stops improving after six rounds. We therefore treat each correction as a training signal and propose LeCor, which performs test-time training on a small set of case adapters that are reset for every case and meta-learned such that a single gradient step driven by a click improves the slices that were not clicked. On the 133 test cases that span at least eight slices, LeCor raises the Dice reached after seven correction rounds from 0.787 with the fine-tuned model to 0.827, reduces the number of cases that never reach a Dice of 0.80 from 47 to 27, and reaches in three correction rounds the accuracy that the fine-tuned model attains in seven.
Yi Luo, Yike Guo, Wenxuan Li +3
Department of Biomedical Engineering, Johns Hopkins University · Department of Radiation Oncology and Molecular Radiation Sciences, Johns Hopkins University · Department of Computer Science, Johns Hopkins University +1