A 3D SAM-Based Progressive Prompting Framework for Multi-Task Segmentation of Radiotherapy-induced Normal Tissue Injuries in Limited-Data Settings

Caiwen Jiang; Lei Zeng; Wei Liu

cs.CVApr 15, 2026

A 3D SAM-Based Progressive Prompting Framework for Multi-Task Segmentation of Radiotherapy-induced Normal Tissue Injuries in Limited-Data Settings

Authors: Caiwen Jiang, Lei Zeng, Wei Liu

Organizations: Department of Radiation Oncology, Mayo Clinic, Phoenix, Arizona, USA

Abstract

Radiotherapy-induced normal tissue injury is a clinically important complication, and accurate segmentation of injury regions from medical images could facilitate disease assessment, treatment planning, and longitudinal monitoring. However, automatic segmentation of these lesions remains largely unexplored because of limited voxel-level annotations and substantial heterogeneity across injury types, lesion size, and imaging modality. To address this gap, we curate a dedicated head-and-neck radiotherapy-induced normal tissue injury dataset covering three manifestations: osteoradionecrosis (ORN), cerebral edema (CE), and cerebral radiation necrosis (CRN). We further propose a 3D SAM-based progressive prompting framework for multi-task segmentation in limited-data settings. The framework progressively incorporates three complementary prompts: text prompts for task-aware adaptation, dose-guided box prompts for coarse localization, and click prompts for iterative refinement. A small-target focus loss is introduced to improve local prediction and boundary delineation for small and sparse lesions. Experiments on ORN, CE, and CRN demonstrate that the proposed method achieves reliable segmentation performance across diverse injury types and outperforms state-of-the-art methods.

Figures & tables

Fig. 1: Representative examples of radiotherapy-induced normal tissue injuries in the curated dataset. Three injury types, osteoradionecrosis (ORN), cerebral edema (CE), and cerebral radiation necrosis (CRN), are displayed in axial, sagittal, and coronal views. ORN is shown on CT, CE on T2-FLAIR MR images, and CRN on contrast-enhanced T1-weighted MR images. Lesion regions are highlighted in red.

Fig. 2: Schematic illustration of the proposed progressive prompting framework. (a) Overall architecture of the proposed method, including the image encoder, prompt encoder, and mask decoder. (b) Dose-guided box prompt generation from the high-dose region, with the target region highlighted in red, for coarse lesion localization. (c) Click-based iterative boundary refinement, where click simulation automatically generates foreground and background points to progressively refine the segmentation boundaries.

Fig. 3: Text prompt construction from patient medical records and diagnostic reports, including task, clinical, and demographic information.

Method	Dice% $\uparrow$	IoU% $\uparrow$	Precision% $\uparrow$	Recall% $\uparrow$	HD95 $\downarrow$	ASSD $\downarrow$
VNet	62.88 $\pm$ 3.92	45.95 $\pm$ 4.28	46.17 $\pm$ 4.70	69.21 $\pm$ 1.51	9.78 $\pm$ 1.76	3.64 $\pm$ 0.55
SegResNet	64.01 $\pm$ 11.59	49.10 $\pm$ 14.78	59.27 $\pm$ 7.51	70.92 $\pm$ 17.13	10.96 $\pm$ 2.97	3.03 $\pm$ 1.37
DynUNet	62.27 $\pm$ 8.14	45.62 $\pm$ 9.13	65.68 $\pm$ 5.80	59.34 $\pm$ 10.22	8.59 $\pm$ 1.57	1.92 $\pm$ 0.30
UNETR	72.19 $\pm$ 4.93	56.65 $\pm$ 5.82	63.74 $\pm$ 5.30	86.73 $\pm$ 18.42	13.06 $\pm$ 10.38	2.87 $\pm$ 0.93
SwinUNETR	76.65 $\pm$ 9.04	62.75 $\pm$ 11.13	76.09 $\pm$ 0.83	74.70 $\pm$ 16.40	9.80 $\pm$ 12.13	2.05 $\pm$ 1.78
Ours	77.11 $\pm$ 3.57	63.23 $\pm$ 4.52	75.93 $\pm$ 5.18	75.17 $\pm$ 8.87	5.70 $\pm$ 0.63	1.39 $\pm$ 0.22

Table 1: Quantitative comparison of the proposed method and five state-of-the-art segmentation methods for radiotherapy-induced injury segmentation, evaluated using Dice, IoU, Precision, Recall, HD95 and ASSD.

Fig. 4: Visual comparison of segmentation results produced by different methods for three representative cases of radiotherapy-induced injuries, including osteoradionecrosis (ORN), cerebral edema (CE), cerebral radiation necrosis (CRN). From left to right are the ground truth (GT), and the segmentation results obtained by VNet, SegResNet, DynUNet, UNETR, SwinUNETR and our method. Red overlays indicate the lesion regions.

Method	Dice% $\uparrow$	IoU% $\uparrow$	Precision% $\uparrow$	Recall% $\uparrow$	HD95 $\downarrow$	ASSD $\downarrow$
Base	65.99 $\pm$ 6.46	49.08 $\pm$ 6.07	51.32 $\pm$ 2.94	62.99 $\pm$ 7.14	10.59 $\pm$ 1.86	1.70 $\pm$ 0.06
Base-T	71.00 $\pm$ 3.31	55.11 $\pm$ 3.88	62.00 $\pm$ 6.48	73.66 $\pm$ 16.99	7.77 $\pm$ 1.26	2.36 $\pm$ 0.31
Base-T-B	72.48 $\pm$ 8.36	58.48 $\pm$ 12.53	69.44 $\pm$ 6.82	68.67 $\pm$ 24.59	7.71 $\pm$ 1.92	1.83 $\pm$ 0.40
Base-T-B-C	74.05 $\pm$ 6.95	57.91 $\pm$ 9.12	78.79 $\pm$ 3.39	72.76 $\pm$ 15.54	6.43 $\pm$ 1.35	1.28 $\pm$ 0.04
Base-T-B-C-L	77.11 $\pm$ 3.57	63.23 $\pm$ 4.52	75.93 $\pm$ 5.18	75.17 $\pm$ 8.87	5.70 $\pm$ 0.63	1.39 $\pm$ 0.22

Table 2: Quantitative ablation study of the progressively introduced components in the proposed framework for radiotherapy-induced injury segmentation, evaluated using Dice, IoU, Precision, Recall, HD95 and ASSD.

Fig. 5: Visual comparison of ablation results for an ORN case in three anatomical views (axial, sagittal and coronal). From left to right are the ground truth (GT) and the results of the baseline model and its successive variants with text prompt (Base-T), text + box prompts (Base-T-B), text + box + click prompts (Base-T-B-C), and the full model with the combined loss (Base-T-B-C-L). Red overlays denote the lesion regions.

Fig. 6: Per-task comparison of Base and Base-T on ORN, CE, and CRN for evaluating the contribution of text prompting to unified multi-task segmentation. The left panel shows Dice, and the right panel shows HD95.

Fig. 7: Visualization of high-dose masks extracted under different relative dose thresholds for dose-guided box prompting in an ORN case. From left to right are the input image, the corresponding dose map, and the high-dose masks obtained using different relative thresholds $(\tau\times D_{\max})$ , with $\tau=0.5,0.6,0.7,0.8,$ and $0.9$ . The lesion region is overlaid in red.

Fig. 8: Quantitative analysis of segmentation performance under different relative dose thresholds for high-dose mask extraction in dose-guided box prompting.

Fig. 9: Sensitivity analysis of the proposed click refinement strategy in terms of Dice and HD95 under different refinement iteration settings and simulated click numbers. (a) Quantitative results with different numbers of refinement iterations. (b) Quantitative results with different numbers of simulated clicks per iteration.

Medical Image Segmentation

Medical image segmentation delineates organs, lesions, and other structures in scans to support measurement, diagnosis, treatment planning, and research.

See all 788 papers

Text-Guided Medical Image Segmentation via Residual Aggregation and Dense Alignment Representations
Oct 10, 2026Jie Gui, Hang Tu, Yunfeng Dou +3Text-Guided Image Segmentation Promptable Image Segmentation
Key Laboratory of Intelligent Computing and Signal Processing, Ministry of Education, Anhui University, Hefei, China · School of Artificial Intelligence, Anhui University, Hefei, China · School of Computer Science and Technology, Anhui University, Hefei, China +1
SARFA: Segment Anything with Radiomic Feature Alignment
Jul 14, 2026Tyler Ward, Abdullah ImranPromptable Image Segmentation Medical Image Segmentation
Computer Science Department University of Kentucky, Lexington, KY, USA
Prompt Group-Aware Training for Robust Text-Guided Nuclei Segmentation
Mar 6, 2026Yonghuang Wu, Zhenyang Liang, Wenwen Zeng +2Text-Guided Image Segmentation Medical Image Segmentation
Fudan University, Shanghai, China

Oct 10, 2026J/K move · Enter open · S save

Text-Guided Medical Image Segmentation via Residual Aggregation and Dense Alignment Representations

Jie Gui, Hang Tu, Yunfeng Dou, Wen Sha+2

Key Laboratory of Intelligent Computing and Signal Processing, Ministry of Education, Anhui University, Hefei, China · School of Artificial Intelligence, Anhui University, Hefei, China · School of Computer Science and Technology, Anhui University, Hefei, China+1

Text-Guided Image Segmentation Promptable Image Segmentation Medical Image Grounding Medical VLMs

Open paper AskPDF arXiv

Specifying segmentation targets with natural language provides flexible semantic conditioning for medical image segmentation. However, existing methods still struggle to reliably ground high-level textual semantics into accurate pixel-level spatial prompts, especially under low-contrast lesions, ambiguous boundaries, and visually similar anatomical contexts. In this paper, we propose RADAR (Residual Aggregation and Dense Alignment Representations), a parameter-efficient framework for text-guided medical image segmentation. RADAR explicitly decouples and synergizes the medical semantic anchoring capability of BiomedCLIP with the prompt-driven mask decoding ability of MedSAM2, while enabling low-cost task adaptation via LoRA. Specifically, Guided Residual Depth Aggregation adaptively aggregates cross-layer features under semantic guidance to alleviate delayed textual injection and deep semantic dilution. Structured Dense Representation Alignment leverages a frozen BiomedCLIP to construct crop-level target, boundary, and context anchors, improving the local discriminability of dense features and the reliability of automatic prompts. Multi-modal Prompt Generation further converts cross-modal semantic responses into composite text, point, and mask prompts for MedSAM2 decoding. Experiments on three public medical image segmentation datasets show that RADAR achieves competitive performance with only 0.74M trainable parameters, demonstrating its effectiveness in parameter-efficient adaptation, semantic grounding, and dense representation enhancement. The source code is available at https://github.com/JGuiidev/RADAR.

Abstract

Medical Image Segmentation

Explore similar work

Text-Guided Medical Image Segmentation via Residual Aggregation and Dense Alignment Representations

SARFA: Segment Anything with Radiomic Feature Alignment

Prompt Group-Aware Training for Robust Text-Guided Nuclei Segmentation