Hierarchical Text-Guided Brain Tumor Segmentation via Sub-Region-Aware Prompts

Bahram Mohammadi; Ta Duc Huy; Afrouz Sheikholeslami; Qi Chen; Vu Minh Hieu Phan; Sam White; Minh-Son To; Xuyun Zhang

cs.CVMar 22, 2026

Hierarchical Text-Guided Brain Tumor Segmentation via Sub-Region-Aware Prompts

Authors: Bahram Mohammadi, Ta Duc Huy, Afrouz Sheikholeslami, Qi Chen, Vu Minh Hieu Phan, Sam White, Minh-Son To, Xuyun Zhang, +3 more

Organizations: Macquarie University, Sydney, NSW, Australia · Adelaide University, Adelaide, SA, Australia · Flinders University, Adelaide, SA, Australia · University of Sydney, Sydney, NSW, Australia

Abstract

Brain tumor segmentation remains challenging because the three standard sub-regions, i.e., whole tumor (WT), tumor core (TC), and enhancing tumor (ET), often exhibit ambiguous visual boundaries. Integrating radiological description texts with imaging has shown promise. However, most multimodal approaches typically compress a report into a single global text embedding shared across all sub-regions, overlooking their distinct clinical characteristics. We propose TextCSP (text-modulated soft cascade architecture), a hierarchical text-guided framework that builds on the TextBraTS baseline with three novel components: (1) a text-modulated soft cascade decoder that predicts WT->TC->ET in a coarse-to-fine manner consistent with their anatomical containment hierarchy. (2) sub-region-aware prompt tuning, which uses learnable soft prompts with a LoRA-adapted BioBERT encoder to generate specialized text representations tailored for each sub-region; (3) text-semantic channel modulators that convert the aforementioned representations into channel-wise refinement signals, enabling the decoder to emphasize features aligned with clinically described patterns. Experiments on the TextBraTS dataset demonstrate consistent improvements across all sub-regions against state-of-the-art methods by 1.7% and 6% on the main metrics Dice and HD95.

Figures & tables

Figure 1: Main architecture of our TextCSP, which contains three key components: (1) a text-modulated soft cascade decoder that predicts WT $\to$ TC $\to$ ET in a coarse-to-fine manner (Sec. 2.2 ); (2) sub-region-aware prompt tuning (Sec. 2.3 ), which uses learnable soft prompts with a LoRA-adapted BioBERT encoder to generate specialized text representations tailored to each sub-region; and (3) text-semantic channel modulators that convert these representations into channel-wise refinement signals, enabling the decoder to emphasize features aligned with clinically described patterns 2.4 .

Methods	Dice (%) $\uparrow$				HD95 $\downarrow$
Methods	ET	WT	TC	Avg.	ET	WT	TC	Avg.
3D-UNet [ 16 ]	80.4	87.3	81.6	83.1	6.11	10.51	8.93	8.17
nnU-Net [ 9 ]	82.2	87.5	82.6	84.1	4.27	11.90	8.52	8.23
SegResNet [ 5 ]	80.9	88.4	82.3	83.8	6.18	7.28	7.41	6.95
Swin UNETR [ 4 ]	81.0	89.5	80.8	83.8	5.95	8.23	7.03	7.07
Nestedformer [ 20 ]	82.6	89.5	80.2	84.1	5.08	10.51	8.93	8.17

Table 1: Comparison of TextCSP with the state-of-the-art methods on the TextBraTS dataset.

\dagger

denotes the results reproduced on the same platform as ours.

Soft Cascade	Sub-Region-Aware Prompt	LoRA	Text Modulation	Dice (%) $\uparrow$	HD95 $\downarrow$
$\checkmark$	$\times$	$\times$	$\times$	85.9	5.67
$\checkmark$	$\checkmark$	$\times$	$\times$	86.4	5.26
$\checkmark$	$\checkmark$	$\checkmark$	$\times$	86.6	5.04
$\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$	87.0	4.81

Table 2: Ablation study of main components of our method.

Table 4

Figure 2: Qualitative comparison with the baseline TextBraTS.

Figure 3: Cascade attention visualization. $A_{WT}$ concentrates within the WT region toward the TC boundary, and $A_{TC}$ further narrows from TC toward the ET boundary.

Explore similar work

Evidential learning driven Breast Tumor Segmentation with Stage-divided Vision-Language Interaction
Mar 11, 2026Jingxing Zhong, Qingtao Pan, Xuchang Zhou +2Medical Image Segmentation Text-Guided Image Segmentation
Center for Precision Medicine, The First Affiliated Hospital of Xiamen University, School of Medicine, Xiamen University, Xiamen, China; Department of Laboratory Medicine, The First Affiliated Hospital of Xiamen University, School of Medicine, Xiamen University, Xiamen, 361003, China · Maynooth International Engineering College, Fuzhou University, Fuzhou, 350108, China · Department of Computer and Data Science, Case Western Reserve University, Cleveland, Ohio, USA, Cleveland, 44106, Ohio, USA +2
SegGuidedNet: Sub-Region-Aware Attention Supervision for Interpretable Brain Tumor Segmentation
May 21, 2026Hasaan Maqsood, Saif Ur Rehman Khan, Sebastian Vollmer +2Image Segmentation BraTS
National University of Sciences and Technology (NUST), Islamabad, Pakistan. · Department of Computer Science, Rhineland-Palatinate Technical University of Kaiserslautern-Landau, Kaiserslautern, 67663, Germany. · German Research Center for Artificial Intelligence, Kaiserslautern, 67663, Germany. +2
Text-Guided Refinement of Multi-sequence Glioma Subregion Segmentation with a Vision-Language Foundation Model
Aug 5, 2026Zach Eidex, Yu-nong Lin, Mojtaba Safari +4Interactive Medical Image Segmentation Medical Imaging Foundation Models
Department of Biomedical Informatics, Emory University School of Medicine, Atlanta, GA, USA · Department of Radiation and Cellular Oncology, The University of Chicago, Chicago, IL, USA · Department of Radiation Oncology, Winship Cancer Institute, Emory University School of Medicine, Atlanta, GA, USA

Mar 11, 2026J/K move · Enter open · S save

Evidential learning driven Breast Tumor Segmentation with Stage-divided Vision-Language Interaction

Jingxing Zhong, Qingtao Pan, Xuchang Zhou, Jiazhen Lin+1

Center for Precision Medicine, The First Affiliated Hospital of Xiamen University, School of Medicine, Xiamen University, Xiamen, China; Department of Laboratory Medicine, The First Affiliated Hospital of Xiamen University, School of Medicine, Xiamen University, Xiamen, 361003, China · Maynooth International Engineering College, Fuzhou University, Fuzhou, 350108, China · Department of Computer and Data Science, Case Western Reserve University, Cleveland, Ohio, USA, Cleveland, 44106, Ohio, USA+2

Medical Image Segmentation Text-Guided Image Segmentation Medical VLMs

Open paper AskPDF arXiv

Breast cancer is one of the most common causes of death among women worldwide, with millions of fatalities annually. Magnetic Resonance Imaging (MRI) can provide various sequences for characterizing tumor morphology and internal patterns, and becomes an effective tool for detection and diagnosis of breast tumors. However, previous deep-learning based tumor segmentation methods have limitations in accurately locating tumor contours due to the challenge of low contrast between cancer and normal areas and blurred boundaries. Leveraging text prompt information holds promise in ameliorating tumor segmentation effect by delineating segmentation regions. Inspired by this, we propose text-guided Breast Tumor Segmentation model (TextBCS) with stage-divided vision-language interaction and evidential learning. Specifically, the proposed stage-divided vision-language interaction facilitates information mutual between visual and text features at each stage of down-sampling, further exerting the advantages of text prompts to assist in locating lesion areas in low contrast scenarios. Moreover, the evidential learning is adopted to quantify the segmentation uncertainty of the model for blurred boundary. It utilizes the variational Dirichlet to characterize the distribution of the segmentation probabilities, addressing the segmentation uncertainties of the boundaries. Extensive experiments validate the superiority of our TextBCS over other segmentation networks, showcasing the best breast tumor segmentation performance on publicly available datasets.

Abstract

Explore similar work

Evidential learning driven Breast Tumor Segmentation with Stage-divided Vision-Language Interaction

SegGuidedNet: Sub-Region-Aware Attention Supervision for Interpretable Brain Tumor Segmentation

Text-Guided Refinement of Multi-sequence Glioma Subregion Segmentation with a Vision-Language Foundation Model