Organizations: Center for Precision Medicine, The First Affiliated Hospital of Xiamen University, School of Medicine, Xiamen University, Xiamen, China; Department of Laboratory Medicine, The First Affiliated Hospital of Xiamen University, School of Medicine, Xiamen University, Xiamen, 361003, China · Maynooth International Engineering College, Fuzhou University, Fuzhou, 350108, China · Department of Computer and Data Science, Case Western Reserve University, Cleveland, Ohio, USA, Cleveland, 44106, Ohio, USA · Department of Rehabilitation Medicine, the First Affiliated Hospital of Xiamen University, School of Medicine, Xiamen University, Xiamen, 361003, China · Department of Pharmacy, Xiang’an Hospital of Xiamen University, School of Medicine, Xiamen University, Xiamen, 361005, China
Breast cancer is one of the most common causes of death among women worldwide, with millions of fatalities annually. Magnetic Resonance Imaging (MRI) can provide various sequences for characterizing tumor morphology and internal patterns, and becomes an effective tool for detection and diagnosis of breast tumors. However, previous deep-learning based tumor segmentation methods have limitations in accurately locating tumor contours due to the challenge of low contrast between cancer and normal areas and blurred boundaries. Leveraging text prompt information holds promise in ameliorating tumor segmentation effect by delineating segmentation regions. Inspired by this, we propose text-guided Breast Tumor Segmentation model (TextBCS) with stage-divided vision-language interaction and evidential learning. Specifically, the proposed stage-divided vision-language interaction facilitates information mutual between visual and text features at each stage of down-sampling, further exerting the advantages of text prompts to assist in locating lesion areas in low contrast scenarios. Moreover, the evidential learning is adopted to quantify the segmentation uncertainty of the model for blurred boundary. It utilizes the variational Dirichlet to characterize the distribution of the segmentation probabilities, addressing the segmentation uncertainties of the boundaries. Extensive experiments validate the superiority of our TextBCS over other segmentation networks, showcasing the best breast tumor segmentation performance on publicly available datasets.
Figures & tables
Figure 1: Various segmentation methods for breast cancer segmentation. (a) Traditional segmentation methods rely solely on the image to perform segmentation tasks. (b) Our TextBCS leverages text prompts related to canceration regions to assist in locating breast cancer areas.
Figure 2: Overview of TextBCS, consisting of the SVLI module and the evidential learning module. SVLI module performs cross fusion and alignment between the image and text feature at each downsampling stage, enhancing sufficient image-text interaction. The evidential learning module performs pixel-level uncertainty estimation based on the decoding results, preventing the model from making unreliable predictions.
Methods
Text
Dice (%)
mIoU (%)
Param (M)
Flops (G)
UNet [ 39 ]
✗
81.54
73.22
14.8
50.3
UNet++ [ 64 ]
✗
81.23
72.69
74.5
94.6
nnUNet [ 16 ]
✗
80.15
71.81
19.1
412.7
TransUNet [ 7 ]
✗
83.14
75.49
105
56.7
SwinUNet [ 5 ]
✗
81.70
70.25
82.3
67.3
UCTransNet [ 51 ]
✗
78.07
69.41
65.6
63.2
Table 1: The quantitative comparison between our method and other comparison methods on the Duke-Breast-Cancer-MRI dataset demonstrates the superiority of our method. The best values are in bold.
Baseline
SVLI
EL
Dice (%)
mIoU (%)
✓
81.54
73.22
✓
✓
84.41
75.24
✓
✓
83.19
74.74
✓
✓
✓
85.33
76.08
Table 2: Ablation studies on the Duke-Breast-Cancer-MRI dataset demonstrate the significant improvements of the proposed innovations, including SVLI and EL. The best values are in bold.
Figure 3: The visual comparison of segmentation results between different methods,such as UNet, nnUNet, TransUNet, CLIP, MGCA and LViT, is displayed. Each row corresponds to one subject.
Figure 4: Saliency map for interpretability study of different layers in encoder. It is obvious that the introduction of interaction between text prompts and images in each stage can better locate the breast cancer region.
Figure 5: A case that our method fail under the text prompt by segmenting incorrect areas and insufficiently identifying the target area.
Coefficient
1.0
0.1
0.01
0.001
λ1
84.79
84.83
85.33
85.14
λ3
85.11
84.90
85.33
85.07
Table 3: Ablation study of parameters ( λ1,λ3 ). Dice values of Duke-Breast-Cancer-MRI datasets are reported. Bold values are the Dice values corresponding to the optimal parameters.
Methods
p-values
Unet [ 39 ]
0.0014
Unet++ [ 64 ]
0.0032
nnUNet [ 16 ]
0.0008
TransUNet [ 7 ]
0.0030
SwinUNet [ 5 ]
0.0018
UCTransNet [ 51 ]
0.0009
Table 4: P-values from t-test between our method and other methods shown that our method exhibits statistical significance. P-values are calculated based on the Dice.
Figure 6: Qualitative experiment of different styles of text prompts shows the robustness of the model to variations in the text prompts.
Brain tumor segmentation remains challenging because the three standard sub-regions, i.e., whole tumor (WT), tumor core (TC), and enhancing tumor (ET), often exhibit ambiguous visual boundaries. Integrating radiological description texts with imaging has shown promise. However, most multimodal approaches typically compress a report into a single global text embedding shared across all sub-regions, overlooking their distinct clinical characteristics. We propose TextCSP (text-modulated soft cascade architecture), a hierarchical text-guided framework that builds on the TextBraTS baseline with three novel components: (1) a text-modulated soft cascade decoder that predicts WT->TC->ET in a coarse-to-fine manner consistent with their anatomical containment hierarchy. (2) sub-region-aware prompt tuning, which uses learnable soft prompts with a LoRA-adapted BioBERT encoder to generate specialized text representations tailored for each sub-region; (3) text-semantic channel modulators that convert the aforementioned representations into channel-wise refinement signals, enabling the decoder to emphasize features aligned with clinically described patterns. Experiments on the TextBraTS dataset demonstrate consistent improvements across all sub-regions against state-of-the-art methods by 1.7% and 6% on the main metrics Dice and HD95.
Bahram Mohammadi, Ta Duc Huy, Afrouz Sheikholeslami +8
Macquarie University, Sydney, NSW, Australia · Adelaide University, Adelaide, SA, Australia · Flinders University, Adelaide, SA, Australia +1
Accurate brain tumor segmentation from MRI scans is critical for diagnosis and treatment planning. Despite the strong performance of recent deep learning approaches, two fundamental limitations remain: (1) the lack of reliable uncertainty quantification in single-model predictions, which is essential for clinical deployment because the level of uncertainty may impact treatment decision-making, and (2) the under-utilization of rich information in radiology reports that can guide segmentation in ambiguous regions. In this paper, we propose the Disagreement-Guided Refinement Network (DGRNet), a novel framework that addresses both limitations through multi-view disagreement-based uncertainty estimation and text-conditioned refinement. DGRNet generates diverse predictions via four lightweight view-specific adapters attached to a shared encoder-decoder, enabling efficient uncertainty quantification within a single forward pass. Afterward, we build disagreement maps to identify regions of high segmentation uncertainty, which are then selectively refined according to clinical reports. Moreover, we introduce a diversity-preserving training strategy that combines pairwise similarity penalties and gradient isolation to prevent view collapse. The experimental results on the TextBraTS dataset show that DGRNet favorably improves state-of-the-art segmentation accuracy by 2.4% and 11% in main metrics Dice and HD95, respectively, while providing meaningful uncertainty estimates.
Bahram Mohammadi, Yanqiu Wu, Vu Minh Hieu Phan +6
Macquarie University, Sydney, NSW, Australia · Adelaide University, Adelaide, SA, Australia · Flinders University, Adelaide, SA, Australia +1
Medical image segmentation is more clinically valuable when it supports diagnosis rather than merely producing lesion masks. However, diagnostically relevant lesion cues are often subtle and localized, while existing models may be distracted by background tissues, acoustic artifacts, and irrelevant visual correlations. To address this problem, we propose Rad-VLSM, a two-stage cross-modal framework for semantics-assisted lesion focusing, robust segmentation, and visually grounded diagnosis. In the first stage, a BLIP-2-based vision-language alignment module identifies lesion-related candidate regions under semantic guidance and converts them into box prompts. In the second stage, these prompts are fed into a SAM-based multitask network, where a multi-candidate region aggregation strategy improves prompt stability and guides lesion segmentation. The predicted masks are then used as spatial priors for diagnosis, and a visual-radiomics fusion head integrates lesion-aware visual features with selected radiomics descriptors. By using semantic information for localization rather than direct prediction, Rad-VLSM reduces text-to-diagnosis dependence and grounds diagnosis in lesion-level evidence. Experiments on a private clinical breast ultrasound dataset and public benchmarks show that Rad-VLSM achieves strong segmentation and diagnostic performance with favorable generalization.