cs.CVAug 18, 2026

TEAMS: Text-prompted spatiotEmporal dual-heAd Mamba Snake

Authors: Ruicheng Zhang, Jianhui Lei, Kaiwen Shen, Haowei Guo, Jun Zhou, Bin Chen, Mengtang Li, Shen Zhao, 

Organizations: School of intelligent systems engineering, Sun Yat-sen University, Guangzhou 510006, China · Affiliated Hangzhou First People’s Hospital, Zhejiang University School of Medicine, Zhejiang, China · School of Engineering, Case Western Reserve University, Cleveland, USA

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Deep snake is a promising family of instance segmentation methods that accurately predicts object-level contours, thereby overcoming common pixel-level misclassification issues such as mask cavities and jagged edges in semantic segmentation approaches. However, existing deep snake methods face challenges in handling complex morphological variations, accurately capturing fine-grained organ details, and correcting base detection errors. To mitigate these limitations, we propose a cohesive Text-prompted spatiotEmporal dual-heAd Mamba Snake (TEAMS), a novel vision-language Mamba snake framework with three key innovations: (1) A Spatiotemporal Snake Evolution Strategy (SSES) is introduced to tackle complex morphological variations by capturing bidirectional spatial dependencies along the snake contour and temporal dynamics across evolution steps in a state space model. (2) A Contour Morphology-Aware Mamba (CMAM) is proposed to quantify local contour morphologies to modulate the structured attention mask in the Mamba2 SSD dual form, which extends Mamba's capability to perceive the relative importance of its input sequence elements for better delineation of fine-grained organ details. (3) A Text-prompted Collaborative Dual-Head Snake (TCDHS) is designed to incorporate cues from textual prompts and transfer the evolved contour information to the base detection head, which enhances the deep snake workflow and mitigates wrong detections. Comprehensive evaluations on five datasets covering different organs and imaging modalities demonstrate that TEAMS outperforms existing semantic and deep snake segmentation methods (e.g., relative mDice/mBF improvements of 6.9%/9.1% in a spinal dataset), underscoring its potential as a reliable tool across diverse medical image segmentation scenarios.

Figures & tables

Fig. 1: Deep snake has the potential to address illogical errors in semantic segmentation caused by pixel misclassifications.
Fig. 2: Limitations of existing deep snake workflow: susceptibility to morphological variations, inadequate modeling of fine-grained organ details, and inability to correct base detection errors.
Fig. 3: Innovation and advantage of our TEAMS: TEAMS is a new deep snake method that captures spatial and temporal context during snake evolution (in SSES), dynamically adapts contour smoothness to fit geometric details (in CMAM), integrates cues from textual prompts, and feeds evolved contour features to enhance base detection (in TCDHS) with dual-head consistency feedback.
Fig. 4: The workflow of TEAMS. It first extracts image/text features via a feature pyramid backbone and ClinicalBERT, and fuses them with cross-modal self-attention for base detection (Fig. 4 (a)) and contour initialization; it then evolves contours with a dedicated snake evolution head (Fig. 4 (b)), where SSES captures spatial and temporal contexts (Fig. 4 (b1)) while CMAM adaptively regulates contour smoothness (Fig. 4 (b2)) for accurate snake evolution. Finally, TCDHS performs post-evolution detection and designs a dual-head detection consistency loss to align the detections of the base and post-evolution branches (Fig. 4 (b3)).
Fig. 5: The Spatiotemporal Snake Evolution Strategy (SSES). Fig. 5 (a) shows the bidirectional spatial state space module. Fig. 5 (b) shows the temporal trajectory state aggregation module.
Fig. 6: The Contour Morphology-Aware Mamba (CMAM). Fig. 6 (a) shows how CMAM quantifies contour morphology complexity. Fig. 6 (b) shows how the CM-SSD is designed to leverage the morphological complexity for modulating the structured attention mask.
Fig. 7: The Text-prompted Collaborative Dual-Head Snake (TCDHS). In Fig. 7 (a), task-level text prompts are fused with multi-scale visual features to condition the base detection head, which predicts Clsbase\mathbf{Cls}_{\text{base}} , Boxbase\mathbf{Box}_{\text{base}} , and initializes P0\mathbf{P}_{0} . In Fig. 7 (b), organ-level text prompts are aligned with RoI features to form multi-modal feature maps Fcsm\mathbf{F}_{\text{cs}}^{m} for snake evolution. In Fig. 7 (c), the evolved contours P\mathbf{P} are converted into contour-aware heatmaps to augment features for a post-evolution detector. In Fig. 7 (d), a dual-head detection consistency loss is designed to align { Clsbase,Boxbase\mathbf{Cls}_{\text{base}},\mathbf{Box}_{\text{base}} } with { Clspost,Boxpost\mathbf{Cls}_{\text{post}},\mathbf{Box}_{\text{post}} }, transferring the contour information to the base detector.
Fig. 8: TEAMS achieves high instance segmentation performance on five challenging datasets of different imaging modalities and target organs (Column 11). It also outperforms state-of-the-art semantic segmentation (Columns 3 ∼\sim 8) and deep snake (Columns 9/10) methods.
Dataset ID/ NameModality / Target organsDataset sizeTarget class numbers
1. MR_AVBCE- ExtendedMRI / Spine∼\sim 480 scans, ∼\sim 1233 slices50
2. VerSeCT / Spine∼\sim 374 scans, ∼\sim 8000 slices26
3. BTCVCT / Abdomen30 scans, ∼\sim 900 slices8
4. RAOSCT / Abdomen∼\sim 317 scans, ∼\sim 7500 slices19
5. PanNukeMicroscopy / Nuclei∼\sim 7900 image patches5
Table 1: Overview of the five evaluation datasets.
MethodMR_AVBCE- ExtendedVerSe19(Pu)VerSe19(Pr)VerSe20(Pu)VerSe20(Pr)BTCVRAOSPanNuke
mIoUmDicemBFmIoUmDicemBFmIoUmDicemBFmIoUmDicemBFmIoUmDicemBFmIoUmDicemBFmIoUmDicemBFmPQ
TEAMS69.2 ∗ ±\pm 0.577.4 ∗ ±\pm 0.469.6 ∗ ±\pm 0.686.5 ∗ ±\pm 0.492.5 ∗ ±\pm 0.282.2 ∗ ±\pm 0.485.9 ∗ ±\pm 0.291.6 ±\pm 0.481.5 ∗ ±\pm 0.487.0 ∗ ±\pm 0.692.9 ∗ ±\pm 0.380.7 ∗ ±\pm 0.585.8 ∗ ±\pm 0.491.8 ∗ ±\pm 0.280.9 ∗ ±\pm 0.385.3 ±\pm 0.291.7 ∗ ±\pm 0.179.8 ∗ ±\pm 0.479.8 ∗ ±\pm 0.387.9 ∗ ±\pm 0.475.9 ∗ ±\pm 0.741.1 ±\pm 0.8
nnU-Net v2 ( Isensee et al. (2021) )62.7 ±\pm 1.170.1 ±\pm 1.061.8 ±\pm 1.382.2 ±\pm 0.589.7 ±\pm 0.376.8 ±\pm 0.681.0 ±\pm 0.388.8 ±\pm 0.475.6 ±\pm 0.880.2 ±\pm 0.488.1 ±\pm 0.373.9 ±\pm 0.881.4 ±\pm 0.489.5 ±\pm 0.375.4 ±\pm 0.881.9 ±\pm 0.686.0 ±\pm 0.971.4 ±\pm 0.773.8 ±\pm 0.285.7 ±\pm 0.368.6 ±\pm 0.834.5 ±\pm 1.1
UNETR ( Hatamizadeh et al. (2022) )56.1 ±\pm 1.167.3 ±\pm 1.157.7 ±\pm 1.278.6 ±\pm 0.585.3 ±\pm 0.671.6 ±\pm 0.979.7 ±\pm 0.785.5 ±\pm 0.671.3 ±\pm 0.578.0 ±\pm 0.685.3 ±\pm 0.571.8 ±\pm 0.779.2 ±\pm 0.586.4 ±\pm 0.472.5 ±\pm 0.981.0 ±\pm 0.684.1 ±\pm 0.769.6 ±\pm 1.174.1 ±\pm 0.785.9 ±\pm 0.868.1 ±\pm 1.031.4 ±\pm 1.3
TransUNet ( Chen et al. (2021) )55.8 ±\pm 1.066.7 ±\pm 1.056.4 ±\pm 1.378.5 ±\pm 0.784.5 ±\pm 0.770.1 ±\pm 0.977.8 ±\pm 0.684.2 ±\pm 0.669.5 ±\pm 0.778.4 ±\pm 0.685.0 ±\pm 0.771.2 ±\pm 0.776.8 ±\pm 0.683.9 ±\pm 0.669.1 ±\pm 0.979.0 ±\pm 0.985.3 ±\pm 0.970.2 ±\pm 1.072.3 ±\pm 0.684.1 ±\pm 0.667.2 ±\pm 1.135.8 ±\pm 1.2
Swin-Unet ( Cao et al. (2023a) )56.4 ±\pm 1.067.0 ±\pm 0.957.3 ±\pm 1.480.8 ±\pm 0.687.3 ±\pm 0.373.4 ±\pm 1.180.4 ±\pm 0.685.7 ±\pm 0.572.6 ±\pm 0.779.5 ±\pm 0.786.6 ±\pm 0.672.9 ±\pm 0.579.8 ±\pm 0.685.1 ±\pm 0.571.7 ±\pm 0.880.8 ±\pm 0.683.9 ±\pm 0.767.8 ±\pm 0.973.0 ±\pm 0.883.6 ±\pm 0.866.7 ±\pm 0.931.3 ±\pm 1.2
Table 2: Superiority of TEAMS over 14 SOTA methods on five datasets. Results are reported as mean ± standard deviation. The best results are highlighted in bold (this annotation holds for the subsequent tables), and the second-best results are underlined. Statistical significance is marked with * when p ¡ 0.05 compared with the strongest competing method under two-sided paired t-tests across folds or repeated runs.
Fig. 9: TEAMS consistently achieves strong results for most organ categories on the two most challenging datasets (Datasets 1 and 4). TEAMS (solid lines) outperforms the best semantic segmentation approach (dashed lines) for each organ.
Model ConfigurationmIoUmDicemBF
Baseline58.267.558.8
+ SSES62.2( ⇑\Uparrow 6.9%)72.9( ⇑\Uparrow 8.0%)62.8( ⇑\Uparrow 6.8%)
+ CMAM59.1( ⇑\Uparrow 1.5%)71.3( ⇑\Uparrow 5.6%)60.3( ⇑\Uparrow 2.6%)
+ Text prompts in TCDHS60.7( ⇑\Uparrow 4.3%)71.3( ⇑\Uparrow 5.6%)61.4( ⇑\Uparrow 4.4%)
+ Consistency feedback in TCDHS61.3( ⇑\Uparrow 5.3%)71.6( ⇑\Uparrow 6.1%)60.8( ⇑\Uparrow 3.4%)
+ TCDHS61.9( ⇑\Uparrow 6.3%)72.4( ⇑\Uparrow 7.3%)62.5( ⇑\Uparrow 6.3%)
Table 3: Ablation study demonstrating the effect of SSES, CMAM, text prompts, detection consistency feedback, and full TCDHS.
Fig. 10: The contributions of each core module. When each core module is enabled, the evolution failures are mitigated (a1 VS a2), the boundary details are better delineated (b1 VS b2), and the wrong detections are avoided (c1 VS c2).
Fig. 11: The necessity of Mamba modeling in deep snakes. Fig. 11 (a)/(b) show SSES/CMAM (green) better benefits overall performance than hyperparameter tuning (blue). Fig. 11 (c) shows SSES (c1) better mitigates evolution failure than increasing iteration numbers (c2). Fig. 11 (d) shows CMAM (d1) better delineates fine-grained organ details than increasing contour points (d2).
Temporal modeling strategymIoUmDicemBF
Standard temporal feature aggregation66.673.666.5
GRU-based recurrent modeling66.172.962.9
SSES69.277.469.6
Table 4: SSES achieves superior performance compared with existing temporal modeling strategies.
Temporal modeling methodmIoUmDicemBFTrainable params (M)Inference time (s)GPU mem- ory (MB)
Attention-based temporal fusion63.073.963.117.671.253066
Exponential decay aggregation (ours)69.277.469.615.741.172852
Table 5: The superiority of exponential decay aggregation over attention-based temporal fusion.
DatasetMethodmIoUmDicemBFmPQ
MR_AVBCE- ExtendedLearned alternative61.969.960.4–
 Our CMAM69.277.469.6–
VerSeLearned alternative80.483.171.0–
 Our CMAM87.092.980.7–
BTCVLearned alternative79.782.470.6–
 Our CMAM85.391.779.8–
Table 6: The morphology-guided learnable structured attention mask modeling in CMAM is better than the learned alternative.
VariantmIoUmDicemBF
Convolution-based snake evolution58.966.060.9
Transformer-based snake evolution63.572.864.9
Mamba-based snake evolution (ours)69.277.469.6
Table 7: Mamba-based snake evolution outperforms convolution-based and Transformer-based alternatives.
Fig. 12: The benefits of the text prompts. The task-level text prompts help to better distinguish different organs (a1 VS a2), and the organ-level text prompts help the snake to better evolve to target organs (b1 VS b2).
SettingmIoUmDicemBF
No prompt65.873.664.3
Random prompt63.371.962.8
Incorrect prompt63.071.362.1
Different correct template 168.977.468.8
Different correct template 268.576.768.5
Correct prompt / TEAMS69.277.469.6
Table 8: Ablation study on different textual prompt settings validates that textual prompts benefit snake segmentation.
MethodParams (M)Inference time (s)Peak GPU memory (MB)mDice/ mBF
Conference version12.480.83185671.3/63.6
Journal version (TEAMS)15.74 (trainable) + 137.43 (frozen)1.17285277.4/69.6
Table 9: The computational costs of TEAMS and the conference version on MR_AVBCE-Extended.
Fig. 13: Representative failure cases of TEAMS under annotation ambiguity. The solid contours denote the ground truth, and the dashed contours denote the evolved snake predictions. Red arrows indicate ambiguous vessel regions near the liver boundary: the vessel is excluded from the liver ground truth in Fig. 13 (a1), but included in the liver ground truth in Fig. 13 (a2). Such inconsistent annotations may lead to suboptimal segmentation results.
More in this topic

Medical Image Segmentation

Medical image segmentation delineates organs, lesions, and other structures in scans to support measurement, diagnosis, treatment planning, and research.

See all 786 papers

Explore similar work

CardsList

  1. SAM-DCE: Decoupled Context Enhancement for Prompt-Free Medical Image Segmentation

    Sep 21, 2025Yingzhen Hu, Yiheng Zhong, Aijing Yang +2Medical Image SegmentationSegment Anything Models

    Mohamed bin Zayed University of AI, Abu Dhabi, UAE · Xi’an Jiaotong-Liverpool University, Suzhou, China · University of Liverpool, Liverpool, United Kingdom

  2. TAMISeg: Text-Aligned Multi-scale Medical Image Segmentation with Semantic Encoder Distillation

    Apr 13, 2026Qiang Gao, Yi Wang, Yong Zhang +4Image SegmentationMedical Image Segmentation

    Department of Data Science and AI, Monash University, Melbourne, Australia · College of Computer Science, Chongqing University, Chongqing, China · Chongqing University Central Hospital, School of Medicine, Chongqing, China

  3. CS-MUNet: A Channel-Spatial Dual-Stream Mamba Network for Multi-Organ Segmentation

    Mar 20, 2026Yuyang Zheng, Mingda Zhang, Jianglong Qin +4Medical Image SegmentationMulti-Organ Segmentation

    School of Software Engineering, Yunnan University, KunMing, 650500, Yunnan, China. · The School of Artificial Intelligence, The Chinese University of Hong Kong, Shenzhen, 518172, Guangdong, China. · Yunnan Provincial Key Laboratory of Software Engineering, Yunnan University, Kunming, 650500, Yunnan, China.