Organizations: IEMN UMR CNRS 8520, Université Polytechnique Hauts-de-France, 59313 Valenciennes, France · University of the Basque Country (UPV/EHU), Donostia-San Sebastián, Spain · Sorbonne Center for Artificial Intelligence, Sorbonne University Abu Dhabi, Abu Dhabi, UAE
Deep learning has achieved remarkable success in medical image segmentation, often reaching expert-level accuracy in delineating tumors and tissues. However, most existing approaches remain task-specific, showing strong performance on individual datasets but limited generalization across diverse imaging modalities. Moreover, many methods focus primarily on the encoder, relying on large pretrained backbones that increase computational complexity. In this paper, we propose a decoder-centric approach for generalized 2D medical image segmentation. The proposed Deco-Mamba follows a U-Net-like structure with a Transformer-CNN-Mamba design. The encoder combines a CNN block and Transformer backbone for efficient feature extraction, while the decoder integrates our novel Co-Attention Gate (CAG), Vision State Space Module (VSSM), and deformable convolutional refinement block to enhance multi-scale contextual representation. Additionally, a windowed distribution-aware KL-divergence loss is introduced for deep supervision across multiple decoding stages. Extensive experiments on diverse medical image segmentation benchmarks yield state-of-the-art performance and strong generalization capability while maintaining moderate model complexity. The source code will be released upon acceptance.
Figures & tables
Figure 1: Comparison of Mean Dice Scores of SOTA methods with Respect to Model Complexity in terms of Parameters (M). Our proposed approaches are denoted Deco-Mamba-V 0 and Deco-Mamba-V 1 .
Figure 2: Comparison of Mean Dice Scores with Respect to Model Complexity in terms of FLOPs (G). Our proposed approaches are denoted Deco-Mamba-V 0 and Deco-Mamba-V 1 .
Table 1: Comparison of Abdominal Multi-Organ Segmentation on the Synapse Dataset. DSC and HD95 represent the average Dice Score and 95% Hausdorff Distance across eight classes, respectively. Columns 4 to 11 present the Dice Score for each class, while the last two columns indicate model complexity in terms of the number of parameters (millions) and the number of FLOPs (GMac), respectively. All methods follow the same splitting and evaluation protocol introduced in [ 7 ] for a fair comparison. Red and blue values indicate the best and second-best results, respectively.
Architecture
Average
Spleen
Kidney (R)
Kidney (L)
Gallbladder
Esophagus
Liver
Stomach
Aorta
IVA
P&S Vein
Pancreas
AG (R)
AG (L)
DSC ↑
HD95 ↓
U-Net [ 31 ]
71.35
25.16
84.71
75.87
82.05
63.38
69.23
92.71
71.66
86.57
71.93
60.82
54.85
57.14
56.58
Att-UNet [ 27 ]
71.75
26.16
84.21
77.40
82.60
65.91
68.72
93.53
73.89
86.12
70.86
61.52
54.10
55.29
58.63
Unet++ [ 47 ]
72.18
21.53
85.70
76.54
84.28
67.23
72.18
93.78
68.51
87.08
72.01
61.91
54.13
60.13
54.79
Swin-Unet [ 23 ]
67.65
21.52
85.48
73.32
80.05
62.76
65.26
92.53
71.15
79.03
63.45
54.26
54.81
51.12
46.26
TransUNet [ 7 ]
71.85
22.02
85.31
79.26
83.38
61.68
67.54
94.03
74.73
87.04
75.75
59.91
58.18
57.96
49.28
Table 2: Comparison of abdominal multi-organ segmentation on the BTCV dataset (13 classes). DSC and HD95 denote the average Dice score and 95% Hausdorff distance over all classes, respectively. Columns four to the last report the Dice score for each organ. All methods follow the same data split and evaluation protocol as in [ 7 ] . Red and blue values indicate the best and second-best results, respectively.
Ex
Architecture
Avg DSC ↑
RV ↑
Myo ↑
LV ↑
1
U-Net [ 31 ]
87.84
86.51
84.66
92.36
2
Att-UNet [ 27 ]
88.04
86.70
84.59
92.83
3
UNet++ [ 47 ]
90.65
88.52
88.31
95.12
4
SwinUNet [ 6 ]
88.07
85.77
84.42
94.03
5
TransUNet [ 7 ]
89.71
86.67
87.27
95.18
6
MT-Unet [ 38 ]
90.43
86.64
89.04
95.62
Table 3: Comparison on Automatic Cardiac Diagnosis Challenge (ACDC) dataset. Red and blue values indicate the best and second-best results, respectively.
Ex
Architecture
ISIC17
ISIC18
GlaS
MoNuSeg
1
U-Net
81.59
85.80
85.45
76.45
2
Att-UNet
80.82
84.99
88.80
76.67
3
UNet++
80.64
85.59
87.56
77.01
4
TransUNet
84.10
86.49
88.40
78.53
5
FAT-Net
85.00
86.91
–
–
6
PAG-TransYnet
83.99
87.99
94.20
79.62
Table 4: Comparison on the ISIC17, ISIC18, GlaS, and MoNuSeg datasets. Red and blue indicate the best and second-best results, respectively. In our experiments, different metrics showed similar trends; only Dice scores are reported here, with additional metrics in the supplementary material (Tables 1 and 2 page 2).
ID
Ablation Scenario
DSC ↑
HD95 ↓
1
w/o CNN branch in encoder
84.07
18.92
2
w/o VSSMB
83.51
15.96
3
Replace CAG with AG [ 27 ]
82.98
15.69
4
Replace CAG with LGAG (EMCAD [ 30 ] )
82.69
20.36
5
Replace CAG with CBAM [ 41 ]
84.01
16.19
6
Replace DefConv with Standard Conv
84.53
16.18
Table 5: Ablation study on the Synapse dataset. We analyze encoder components, attention design, reconstruction strategies, and supervision losses under a unified setting.
Medical image segmentation delineates organs, lesions, and other structures in scans to support measurement, diagnosis, treatment planning, and research.
School of Computing, Korea Advanced Institute of Science and Technology (KAIST), Daejeon, Republic of Korea · Department of Energy, Aalborg University, Aalborg, Denmark