U-Net architectures have been instrumental in advancing biomedical image segmentation (BIS) but often struggle with capturing long-range information. One reason is the conventional down-sampling techniques that prioritize computational efficiency at the expense of information retention. This paper introduces a simple but effective strategy, we call it Stair Pooling, which moderates the pace of down-sampling and reduces information loss by leveraging a sequence of concatenated small and narrow pooling operations in varied orientations. Specifically, our method modifies the reduction in dimensionality within each 2D pooling step from 41 to 21. This approach can also be adapted for 3D pooling to preserve even more information. Such preservation aids the U-Net in more effectively reconstructing spatial details during the up-sampling phase, thereby enhancing its ability to capture long-range information and improving segmentation accuracy. Extensive experiments on three BIS benchmarks demonstrate that the proposed Stair Pooling can increase both 2D and 3D U-Net performance by an average of 3.8% in Dice scores. Moreover, we leverage the transfer entropy to select the optimal down-sampling paths and quantitatively show how the proposed Stair Pooling reduces the information loss.
Figures & tables
Figure 1: The overview of our proposed Stair Pooling. It splits the original max pooling layer into a series of concatenated small and narrow pooling kernels. To break the linear relationship, each pooling operation is followed by a convolutional layer and a ReLU activation.
Methods
DSC↑
HD↓
Aorta
Gallbladder
Kidney(L)
Kidney(R)
Liver
Pancreas
Spleen
Stomach
R50 U-Net
74.68
36.87
87.74
63.66
80.6
78.19
93.74
56.9
85.87
74.16
U-Net
76.85
39.7
89.07
69.72
77.77
68.6
93.43
53.98
86.67
75.58
R50 Att-UNet
75.57
37.66
55.92
63.91
79.2
72.71
93.56
49.37
87.19
74.95
Att-UNet
77.77
36.87
89.55
68.88
77.98
71.11
93.57
58.04
87.3
75.75
R50 ViT
71.29
32.27
73.73
55.13
73.52
72.2
91.51
45.99
81.99
73.95
TransUnet
77.48
31.69
87.23
63.13
81.87
77.02
94.08
55.86
85.08
75.62
Table 1: Segmentation accuracy of different methods on the Synapse multi-organ CT dataset. The highest value in each column is highlighted in bold. Gray rows refer to methods whose sizes are significantly larger than others.
Methods
DSC
RV
Myo
LV
R50 U-Net
87.55
87.1
80.63
94.92
U-Net
86.63
85.46
81.08
93.35
R50 ViT
87.57
86.07
81.88
94.75
TransUnet
89.71
88.86
84.53
95.73
SwinUnet
90.00
88.55
85.62
95.83
Pyramid UNet
89.17
88.14
83.85
95.52
Table 2: Segmentation accuracy of different methods on the ACDC dataset in DSC( % ).
Figure 2: Qualitative comparison of different approaches on the Synapse dataset. From left to right: Ground Truth, U-Net, SwinUnet, UNet with HWT pooling, our SP UNet and the TE selected variant.
Methods
DSC
Kidney
Masses
Tumor
UNETER
73.4
95.54
69.07
60.56
UNETER+SENet
74.9
95.70
69.73
60.50
UNETERt+CBAM
75.2
95.33
70.05
62.46
U-Net++
75.9
95.65
70.94
62.64
Attention UNETER
76.6
95.47
70.65
62.73
SP UNETER
77.1
95.71
71.77
63.85
Table 3: Segmentation accuracy of different methods on the KiT23 dataset in DSC( % ).
Figure 3: Segmentation results and sizes of different models on the Synapse dataset.
Dataset
Step1
Step2
Step3
Synpase
P21′ , P12′
P21′ , P12′
P12′ , P21′
ACDC
P21′ , P12′
P21′ , P12′
P21′ , P12′
KiT23
P122′ , P212′ , P221′
P122′ , P221′ , P212′
P122′ , P212′ , P221′
Table 4: Optimal paths selected by calculating TE on three datasets, where Pvhw′ refers to a v×h×w pooling kernel.
Figure 4: Correlations between DSC and TE on the KiT23 dataset.
Figure 5: Violin plots for visualizing distributions of transfer entropy across different paths on three datasets.
Segmentation of biomarkers in medical images is frequently viewed as a first step towards medical image analysis in any bioinformatics or biomedical application. Despite progress, existing methods still struggle to capture information at multiple scales and to perform upsampling effectively across different datasets. These shortcomings often result in suboptimal generalization capabilities. Recently, architectures belonging to the Nested-UNet family excel in capturing multiscale contextual information and upsample them effectively. In this work, We propose a novel Nested-UNet architecture that effectively captures multi-scale contextual information. It includes inner and outer attention units to enhance focus during upsampling, along with channel-wise feature recalibration using squeeze-and-excitation modules, leading to improved segmentation performance. Additionally, the architecture integrates an edge-aware loss to emphasize boundary accuracy by assigning greater importance to edge regions. Tested extensively on three publicly available benchmark datasets. Our method demonstrates a generalization performance superior to existing Nested-UNet methods. Code: https://github.com/saadwazir/histosegplusplus
Saad Wazir, Rao Faizan, Daeyoung Kim
School of Computing, KAIST Daejeon, Republic of Korea · Kyung Hee University (Global Campus) Yongin-si, Republic of Korea
Medical image segmentation plays an important role in computer aided diagnosis, treatment planning, and disease monitoring. U-Net has been widely used for biomedical image segmentation because of its encoder decoder structure and skip connections. However, conventional convolution based U-Net models may have limited ability to capture long range dependencies and global contextual information, which can affect performance in complex segmentation tasks. This paper presents a comparative study of five U-Net based architectures: U-Net 3D, Residual U-Net, Attention U-Net, UNETR, and Swin UNETR. The models are evaluated on two benchmark datasets: BraTS 2023 for brain tumor segmentation and DRIVE for retinal vessel segmentation. Experimental results show that Swin UNETR achieves the best overall performance, with Dice scores of 0.8965 on BraTS 2023 and 0.8078 on DRIVE. The results suggest that transformer based U-Net variants are effective for segmentation tasks requiring global contextual modeling, while residual learning remains useful for fine structure segmentation. This study provides practical insights into model selection for medical image segmentation across volumetric MRI and retinal imaging tasks.
Khoa Pham, Sindhuja Penchala, Jiacheng Li +2
Mississippi State University · The University of Alabama
Medical image segmentation requires high accuracy and robustness, yet practical commercial deployment also demands privacy preservation and computational efficiency. In this context, the U-Net architecture, which can be inherently decoupled into independent encoder and decoder components, serves as a natural commercial choice. However, pure Transformer-based variants like Swin-UNet often suffer from insufficient local detail capture and limited interpretability. In this paper, we propose a lightweight hybrid architecture built upon the Swin-UNet framework. Our model integrates a parallel CNN encoder to complement the shallow layer reasoning of Swin Transformers with local texture features. To bridge the semantic gap and enhance fine-grained spatial detail recovery, we design an asymmetric feature fusion strategy and introduce cross-layer skip (XSkip) connections that explicitly propagate shallow CNN features into the decoder. We further incorporate novel loss functions and an auxiliary supervision head (Aux-Head) to strengthen training stability, boundary delineation, and intermediate feature interpretability. Extensive experiments on the Synapse multi-organ segmentation dataset demonstrate that our approach achieves state-of-the-art competitive Dice scores and Hausdorff distances, offering an accurate, efficient, and interpretable solution for clinical deployment.