Organizations: Department of Computer Science, Brown University, Providence, RI, USA · Department of Computer Science, Bowdoin College, Brunswick, ME, USA
Squeeze-and-Excitation (SE) modules improve convolutional networks by modeling channel-wise dependencies, but global average pooling collapses spatial structure into a single per-channel descriptor, limiting their effectiveness for dense prediction tasks such as image segmentation. We generalize SE by replacing global pooling with spatially guided pooling, enabling region-aware channel recalibration through four modules with complementary cost profiles. BASE formalizes Tiled Squeeze-and-Excite using a regular grid; SPASE pools over SLIC superpixels for image-adaptive regions; MAPSE obtains regions from semantic maps produced by a frozen, general-purpose network; and PLEASE learns soft region assignments end-to-end through a lightweight assignment network. Each channel receives one gate per region, so attention at each pixel reflects its region membership, bridging channel and spatial attention. PLEASE also produces its soft assignments as a byproduct of the forward pass, providing structured spatial interpretability without post-hoc analysis. The modules offer complementary computational tradeoffs: some add essentially no parameters, while others add no measurable latency. Across binary segmentation on CUB, DUTS, and ECSSD, semantic segmentation on PASCAL VOC 2012 and ADE20K, and classification on mini-ImageNet, our modules outperform SE and related spatial-channel attention variants on most benchmarks, achieving the majority of best results.
Figures & tables
Figure 1: Relevant Squeeze-and-Excite Modules. Learnable operations are marked with .
Table 1: Interpretability visualizations for representative good and bad segmentation predictions. The top two rows correspond to examples selected from the top 5% of examples by mIoU, while the bottom two rows correspond to examples selected from the bottom 5% by mIoU. CUB/DUTS/ECSSD use U-Net, while ADE20K uses Semantic U-Net.
Model
Params
Added params
GFLOPs
Added GFLOPs
CPU latency
GPU latency
Baseline
7.763M
0
18.551
0.000
237.69
2.10
SE
7.774M
+11,390
18.557
+0.006
245.33
2.83
scSE
7.775M
+11,874
18.569
+0.018
244.90
3.16
ECA
7.763M
+16
18.557
+0.006
241.73
2.58
CA
7.782M
+18,704
18.565
+0.015
243.62
3.42
EMA
7.764M
+910
18.738
+0.187
270.51
4.04
Table 2: U-Net complexity and latency at input size 3×224×224 . Latency is reported in ms/image for FP32 inference at batch size 1, using 50 warm-up and 200 measured iterations. GPU latency was measured on a single NVIDIA GeForce RTX 3090 (24 GB); CPU latency was measured single-threaded on one core of a dual-socket AMD EPYC 7532 node ( 2×32 physical cores; SMT disabled).
Figure S1 : Architectural details of the CNNs used for experimentation.
Figure S2 : Example images and segmentation masks from CUB, DUTS, ECSSD, PASCAL VOC 2012, and ADE20K. (a) Input images. (b) Ground truth masks.
Table S2 : DUTS K ablation results for SPASE , E1–E4, ϕ(⋅)=MLP , U-Net.
MLP [ 8 ]
Conv1D [ 32 ]
Module
mIoU / Boundary F1
Added params
Added params (%)
mIoU / Boundary F1
Added params
Added params (%)
Baseline (SE/ECA)
82.41 ± 0.47 / 60.35 ± 0.81
11,390
+0.147
84.07 ± 0.39 / 63.63 ± 0.17
16
+0.000
BASE
82.73 ± 0.77 / 60.27 ± 1.67
11,390
+0.147
83.28 ± 0.82 / 61.97 ± 1.30
16
+0.000
SPASE
82.95 ± 0.19 / 61.02 ± 0.93
11,390
+0.147
83.51 ± 0.24 / 61.81 ± 0.83
16
+0.000
PLEASE
83.08 ± 0.27 / 61.62 ± 0.70
34,142
+0.440
84.08 ± 0.63 / 62.78 ± 0.09
22,768
+0.293
MAPSE (GradCAM)
84.60 ± 0.31 / 63.71 ± 0.93
11,390
+0.147
85.45 ± 0.49 / 65.67 ± 0.36
16
+0.000
Table S3 : DUTS ϕ(⋅) ablation for U-Net encoder E1–E4 modules. Metric cells report mIoU / Boundary F1. For the baseline row, the MLP setting is vanilla SE and the 1D conv setting is ECA. The better of the two settings in each row is in bold .
Backbone
Module
Residual False
Residual True
U-Net
Baseline (ECA)
84.05 ± 0.35 / 63.59 ± 0.24
83.20 ± 0.91 / 62.00 ± 0.85
U-Net
BASE
83.26 ± 0.54 / 61.88 ± 1.12
82.53 ± 0.73 / 60.78 ± 1.38
U-Net
SPASE
83.59 ± 0.48 / 61.76 ± 0.56
83.36 ± 0.19 / 61.78 ± 0.35
U-Net
PLEASE
84.71 ± 0.58 / 64.76 ± 0.69
83.77 ± 0.64 / 62.67 ± 0.28
FCN
Baseline (ECA)
73.70 ± 0.83 / 41.62 ± 0.58
73.33 ± 0.45 / 40.39 ± 0.17
FCN
BASE
72.46 ± 0.64 / 40.35 ± 0.68
71.54 ± 0.43 / 38.84 ± 0.56
Table S4 : DUTS residual-connection ablation for the U-Net (E1–E4) and FCN (S1–S2). All modules use ECA excitation. Metric cells report mIoU / Boundary F1, and the better of the two settings in each row is in bold .
Location
Blocks
mIoU
Boundary F1
Encoder
E1
81.36 ± 0.67
58.71 ± 1.50
Encoder
E1–E2
81.83 ± 0.17
60.20 ± 0.15
Encoder
E1–E3
82.05 ± 0.21
60.53 ± 0.82
Encoder
E1–E4
83.40 ± 0.63
62.40 ± 0.44
Skip
S1
81.75 ± 0.27
58.25 ± 0.65
Skip
S1–S2
82.12 ± 0.31
58.72 ± 0.23
Table S5 : DUTS attention location ablation using PLEASE with K=8 , ϕ=MLP , no residual connection, and a U-Net backbone.
K
mIoU
Boundary F1
2
82.73 ± 1.09
61.38 ± 1.17
8
82.87 ± 0.81
61.70 ± 0.76
16
83.24 ± 0.82
61.52 ± 0.95
64
83.48 ± 0.28
62.47 ± 0.33
128
83.03 ± 0.45
61.80 ± 0.81
500
82.71 ± 0.41
61.37 ± 0.77
Table S6 : DUTS region-count ablation using PLEASE at E1–E4 with ϕ=MLP , no residual connection, and a U-Net backbone.
K
mIoU
Dice/F1
Boundary F1
2
83.98 ± 0.17
91.28 ± 0.10
62.49 ± 0.61
4
84.54 ± 0.73
91.61 ± 0.43
63.44 ± 0.74
8
84.28 ± 0.62
91.45 ± 0.37
62.95 ± 0.76
16
83.85 ± 0.28
91.20 ± 0.17
62.15 ± 1.04
64
84.18 ± 0.12
91.40 ± 0.07
63.16 ± 0.93
128
84.60 ± 0.31
91.64 ± 0.19
63.71 ± 0.93
Table S7 : DUTS MAPSE (GradCAM, TopK) region-count ablation using class activation maps from a frozen ImageNet-pretrained ResNet-50 teacher at E1–E4, ϕ=MLP , no residual connection, and a U-Net backbone. Results are mean ± sample standard deviation over three seeds.
K
mIoU
Dice/F1
Boundary F1
2
85.20 ± 0.17
91.99 ± 0.10
65.82 ± 1.10
4
86.54 ± 0.70
92.77 ± 0.40
67.75 ± 0.52
8
86.51 ± 0.58
92.75 ± 0.33
68.29 ± 0.68
16
82.82 ± 0.45
90.58 ± 0.27
60.85 ± 0.54
64
83.10 ± 0.50
90.75 ± 0.30
61.22 ± 0.55
128
83.22 ± 0.24
90.82 ± 0.14
61.38 ± 0.35
Table S8 : DUTS MAPSE (DINO) region-count ablation using final normalized DINOv3 patch tokens at E1–E4, ϕ=MLP , no residual connection, and a U-Net backbone. Results are mean ± sample standard deviation over three seeds.
Embedding source
K
mIoU
Dice/F1
Boundary F1
Patch tokens
2
85.74 ± 0.32
92.31 ± 0.19
66.54 ± 0.38
Patch tokens
4
86.77 ± 0.40
92.90 ± 0.23
68.18 ± 0.84
Key features
2
85.56 ± 0.54
92.20 ± 0.32
65.47 ± 0.13
Key features
4
85.60 ± 0.06
92.23 ± 0.04
66.34 ± 0.63
Table S9 : DUTS MAPSE (DINO) embedding-source and region-count ablation using E1–E4, ϕ=Conv1D , no residual connection, and a U-Net backbone. Results are mean ± sample standard deviation over three seeds.
CUB
DUTS
ECSSD
Model
Added params
mIoU
Boundary F1
mIoU
Boundary F1
mIoU
Boundary F1
Baseline
0.000
82.06 ± 1.08
54.12 ± 0.80
66.39 ± 0.02
35.04 ± 0.78
62.54 ± 0.70
29.97 ± 1.24
SE
+0.180
84.04 ± 1.07
56.18 ± 0.87
70.91 ± 0.70
39.30 ± 0.29
61.63 ± 0.91
28.32 ± 1.49
scSE
+0.193
84.18 ± 0.15
56.31 ± 0.73
71.34 ± 0.89
39.35 ± 1.80
61.68 ± 1.45
29.17 ± 2.31
ECA
+0.001
83.93 ± 0.42
55.35 ± 1.28
73.10 ± 0.77
41.34 ± 0.66
61.71 ± 2.07
29.17 ± 2.12
CA
+0.328
84.08 ± 0.63
56.25 ± 0.55
71.20 ± 0.77
37.47 ± 0.45
60.83 ± 1.54
27.72 ± 1.08
Table S10 : FCN quantitative comparison across the binary segmentation datasets. All values are percentages. Best results are in bold , second best underlined .
Model
Params
Added params
GFLOPs
Added GFLOPs
CPU latency
GPU latency
Baseline
1.533M
0
16.399
0.000
222.07
1.29
SE
1.536M
+2,764
16.400
+0.001
213.37
1.33
scSE
1.536M
+2,958
16.403
+0.004
219.19
1.47
ECA
1.533M
+8
16.400
+0.001
225.06
1.35
CA
1.538M
+5,024
16.400
+0.001
267.28
1.94
EMA
1.533M
+224
16.471
+0.072
236.37
2.00
Table S11 : FCN complexity and inference latency at input size 1×3×224×224 , with attention applied to the two skip connections.
Model
Params
Added params
GFLOPs
Added GFLOPs
CPU latency
GPU latency
Baseline
24.437M
0
62.733
0.000
732.67
5.10
SE
24.448M
+11,808
62.739
+0.006
738.71
5.26
scSE
24.449M
+12,324
62.751
+0.018
736.22
5.45
ECA
24.437M
+16
62.739
+0.006
738.05
5.23
CA
24.456M
+19,536
62.736
+0.003
1008.59
6.97
EMA
24.438M
+944
63.070
+0.337
889.31
6.49
Table S12 : Semantic U-Net complexity and inference latency at input size 1×3×512×512 with a 21-class output head.
Model
Params
Added params
GFLOPs
Added GFLOPs
CPU latency
GPU latency
Baseline
23.713M
0
8.207
0.000
142.65
5.20
SE
26.244M
+2,530,992
8.218
+0.011
147.99
9.07
scSE
26.259M
+2,546,112
8.229
+0.022
169.87
10.41
ECA
23.713M
+86
8.213
+0.006
156.72
8.02
CA
27.517M
+3,804,512
8.316
+0.109
160.00
12.59
EMA
23.911M
+198,368
10.151
+1.943
205.79
13.88
Table S13 : ResNet-50 mini-ImageNet [ 28 ] complexity and inference latency at input size 1×3×224×224 with a 100-class output head.
Table S14 : Interpretability visualizations for good segmentation predictions, selected from the top 5% of examples by mIoU for each dataset. Each dataset is shown with two representative examples. CUB/DUTS/ECSSD use U-Net, while VOC and ADE20K use Semantic U-Net. Columns show the input, ground truth, prediction, and argmax/PCA visualizations for encoder layers 1–4.
Table S15 : Interpretability visualizations for bad segmentation predictions, selected from the bottom 5% of examples by mIoU for each dataset. Each dataset is shown with two representative examples. CUB/DUTS/ECSSD use U-Net, while VOC and ADE20K use Semantic U-Net. Columns show the input, ground truth, prediction, and argmax/PCA visualizations for encoder layers 1–4.
Spatially variant dynamic convolution provides a principled approach of integrating spatial adaptivity into deep neural networks. However, mainstream designs in medical segmentation commonly generate dynamic kernels through average pooling, which implicitly collapses high-frequency spatial details into a coarse, spatially-compressed representation, leading to over-smoothed predictions that degrade the fidelity of fine-grained clinical structures. To address this limitation, we propose a novel Structure-Guided Dynamic Convolution (SGDC) mechanism, which leverages an explicitly supervised structure-extraction branch to guide the generation of dynamic kernels and gating signals for structure-aware feature modulation. Specifically, the high-fidelity boundary information from this auxiliary branch is fused with semantic features to enable spatially-precise feature modulation. By replacing context aggregation with pixel-wise structural guidance, the proposed design effectively prevents the information loss introduced by average pooling. Experimental results show that SGDC achieves state-of-the-art performance on ISIC 2016, PH2, ISIC 2018, and CoNIC datasets, delivering superior boundary fidelity by reducing the Hausdorff Distance (HD95) by 2.05, and providing consistent IoU gains of 0.99%-1.49% over pooling-based baselines. Moreover, the mechanism exhibits strong potential for extension to other fine-grained, structure-sensitive vision tasks, such as small-object detection, offering a principled solution for preserving structural integrity in medical image analysis. To facilitate reproducibility and encourage further research, the implementation code for both our SGE and SGDC modules has been is publicly released at https://github.com/solstice0621/SGDC.
Bo Shi, Wei-ping Zhu, M. N. S. Swamy
Department of Electrical and Computer Engineering, Concordia University
This paper presents Focus-on-Regions Network (FoR-Net), an efficient semantic segmentation framework that explicitly focuses on hard regions through a selector-driven Top-K mechanism. Instead of relying on heavy global modeling, FoR-Net selectively enhances structurally informative regions. Multi-scale reasoning branches are introduced to aggregate spatial context efficiently. Experiments on the Cityscapes dataset demonstrate that FoR-Net achieves competitive performance while maintaining a lightweight architecture.
Sheng-Wei Chan, Hsin-Jui Pan, Jen-Shiun Chiang
Department of Electrical and Computer Engineering, Tamkang University, No.151, Yingzhuan Rd., Tamsui Dist., New Taipei City, 251301, Taiwan
U-Net remains a practical baseline for image segmentation because of its simple encoder-decoder structure and skip connections. However, the bottleneck representation is still dominated by a limited set of receptive fields, while decoder features are propagated without explicitly emphasizing the most informative channels and spatial locations. This paper presents MSCA-UNet, a U-Net-based segmentation architecture that combines multi-scale contextual aggregation at the bottleneck with channel-spatial attention refinement in the decoder. The multi-scale module uses parallel atrous convolutions to capture contextual features at different receptive fields, while Convolutional Block Attention Modules (CBAMs) progressively recalibrate decoder features. Under identical experimental settings, the baseline U-Net achieves 96.9% mIoU on a held-out test set. Adding multi-scale context improves mIoU to 97.5%, while attention alone reaches 98.4%. Combining both mechanisms yields 99.1% mIoU, a 2.2 percentage-point improvement over the baseline. Parameter analysis further shows that the attention-only variant adds approximately 0.044M parameters, whereas the multi-scale module contributes most of the additional model capacity. The results support the view that multi-scale context enrichment and attention-based feature refinement provide complementary benefits within a U-Net framework.
Sheng-Wei Chan
Dept. of Electrical Engineering Tamkang University