Region Aware Squeeze-and-Excitation Networks for Image Segmentation

Jack Roberts; Praneeth Tummala; Kavi Sarna; Jeova Farias Sales Rocha Neto

cs.CVOct 10, 2026

Region Aware Squeeze-and-Excitation Networks for Image Segmentation

Authors: Jack Roberts, Praneeth Tummala, Kavi Sarna, Jeova Farias Sales Rocha Neto

Organizations: Department of Computer Science, Brown University, Providence, RI, USA · Department of Computer Science, Bowdoin College, Brunswick, ME, USA

Abstract

Squeeze-and-Excitation (SE) modules improve convolutional networks by modeling channel-wise dependencies, but global average pooling collapses spatial structure into a single per-channel descriptor, limiting their effectiveness for dense prediction tasks such as image segmentation. We generalize SE by replacing global pooling with spatially guided pooling, enabling region-aware channel recalibration through four modules with complementary cost profiles. BASE formalizes Tiled Squeeze-and-Excite using a regular grid; SPASE pools over SLIC superpixels for image-adaptive regions; MAPSE obtains regions from semantic maps produced by a frozen, general-purpose network; and PLEASE learns soft region assignments end-to-end through a lightweight assignment network. Each channel receives one gate per region, so attention at each pixel reflects its region membership, bridging channel and spatial attention. PLEASE also produces its soft assignments as a byproduct of the forward pass, providing structured spatial interpretability without post-hoc analysis. The modules offer complementary computational tradeoffs: some add essentially no parameters, while others add no measurable latency. Across binary segmentation on CUB, DUTS, and ECSSD, semantic segmentation on PASCAL VOC 2012 and ADE20K, and classification on mini-ImageNet, our modules outperform SE and related spatial-channel attention variants on most benchmarks, achieving the majority of best results.

Figures & tables

Panel of Figure 1 — Figure 1: Relevant Squeeze-and-Excite Modules. Learnable operations are marked with .

Panel of Table 1 — Table 1: Interpretability visualizations for representative good and bad segmentation predictions. The top two rows correspond to examples selected from the top 5% of examples by mIoU, while the bottom two rows correspond to examples selected from the bottom 5% by mIoU. CUB/DUTS/ECSSD use U-Net, while ADE20K uses Semantic U-Net.

Model	Params	Added params	GFLOPs	Added GFLOPs	CPU latency	GPU latency
Baseline	7.763M	0	18.551	0.000	237.69	2.10
SE	7.774M	+11,390	18.557	+0.006	245.33	2.83
scSE	7.775M	+11,874	18.569	+0.018	244.90	3.16
ECA	7.763M	+16	18.557	+0.006	241.73	2.58
CA	7.782M	+18,704	18.565	+0.015	243.62	3.42
EMA	7.764M	+910	18.738	+0.187	270.51	4.04

Table 2: U-Net complexity and latency at input size

3{\times}224{\times}224

. Latency is reported in ms/image for FP32 inference at batch size 1, using 50 warm-up and 200 measured iterations. GPU latency was measured on a single NVIDIA GeForce RTX 3090 (24 GB); CPU latency was measured single-threaded on one core of a dual-socket AMD EPYC 7532 node (

2{\times}32

physical cores; SMT disabled).

Figure S1 : Architectural details of the CNNs used for experimentation.

Panel of Figure 5 — Figure S2 : Example images and segmentation masks from CUB, DUTS, ECSSD, PASCAL VOC 2012, and ADE20K. (a) Input images. (b) Ground truth masks.

Attention Location	Blocks	mIoU	Boundary F1
Encoder	E1	82.07 $\pm$ 0.41	59.75 $\pm$ 0.91
Encoder	E1–E2	82.16 $\pm$ 0.10	60.24 $\pm$ 0.55
Encoder	E1–E3	82.63 $\pm$ 0.56	60.66 $\pm$ 1.31
Encoder	E1–E4	82.97 $\pm$ 0.70	61.21 $\pm$ 1.43
Skip	S1	81.97 $\pm$ 0.65	60.00 $\pm$ 0.95
Skip	S1–S2	82.09 $\pm$ 0.95	58.95 $\pm$ 1.28

Table S1 : DUTS attention location ablation for SPASE ,

K=8

\phi(\cdot)=\mathrm{MLP}

, U-Net.

$K$	mIoU	Boundary F1
2	82.88 $\pm$ 0.12	60.91 $\pm$ 0.55
8	82.95 $\pm$ 0.19	61.02 $\pm$ 0.93
16	82.61 $\pm$ 0.60	60.37 $\pm$ 1.53
64	81.76 $\pm$ 0.38	59.09 $\pm$ 0.89
128	81.87 $\pm$ 0.29	59.01 $\pm$ 0.42
500	81.50 $\pm$ 0.76	58.11 $\pm$ 1.56

Table S2 : DUTS

K

ablation results for SPASE , E1–E4,

\phi(\cdot)=\mathrm{MLP}

, U-Net.

	MLP [ 8 ]			Conv1D [ 32 ]
Module	mIoU / Boundary F1	Added params	Added params (%)	mIoU / Boundary F1	Added params	Added params (%)
Baseline (SE/ECA)	82.41 $\pm$ 0.47 / 60.35 $\pm$ 0.81	$11{,}390$	$+0.147$	84.07 $\pm$ 0.39 / 63.63 $\pm$ 0.17	$16$	$+0.000$
BASE	82.73 $\pm$ 0.77 / 60.27 $\pm$ 1.67	$11{,}390$	$+0.147$	83.28 $\pm$ 0.82 / 61.97 $\pm$ 1.30	$16$	$+0.000$
SPASE	82.95 $\pm$ 0.19 / 61.02 $\pm$ 0.93	$11{,}390$	$+0.147$	83.51 $\pm$ 0.24 / 61.81 $\pm$ 0.83	$16$	$+0.000$
PLEASE	83.08 $\pm$ 0.27 / 61.62 $\pm$ 0.70	$34{,}142$	$+0.440$	84.08 $\pm$ 0.63 / 62.78 $\pm$ 0.09	$22{,}768$	$+0.293$
MAPSE (GradCAM)	84.60 $\pm$ 0.31 / 63.71 $\pm$ 0.93	$11{,}390$	$+0.147$	85.45 $\pm$ 0.49 / 65.67 $\pm$ 0.36	$16$	$+0.000$

Table S3 : DUTS

\phi(\cdot)

ablation for U-Net encoder E1–E4 modules. Metric cells report mIoU / Boundary F1. For the baseline row, the MLP setting is vanilla SE and the 1D conv setting is ECA. The better of the two settings in each row is in bold .

Backbone	Module	Residual False	Residual True
U-Net	Baseline (ECA)	84.05 $\pm$ 0.35 / 63.59 $\pm$ 0.24	83.20 $\pm$ 0.91 / 62.00 $\pm$ 0.85
U-Net	BASE	83.26 $\pm$ 0.54 / 61.88 $\pm$ 1.12	82.53 $\pm$ 0.73 / 60.78 $\pm$ 1.38
U-Net	SPASE	83.59 $\pm$ 0.48 / 61.76 $\pm$ 0.56	83.36 $\pm$ 0.19 / 61.78 $\pm$ 0.35
U-Net	PLEASE	84.71 $\pm$ 0.58 / 64.76 $\pm$ 0.69	83.77 $\pm$ 0.64 / 62.67 $\pm$ 0.28
FCN	Baseline (ECA)	73.70 $\pm$ 0.83 / 41.62 $\pm$ 0.58	73.33 $\pm$ 0.45 / 40.39 $\pm$ 0.17
FCN	BASE	72.46 $\pm$ 0.64 / 40.35 $\pm$ 0.68	71.54 $\pm$ 0.43 / 38.84 $\pm$ 0.56

Table S4 : DUTS residual-connection ablation for the U-Net (E1–E4) and FCN (S1–S2). All modules use ECA excitation. Metric cells report mIoU / Boundary F1, and the better of the two settings in each row is in bold .

Location	Blocks	mIoU	Boundary F1
Encoder	E1	81.36 $\pm$ 0.67	58.71 $\pm$ 1.50
Encoder	E1–E2	81.83 $\pm$ 0.17	60.20 $\pm$ 0.15
Encoder	E1–E3	82.05 $\pm$ 0.21	60.53 $\pm$ 0.82
Encoder	E1–E4	83.40 $\pm$ 0.63	62.40 $\pm$ 0.44
Skip	S1	81.75 $\pm$ 0.27	58.25 $\pm$ 0.65
Skip	S1–S2	82.12 $\pm$ 0.31	58.72 $\pm$ 0.23

Table S5 : DUTS attention location ablation using PLEASE with

K=8

\phi=\mathrm{MLP}

, no residual connection, and a U-Net backbone.

$K$	mIoU	Boundary F1
2	82.73 $\pm$ 1.09	61.38 $\pm$ 1.17
8	82.87 $\pm$ 0.81	61.70 $\pm$ 0.76
16	83.24 $\pm$ 0.82	61.52 $\pm$ 0.95
64	83.48 $\pm$ 0.28	62.47 $\pm$ 0.33
128	83.03 $\pm$ 0.45	61.80 $\pm$ 0.81
500	82.71 $\pm$ 0.41	61.37 $\pm$ 0.77

Table S6 : DUTS region-count ablation using PLEASE at E1–E4 with

\phi=\mathrm{MLP}

, no residual connection, and a U-Net backbone.

$K$	mIoU	Dice/F1	Boundary F1
2	83.98 $\pm$ 0.17	91.28 $\pm$ 0.10	62.49 $\pm$ 0.61
4	84.54 $\pm$ 0.73	91.61 $\pm$ 0.43	63.44 $\pm$ 0.74
8	84.28 $\pm$ 0.62	91.45 $\pm$ 0.37	62.95 $\pm$ 0.76
16	83.85 $\pm$ 0.28	91.20 $\pm$ 0.17	62.15 $\pm$ 1.04
64	84.18 $\pm$ 0.12	91.40 $\pm$ 0.07	63.16 $\pm$ 0.93
128	84.60 $\pm$ 0.31	91.64 $\pm$ 0.19	63.71 $\pm$ 0.93

Table S7 : DUTS MAPSE (GradCAM, TopK) region-count ablation using class activation maps from a frozen ImageNet-pretrained ResNet-50 teacher at E1–E4,

\phi=\mathrm{MLP}

, no residual connection, and a U-Net backbone. Results are mean

\pm

sample standard deviation over three seeds.

$K$	mIoU	Dice/F1	Boundary F1
2	85.20 $\pm$ 0.17	91.99 $\pm$ 0.10	65.82 $\pm$ 1.10
4	86.54 $\pm$ 0.70	92.77 $\pm$ 0.40	67.75 $\pm$ 0.52
8	86.51 $\pm$ 0.58	92.75 $\pm$ 0.33	68.29 $\pm$ 0.68
16	82.82 $\pm$ 0.45	90.58 $\pm$ 0.27	60.85 $\pm$ 0.54
64	83.10 $\pm$ 0.50	90.75 $\pm$ 0.30	61.22 $\pm$ 0.55
128	83.22 $\pm$ 0.24	90.82 $\pm$ 0.14	61.38 $\pm$ 0.35

Table S8 : DUTS MAPSE (DINO) region-count ablation using final normalized DINOv3 patch tokens at E1–E4,

\phi=\mathrm{MLP}

, no residual connection, and a U-Net backbone. Results are mean

\pm

sample standard deviation over three seeds.

Embedding source	$K$	mIoU	Dice/F1	Boundary F1
Patch tokens	2	85.74 $\pm$ 0.32	92.31 $\pm$ 0.19	66.54 $\pm$ 0.38
Patch tokens	4	86.77 $\pm$ 0.40	92.90 $\pm$ 0.23	68.18 $\pm$ 0.84
Key features	2	85.56 $\pm$ 0.54	92.20 $\pm$ 0.32	65.47 $\pm$ 0.13
Key features	4	85.60 $\pm$ 0.06	92.23 $\pm$ 0.04	66.34 $\pm$ 0.63

Table S9 : DUTS MAPSE (DINO) embedding-source and region-count ablation using E1–E4,

\phi=\mathrm{Conv1D}

, no residual connection, and a U-Net backbone. Results are mean

\pm

sample standard deviation over three seeds.

		CUB		DUTS		ECSSD
Model	Added params	mIoU	Boundary F1	mIoU	Boundary F1	mIoU	Boundary F1
Baseline	$0.000$	82.06 $\pm$ 1.08	54.12 $\pm$ 0.80	66.39 $\pm$ 0.02	35.04 $\pm$ 0.78	62.54 $\pm$ 0.70	29.97 $\pm$ 1.24
SE	$+0.180$	84.04 $\pm$ 1.07	56.18 $\pm$ 0.87	70.91 $\pm$ 0.70	39.30 $\pm$ 0.29	61.63 $\pm$ 0.91	28.32 $\pm$ 1.49
scSE	$+0.193$	84.18 $\pm$ 0.15	56.31 $\pm$ 0.73	71.34 $\pm$ 0.89	39.35 $\pm$ 1.80	61.68 $\pm$ 1.45	29.17 $\pm$ 2.31
ECA	$+0.001$	83.93 $\pm$ 0.42	55.35 $\pm$ 1.28	73.10 $\pm$ 0.77	41.34 $\pm$ 0.66	61.71 $\pm$ 2.07	29.17 $\pm$ 2.12
CA	$+0.328$	84.08 $\pm$ 0.63	56.25 $\pm$ 0.55	71.20 $\pm$ 0.77	37.47 $\pm$ 0.45	60.83 $\pm$ 1.54	27.72 $\pm$ 1.08

Table S10 : FCN quantitative comparison across the binary segmentation datasets. All values are percentages. Best results are in bold , second best underlined .

Model	Params	Added params	GFLOPs	Added GFLOPs	CPU latency	GPU latency
Baseline	1.533M	0	16.399	0.000	222.07	1.29
SE	1.536M	+2,764	16.400	+0.001	213.37	1.33
scSE	1.536M	+2,958	16.403	+0.004	219.19	1.47
ECA	1.533M	+8	16.400	+0.001	225.06	1.35
CA	1.538M	+5,024	16.400	+0.001	267.28	1.94
EMA	1.533M	+224	16.471	+0.072	236.37	2.00

Table S11 : FCN complexity and inference latency at input size

1{\times}3{\times}224{\times}224

, with attention applied to the two skip connections.

Model	Params	Added params	GFLOPs	Added GFLOPs	CPU latency	GPU latency
Baseline	24.437M	0	62.733	0.000	732.67	5.10
SE	24.448M	+11,808	62.739	+0.006	738.71	5.26
scSE	24.449M	+12,324	62.751	+0.018	736.22	5.45
ECA	24.437M	+16	62.739	+0.006	738.05	5.23
CA	24.456M	+19,536	62.736	+0.003	1008.59	6.97
EMA	24.438M	+944	63.070	+0.337	889.31	6.49

Table S12 : Semantic U-Net complexity and inference latency at input size

1{\times}3{\times}512{\times}512

with a 21-class output head.

Model	Params	Added params	GFLOPs	Added GFLOPs	CPU latency	GPU latency
Baseline	23.713M	0	8.207	0.000	142.65	5.20
SE	26.244M	+2,530,992	8.218	+0.011	147.99	9.07
scSE	26.259M	+2,546,112	8.229	+0.022	169.87	10.41
ECA	23.713M	+86	8.213	+0.006	156.72	8.02
CA	27.517M	+3,804,512	8.316	+0.109	160.00	12.59
EMA	23.911M	+198,368	10.151	+1.943	205.79	13.88

Table S13 : ResNet-50 mini-ImageNet [ 28 ] complexity and inference latency at input size

1{\times}3{\times}224{\times}224

with a 100-class output head.

Panel of Table 19 — Table S14 : Interpretability visualizations for good segmentation predictions, selected from the top 5% of examples by mIoU for each dataset. Each dataset is shown with two representative examples. CUB/DUTS/ECSSD use U-Net, while VOC and ADE20K use Semantic U-Net. Columns show the input, ground truth, prediction, and argmax/PCA visualizations for encoder layers 1–4.

Panel of Table 20 — Table S15 : Interpretability visualizations for bad segmentation predictions, selected from the bottom 5% of examples by mIoU for each dataset. Each dataset is shown with two representative examples. CUB/DUTS/ECSSD use U-Net, while VOC and ADE20K use Semantic U-Net. Columns show the input, ground truth, prediction, and argmax/PCA visualizations for encoder layers 1–4.

Explore similar work

SGDC: Structurally-Guided Dynamic Convolution for Medical Image Segmentation
Feb 26, 2026Bo Shi, Wei-ping Zhu, M. N. S. SwamyMedical Image Segmentation Dynamic Convolution
Department of Electrical and Computer Engineering, Concordia University
FoR-Net: Focus-on-Regions Network for Semantic Segmentation
May 4, 2026Sheng-Wei Chan, Hsin-Jui Pan, Jen-Shiun ChiangSemantic Segmentation
Department of Electrical and Computer Engineering, Tamkang University, No.151, Yingzhuan Rd., Tamsui Dist., New Taipei City, 251301, Taiwan
MSCA-UNet: Multi-Scale Context and Attention U-Net for Image Segmentation
Sep 6, 2026Sheng-Wei ChanImage Segmentation U-Net
Dept. of Electrical Engineering Tamkang University

Feb 26, 2026J/K move · Enter open · S save

SGDC: Structurally-Guided Dynamic Convolution for Medical Image Segmentation

Bo Shi, Wei-ping Zhu, M. N. S. Swamy

Department of Electrical and Computer Engineering, Concordia University

Medical Image Segmentation Dynamic Convolution Boundary-Aware Segmentation Skin Lesion Segmentation Histopathology Image Segmentation

Open paper AskPDF arXiv

Spatially variant dynamic convolution provides a principled approach of integrating spatial adaptivity into deep neural networks. However, mainstream designs in medical segmentation commonly generate dynamic kernels through average pooling, which implicitly collapses high-frequency spatial details into a coarse, spatially-compressed representation, leading to over-smoothed predictions that degrade the fidelity of fine-grained clinical structures. To address this limitation, we propose a novel Structure-Guided Dynamic Convolution (SGDC) mechanism, which leverages an explicitly supervised structure-extraction branch to guide the generation of dynamic kernels and gating signals for structure-aware feature modulation. Specifically, the high-fidelity boundary information from this auxiliary branch is fused with semantic features to enable spatially-precise feature modulation. By replacing context aggregation with pixel-wise structural guidance, the proposed design effectively prevents the information loss introduced by average pooling. Experimental results show that SGDC achieves state-of-the-art performance on ISIC 2016, PH2, ISIC 2018, and CoNIC datasets, delivering superior boundary fidelity by reducing the Hausdorff Distance (HD95) by 2.05, and providing consistent IoU gains of 0.99%-1.49% over pooling-based baselines. Moreover, the mechanism exhibits strong potential for extension to other fine-grained, structure-sensitive vision tasks, such as small-object detection, offering a principled solution for preserving structural integrity in medical image analysis. To facilitate reproducibility and encourage further research, the implementation code for both our SGE and SGDC modules has been is publicly released at https://github.com/solstice0621/SGDC.

Abstract

Explore similar work

SGDC: Structurally-Guided Dynamic Convolution for Medical Image Segmentation

FoR-Net: Focus-on-Regions Network for Semantic Segmentation

MSCA-UNet: Multi-Scale Context and Attention U-Net for Image Segmentation