Gated Differential Linear Attention: A Linear-Time Decoder for High-Fidelity Medical Segmentation

Hongbo Zheng; Afshin Bozorgpour; Dorit Merhof; Minjia Zhang

cs.CVMar 3, 2026

Gated Differential Linear Attention: A Linear-Time Decoder for High-Fidelity Medical Segmentation

Authors: Hongbo Zheng, Afshin Bozorgpour, Dorit Merhof, Minjia Zhang

Organizations: University of Illinois Urbana-Champaign · University of Regensburg

Abstract

Medical image segmentation requires models that preserve fine anatomical boundaries while remaining practical for clinical deployment. Transformers capture long-range dependencies but incur quadratic attention cost, whereas CNNs are efficient but less effective at global reasoning. Linear attention offers $\mathcal{O}(N)$ scaling, but often produces diffuse feature aggregation that weakens boundary-sensitive prediction. We introduce a gated differential linear-attention mixer for medical image segmentation. Its global path, Gated Differential Linear Attention (GDLA), performs differential subtraction between two kernelized attention branches over complementary query/key subspaces to suppress redundant responses, and employs a data-dependent gate for token refinement. A parallel local token-mixing branch with depthwise convolution strengthens neighborhood interactions for better refinement, and the two branches are fused while preserving $\mathcal{O}(N)$ complexity. When instantiated in a pretrained Pyramid Vision Transformer (PVT)-based encoder--decoder model, \name achieves state-of-the-art results on the evaluated 2D medical segmentation benchmarks spanning CT, MRI, ultrasound, and dermoscopy, with a favorable accuracy--efficiency trade-off over closely related baselines. The code is publicly available at https://github.com/xmindflow/gdla.

Figures & tables

Panel of Figure 1 — Figure 1: Comparison of model performance on Synapse dataset and compute–accuracy trade-off. Our approach achieves the highest Dice score with lower # parameters and lower # FLOPs.

Figure 2: PVT-GDLA overview. A pretrained Pyramid Vision Transformer (PVT) encoder extracts multi-scale features that are progressively decoded by GDLA blocks. Each GDLA block contains a GDLA mixer and a feed-forward network (FFN). Within the mixer, a gated differential linear-attention path captures globally selective context, while a parallel local token-mixing branch reinforces neighborhood interactions. The fused features are projected and upsampled with skip connections to recover spatial resolution.

Figure 3: Gated Differential Linear Attention (GDLA). Two complementary query/key subspaces,

(\boldsymbol{Q}_{1},\boldsymbol{K}_{1})

and

(\boldsymbol{Q}_{2},\boldsymbol{K}_{2})

, are derived from

\boldsymbol{X}

. Each branch applies kernelized linear attention with

\phi(\cdot)=\mathrm{ELU}(\cdot)+1

, and the resulting outputs are combined by learnable channel-wise subtraction,

\boldsymbol{A}_{1}-\boldsymbol{\lambda}\odot\boldsymbol{A}_{2}

, followed by

\mathrm{RMSNorm}

and a data-dependent SiLU gate.

Methods	Params	FLOPs	Spl.	RKid.	LKid.	Gal.	Liv.	Sto.	Aor.	Pan.	Average
Methods	Params	FLOPs	Spl.	RKid.	LKid.	Gal.	Liv.	Sto.	Aor.	Pan.	DSC $\uparrow$	HD95 $\downarrow$
R50+U-Net [ 8 ]	30.42 M	-	85.87	78.19	80.60	63.66	93.74	74.16	87.74	56.90	74.68	36.87
TransUNet [ 8 ]	96.07 M	88.91 G	85.08	77.02	81.87	63.16	94.08	75.62	87.23	55.86	77.49	31.69
Swin-UNet [ 7 ]	27.17 M	6.16 G	90.66	79.61	83.28	66.53	94.29	76.60	85.47	56.58	79.13	21.55
HiFormer-B [ 15 ]	25.51 M	8.05 G	90.99	79.77	85.23	65.23	94.61	81.08	86.21	59.52	80.39	14.70
VM-UNet [ 31 ]	44.27 M	6.52 G	89.51	82.76	86.16	69.41	94.17	81.40	86.40	58.80	81.08	19.21

Table 1: Evaluation results on the Synapse dataset ( blue indicates the best and red the second best results). Results are averaged over three runs.

Figure 4: Qualitative results of the proposed method versus other approaches on the Synapse dataset.

Methods Avg. Dice RV MYO LV R50+UNet [ 8 ] 87.55 87.10 80.63 94.92 R50+AttnUNet [ 8 ] 86.75 87.58 79.20 93.47 TransUNet [ 8 ] 89.71 88.86 84.53 95.73 Swin-UNet [ 7 ] 90.00 88.55 85.62 95.83 R50+ViT+CUP [ 8 ] 87.57 86.07 81.88 94.75 MT-UNet [ 38 ] 90.43 86.64 89.04 95.62 MISSFormer [ 17 ] 90.86 89.55 88.04 94.99 PVT-EMCAD-B2 [ 29 ] 92.12 90.65 89.68 96.02 CENet [ 5 ] 92.18 90.90 89.63 95.99 PVT-DiffAttn [ 41 ] 92.20 90.64 89.90 96.06 PVT-SelfAttn [ 36 ] 92.10 90.47 89.76 96.06 PVT-LinearAttn [ 18 ] 91.77 89.80 89.52 96.00 PVT-GDLA 92.53 91.30 90.21 96.07

Methods Avg. Dice U-Net [ 30 ] 74.04 PraNet [ 11 ] 75.41 CaraNet [ 22 ] 77.34 AttnUNet [ 25 ] 74.48 DeepLabv3+ [ 2 ] 76.81 TransUNet [ 8 ] 78.30 TransFuse [ 42 ] 79.36 Swin-UNet [ 7 ] 77.38 PVT-EMCAD-B2 [ 29 ] 80.25 PVT-DiffAttn [ 41 ] 79.84 PVT-SelfAttn [ 36 ] 79.42 PVT-LinearAttn [ 18 ] 79.14 PVT-GDLA 80.54

Methods PH 2 HAM10000 Dice Acc. Dice Acc. U-Net [ 30 ] 89.36 92.33 91.67 95.67 TransUNet [ 8 ] 88.40 92.00 93.53 96.49 Swin-Unet [ 7 ] 94.49 96.78 92.63 96.16 DeepLabv3+ [ 9 ] 92.02 95.03 92.51 96.07 Att-UNet [ 25 ] 90.03 92.76 92.68 96.10 UCTransNet [ 37 ] 90.93 94.08 93.46 96.84 MissFormer [ 17 ] 85.50 90.50 92.11 96.21 CENet [ 5 ] 95.04 97.19 94.71 97.04 PVT-DiffAttn [ 41 ] 95.10 97.13 94.73 97.10 PVT-SelfAttn [ 36 ] 95.08 97.18 94.78 97.06 PVT-LinearAttn [ 18 ] 94.78 96.68 94.68 96.87 PVT-GDLA 95.59 97.20 95.01 97.38

Table 2: Evaluation results on the ACDC (left), BUSI (middle), and skin–lesion benchmarks PH 2 and HAM10000 (right). All metrics are averaged over three runs.

Panel of Figure 5 — Figure 5: Visual comparison of the proposed method versus others on the Skin datasets. HAM10000 (top) and PH 2 (bottom). The blue and the green lines represent prediction and ground truth respectively.

Panel of Figure 6 — Figure 6: Linear Attention (LA) vs. Gated Differential Linear Attention (GDLA) on a Synapse CT slice. Rows correspond to decoder stages (coarse $\rightarrow$ fine); columns show token-norm maps $\|\text{input}\|$ and update magnitudes $\|\Delta\text{attn}\|$ for LA (left) and GDLA (right), with a difference panel visualizing normalized $\|\Delta\text{attn}\|$ (GDLA $-$ LA).

Panel of Figure 7 — Figure 7: Channel saliency across decoder stages. Normalized RMS feature–energy per channel for Decoder 1–3 (coarse to fine) comparing linear attention (LA) and GDLA. The left column includes the input data and ground truth segmentation.

Components			Params	FLOPs	Dice
DiffAttn	Mixer	Gate	Params	FLOPs	Dice
✗	✗	✗	28.25 M	5.38 G	94.78
✓	✗	✗	29.89 M	6.05 G	95.08
✓	✗	✓	30.01 M	6.10 G	95.20
✓	✓	✗	31.13 M	6.52 G	95.46
✓	✓	✓	32.13 M	6.85 G	95.59

Table 3: Effect of different components of PVT-GDLA on the PH 2 datasets. All results are averaged over three runs. The best result is shown in bold.

Kernel Size		Avg. Dice
$\mathrm{DWC}$	$\mathrm{ConvTranspose}$	Synapse	ACDC	HAM1000
$3{\times}3$	$3{\times}3$	85.32	92.53	95.01
$3{\times}3$	$5{\times}5$	84.94	92.23	94.73
$5{\times 5}$	$3{\times}3$	84.73	92.21	94.68
$5{\times 5}$	$5{\times}5$	84.63	92.17	94.62

Table 4: Effect of

\mathrm{ConvTranspose}

kernel size in Upsample and depth-wise convolution kernel size in local token mixing on Synapse multi-organ, ACDC , HAM10000 datasets. All results are averaged over three runs. Best results are highlighted in bold.

Gate	Avg. Dice
Gate	Synapse	ACDC	HAM10000
Sigmoid	84.94	92.24	94.70
SiLU	85.32	92.53	95.01

Table 5: Effect of gating functions in GDLA. Comparison of sigmoid and SiLU gating, evaluated by average Dice (%, mean

\pm

std) on Synapse , ACDC , and HAM10000 .

FFN	Avg. Dice
FFN	Synapse	ACDC	HAM10000
2-layer MLP	84.73	92.33	94.84
SwiGLU	84.78	92.25	94.90
Mix-FFN	85.32	92.53	95.01

Table 6: Effect of FFN design within GDLA. Comparison of three feed-forward variants, 2-layer MLP (

4\times

expansion), SwiGLU, and Mix-FFN, evaluated by average Dice (%) on Synapse , ACDC , and HAM10000 .

Figure 8: GDLA vs. baseline attentions on $\|\Delta\text{attn}\|$ . Rows show decoder stages (S1–S3, coarse $\rightarrow$ fine). Columns display per-method update magnitudes for GDLA, Differential Attention (DA), Self Attention (SA), and Linear Attention (LA), alongside normalized difference maps ( $\|\Delta\text{attn}\|_{\text{GDLA}}-\|\Delta\text{attn}\|_{\text{baseline}}$ ).

Medical Image Segmentation

Medical image segmentation delineates organs, lesions, and other structures in scans to support measurement, diagnosis, treatment planning, and research.

See all 768 papers

MedCAGD: Context-Aware Gated Decoder for Efficient Medical Image Segmentation
Jul 1, 2026Saad Wazir, Patrick Dominique Vibild, Dinh Phu Tran +2Medical Image Segmentation
School of Computing, Korea Advanced Institute of Science and Technology (KAIST), Daejeon, Republic of Korea · Department of Energy, Aalborg University, Aalborg, Denmark
DCAU-Net: Differential Cross Attention and Channel-Spatial Feature Fusion for Medical Image Segmentation
Mar 10, 2026Yanxin Li, Hui Wan, Libin LanMedical Image Segmentation Cross-Attention
College of Computer Science and Engineering, Chongqing University of Technology, Chongqing, China
RF-HiT: Rectified Flow Hierarchical Transformer for General Medical Image Segmentation
Apr 21, 2026Ahmed Marouane Djouamaa, Abir Belaala, Abdellah Zakaria Sellam +3Image Segmentation Vision Transformer
Dept. of Computer Science, Mohamed Khider University, Biskra, Algeria · Inst. of Applied Sciences and Intelligent Systems, National Research Council of Italy, 73100 Lecce, Italy · Dept. of Computer Science and AI, University of the Basque Country, UPV/EHU, San Sebastian, Spain +1

Jul 1, 2026J/K move · Enter open · S save

MedCAGD: Context-Aware Gated Decoder for Efficient Medical Image Segmentation

Saad Wazir, Patrick Dominique Vibild, Dinh Phu Tran, Seongah Kim+1

School of Computing, Korea Advanced Institute of Science and Technology (KAIST), Daejeon, Republic of Korea · Department of Energy, Aalborg University, Aalborg, Denmark

Medical Image Segmentation

Open paper AskPDF arXiv

Medical image segmentation relies on the ability of encoder-decoder architectures to translate rich feature representations into accurate pixel-level predictions under challenging conditions such as low contrast, structural ambiguity, and scale variability. While recent advances in large-scale pretraining and transformer-based encoders have substantially improved feature extraction, segmentation accuracy remains constrained by decoder design, particularly in terms of cross-scale alignment, contextual integration, and boundary preservation. In this work, we revisit medical image segmentation from a decoder-centric perspective and propose a context-aware gated decoder that systematically regulates feature fusion and contextual aggregation throughout the decoding process. The proposed decoder integrates lightweight multi-scale channel recalibration, gated skip fusion with spatial competition and a global context aggregation mechanism that injects encoder-wide information into intermediate decoding stages. This design enables effective translation of strong pretrained encoder representations into spatially consistent predictions. Extensive experiments across 11 medical image segmentation benchmarks validate the effectiveness and demonstrate that the proposed approach consistently outperforms strong baselines while remaining computationally practical. Code: https://github.com/saadwazir/MedCAGD

Abstract

Medical Image Segmentation

Explore similar work

MedCAGD: Context-Aware Gated Decoder for Efficient Medical Image Segmentation

DCAU-Net: Differential Cross Attention and Channel-Spatial Feature Fusion for Medical Image Segmentation

RF-HiT: Rectified Flow Hierarchical Transformer for General Medical Image Segmentation