Organizations: Center for Excellence in Brain Science and Intelligence Technology, State Key Laboratory of Brain Cognition and Brain-inspired Intelligence Technology, Institute of Neuroscience, Chinese Academy of Sciences · China Electric Power Research Institute Co., Ltd. (CEPRI)
Spiking Transformers model token interactions primarily through spiking self-attention (SSA). However, binary query and key representations map continuous similarities to sparse and discrete relation responses, which may suppress weak relations and limit the propagation of local spatial context. To address this limitation, we introduce Spiking Local Interaction (SLI) and Adaptive Complementary Fusion (ACF). SLI establishes an attention-independent pathway for direct information exchange among neighboring spiking tokens using lightweight depthwise--pointwise transformations. ACF integrates SSA and SLI through layer-specific, channel-wise coefficients that adaptively balance their contributions at different network depths. The proposed design preserves the original attention formulation and can be incorporated into different Spiking Transformer architectures with modest parameter overhead. Experiments on ImageNet-1K, CIFAR-10, CIFAR-100, CIFAR10-DVS, and ADE20K show consistent improvements across image classification, event-based recognition, and semantic segmentation. In particular, QKFormer with SLI and ACF achieves 84.37% Top-1 accuracy on ImageNet-1K and 37.5% mIoU on ADE20K, where the segmentation model is trained without ImageNet pretraining. Ablation studies and qualitative analyses further indicate that SSA and SLI capture complementary interaction patterns and that learnable fusion consistently outperforms fixed weighting.
Figures & tables
Figure 1: Architecture of the proposed spiking Transformer with Adaptive Complementary Fusion (ACF). Each block combines spiking self-attention (SSA) with a parallel Spiking Local Interaction (SLI) pathway. SSA constructs input-dependent token relations, while SLI aggregates local spatial context independently of QK co-activation. Their responses are adaptively calibrated by learnable channel-wise coefficients, and the fused representation is propagated to subsequent blocks.
Figure 2: Sparse and discrete relation responses in spiking attention. (a) Comparison of vanilla self-attention and spiking self-attention. (b) Attention-response non-zero ratios across eight blocks of Spikingformer and QKFormer.
Methods
Type
Architecture
Params (M)
Time Steps
Top-1 Acc. (%)
ViT ( Dosovitskiy et al. 2020 )
ANN
ViT-B/16
86.59
1
77.90
DeiT ( Touvron et al. 2021 )
ANN
DeiT-B
86.59
1
81.80
ANN
DeiT-B †
86.59
1
83.10
Spikformer ( Zhou et al. 2022 )
SNN
Spikformer-8-512
29.68
4
73.38
SNN
Spikformer-8-768
66.34
4
74.81
SDT ( Yao et al. 2023 )
SNN
SDT-8-512
29.68
4
74.57
Table 1: ImageNet-1K classification results. ∗ denotes our reproduction under the original experimental settings, and † denotes an input resolution of 288×288 ; all other models use 224×224 . HST denotes Hierarchical Spiking Transformer.
Method
CIFAR10
CIFAR100
CIFAR10-DVS
Params (M)
T
Acc. (%)
Params (M)
T
Acc. (%)
Params (M)
T
Acc. (%)
α -SSA ( Xiao et al. 2025 )
9.32
4
95.97
9.32
4
80.22
2.57
16
81.64
SacSSA ( Wang et al. 2025 )
5.57
4
96.1
5.57
4
80.1
1.52
16
82.3
A2OS2 ( Guo et al. 2025 )
10.23
4
96.42
10.23
4
79.90
-
-
-
STAtten+ ( Lee et al. 2025 )
-
4
95.35
-
4
80.20
-
16
83.9
SEMM ( Lee et al. 2025 )
-
4
96.16
-
4
80.24
-
16
82.1
Table 2: Comparison on CIFAR-10, CIFAR-100, and CIFAR10-DVS. The Spikingformer and QKFormer baselines in the lower block are reproduced under a unified benchmark for direct comparison with their ACF-enhanced counterparts.
Figure 3: Qualitative ADE20K results of Spikingformer and QKFormer with and without SLI and ACF.
Figure 4: Component ablation on CIFAR-100 using Spikingformer-4-384 and QKFormer-4-384.
Backbone
Mean Top-1 (%) ↑
Std. ↓
Fixed
Learnable
Fixed
Learnable
QKFormer
81.64
81.74
0.33
0.08
Spikingformer
80.62
80.77
0.17
0.06
Table 4: Mean and standard deviation of Top-1 accuracy across five fusion-coefficient settings on CIFAR-100.
Figure 5: Channel-wise distributions of the learned γssa and γsli across blocks, backbones, and datasets.
Figure 6: Grad-CAM++ visualizations of SSA, SLI, and their ACF-fused responses on ImageNet-1K.
Stage
Spatial resolution
Layer specification
8-512
10-512
8-768
Spikingformer (columnar)
Spiking Tokenizer
H×W
Downsampling 1
Conv 3×3
64
—
96
2H×2W
Downsampling 2
MP + Conv 3×3
128
—
192
4H×4W
Downsampling 3
MP + Conv 3×3
256
—
384
8H×8W
Downsampling 4
MP + Conv 3×3
512
—
768
16H×16W
Downsampling 5
MP + Conv 3×3
512
—
768
Table 5: Architecture specifications of the ImageNet-scale Spikingformer and QKFormer backbones with SLI and ACF. In each Transformer block, SLI operates in parallel with SSA or TSSA, and the two pathways are fused by ACF before the spiking MLP. MP denotes 3×3 max-pooling with stride 2 , and each QKFormer downsampling module includes a 1×1 stride- 2 convolutional shortcut.
Stage
Spatial resolution
Layer specification
Spikingformer
QKFormer
1
2H×2W
Downsampling
Conv 7×7 , stride 2 , dim 32
Blocks
Conv-based SNN block
×1→P1
×1→P1
2
4H×4W
Downsampling
Conv 3×3 , stride 2 , dim 64
Blocks
Conv-based SNN block
×1→P2
×1→P2
3
8H×8W
Downsampling
Conv 3×3 , stride 2 , dim 128
Blocks
Conv-based SNN block
×2→P3
×2→P3
Table 6: Architecture specifications of the ADE20K segmentation backbones with SLI and ACF. In each Transformer SNN block, SLI operates in parallel with SSA or TSSA, and their outputs are fused by ACF before the spiking MLP. P1 – P4 denote the multi-scale features passed to the FPN neck. The downsampling modules follow SDT-V2 and contain convolutional shortcuts. PW and DW denote pointwise and depthwise convolutions, respectively.
Configuration
CIFAR-10/100
CIFAR10-DVS
ImageNet-1K
Input size
322
1282
2242
Time steps T
4
16
4
Epochs
400
96
300 / 200
Warmup epochs
20
10
20 / 5
Batch size
64
16
24 per GPU
Learning rate
10−3
10−3
5×10−4
Table 7: Training configurations for the classification experiments. Values separated by “/” correspond to Spikingformer and QKFormer, respectively. The reported epoch counts exclude the subsequent 10 -epoch cooldown phase.
Spiking Transformers combine attention-based architectures with event-driven spiking neurons to process information using sparse, temporally coded computation.
Spiking Transformers provide a promising paradigm for efficient visual processing with spike-driven computation, yet their Softmax-free Spiking Self-Attention (SSA) struggles to establish spatially localized token interactions. Although existing locality-enhanced SSA methods improve accuracy, it remains unclear whether they consistently induce spatial locality across layers and different Spiking Transformer architectures. Through Mean Attention Distance (MAD) analysis, we reveal that computational locality does not necessarily translate into spatial locality and show that uniformly applying the same locality enhancement overlooks architecture-dependent deployment requirements. Motivated by these observations, we propose Spatially Contiguous Local Attention with Boundary Continuity Pathway (SCLA-BCP). SCLA computes attention within non-overlapping regions of spatially adjacent tokens, while BCP facilitates cross-boundary information exchange through a lightweight convolutional pathway. Furthermore, we develop a hierarchical locality deployment strategy to effectively apply SCLA-BCP across the two major Spiking Transformer architectures. Extensive experiments on seven static and neuromorphic datasets covering classification, detection, and segmentation demonstrate consistent improvements with limited parameter and energy overhead. Notably, our approach improves mAP@50 by up to 9.50% on COCO 2017 and mIoU by up to 3.42% on ADE20K. Visualizations, MAD analysis, and ablation studies further validate its effectiveness.
Zeqi Zheng, Zizheng Zhu, Yuping Yan +3
1Zhejiang University · 2Westlake University · 3Peking University
Transformer-based Spiking Neural Networks (SNNs) integrate SNNs with global self-attention and have demonstrated impressive performance. However, existing Transformer-based SNNs suffer from two fundamental limitations. First, they typically employ max pooling layers to reduce the size of feature maps, but the max pooling captures only the strongest response and fails to comprehensively preserve representative regional features. Second, the global self-attention involves all global feature interactions, resulting in computational redundancy and quadratic computational complexity, thus conflicting with the sparse and energy-efficient characteristics of SNNs. To address these challenges, we develop Local Structure-Aware Spiking Transformer (LSFormer), a novel Transformer-based Spiking Neural Network that incorporates Spiking Response Pooling (SPooling) and Local Structure-Aware Spiking Self-Attention (LS-SSA). For the first time, our LSFormer leverages a local dilated window mechanism to capture both local details and long-range dependencies. Experimental results demonstrate that our LSFormer achieves state-of-the-art performance compared to existing advanced Transformer-based SNNs. Notably, on the more challenging static dataset Tiny-ImageNet and neuromorphic dataset N-CALTECH101, LSFormer substantially outperforms state-of-the-art baselines by 4.3% and 8.6% in top-1 classification accuracy, respectively. These results highlight the potential of LSFormer to advance energy-efficient spiking models toward practical deployment in large-scale vision applications.
Lingdong Li, Hangming Zhang, Qiang Yu
School of Future Technology, Tianjin Key Laboratory of Cognitive Computing and Application, Tianjin University, Tianjin 300354, China · School of Artificial Intelligence, Tianjin Key Laboratory of Cognitive Computing and Application, College of Intelligence and Computing, Tianjin University, Tianjin 300354, China
Spiking Neural Networks (SNNs) offer notable advantages in biological plausibility and energy efficiency, making them promising candidates for building low-power Transformers. However, existing Spiking Transformers largely adhere to a passive reactive paradigm, which struggles to focus on task-relevant information and incurs substantial computational overhead when processing redundant visual data. To overcome this fundamental yet underexplored limitation, we propose SAFformer, a novel Spiking Transformer architecture based on an active predictive filtering paradigm. Inspired by the brain's predictive coding mechanism, SAFformer actively suppresses predictable signals and focuses on salient visual features. Extensive experiments show that SAFformer establishes new state-of-the-art performance on CIFAR-10/100 and CIFAR10-DVS. Remarkably, on ImageNet-1K, it achieves 80.44% Top-1 accuracy with only 26.58M parameters and an energy consumption of 5.88 mJ, demonstrating an exceptional balance between accuracy and efficiency.
Zequan Xie, Weiming Zeng, Yunhua Chen +3
School of Computer Science and Technology, Guangdong University of Technology, Guangzhou, China · Faculty of Science, Hong Kong Baptist University, China · School of Electronic Information, Wuhan University, China