Ge$^\text{2}$mS-T: Multi-Dimensional Grouping for Ultra-High Energy Efficiency in Spiking Transformer

Zecheng Hao; Shenghao Xie; Kang Chen; Wenxuan Liu; Zhaofei Yu; Tiejun Huang

cs.NEApr 10, 2026

Ge $^\text{2}$ mS-T: Multi-Dimensional Grouping for Ultra-High Energy Efficiency in Spiking Transformer

Authors: Zecheng Hao, Shenghao Xie, Kang Chen, Wenxuan Liu, Zhaofei Yu, Tiejun Huang

Organizations: Peking University, School of Computer Science, State Key Laboratory for Multimedia Information Processing, Beijing, China · Peking University, School of Computer Science, Beijing Key Laboratory of Brain-inspired Spiking Large Models, Beijing, China

Abstract

Spiking Neural Networks (SNNs) offer superior energy efficiency over Artificial Neural Networks (ANNs). However, they encounter significant deficiencies in training and inference metrics when applied to Spiking Vision Transformers (S-ViTs). Existing paradigms including ANN-SNN Conversion and Spatial-Temporal Backpropagation (STBP) suffer from inherent limitations, precluding concurrent optimization of memory, accuracy and energy consumption. To address these issues, we propose Ge $^\text{2}$ mS-T, a novel architecture implementing grouped computation across temporal, spatial and network structure dimensions. Specifically, we introduce the Grouped-Exponential-Coding-based IF (ExpG-IF) model, enabling lossless conversion with constant training overhead and precise regulation for spike patterns. Additionally, we develop Group-wise Spiking Self-Attention (GW-SSA) to reduce computational complexity via multi-scale token grouping and multiplication-free operations within a hybrid attention-convolution framework. Experiments confirm that our method can achieve superior performance with ultra-high energy efficiency on challenging benchmarks. To our best knowledge, this is the first work to systematically establish multi-dimensional grouped computation for resolving the triad of memory overhead, learning capability and energy budget in S-ViTs. Code is available at https://github.com/hzc1208/Ge2mST.

Figures & tables

Figure 1. Comparison of network parameter count, inference accuracy and energy consumption between prior state-of-the-art (SoTA) S-ViT & S-CNN approaches and this work on the ImageNet-1k dataset. Smaller circles denote fewer network parameters and upper-left positions indicate higher energy efficiency.

Learning Paradigm	Model	Train. Load	Train. Mem.	Inf. Latency	Inf. Acc.	SSA SOPs	SFFN SOPs
ANN-SNN Conversion	IF	$\mathcal{O}(1)$	$\mathcal{O}(1)$	High	High	$\geq\!\mathcal{O}(T^{2}N^{2}C)$	$\geq\!\mathcal{O}(TNC^{2})$
STBP Training	IF/LIF	$\mathcal{O}(T)$	$\mathcal{O}(T)$	Low	Low	$\mathcal{O}(TN^{2}C)$	$\mathcal{O}(TNC^{2})$
This Work	ExpG-IF	$\bm{O(1)}$	$\bm{O(1)}$	Low	High	$\bm{O\left(\frac{TN^{2}C}{\|G_{T}\|\|G_{S}\|}\right)}$	$\bm{O\left(\frac{TNC^{2}}{\|G_{T}\|}\right)}$

Table 1. Performance comparison of SNN learning paradigms on S-ViT with respect to training and inference metrics.

Figure 2. Overall architecture of Ge²mS-T, comprising a series of stage-oriented blocks constructed from SSA, SConv and SFFN. Compared with Conversion-based and STBP-based SSAs, the energy efficiency of GW-SSA is substantially enhanced across multiple dimensions.

Stage	Feature Size	Block Structure	Ge²mS-T Small	Ge²mS-T Base	Ge²mS-T Large
Stem	224 $\times$ 224	Convolution	Conv 7 $\times$ 7, stride 2
Stage 1	112 $\times$ 112	ExpG-IF	$\begin{bmatrix}C=24,R_{1},R_{2}=2,4\end{bmatrix}$	$\begin{bmatrix}C=32,R_{1},R_{2}=2,4\end{bmatrix}$	$\begin{bmatrix}C=40,R_{1},R_{2}=2,4\end{bmatrix}$
Stage 1	56 $\times$ 56	SConv, SFFN	$\begin{bmatrix}C=48,R_{1},R_{2}=2,4\end{bmatrix}$	$\begin{bmatrix}C=64,R_{1},R_{2}=2,4\end{bmatrix}$	$\begin{bmatrix}C=80,R_{1},R_{2}=2,4\end{bmatrix}$
Stage 2	28 $\times$ 28	ExpG-IF	$\begin{bmatrix}C=96,r_{\text{g,w}}=0.5\\ \|G_{S}\|=4^{2},\|G_{T}\|=2\\ h=4,R=4\end{bmatrix}\times 2$	$\begin{bmatrix}C=128,r_{\text{g,w}}=0.5\\ \|G_{S}\|=4^{2},\|G_{T}\|=2\\ h=4,R=4\end{bmatrix}\times 2$	$\begin{bmatrix}C=160,r_{\text{g,w}}=0.5\\ \|G_{S}\|=4^{2},\|G_{T}\|=2\\ h=4,R=4\end{bmatrix}\times 2$
		GW-SSA
		Conv-SFFN

Table 2. Detailed configurations of Ge²mS-T on the ImageNet-1k dataset. Here

R_{1}

denotes the expansion ratio of SConv within

\text{Conv}_{\text{B}}

R_{2},R

denote the expansion ratio of Conv-SFFN (SFFN),

r_{\text{g,w}},h

represent the split ratio of global & window attention dimensions in GW-SSA and the number of attention heads.

Method	Type	Architecture	Param.(M)	T	SOPs(G)	Energy(mJ)	Acc.(%)
Spiking ResNet ( Hu et al., 2021 )	ANN-SNN Conversion	ResNet-34	21.79	350	65.28	59.30	71.61
Spiking ResNet ( Hu et al., 2021 )	ANN-SNN Conversion	ResNet-50	25.56	350	78.29	70.93	72.75
STBP-tdBN ( Zheng et al., 2021 )	STBP Training	Spiking ResNet-34	21.79	6	6.50	6.39	63.72
SEW ResNet ( Fang et al., 2021a )	STBP Training	SEW ResNet-34	21.79	4	3.88	4.04	67.04
		SEW ResNet-50	25.56	4	4.83	4.89	67.78
		SEW ResNet-101	44.55	4	9.30	8.91	68.76

Table 3. Performance evaluation of relevant approaches based on different learning paradigms and network architectures on the ImageNet-1k dataset. Here SOPs denotes the average number of synaptic operations per input sample, and Energy is computed following the calculation standards outlined in ( Zhou et al., 2023 ) .

Method	Type	CIFAR-10		CIFAR-100		CIFAR10-DVS
Method	Type	Time-step	Acc.(%)	Time-step	Acc.(%)	Time-step	Acc.(%)
STBP-tdBN ( Zheng et al., 2021 )	STBP Training	4	92.92	-	-	10	67.8
TET ( Deng et al., 2022 )	STBP Training	4	94.44	4	74.47	10	83.17
SLTT ( Meng et al., 2023 )	Online Training	6	94.44	6	74.38	10	82.20
GAC-SNN ( Qiu et al., 2024 )	STBP Training	4	96.24	4	79.83	-	-
Spikformer ( Zhou et al., 2023 )	STBP Training	4	95.19	4	77.86	16	80.6

Table 4. Performance evaluation on downstream benchmarks, which encompass both static and neuromorphic datasets, including CIFAR-10, CIFAR-100 and CIFAR10-DVS.

Panel of Figure 3 — Figure 3. Distribution of energy-related metrics for modules (SSA, SConv and SFFN) on the ImageNet-1k dataset, including average firing rate, SOPs and its theoretical upper-bounds.

Spiking Transformers

Spiking Transformers combine attention-based architectures with event-driven spiking neurons to process information using sparse, temporally coded computation.

See all 42 papers

BSViT: A Burst Spiking Vision Transformer for Expressive and Efficient Visual Representation Learning
Apr 25, 2026Hongxiang Peng, Dewei Bai, Hong QuSpiking Transformers Efficient ViTs
School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu 610054, China
Neural Dynamics Self-Attention for Spiking Transformers
Mar 9, 2026Dehao Zhang, Fukai Guo, Shuai Wang +6Spiking Transformers Spiking Neural Networks
University of Electronic Science and Technology of China · Shenzhen Loop Area Institute · The Chinese University of Hong Kong (Shenzhen)
SAFformer:Improving Spiking Transformer via Active Predictive Filtering
May 8, 2026Zequan Xie, Weiming Zeng, Yunhua Chen +3Spiking Transformers Efficient ViTs
School of Computer Science and Technology, Guangdong University of Technology, Guangzhou, China · Faculty of Science, Hong Kong Baptist University, China · School of Electronic Information, Wuhan University, China

Apr 25, 2026J/K move · Enter open · S save

BSViT: A Burst Spiking Vision Transformer for Expressive and Efficient Visual Representation Learning

Hongxiang Peng, Dewei Bai, Hong Qu

School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu 610054, China

Spiking Transformers Efficient ViTs Spiking Neural Networks Sparse Attention

Open paper AskPDF arXiv

Spiking Vision Transformers (S-ViTs) offer a promising framework for energy-efficient visual learning. However, existing designs remain limited by two fundamental issues: the restricted information capacity of binary spike coding and the dense token interactions introduced by global self-attention. To address these challenges, this work proposes BSViT, a burst spiking-driven Vision Transformer featuring a Dual-Channel Burst Spiking Self-Attention (DBSSA) mechanism. DBSSA encodes queries with binary spikes and keys with burst spikes to enhance representational capacity. The value pathway adopts dual excitatory and inhibitory binary channels, enabling signed modulation and richer spike interactions. Importantly, the entire attention operation preserves addition-only computation, ensuring compatibility with energy-efficient neuromorphic hardware. To further reduce spike activity and incorporate spatial priors, a patch adjacency masking strategy is introduced to restrict attention to local neighborhoods, resulting in structure-aware sparsity and reduced computational overhead. In addition, burst spike coding is systematically integrated across the network to increase spike-level representational capacity beyond conventional binary spiking. Extensive experiments on both static and event-based vision benchmarks demonstrate that BSViT consistently outperforms existing spiking Transformers in accuracy while maintaining competitive energy efficiency.

Abstract

Spiking Transformers

Explore similar work

BSViT: A Burst Spiking Vision Transformer for Expressive and Efficient Visual Representation Learning

Neural Dynamics Self-Attention for Spiking Transformers

SAFformer:Improving Spiking Transformer via Active Predictive Filtering