MorphAtt: A Neuromorphic Accelerator for Efficient Multi-Head Attention Processing in Spiking Vision Transformers

Rachmad Vidya Wicaksana Putra; Amirhesam Jafari Rad; Muhammad Shafique

cs.ARSep 27, 2026

MorphAtt: A Neuromorphic Accelerator for Efficient Multi-Head Attention Processing in Spiking Vision Transformers

Authors: Rachmad Vidya Wicaksana Putra, Amirhesam Jafari Rad, Muhammad Shafique

Organizations: eBRAIN Lab, New York University (NYU) Abu Dhabi, Abu Dhabi, United Arab Emirates · University of Tehran, Tehran, Iran

Abstract

Spiking Vision Transformers (SViTs) are developed as an energy-efficient alternative to conventional ViTs for computer vision tasks at the edge. However, huge parameter counts and complex multi-head self-attention (MHSA) operations make it challenging to achieve high energy efficiency in SViT inference, especially in tightly constrained applications. To maximize efficiency gains of SViT processing, we propose MorphAtt, a novel digital accelerator that expedites SViT inference through streamlined processing. Specifically, it processes MHSA operations using cascaded hardware modules: a Spiking Query-Key-Value generator (SpikeQKV), a low-complexity Spiking Multi-Head Self-Attention engine (SpikeAtten), and Reparameterization Convolution (RepConv) modules. To mitigate traffic congestion in on-chip memory accesses and data reuse, specialized inter-module buffers are integrated within the dataflow. Under synthesis using 32nm CMOS technology, MorphAtt achieves 792-1605 GOPS of throughput, while incurring ~39-55 mW of power consumption and 1.5 mm^2 of area, which lead to 20.3-29.1 TOPS/W of energy efficiency. These results also demonstrate that our MorphAtt offers better performance and efficiency trade-offs than state-of-the-art, thereby enabling highly energy-efficient vision-based AI systems at the edge.

Figures & tables

Figure 1: A typical SViT architecture: (a) a network overview, (b) spike-driven transformer block, and (c) multi-head self-attention (MHSA).

Figure 2: Our MorphAtt accelerator: (a) the overall architecture, and the detailed architectures for key modules: (b) Spiking Query-Key-Value generator (SpikeQKV), (c) QKV buffers (QKV_Buffers), (d) Spiking Multi-Head Self-Attention engine (SpikeAtten), (e) data buffer (Data_Buffer), and (f) Reparameterization Convolution module (RepConv).

Figure 3: SpikeGen internal architecture.

Figure 4: Single-Head Self-Attention (SHSA) architecture.

Figure 5: Overview of Convolutional Architectures in MorphAtt: (a) Conv1D, (b) Conv2D, and (c) Non-Spiking Conv1D.

Evaluation	Design of ( Li et al. 2024 )	SpikeTA ( Gao and others 2025 )	FireFly-T ( Li and others 2025 )	VESTA ( Chen and others 2024 )	Bishop ( Xu and others 2025 )	Design of ( Fang et al. 2025 )	Design of ( Xu and others 2024 )	MorphAtt (ours)
Paradigm	2D	2D	2D	2D	2D	2D	3D	2D
Technology (nm)	FPGA	FPGA	FPGA	28	28	28	28	32
Precision (bit)	10	8	4	8	multi-bit	8	8	8
Area (mm 2 )	N/A	N/A	N/A	0.84	2.96	6.5	0.20	1.49
Power (mW)	$\sim$ 12000	$\sim$ 71759	$\sim$ 4349	416	627	222.65	201.63	38.96 - 55.12
Frequency (MHz)	200	450	300	500	500	200	500	100

Table 1: Comparison against the state-of-the-art digital SViT accelerators.

	Total	Total	Power
Module	Area	Power	Distribution
	(10 3 $\times$ $\mu$ m 2 )	(mW)	(% of total)
SpikeQKV	686.0	20.095	51.6%
SpikeAtten	9.4	0.209	0.6%
RepConv	42.6	1.267	3.2%
On-chip Memories	758.3	17.394	44.6%

Table 2: Post-synthesis breakdown of MorphAtt modules (32nm HVT, 1.05V, 100 MHz).

Figure 6: Pie-chart breakdown of MorphAtt for (a) area and (b) power consumption, across different modules.

Figure 7: (a) The timing diagram of the main FSM for sequential-based processing in MorphAtt. (b) The hypothetical timing diagram of the main FSM for pipeline-based processing in MorphAtt.

AI Accelerator Inference

AI accelerator inference studies hardware and software for running neural networks faster and more efficiently across data-center, edge, and mobile devices.

See all 102 papers

BSViT: A Burst Spiking Vision Transformer for Expressive and Efficient Visual Representation Learning
Apr 25, 2026Hongxiang Peng, Dewei Bai, Hong QuSpiking Transformers Efficient ViTs
School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu 610054, China
Ge $^\text{2}$ mS-T: Multi-Dimensional Grouping for Ultra-High Energy Efficiency in Spiking Transformer
Apr 10, 2026Zecheng Hao, Shenghao Xie, Kang Chen +3Spiking Transformers Spiking Neural Networks
Peking University, School of Computer Science, State Key Laboratory for Multimedia Information Processing, Beijing, China · Peking University, School of Computer Science, Beijing Key Laboratory of Brain-inspired Spiking Large Models, Beijing, China
Neural Dynamics Self-Attention for Spiking Transformers
Mar 9, 2026Dehao Zhang, Fukai Guo, Shuai Wang +6Spiking Transformers Spiking Neural Networks
University of Electronic Science and Technology of China · Shenzhen Loop Area Institute · The Chinese University of Hong Kong (Shenzhen)

Apr 25, 2026J/K move · Enter open · S save

BSViT: A Burst Spiking Vision Transformer for Expressive and Efficient Visual Representation Learning

Hongxiang Peng, Dewei Bai, Hong Qu

School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu 610054, China

Spiking Transformers Efficient ViTs Spiking Neural Networks Sparse Attention

Open paper AskPDF arXiv

Spiking Vision Transformers (S-ViTs) offer a promising framework for energy-efficient visual learning. However, existing designs remain limited by two fundamental issues: the restricted information capacity of binary spike coding and the dense token interactions introduced by global self-attention. To address these challenges, this work proposes BSViT, a burst spiking-driven Vision Transformer featuring a Dual-Channel Burst Spiking Self-Attention (DBSSA) mechanism. DBSSA encodes queries with binary spikes and keys with burst spikes to enhance representational capacity. The value pathway adopts dual excitatory and inhibitory binary channels, enabling signed modulation and richer spike interactions. Importantly, the entire attention operation preserves addition-only computation, ensuring compatibility with energy-efficient neuromorphic hardware. To further reduce spike activity and incorporate spatial priors, a patch adjacency masking strategy is introduced to restrict attention to local neighborhoods, resulting in structure-aware sparsity and reduced computational overhead. In addition, burst spike coding is systematically integrated across the network to increase spike-level representational capacity beyond conventional binary spiking. Extensive experiments on both static and event-based vision benchmarks demonstrate that BSViT consistently outperforms existing spiking Transformers in accuracy while maintaining competitive energy efficiency.

Abstract

AI Accelerator Inference

Explore similar work

BSViT: A Burst Spiking Vision Transformer for Expressive and Efficient Visual Representation Learning

Ge2^\text{2}2mS-T: Multi-Dimensional Grouping for Ultra-High Energy Efficiency in Spiking Transformer

Neural Dynamics Self-Attention for Spiking Transformers

Ge $^\text{2}$ mS-T: Multi-Dimensional Grouping for Ultra-High Energy Efficiency in Spiking Transformer