Spike-driven Vision-Language-Action Model

Shuai Wang; Malu Zhang; Mingquan Liu; Weihui Dai; Dehao Zhang; Jieyuan Zhang; Yimeng Shan; Zijian Zhou

cs.CLSep 30, 2026

Spike-driven Vision-Language-Action Model

Authors: Shuai Wang, Malu Zhang, Mingquan Liu, Weihui Dai, Dehao Zhang, Jieyuan Zhang, Yimeng Shan, Zijian Zhou, +1 more

Organizations: University of Electronic Science and Technology of China.

Abstract

Vision-language-action (VLA) models bridge multimodal understanding and robotic control, advancing the dominant paradigm for embodied intelligence. However, most existing models rely on large Transformers, whose latency and energy costs hinder deployment on resource-constrained platforms. Through sparse event-driven computation, spiking neural networks offer a promising paradigm for high-performance and energy-efficient computing. Here, we propose the first Spike-driven VLA framework enabling end-to-end direct training for robotic manipulation, which mainly comprises three core components. First, we develop spiking visual and instruction encoders for multimodal perception, encoding visual observations and language instructions into sparse, reliable spike representations for subsequent cross-modal fusion. Then, we introduce Multi-Winner Spike Fusion for instruction-guided scene understanding, using bidirectional top- $k$ winner-take-all spike routing to suppress background interference and yield fused memory. Finally, we propose a Spike Action Chunking Transformer that incorporates spiking cross-attention over the fused memory and the current robot state, enabling efficient end-to-end generation of continuous action chunks for robotic control. Extensive experiments on LIBERO and Meta-World demonstrate that Spike-driven VLA achieves competitive performance with fewer parameters and lower estimated inference energy than conventional VLA models. This work establishes a foundational framework for neuromorphic VLA modeling, paving the way for future advances in resource-efficient embodied intelligence.

Figures & tables

Figure 2: (a) SVE encodes main- and wrist-view observations into visual spike tokens capturing appearance and spatial relations; (b) SIE encodes language instructions into contextual semantic spike tokens that condition visual perception.

Figure 3: (a) MWSF uses bidirectional Top- $k$ WTA routing to retain instruction-relevant visual evidence, suppress background interference, and reduce computation. (b) SpikeACT uses learnable action tokens to predict continuous action chunks in parallel from fused memory and the robot state.

Method	Spike- driven	Computation Efficiency			Meta-World Success Rate (%)
		Params	FLOPs	Energy	Easy	Medium	Hard	V. Hard	Avg. $\uparrow$
		(B) $\downarrow$	(G) $\downarrow$	(mJ) $\downarrow$	Easy	Medium	Hard	V. Hard	Avg. $\uparrow$
TinyVLA ( Wen and others, 2024 )	$\times$	0.42	1259.1	2896.0	77.6	21.5	11.4	15.8	31.6
$\pi_{0}$ ( Black et al., 2025 )	$\times$	3.2	3069.8	7060.6	71.8	48.2	41.7	30.0	47.9
SmolVLA (0.45B) ( Shukor et al., 2025 )	$\times$	0.45	598.4	1376.4	82.5	41.8	45.0	60.0	57.3
SmolVLA (2.25B) ( Shukor et al., 2025 )	$\times$	2.25	4239.5	9751.1	87.1	51.8	70.0	64.0	68.2

Table 1: Comparison on the Meta-World benchmark ( Yu et al., 2020 ) . Baseline success rates are taken from ( Shukor et al., 2025 ) . Our method is evaluated over 50 episodes per task.

Method	Spike- driven	Computational Efficiency			LIBERO Success Rate (%)
		Params	FLOPs	Energy	Spa.	Obj.	Goal	Long	Avg. $\uparrow$
		(B) $\downarrow$	(G) $\downarrow$	(mJ) $\downarrow$	Spa.	Obj.	Goal	Long	Avg. $\uparrow$
OpenVLA ( Kim et al., 2025 )	$\times$	7.5	4158.3	9564	84.7	88.4	79.2	53.7	76.5
$\pi_{0}$ ( Black et al., 2025 )	$\times$	3.2	4371.9	10055	96.8	98.8	95.8	85.2	94.2
$\pi_{0.5}$ ( Physical Intelligence et al., 2025 )	$\times$	3.4	5014.2	11533	98.8	98.2	98.0	92.4	96.9
CogVLA ( Li et al., 2025 )	$\times$	8.3	3299.2	7588	98.6	98.8	96.6	95.4	97.4

Table 2: Comparison on LIBERO ( Liu et al., 2023 ) benchmark. We report suite-level and overall average success rates based on 50 rollouts per task.

Method	Spike- driven	Params (B) $\downarrow$	LIBERO-Plus Success Rate (%)
	Spike- driven	Params (B) $\downarrow$	Camera	Robot	Language	Light	Background	Noise	Layout	Avg. $\uparrow$
OpenVLA ( Kim et al., 2025 )	$\times$	7.5	1.1	4.1	26.8	4.4	25.3	19.3	31.6	16.1
NORA ( Hung et al., 2025 )	$\times$	3.8	4.0	41.1	67.0	31.0	50.5	17.6	63.9	39.3
UniVLA ( Bu et al., 2025 )	$\times$	7.0	4.3	50.3	71.8	59.1	80.0	25.3	34.3	46.4
$\pi_{0}$ ( Black et al., 2025 )	$\times$	3.2	15.8	6.6	61.0	79.6	78.5	79.4	70.4	55.9
Spike-driven VLA (Ours)	$\checkmark$	0.15	41.2	46.7	48.7	84.2	61.3	34.5	67.5	54.9

Table 3: Comparison on LIBERO-Plus ( Fei et al., 2026 ) across seven perturbation dimensions.

Suite	w/o WTA	w/ WTA	$\Delta$
Spatial	90.2	95.0	$+4.8$
Object	95.6	95.4	$-0.2$
Goal	96.2	96.6	$+0.4$
Long	78.6	82.4	$+3.8$
Overall	90.2	92.4	$+2.2$

Table 4: Ablation study of WTA.

Variant	VE	LE	Params (M)	Energy (mJ)	LIBERO Success Rate (%)
Variant	VE	LE	Params (M)	Energy (mJ)	Spatial	Object	Goal	Long	Avg.
B0	DINOv3-B	BERT	216.1	179.7	98.6	99.2	93.2	94.0	96.3
B1	DINOv3-B	SIE	216.1	173.0	98.0	98.6	95.4	92.4	96.1
B2	SVE	BERT	148.6	28.8	95.8	98.2	95.6	80.8	92.6
B3	SVE	SIE	148.6	20.9	95.0	95.4	96.6	82.4	92.4

Table 5: Ablation of visual and language encoders on LIBERO. VE and LE denote the visual and language encoders, respectively. DINOv3-B ( Siméoni et al., 2025 ) and BERT ( Devlin et al., 2019 ) serve as the corresponding ANN baselines.

Figure 4: Attention heatmaps showing that our MWSF module can better focus on instruction-relevant visual regions, highlighted by red boxes.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Hyperparameter	Setting
Time steps	4
Training epochs	200
Input resolution	$224\times 224$
Batch size	600
Optimizer	LAMB
Base learning rate	$6\times 10^{-4}$

Appendix

Table 6: ImageNet-1K pretraining settings for the spiking visual encoder.

Hyperparameter	MLM pre-training	Spike-driven VLA training
Objective	MLM (15% masking)	Frozen encoder
Tokenizer	bert-base-uncased	Same
Maximum sequence length	$128$	$N_{\ell}\leq 128$
Optimizer	AdamW	–
Peak learning rate	$2\times 10^{-4}$	–
Weight decay	$0$	–

Appendix

Table 7: Training hyperparameters of the SmoothSpike backbone used as SIE. The VLA column summarizes its configuration in Spike-driven VLA, where the backbone remains frozen.

stage	# Tokens	Layer Specification			SIE
Emb.	$N_{\ell}$	Token / position / segment Emb.		Dim	$768$
		Vocabulary / max position		–	$30522$ / $512$
		Embedding Norm		–	LayerNorm
		Shared orthogonal encoding		–	$\mathbf{H}_{1}$
		Temporal repeat		$T$	$4$
Encoder block	$N_{\ell}$	SSA	RMSNorm + LIF + $\mathbf{H}_{1}^{\top}$	$\tau$	$2.0$

Appendix

Table 8: Configuration of the Spike Instruction Encoder (SIE) used in Spike-driven VLA. The backbone is SmoothSpike-12-768 ( Zhou et al., 2026 ) .

N_{\ell}

is the instruction length (

N_{\ell}\leq 128

\mathbf{H}_{1}

is shared across residual branches;

\mathbf{H}_{2}

is per-layer on the value branch;

\mathbf{H}_{3}

is a

4

-block diagonal transform in the MLP. Query/key LIF neurons are not transformed.

Figure 5: Successful Spike-driven VLA rollouts on LIBERO. Rows (a)–(d) show four tasks. Columns show frames sampled at T= $0$ , $0.5$ , $1.0$ , $1.5$ , and $2.0$ .

Item	Configuration
Effective batch size	$8$ GPUs $\times$ $32$ samples/GPU $\times$ $1$ accumulation step $=256$
Training budget	$80{,}000$ updates; $20{,}480{,}000$ sample exposures
Optimizer	AdamW, $\beta=(0.9,0.95)$ , $\epsilon=10^{-8}$
Visual backbone LR	$1\times 10^{-5}$
Other trainable parameters LR	$5\times 10^{-5}$
LR schedule	$10{,}000$ steps of linear warm-up, followed by a constant LR

Appendix

Table 9: Training configuration of Spike-driven VLA on LIBERO.

Suite	Overall	Background Textures	Robot Initial States	Camera Viewpoints	Language Instructions	Sensor Noise	Object Layout	Light Conditions
Spatial	65.7	72.1 (186/258)	57.7 (202/350)	63.0 (237/376)	40.0 (156/390)	48.4 (170/351)	87.0 (335/385)	99.7 (291/292)
Object	56.0	70.6 (175/248)	42.2 (168/398)	42.9 (170/396)	58.2 (206/354)	32.7 (138/422)	65.8 (265/403)	97.3 (289/297)
Goal	54.0	69.4 (195/281)	50.9 (208/409)	46.8 (191/408)	35.4 (145/410)	43.5 (165/379)	60.9 (259/425)	84.6 (236/279)
Long	37.6	36.0 (104/289)	37.2 (146/393)	14.6 (61/419)	63.2 (242/383)	17.6 (79/449)	54.5 (170/312)	52.9 (145/274)

Appendix

Table 10: Success rates (%) on LIBERO-Plus. Parentheses give successful and total rollout counts. Overall rates are computed from pooled rollout counts.

Figure 6: Successful Spike-driven VLA rollouts on Meta-World MT50. Rows (a)–(d) show four tasks. Columns show frames sampled at T= $0$ , $0.5$ , $1.0$ , $1.5$ , and $2.0$ .

Explore similar work

SpikeVLA: Vision-Language-Action Models with Spiking Neural Networks
Jun 26, 2026Ruiqi Song, Dujun Nie, Siyu Teng +7Spiking Neural Networks Vision-Language-Action Models
College of Surveying and Geo-informatics, Tongji University, Shanghai, China · State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, China · OpenSpace Lab, China +6
VL2Spike: Spike-driven Distillation from VLMs for Low-Power Visual Perception in Embodied AI
Jun 14, 2026Zinan Liu, Eric Zheng, Soumyaratna Debnath +3Cross-Modal Knowledge Distillation Spiking Transformers
School of EEE, Nanyang Technological University (NTU), Singapore · Department of Computer Science, University of Toronto and Advanced Micro Devices, Inc., Canada · State Key Laboratory of Extreme Photonics and Instrumentation, Zhejiang University, Hangzhou 310027, China +1
TurboVLA: Real-Time Vision-Language-Action Model at 32 Hz on an RTX 4090 with <1 GB VRAM
Jul 29, 2026Hengyi Xie, Chenfei Yao, Xianjin Wu +4Efficient VLA Model Inference Efficient VLA Models
Huazhong University of Science and Technology

Jun 26, 2026J/K move · Enter open · S save

SpikeVLA: Vision-Language-Action Models with Spiking Neural Networks

Ruiqi Song, Dujun Nie, Siyu Teng, Baiyong Ding+6

College of Surveying and Geo-informatics, Tongji University, Shanghai, China · State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, China · OpenSpace Lab, China+6

Spiking Neural Networks Vision-Language-Action Models VLMs for Robotics

Open paper AskPDF arXiv

Vision-Language-Action (VLA) models have become a dominant paradigm for embodied intelligence. However, most existing approaches are built on large-scale transformers, resulting in substantial inference latency and energy consumption that limit their practical deployment in low-power, real-time scenarios. We propose SpikeVLA, a spiking VLA architecture for embodied navigation with energy-efficient inference, consisting of three key components. (i) A spiking vision encoder, Spike-V, that replaces dense continuous layers with event-driven spiking layers to reduce the energy consumption of visual representation learning. (ii) A multi-modal spiking large language model, Spike-L, that reformulates cross-modal reasoning with spiking dynamics and token-level event-driven sparsity to further lower computational cost. (iii) A spiking action policy network, Spike-A employs Laplacian-kernel population coding with a multi-layer fully connected SNN, and decodes spiking activities into stable and robust continuous control with energy-efficient inference under low-power constraints. Experiments on navigation and robotic control tasks show that SpikeVLA significantly reduces energy consumption and computational cost while maintaining competitive performance, highlighting its potential for low-power, real-time embodied intelligence.

Abstract

Explore similar work

SpikeVLA: Vision-Language-Action Models with Spiking Neural Networks

VL2Spike: Spike-driven Distillation from VLMs for Low-Power Visual Perception in Embodied AI

TurboVLA: Real-Time Vision-Language-Action Model at 32 Hz on an RTX 4090 with <1 GB VRAM