NeuroVLA: A Brain-inspired Embodied Intelligence for Fluid and Fast Reflexive Robotics Control
Authors: Weiyu Guo, He Zhang, Pengteng Li, Tiefu Cai, Ziyang Chen, Yandong Guo, Xiao He, Yongkui Yang, +2 more
Organizations: The Thrust of Artificial Intelligence, The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China. · AI2 Robotics, Shenzhen, China · Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China · The Department of Computer Science and Engineering, The Hong Kong University of Science and Technology, Hong Kong, China
Vision-language-action (VLA) models inherit powerful semantic representations from large language models, but carrying over their design priorities can leave the demands of physical control under-specified. Stable manipulation requires high-frequency proprioceptive feedback and persistent motor state, in addition to instruction understanding. Here we introduce Neuromorphic Vision-Language-Action (NeuroVLA), a brain-inspired architecture that couples a cortical vision-language planner to recurrent cerebellar modulation and a stateful spiking spinal controller. These components translate semantic intent into actions while incorporating high-frequency proprioceptive and force feedback, maintaining short-term motor memory, and enabling local responses to unexpected contact. In simulated and physical manipulation tasks, the cerebellar module reduces mean commanded jerk by 75.6%, and NeuroVLA achieves a 54.8% collision-recovery rate compared with 0.0% for the evaluated VLA baselines. The spinal controller implemented on an FPGA-based neuromorphic processor achieves 2.19 ms inference latency and 0.87 mJ per inference, corresponding to approximately 0.4 W during inference. These results motivate moving beyond inherited language-model design priorities toward embodied architectures that explicitly integrate semantic planning, high-frequency bodily feedback, motor memory, and local correction.
Figures & tables
Figure 1: Semantic planning coupled to feedback-driven motor control. a, Conceptual separation of high-dimensional semantic computation and local sensorimotor processing. The schematic illustrates the intended CUDA/neuromorphic partition; in the evaluated implementation, the cortical and cerebellar modules run on the CUDA tier, and hardware measurements concern the FPGA-based spinal controller. b, Information flow in NeuroVLA. A cortical vision-language module generates compact intention tokens; recurrent cerebellar modulation combines these tokens with proprioceptive history; and a stateful spiking spinal module produces continuous actions. The force-feedback pathway supports local contact responses between cortical updates.
Figure 2: Instruction-dependent spatial selection by the Q-Former. a, b, Attention during “Put the wine bottle on the cabinet”: focus shifts from the manipulation target to the placement surface as execution progresses. c, d, Under “Open the middle drawer” in the same scene, attention concentrates on the drawer handle despite the nearby bottle. These maps illustrate task-dependent selection of spatial features for downstream motor control.
Figure 3: Temporal coordination and force-aware trajectory correction. a, Wrench and joint-angle traces during “Shake the cup” show synchronized oscillatory cycles, providing a sensorimotor description of rhythmic execution. b, Contact produces a wrench transient. The local feedback pathway adjusts the subsequent trajectory (blue solid line), whereas the baseline continues along the obstructed path (red dashed line). These examples assess temporal coordination and contact-aware correction in the engineered controller.
Figure 4: Proprioceptive modulation reduces fluctuations in commanded motion. a, Commanded acceleration traces in translational and rotational dimensions with the baseline policy (red) and cerebellar modulation (blue). b, c, Mean Absolute Commanded Jerk and Mean Absolute Commanded Acceleration. Mean jerk decreases by 75.6%, and acceleration decreases by 32.8%–58.0% across axes. These metrics characterize command smoothness under the evaluated conditions.
Figure 5: Selective spiking activity and temporal ablation. a, Membrane potentials and spike trains during “Static Pose & Dynamic Gripper” show relatively quiescent end-effector pose neurons and active gripper-control neurons. b, Firing rates across end-effector and gripper conditions quantify demand-dependent activity. c, Mean activation rates are lower in static than dynamic phases. d, LIBERO task success with a multi-step SNN, a single-step variant, and a no-cerebellum baseline. e, Task success under noisy observations compares NeuroVLA with the evaluated VLA baselines. Temporal ablation and noisy-observation evaluation test different aspects of the control architecture.
Figure 6: Action-associated structure in spinal representations. Neural Representation of Action: Rearranged firing-rate maps show different activation patterns for kinematic dimensions such as ∣ΔRoll∣ and ∣Gripper∣ . Action Latent Cluster: A t-SNE projection of spinal hidden states separates several action-associated groups. These visualizations describe action-associated organization in the learned motor representation.
Figure 7: Neuromorphic processor implemented on an FPGA platform for deploying the Spiking Spinal Module. a, FPGA board used for implementation. b, Neuromorphic processor incorporating an LIF systolic-array architecture and spike-sparsity-aware computation to reduce inference latency and energy consumption. c, FPGA resource utilization, with 51,953 LUTs, 27,880 FFs and 169 BRAMs. d, Performance summary, showing an inference latency of 2.19 ms at 20 MHz and an energy cost of 0.87 mJ per inference.
Figure 8: Physical manipulation and recovery after unexpected contact. a, Bimanual laboratory platform and experimental scene. b–f, Task success for relocating test tubes (b), pouring liquid (c), organizing laboratory items (d), shaking flasks (e), and discarding test tubes (f), comparing NeuroVLA with OpenVLA, OpenVLA-OFT, UniVLA, and WorldVLA. g, Recovery after collision followed by test-tube shaking. NeuroVLA achieves 54.8% recovery compared with 0.0% for the evaluated baselines. The task tests withdrawal and subsequent continuation under an unexpected physical obstruction.
Vision-language-action (VLA) models have achieved strong performance in language-conditioned manipulation, yet success under nominal evaluation does not necessarily translate into robust closed-loop behavior when executed actions are transiently corrupted. We introduce DS-VLA, a dendritic-inspired action architecture that incorporates dendritic spiking dynamics into VLA control to address this limitation. Specifically, to enable modularized feature processing and temporal information integration, DS-VLA equips action neurons with multiple sparsely connected dendritic branches, each featuring heterogeneous, learned decay factors. Furthermore, to suppress unreliable state updates while preserving task-relevant historical information, we introduce a neuron-wise inhibitory gate that adaptively regulates the admission of new multimodal evidence into dendritic states prior to somatic dynamics. We evaluate DS-VLA on all four LIBERO suites under both nominal rollouts and a unified closed-loop action-perturbation protocol. DS-VLA achieves a 91.6% average nominal success rate and an 87.35% average perturbed success rate, retaining 95.4% of its nominal performance. Under the same reported perturbation setting, OpenVLA-OFT, FAST, π0, and GR00T achieve 39.45%, 23.90%, 28.55%, and 30.75%, respectively. A controlled ablation isolates the contribution of neuron-wise shared inhibition, while analyses of neural dynamics and post-perturbation trajectories associate robust performance with selective evidence suppression and effective behavioral recovery. Together, these results demonstrate that integrating brain-inspired computational mechanisms offers a promising architectural prior for robust embodied intelligence beyond merely scaling vision-language backbones or generative action decoders.
Yaxing Lyu, Jingyi Li, Mingkun Xu +1
Guangdong Institute of Intelligence Science and Technology · The University of Hong Kong · University of Illinois Urbana-Champaign +1
Vision-language-action (VLA) models bridge multimodal understanding and robotic control, advancing the dominant paradigm for embodied intelligence. However, most existing models rely on large Transformers, whose latency and energy costs hinder deployment on resource-constrained platforms. Through sparse event-driven computation, spiking neural networks offer a promising paradigm for high-performance and energy-efficient computing. Here, we propose the first Spike-driven VLA framework enabling end-to-end direct training for robotic manipulation, which mainly comprises three core components. First, we develop spiking visual and instruction encoders for multimodal perception, encoding visual observations and language instructions into sparse, reliable spike representations for subsequent cross-modal fusion. Then, we introduce Multi-Winner Spike Fusion for instruction-guided scene understanding, using bidirectional top-k winner-take-all spike routing to suppress background interference and yield fused memory. Finally, we propose a Spike Action Chunking Transformer that incorporates spiking cross-attention over the fused memory and the current robot state, enabling efficient end-to-end generation of continuous action chunks for robotic control. Extensive experiments on LIBERO and Meta-World demonstrate that Spike-driven VLA achieves competitive performance with fewer parameters and lower estimated inference energy than conventional VLA models. This work establishes a foundational framework for neuromorphic VLA modeling, paving the way for future advances in resource-efficient embodied intelligence.
Shuai Wang, Malu Zhang, Mingquan Liu +6
University of Electronic Science and Technology of China.
Real-world deployment of Vision-Language-Action (VLA) models is often bottlenecked by efficiency-performance trade-offs, cross-embodiment generalization, and execution smoothness. We present NebulaVLA, an asynchronous dual-frequency architecture that decouples high-level semantic reasoning from low-level action control, optimizing computational resources and modularity. To bridge semantic gaps across heterogeneous robots, we introduce GESTURE-7, a unified language-grounded action representation. Furthermore, our Guide Action algorithm enforces kinematic continuity via mask-based smoothness constraints. Comprehensive evaluations demonstrate that NebulaVLA significantly outperforms synchronous baselines, achieving an 85.5% average success rate on LIBERO-Plus and accelerating action generation by \textasciitilde 2.7×. This asynchronous design enables highly efficient and responsive control for practical robotics.