NeuroVLA: A Brain-inspired Embodied Intelligence for Fluid and Fast Reflexive Robotics Control
Authors: Weiyu Guo, He Zhang, Pengteng Li, Tiefu Cai, Ziyang Chen, Yandong Guo, Xiao He, Yongkui Yang, +2 more
Organizations: The Thrust of Artificial Intelligence, The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China. · AI2 Robotics, Shenzhen, China · Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China · The Department of Computer Science and Engineering, The Hong Kong University of Science and Technology, Hong Kong, China
Vision-language-action (VLA) models inherit powerful semantic representations from large language models, but carrying over their design priorities can leave the demands of physical control under-specified. Stable manipulation requires high-frequency proprioceptive feedback and persistent motor state, in addition to instruction understanding. Here we introduce Neuromorphic Vision-Language-Action (NeuroVLA), a brain-inspired architecture that couples a cortical vision-language planner to recurrent cerebellar modulation and a stateful spiking spinal controller. These components translate semantic intent into actions while incorporating high-frequency proprioceptive and force feedback, maintaining short-term motor memory, and enabling local responses to unexpected contact. In simulated and physical manipulation tasks, the cerebellar module reduces mean commanded jerk by 75.6%, and NeuroVLA achieves a 54.8% collision-recovery rate compared with 0.0% for the evaluated VLA baselines. The spinal controller implemented on an FPGA-based neuromorphic processor achieves 2.19 ms inference latency and 0.87 mJ per inference, corresponding to approximately 0.4 W during inference. These results motivate moving beyond inherited language-model design priorities toward embodied architectures that explicitly integrate semantic planning, high-frequency bodily feedback, motor memory, and local correction.
Figures & tables
Figure 1: Semantic planning coupled to feedback-driven motor control. a, Conceptual separation of high-dimensional semantic computation and local sensorimotor processing. The schematic illustrates the intended CUDA/neuromorphic partition; in the evaluated implementation, the cortical and cerebellar modules run on the CUDA tier, and hardware measurements concern the FPGA-based spinal controller. b, Information flow in NeuroVLA. A cortical vision-language module generates compact intention tokens; recurrent cerebellar modulation combines these tokens with proprioceptive history; and a stateful spiking spinal module produces continuous actions. The force-feedback pathway supports local contact responses between cortical updates.
Figure 2: Instruction-dependent spatial selection by the Q-Former. a, b, Attention during “Put the wine bottle on the cabinet”: focus shifts from the manipulation target to the placement surface as execution progresses. c, d, Under “Open the middle drawer” in the same scene, attention concentrates on the drawer handle despite the nearby bottle. These maps illustrate task-dependent selection of spatial features for downstream motor control.
Figure 3: Temporal coordination and force-aware trajectory correction. a, Wrench and joint-angle traces during “Shake the cup” show synchronized oscillatory cycles, providing a sensorimotor description of rhythmic execution. b, Contact produces a wrench transient. The local feedback pathway adjusts the subsequent trajectory (blue solid line), whereas the baseline continues along the obstructed path (red dashed line). These examples assess temporal coordination and contact-aware correction in the engineered controller.
Figure 4: Proprioceptive modulation reduces fluctuations in commanded motion. a, Commanded acceleration traces in translational and rotational dimensions with the baseline policy (red) and cerebellar modulation (blue). b, c, Mean Absolute Commanded Jerk and Mean Absolute Commanded Acceleration. Mean jerk decreases by 75.6%, and acceleration decreases by 32.8%–58.0% across axes. These metrics characterize command smoothness under the evaluated conditions.
Figure 5: Selective spiking activity and temporal ablation. a, Membrane potentials and spike trains during “Static Pose & Dynamic Gripper” show relatively quiescent end-effector pose neurons and active gripper-control neurons. b, Firing rates across end-effector and gripper conditions quantify demand-dependent activity. c, Mean activation rates are lower in static than dynamic phases. d, LIBERO task success with a multi-step SNN, a single-step variant, and a no-cerebellum baseline. e, Task success under noisy observations compares NeuroVLA with the evaluated VLA baselines. Temporal ablation and noisy-observation evaluation test different aspects of the control architecture.
Figure 6: Action-associated structure in spinal representations. Neural Representation of Action: Rearranged firing-rate maps show different activation patterns for kinematic dimensions such as ∣ΔRoll∣ and ∣Gripper∣ . Action Latent Cluster: A t-SNE projection of spinal hidden states separates several action-associated groups. These visualizations describe action-associated organization in the learned motor representation.
Figure 7: Neuromorphic processor implemented on an FPGA platform for deploying the Spiking Spinal Module. a, FPGA board used for implementation. b, Neuromorphic processor incorporating an LIF systolic-array architecture and spike-sparsity-aware computation to reduce inference latency and energy consumption. c, FPGA resource utilization, with 51,953 LUTs, 27,880 FFs and 169 BRAMs. d, Performance summary, showing an inference latency of 2.19 ms at 20 MHz and an energy cost of 0.87 mJ per inference.
Figure 8: Physical manipulation and recovery after unexpected contact. a, Bimanual laboratory platform and experimental scene. b–f, Task success for relocating test tubes (b), pouring liquid (c), organizing laboratory items (d), shaking flasks (e), and discarding test tubes (f), comparing NeuroVLA with OpenVLA, OpenVLA-OFT, UniVLA, and WorldVLA. g, Recovery after collision followed by test-tube shaking. NeuroVLA achieves 54.8% recovery compared with 0.0% for the evaluated baselines. The task tests withdrawal and subsequent continuation under an unexpected physical obstruction.