cs.ROOct 15, 2026

Towards bio-inspired visual navigation: a neuromorphic navigation paradigm for MAVs

Authors: Junjie Jiang, Ankang Zhou, Zheng Fang

Organizations: Faculty of Robot Science and Engineering, Northeastern University, Shenyang 110819, China. · College of Information Science and Engineering, Northeastern University, Shenyang 110819, China.

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Biological visual navigation achieves remarkable robustness at extremely low power, whereas frame-based robotic systems relying on artificial neural networks remain limited in both respects, particularly for energy-constrained micro aerial vehicles (MAVs) operating in complex environments. Inspired by the event-driven, sparse, and asynchronous principles of biological visual systems, neuromorphic sensing and computation provide a promising alternative for robust and energy-efficient navigation. In this paper, we propose the first end-to-end event-driven neuromorphic paradigm for MAV visual navigation in complex environments by integrating neuromorphic vision, spiking neural networks, and neuromorphic processors. The proposed method first extracts the environmental representation from event streams using a spiking perceptual representation module. This representation is fed into a spiking navigation module, where guidance learning and population coding enable efficient policy learning with few time steps. The navigation module then outputs a waypoint converted into executable trajectory points. Experiments show that the proposed neuromorphic method achieves navigation performance comparable to ANN-based methods while markedly reducing power consumption. The spiking perceptual module consumes only 19.57% of the power of its ANN counterpart, while the spiking navigation module deployed on Intel Loihi consumes only 12.37% of that required by its ANN counterpart running on NVIDIA Orin NX. Beyond prior neuromorphic systems focused mainly on relatively simple navigation subtasks, we demonstrate a complete neuromorphic visual navigation system spanning simulation and real-world deployment. Our results establish the feasibility of the neuromorphic paradigm for autonomous navigation and provide a bio-inspired pathway toward robust and low-power MAV navigation.

Figures & tables

Figure 1: Visual navigation mechanisms across biological, conventional, and neuromorphic paradigms. ( A ) The biological visual navigation paradigm relies on the retina and neural circuits. Visual information is sensed asynchronously at the pixel level and processed through sparse, event-driven neural activity to produce navigation behaviors. ( B ) Comparison between conventional and neuromorphic visual navigation paradigms for MAVs. The conventional visual navigation paradigm relies on frame-based cameras and artificial neural networks. Visual information is acquired synchronously as image frames and processed through dense, clock-driven computation to generate navigation actions. The neuromorphic visual navigation paradigm employs event-based vision sensors and spiking neural networks. Inspired by biological visual processing, it captures visual information asynchronously at the pixel level and processes it through sparse, event-driven computation to generate navigation actions.
Figure 2: End-to-end event-driven neuromorphic MAV navigation system. ( A ) Architecture of the proposed end-to-end event-driven neuromorphic navigation system. The system comprises a spiking perceptual representation module and a spiking navigation policy module. The perceptual module converts event streams into compact, geometry-aware features, while the navigation policy module maps these visual features, together with the robot state and goal direction, to navigation actions. ( B ) Key hardware components of the neuromorphic system. The platform consists of a neuromorphic vision sensor (Prophesee EVK4), an NVIDIA Jetson Orin NX, an x86 host computer, a neuromorphic processor (Intel Kapoho Bay), and a micro aerial vehicle. The NVIDIA Jetson Orin NX communicates wirelessly with the x86 host computer. ( C ) Deployment and communication pipeline of the proposed system. Owing to the limited neuron models supported by Intel Kapoho Bay, the spiking perceptual representation module is deployed on the NVIDIA Jetson Orin NX, whereas the spiking navigation policy module is deployed on Intel Kapoho Bay. The Jetson Orin NX acquires the event stream and transmits control commands to the Pixhawk flight controller. The Jetson Orin NX and the x86 host computer exchange system information wirelessly. The x86 host computer exchanges spike trains with Intel Kapoho Bay through NxSDK.
Figure 3: Training pipeline of the proposed end-to-end event-driven neuromorphic navigation method. Continuous grayscale image sequences are first transformed into asynchronous event streams using an event simulator. Depth estimation is then introduced as an auxiliary task to enable geometry-aware spiking features learning from event streams. The learned event features are fused with the MAV state and goal direction, and then encoded into spike trains via population coding. The resulting spikes are processed by the spiking navigation network and decoded to predict the action. Finally, collision loss, smoothness loss, and goal-reaching loss are defined based on the predicted action, providing physically meaningful supervision for end-to-end optimization of all network modules.
SchemeMax Speed (m/s)Success RateAverage Distance (m)Average Time (s)
ANN-based Ours0.545/48 45/4823.35 24.3567.60 66.14
ANN-based Ours1.047/48 46/4824.07 24.8733.87 32.45
ANN-based Ours2.047/48 48/4824.82 25.9816.85 17.78
Table 1: Quantitative comparison between the ANN-based method and the proposed method. Navigation performance is evaluated at maximum flight speeds of 0.5, 1.0, and 2.0 m/s across six simulation environments. Success rate denotes the number of successful trials out of 48 total navigation trials, and average distance and average time denote the mean trajectory length and navigation time of the evaluated trials, respectively.
Figure 4: Navigation trajectories of the proposed method in simulation environments. (A) Continuous navigation performance is evaluated by requiring the MAV to sequentially pass through multiple waypoints in a simulated environment. (B) Robustness is evaluated by placing target points in eight directions within each of six testing environments.
Figure 5: Average trajectory distance comparison between our method and the ANN-based method across six environments. (A) Average trajectory distance at a maximum speed of 0.5 m/s. (B) Average trajectory distance at a maximum speed of 1 m/s. (C) Average trajectory distance at a maximum speed of 2 m/s.
Figure 6: Average navigation time comparison between our method and the ANN-based method across six environments. (A) Average navigation time at a maximum speed of 0.5 m/s. (B) Average navigation time at a maximum speed of 1 m/s. (C) Average navigation time at a maximum speed of 2 m/s.
Figure 7: Navigation trajectories comparison between the proposed method and the ANN-based method. The navigation trajectories of the two methods are compared across six different simulation environments, with the trajectories of the proposed method shown in red and those of the ANN-based method shown in blue.
Figure 8: Navigation trajectories at different maximum speeds. The green, blue, and red lines indicate the navigation trajectories at maximum speeds of 0.5 m/s, 1.0 m/s, and 2.0 m/s, respectively.
Figure 9: Comparison of navigation trajectories with and without depth supervision. The green and red lines indicate trajectories without and with depth supervision, respectively. Removing depth supervision leads to collisions, ground crashes, and substantial target deviations.
Figure 10: Navigation performance of the proposed method in real-world environments. Indoor and outdoor navigation tests were conducted by placing the target point behind a series of obstacles, requiring the MAV to avoid the obstacles and successfully reach the target.
DeviceSchemeDynamic Power (mW)Inference Rate (s -1 )Energy per Inference (mJ/inf.)
NVIDIA GeForce RTX 4090ANN-based43000207820.69
NVIDIA Jetson Orin NX (25 W)ANN-based52758056.55
NVIDIA Jetson Orin NX (10 W)ANN-based19586493.02
Intel Kapoho BayOurs1401720.81
Table 2: Per-inference energy consumption of SNN-based and ANN-based navigation networks across different devices. Dynamic power is defined as the difference between the operating and idle power of each device. The inference rate denotes the number of forward inferences performed per second, and the energy per inference is calculated by dividing the dynamic power by the inference rate. The NVIDIA Jetson Orin NX is evaluated in both high-performance mode (25 W) and power-saving mode (10 W).
Figure 11: Operations and per-inference energy consumption of ANN-based and SNN-based perceptual representation networks. The ANN-based network is evaluated based on multiply–accumulate operations (MACs), whereas the SNN-based network is evaluated based on spike-triggered accumulate operations (ACs). Per-inference energy consumption is estimated from the corresponding operation counts using energy costs of 4.6 pJ per MAC and 0.9 pJ per AC under a 45-nm CMOS technology assumption.
Figure S1 : Composition of the training dataset. Each training dataset includes a complete point cloud map of the environment, along with continuous grayscale images, corresponding depth maps, and MAV poses collected from multiple locations.
Figure S2 : Event–depth calibration for fine-tuning in the real world. The Prophesee EVK4 and Intel RealSense D435i are rigidly mounted together to enable joint camera calibration. The figure also shows representative event frames and their corresponding depth maps acquired in indoor and outdoor environments for real-world fine-tuning of the spiking depth estimation network.
TypesParametersDescriptionsValues
 Training epoch–100
Experimental settingBatch size–16
 r0r_{0}Initial learning rate1.5×10−41.5\times 10^{-4}
 C\mathrm{C}Trajectory time constant88
 wsw_{s}The coefficient for smooth item4×10−44\times 10^{-4}
Loss functionwow_{o}The coefficient for collision item10−310^{-3}
Table S1 : Parameter settings used in the experiments. The table summarizes the principal hyperparameters used for network training, trajectory-loss computation, spiking depth estimation, spiking feature extraction, and the spiking navigation network.
ModuleOperationsCorresponding MACsAverage input firing rateACs
 Conv-14.72M0.36966.98M
Encode-1Conv-237.75M0.223433.73M
 Conv-118.87M0.371128.01M
Encode-2Conv-237.75M0.196829.72M
 Conv-118.87M0.361327.27M
Encode-3Conv-237.75M0.210231.74M
Table S2 : Layer-wise computational statistics of the spiking U-Net. For each layer, the table reports the corresponding ANN multiply–accumulate operations (MACs), the average input firing rate measured in the SNN, and the resulting number of spike-triggered accumulate operations (ACs). The AC count is obtained by weighting the corresponding layer operations by the average input firing rate and the number of SNN time steps.
ModuleOperationsCorresponding MACsAverage input firing rateACs
StemConv12.85M151.40M
 Conv-19.44M0.287010.84M
ResBlock-1.1Conv-29.44M0.267210.09M
 Conv(Downsample)1.05M0.28701.21M
ResBlock-1.2Conv-19.44M0.458817.32M
 Conv-29.44M0.21748.21M
Table S3 : Layer-wise computational statistics of the spiking feature extraction network. For each layer, the table reports the corresponding ANN multiply–accumulate operations (MACs), the average input firing rate of the SNN, and the resulting number of spike-triggered accumulate operations (ACs). The AC count is calculated based on the corresponding operation count, average input firing rate, and number of SNN time steps.
Figure S3 : Event frames visualization across different environments. (A) Simulated event frames. (B) Event frames captured in indoor environments. (C) Event frames captured in outdoor environments. For improved visualization, the event-frame values are amplified by a factor of 10 and then displayed as positive and negative deviations from a baseline intensity value of 127.
Figure S4 : Visualization of predicted depth maps. We present the depth ground truth, event frames, and predicted depth maps in simulation, indoor, and outdoor environments, respectively. For improved visualization, the event-frame values are amplified by a factor of 10 and then displayed as positive and negative deviations from a baseline intensity value of 127.
More in this topic

Visual Navigation

Visual navigation enables robots and embodied agents to reach goals by interpreting camera images and acting in unfamiliar or changing environments.

See all 71 papers

Explore similar work

CardsList
Aug 5, 2026

SpikingNav: Robust Embodied Navigation with Spiking Neural Policies

Embodied navigation requires an agent to make sequential decisions from egocentric observations in a physical environment. Existing Artificial Neural Network (ANN)-based navigation models have achieved strong performance, yet they often rely on dense computation and may degrade under visual corruptions. Spiking neural networks (SNNs) provide event-driven computation and intrinsic temporal dynamics, which are promising for compact and robust navigation on resource-constrained platforms. However, whether spike-based sensing and policy dynamics can improve robustness in visually rich embodied navigation remains an open problem. This paper proposes SpikingNav, a spiking framework for robust indoor embodied navigation. It contains a Spiking Sensing Encoder (SSE) and a Spiking Policy Network (SPN). The SSE extracts task-conditioned visual features with a spike-based backbone. The SPN maintains a recurrent policy state through membrane integration, thresholding, and spike-triggered reset. In this way, SpikingNav exploits the dynamic properties and spike activations of SNNs to improve navigation performance and robustness. We evaluate SpikingNav on PointNav and ObjectNav under clean observations and visual corruptions. SpikingNav achieves competitive clean performance and stronger robustness with fewer parameters and lower per-step computation than a matched ANN baseline. For instance, SpikingNav improves ObjectNav success from 31.05% to 34.12%, and raises the average success under visual corruptions from 8.45% to 13.71%, demonstrating the benefits of spike-based sensing and policy dynamics. We further validate the deployability of our spike-based sensing method on the Thruster-V2 neuromorphic chip. This physical hardware validation shows that SpikingNav can be instantiated on a real neuromorphic substrate for cyber-physical systems.

Jiahong Zhang, Sijun Shen, Dehua Wu +5

Institute of Automation, Chinese Academy of Sciences, Beijing 100045, China · School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing 100049, China · State Key Laboratory of Media Convergence and Communication, Communication University of China, Beijing 100024, China +2

Embodied NavigationSpiking Neural NetworksNeuromorphic Computing
AskPDF
Mar 24, 2026

Event-Driven Neuromorphic Vision Enables Energy-Efficient Visual Place Recognition

Reliable visual place recognition (VPR) under dynamic real-world conditions is critical for autonomous robots, yet conventional deep networks remain limited by high computational and energy demands. Inspired by the mammalian navigation system, we introduce SpikeVPR, a bio-inspired and neuromorphic approach combining event-based cameras with spiking neural networks (SNNs) to generate compact, invariant place descriptors from few exemplars, achieving robust recognition under extreme changes in illumination, viewpoint, and appearance. SpikeVPR is trained end-to-end using surrogate gradient learning and incorporates EventDilation, a novel augmentation strategy enhancing robustness to speed and temporal variations. Evaluated on two challenging benchmarks (Brisbane-Event-VPR and NSAVP), SpikeVPR achieves performance comparable to state-of-the-art deep networks while using 50 times fewer parameters and consuming 30 and 250 times less energy, enabling real-time deployment on mobile and neuromorphic platforms. These results demonstrate that spike-based coding offers an efficient pathway toward robust VPR in complex, changing environments.

Geoffroy Keime, Nicolas Cuperlier, Benoit R. Cottereau

IPAL, CNRS IRL 2955, Singapore · Univ Toulouse, CNRS, CerCo UMR 5549, Toulouse, France · Laboratoire ETIS UMR 8051, CY Cergy-Paris Université, ENSEA, CNRS, Cergy, France

Visual Place RecognitionNeuromorphic ComputingSpiking Neural NetworksEfficient Neural Network InferenceEvent-Based Vision
AskPDF
May 19, 2026

Neuromorphic Control of a Flapping-Wing Robot on Resource-Constrained Hardware

Flapping-Wing Micro Aerial Vehicles (FWMAVs) provide exceptional maneuverability and aerodynamic efficiency but pose significant challenges for onboard control due to nonlinear dynamics and stringent Size, Weight, and Power (SWaP) constraints, as exemplified by a butterfly-inspired robot less than 30 gram. To this end, we present a hierarchical neuromorphic control framework that enables fully onboard, closed-loop flight on a widely available, resource-constrained ESP32 microcontroller with a unit cost of approximately $5. Specifically, our method deploys two lightweight Spiking Neural Networks (SNNs) onboard: one for state estimation from raw sensory feedback and another for control via modulation of a Central Pattern Generator (CPG) for wing actuation. Trained by imitation learning, the system achieves stable pitch and heading angle tracking during untethered real-world flight. Experimental results further reveal that the SNN-based controller reduces latency by 36% (1059us to 680us) and power by 18% (0.033W to 0.027W) for inference compared to the conventional Artificial Neural Network (ANN) baseline, demonstrating the viability of spike-based computation without specialized hardware. To the best of our knowledge, this work constitutes the first demonstration of fully onboard neuromorphic control for autonomous flight of a FWMAV, highlighting the potential of SNNs to enable energy-efficient autonomy under stringent SWaP constraints. Visual abstract: http://bit.ly/4nI8ECY Code: https://anonymous.4open.science/r/Espikify-76E3/

Rim El Filali, Chenrui Feng, Chao Gao +1

Institute for AI Industry Research (AIR), Tsinghua University, Beijing, PR China · Department of Computer Science and Technology, Tsinghua University, Beijing, PR China · Xinchen Qihang Inc., Beijing, PR China

Efficient Neural Network InferenceAerial RoboticsSpiking Neural NetworksNeuromorphic Computing
AskPDF