cs.ROOct 15, 2026
SaveTowards bio-inspired visual navigation: a neuromorphic navigation paradigm for MAVs
Organizations: Faculty of Robot Science and Engineering, Northeastern University, Shenyang 110819, China. · College of Information Science and Engineering, Northeastern University, Shenyang 110819, China.
Abstract
Biological visual navigation achieves remarkable robustness at extremely low power, whereas frame-based robotic systems relying on artificial neural networks remain limited in both respects, particularly for energy-constrained micro aerial vehicles (MAVs) operating in complex environments. Inspired by the event-driven, sparse, and asynchronous principles of biological visual systems, neuromorphic sensing and computation provide a promising alternative for robust and energy-efficient navigation. In this paper, we propose the first end-to-end event-driven neuromorphic paradigm for MAV visual navigation in complex environments by integrating neuromorphic vision, spiking neural networks, and neuromorphic processors. The proposed method first extracts the environmental representation from event streams using a spiking perceptual representation module. This representation is fed into a spiking navigation module, where guidance learning and population coding enable efficient policy learning with few time steps. The navigation module then outputs a waypoint converted into executable trajectory points. Experiments show that the proposed neuromorphic method achieves navigation performance comparable to ANN-based methods while markedly reducing power consumption. The spiking perceptual module consumes only 19.57% of the power of its ANN counterpart, while the spiking navigation module deployed on Intel Loihi consumes only 12.37% of that required by its ANN counterpart running on NVIDIA Orin NX. Beyond prior neuromorphic systems focused mainly on relatively simple navigation subtasks, we demonstrate a complete neuromorphic visual navigation system spanning simulation and real-world deployment. Our results establish the feasibility of the neuromorphic paradigm for autonomous navigation and provide a bio-inspired pathway toward robust and low-power MAV navigation.
Figures & tables
Figure 1: Visual navigation mechanisms across biological, conventional, and neuromorphic paradigms. ( A ) The biological visual navigation paradigm relies on the retina and neural circuits. Visual information is sensed asynchronously at the pixel level and processed through sparse, event-driven neural activity to produce navigation behaviors. ( B ) Comparison between conventional and neuromorphic visual navigation paradigms for MAVs. The conventional visual navigation paradigm relies on frame-based cameras and artificial neural networks. Visual information is acquired synchronously as image frames and processed through dense, clock-driven computation to generate navigation actions. The neuromorphic visual navigation paradigm employs event-based vision sensors and spiking neural networks. Inspired by biological visual processing, it captures visual information asynchronously at the pixel level and processes it through sparse, event-driven computation to generate navigation actions.
Figure 2: End-to-end event-driven neuromorphic MAV navigation system. ( A ) Architecture of the proposed end-to-end event-driven neuromorphic navigation system. The system comprises a spiking perceptual representation module and a spiking navigation policy module. The perceptual module converts event streams into compact, geometry-aware features, while the navigation policy module maps these visual features, together with the robot state and goal direction, to navigation actions. ( B ) Key hardware components of the neuromorphic system. The platform consists of a neuromorphic vision sensor (Prophesee EVK4), an NVIDIA Jetson Orin NX, an x86 host computer, a neuromorphic processor (Intel Kapoho Bay), and a micro aerial vehicle. The NVIDIA Jetson Orin NX communicates wirelessly with the x86 host computer. ( C ) Deployment and communication pipeline of the proposed system. Owing to the limited neuron models supported by Intel Kapoho Bay, the spiking perceptual representation module is deployed on the NVIDIA Jetson Orin NX, whereas the spiking navigation policy module is deployed on Intel Kapoho Bay. The Jetson Orin NX acquires the event stream and transmits control commands to the Pixhawk flight controller. The Jetson Orin NX and the x86 host computer exchange system information wirelessly. The x86 host computer exchanges spike trains with Intel Kapoho Bay through NxSDK.
Figure 3: Training pipeline of the proposed end-to-end event-driven neuromorphic navigation method. Continuous grayscale image sequences are first transformed into asynchronous event streams using an event simulator. Depth estimation is then introduced as an auxiliary task to enable geometry-aware spiking features learning from event streams. The learned event features are fused with the MAV state and goal direction, and then encoded into spike trains via population coding. The resulting spikes are processed by the spiking navigation network and decoded to predict the action. Finally, collision loss, smoothness loss, and goal-reaching loss are defined based on the predicted action, providing physically meaningful supervision for end-to-end optimization of all network modules.
| Scheme | Max Speed (m/s) | Success Rate | Average Distance (m) | Average Time (s) |
|---|---|---|---|---|
| ANN-based Ours | 0.5 | 45/48 45/48 | 23.35 24.35 | 67.60 66.14 |
| ANN-based Ours | 1.0 | 47/48 46/48 | 24.07 24.87 | 33.87 32.45 |
| ANN-based Ours | 2.0 | 47/48 48/48 | 24.82 25.98 | 16.85 17.78 |
Table 1: Quantitative comparison between the ANN-based method and the proposed method. Navigation performance is evaluated at maximum flight speeds of 0.5, 1.0, and 2.0 m/s across six simulation environments. Success rate denotes the number of successful trials out of 48 total navigation trials, and average distance and average time denote the mean trajectory length and navigation time of the evaluated trials, respectively.
Figure 4: Navigation trajectories of the proposed method in simulation environments. (A) Continuous navigation performance is evaluated by requiring the MAV to sequentially pass through multiple waypoints in a simulated environment. (B) Robustness is evaluated by placing target points in eight directions within each of six testing environments.
Figure 5: Average trajectory distance comparison between our method and the ANN-based method across six environments. (A) Average trajectory distance at a maximum speed of 0.5 m/s. (B) Average trajectory distance at a maximum speed of 1 m/s. (C) Average trajectory distance at a maximum speed of 2 m/s.
Figure 6: Average navigation time comparison between our method and the ANN-based method across six environments. (A) Average navigation time at a maximum speed of 0.5 m/s. (B) Average navigation time at a maximum speed of 1 m/s. (C) Average navigation time at a maximum speed of 2 m/s.
Figure 7: Navigation trajectories comparison between the proposed method and the ANN-based method. The navigation trajectories of the two methods are compared across six different simulation environments, with the trajectories of the proposed method shown in red and those of the ANN-based method shown in blue.
Figure 8: Navigation trajectories at different maximum speeds. The green, blue, and red lines indicate the navigation trajectories at maximum speeds of 0.5 m/s, 1.0 m/s, and 2.0 m/s, respectively.
Figure 9: Comparison of navigation trajectories with and without depth supervision. The green and red lines indicate trajectories without and with depth supervision, respectively. Removing depth supervision leads to collisions, ground crashes, and substantial target deviations.
Figure 10: Navigation performance of the proposed method in real-world environments. Indoor and outdoor navigation tests were conducted by placing the target point behind a series of obstacles, requiring the MAV to avoid the obstacles and successfully reach the target.
| Device | Scheme | Dynamic Power (mW) | Inference Rate (s -1 ) | Energy per Inference (mJ/inf.) |
|---|---|---|---|---|
| NVIDIA GeForce RTX 4090 | ANN-based | 43000 | 2078 | 20.69 |
| NVIDIA Jetson Orin NX (25 W) | ANN-based | 5275 | 805 | 6.55 |
| NVIDIA Jetson Orin NX (10 W) | ANN-based | 1958 | 649 | 3.02 |
| Intel Kapoho Bay | Ours | 140 | 172 | 0.81 |
Table 2: Per-inference energy consumption of SNN-based and ANN-based navigation networks across different devices. Dynamic power is defined as the difference between the operating and idle power of each device. The inference rate denotes the number of forward inferences performed per second, and the energy per inference is calculated by dividing the dynamic power by the inference rate. The NVIDIA Jetson Orin NX is evaluated in both high-performance mode (25 W) and power-saving mode (10 W).
Figure 11: Operations and per-inference energy consumption of ANN-based and SNN-based perceptual representation networks. The ANN-based network is evaluated based on multiply–accumulate operations (MACs), whereas the SNN-based network is evaluated based on spike-triggered accumulate operations (ACs). Per-inference energy consumption is estimated from the corresponding operation counts using energy costs of 4.6 pJ per MAC and 0.9 pJ per AC under a 45-nm CMOS technology assumption.
Figure S1 : Composition of the training dataset. Each training dataset includes a complete point cloud map of the environment, along with continuous grayscale images, corresponding depth maps, and MAV poses collected from multiple locations.
Figure S2 : Event–depth calibration for fine-tuning in the real world. The Prophesee EVK4 and Intel RealSense D435i are rigidly mounted together to enable joint camera calibration. The figure also shows representative event frames and their corresponding depth maps acquired in indoor and outdoor environments for real-world fine-tuning of the spiking depth estimation network.
| Types | Parameters | Descriptions | Values |
| Training epoch | – | 100 | |
| Experimental setting | Batch size | – | 16 |
| Initial learning rate | |||
| Trajectory time constant | |||
| The coefficient for smooth item | |||
| Loss function | The coefficient for collision item |
Table S1 : Parameter settings used in the experiments. The table summarizes the principal hyperparameters used for network training, trajectory-loss computation, spiking depth estimation, spiking feature extraction, and the spiking navigation network.
| Module | Operations | Corresponding MACs | Average input firing rate | ACs |
| Conv-1 | 4.72M | 0.3696 | 6.98M | |
| Encode-1 | Conv-2 | 37.75M | 0.2234 | 33.73M |
| Conv-1 | 18.87M | 0.3711 | 28.01M | |
| Encode-2 | Conv-2 | 37.75M | 0.1968 | 29.72M |
| Conv-1 | 18.87M | 0.3613 | 27.27M | |
| Encode-3 | Conv-2 | 37.75M | 0.2102 | 31.74M |
Table S2 : Layer-wise computational statistics of the spiking U-Net. For each layer, the table reports the corresponding ANN multiply–accumulate operations (MACs), the average input firing rate measured in the SNN, and the resulting number of spike-triggered accumulate operations (ACs). The AC count is obtained by weighting the corresponding layer operations by the average input firing rate and the number of SNN time steps.
| Module | Operations | Corresponding MACs | Average input firing rate | ACs |
| Stem | Conv | 12.85M | 1 | 51.40M |
| Conv-1 | 9.44M | 0.2870 | 10.84M | |
| ResBlock-1.1 | Conv-2 | 9.44M | 0.2672 | 10.09M |
| Conv(Downsample) | 1.05M | 0.2870 | 1.21M | |
| ResBlock-1.2 | Conv-1 | 9.44M | 0.4588 | 17.32M |
| Conv-2 | 9.44M | 0.2174 | 8.21M |
Table S3 : Layer-wise computational statistics of the spiking feature extraction network. For each layer, the table reports the corresponding ANN multiply–accumulate operations (MACs), the average input firing rate of the SNN, and the resulting number of spike-triggered accumulate operations (ACs). The AC count is calculated based on the corresponding operation count, average input firing rate, and number of SNN time steps.
Figure S3 : Event frames visualization across different environments. (A) Simulated event frames. (B) Event frames captured in indoor environments. (C) Event frames captured in outdoor environments. For improved visualization, the event-frame values are amplified by a factor of 10 and then displayed as positive and negative deviations from a baseline intensity value of 127.
Figure S4 : Visualization of predicted depth maps. We present the depth ground truth, event frames, and predicted depth maps in simulation, indoor, and outdoor environments, respectively. For improved visualization, the event-frame values are amplified by a factor of 10 and then displayed as positive and negative deviations from a baseline intensity value of 127.
More in this topic
Visual Navigation
Visual navigation enables robots and embodied agents to reach goals by interpreting camera images and acting in unfamiliar or changing environments.
See all 71 papers