Image-goal navigation (ImageNav) requires an agent to reach a location specified by a goal image in an unknown environment. For drones, this task demands exploration, goal- directed navigation, and collision avoidance under limited onboard computing resources. Existing aerial ImageNav methods are built on single-view observations, requiring additional rotation and exploration to search for goal-relevant cues outside the field of view. In this paper, we present Omni-IGN, a fully onboard ImageNav system that integrates quad-fisheye perception, an attention-based reinforcement learning (RL) policy, and a collision-aware safety layer. The policy adaptively fuses goal-relevant features across multiple views to provide directional guidance for continuous navigation. The quad-fisheye RGB images support both ImageNav and obstacle avoidance without a dedicated depth sensor. On Habitat benchmarks, Omni-IGN achieves the best performance among continuous-action baselines. Moreover, flight tests in IsaacSim validate the effectiveness of integrating learning-based depth estimation with the action safety layer for collision avoidance. Finally, real-world experiments demonstrate zero-shot sim-to-real transfer without any fine-tuning or external localization. The code and model checkpoints will be released upon acceptance.
Figures & tables
Fig. 1: System design and data flow of Omni-IGN. The drone is equipped with a Jetson Orin NX 16G onboard computer, an OAK-4P-New quad-fisheye camera rig, and a Livox Mid-360s LiDAR. Rectified images from the four fisheye cameras are fed to the RL navigation policy, while overlapping views between the front camera and its left and right neighbors are used for stereo depth estimation and obstacle avoidance. The velocity command passes through a safety layer before being sent to the flight controller. Meanwhile, LiDAR provides state estimates required for closed-loop control.
Fig. 2: Overview of the navigation model. The RGB image from each direction is stacked with the target image and encoded by a shared ResNet. The resulting per-view features are adaptively fused by a self-attention module and passed to a GRU together with the previous action. The actor-critic head then predicts continuous actions.
Fig. 3: Stereo depth estimation pipeline. (a) Fisheye images are frame-synchronized and captured at 30 Hz. (b) The front camera forms two stereo pairs, P1=(C2,C1) and P2=(C2,C3) , with the left and right cameras, respectively. Each pair is rectified into aligned pinhole images. (c) Fast-FoundationStereo produces disparity maps D1 and D2 for both pairs. (d) The disparity maps are converted to depth and stitched together, then reduced over the central 90∘ FoV to a 16-bin range scan for collision prediction.
Fig. 4: Stereo rectification of the quad-fisheye views. (a) Rectified frame construction for the two stereo pairs. For P1 , C1 and C2 supply the left and right rectified views, respectively. (b) Real fisheye observations and their rectified views for P1 . The dashed curves in the fisheye images are mapped to aligned image rows.
Fig. 5: Action safety layer based on estimated depth for collision avoidance.
Fig. 6: Comparison experiments between Omni-IGN and single-view baseline on the same episodes from HM3D, Gibson, and MP3D. Left: navigation trajectories on the top-down map. Right: the goal image, the single-view and panoramic-view observations at the endpoint.
Method
Backbone
Sensor(s)
Action Space
SPL( ↑ )
SR( ↑ )
ZER [ 14 ]
ResNet9
RGB
Discrete
21.6%
29.2%
OVRL-V2 [ 15 ]
ViT-Base
RGB
Discrete
58.7%
82.0%
FGPrompt [ 16 ]
ResNet9
RGB
Discrete
66.5%
90.4%
Mem-Aug [ 21 ]
ResNet18
4 RGB
Discrete
56.0%
69.0%
VGM [ 20 ]
ResNet18
4 RGB
Discrete
64.0%
76.0%
TSGM [ 10 ]
ResNet18
4 RGB
Discrete
67.2%
81.1%
TABLE I: Comparison with SOTA methods on Gibson.
Methods
Action Space
MP3D
HM3D
SPL( ↑ )
SR( ↑ )
SPL( ↑ )
SR( ↑ )
ZER [ 14 ]
Discrete
10.8%
14.6%
6.3%
9.6%
TSGM [ 10 ]
Discrete
47.3%
58.0%
46.9%
64.1%
FGPrompt (1-RGB)
Discrete
50.4%
77.6%
49.6%
76.1%
FGPrompt (1-RGB)
Continuous
22.9%
59.4%
20.1%
56.5%
SIGN [ 4 ]
Continuous
35.0%
65.4%
30.9%
64.5%
TABLE II: Cross-domain evaluation on MP3D and HM3D.
Fig. 7: Reliability test in the IsaacSim pole environment. (a) The layout of the pole environment. (b) Flight trajectories for each start-end pair, with collision cases marked in red. (c) Stereo-based depth estimation at the location marked by the red star in (b). Top: left and right RGB views of the front-left and front-right stereo pairs and their estimated depth. Bottom: the two depth maps are stitched and cropped to form the range scan.
Navigation
Depth Error (m)
Pair
SR( ↑ )
SPL( ↑ )
Correction Rate(%)
Mean( ↓ )
Max( ↓ )
S1-E1
10/10
0.93
30.4
0.17
1.13
S1-E2
10/10
0.96
14.0
0.10
0.80
S1-E3
10/10
0.91
28.8
0.17
1.09
S2-E1
10/10
0.99
18.1
0.13
1.04
S2-E2
10/10
0.94
23.5
0.13
1.00
TABLE III: Reliability test of collision avoidance and depth estimation over 90 flights in IsaacSim.
Fig. 8: Evolution of view-attention weights during two real-world ImageNav flights in the Lab. Panels (I-a) and (II-a) show the trajectories and goal images. (I-b) and (II-b) show the four camera views and normalized attention scores at five sampled steps. (I-c) and (II-c) show the attention distributions over time. Red boxes highlight the view with the highest attention score.
Fig. 9: Evolution of view attention during a real-world ImageNav flight in a cluttered office. Attention is broadly distributed during exploration and becomes increasingly focused on the forward view during the final approach.
Fig. 10: Real-world stereo depth estimation in the lab and office environments. The left, forward, and right RGB views are shown together with the corresponding stitched stereo depth predictions.
Object-Goal Navigation (ObjectNav) requires an agent to autonomously explore an unknown environment and navigate toward target objects specified by a semantic label. While prior work has primarily studied zero-shot ObjectNav under 2D locomotion, extending it to aerial platforms with 3D locomotion capability remains underexplored. Aerial robots offer superior maneuverability and search efficiency, but also introduce new challenges in spatial perception, dynamic control, and safety assurance. In this paper, we propose AION for vision-based aerial ObjectNav without relying on external localization or global maps. AION is an end-to-end dual-policy reinforcement learning (RL) framework that decouples exploration and goal-reaching behaviors into two specialized policies. We evaluate AION on the AI2-THOR benchmark and further assess its real-time performance in IsaacSim using high-fidelity drone models. Experimental results show that AION achieves superior performance across comprehensive evaluation metrics in exploration, navigation efficiency, and safety. The project is available at https://github.com/Zichen-Yan/AION.
Zichen Yan, Yuchen Hou, Shenao Wang +3
Department of Electrical and Computer Engineering, National University of Singapore, Singapore.
Navigating a drone in unseen and cluttered environments requires reliable generalization to unseen scene layouts and understanding of environmental structure relative to the robot's capabilities. Previous methods, which assume the same environment configuration, often rely heavily on human-designed perception pipelines and predefined rules to guide the robot toward the target. This process is environment-dependent and generalizes poorly across environments. Inspired by animal navigation behavior, we design a navigation framework that navigates with a reinforcement-learning-based policy on top of a world-model-based environment understanding to overcome these issues. In addition, a sparse reward function without hand-crafted shaping terms is designed to avoid local minima traps and encourage yaw control behaviors. In simulation and on real drones, our method exhibits emergent capabilities for navigating complex, unseen environments and escaping local optima where other methods fail. In challenging maps, it achieves a 5.3% higher navigation success rate than best baseline. Furthermore, the proposed framework achieves effective sim-to-real transfer without any tuning during deployment. The code will be publicly available.
Zian Liu, Andong Yang, Chunkai Yang +3
Institute for AI Industry Research, Tsinghua University, Beijing, China. · Department of Electronic Engineering, Tsinghua University, Beijing, China. · School of Remote Sensing and Information Engineering, Wuhan University, Wuhan, China.
Obstacle avoidance in unmanned aerial vehicles (UAVs), as a fundamental capability, has gained increasing attention with the growing focus on spatial intelligence. However, current obstacle-avoidance methods mainly depend on limited field-of-view sensors and are ill-suited for UAV scenarios which require full-spatial awareness when the movement direction differs from the UAV's heading. This limitation motivates us to explore omnidirectional obstacle avoidance for panoramic drones with full-view perception. We first study an under explored problem setting in which a UAV must generate collision-free motion in environments with obstacles from arbitrary directions, and then construct a benchmark that consists of three representative flight tasks. Based on such settings, we propose Fly360, a two-stage perception-decision pipeline with a fixed random-yaw training strategy. At the perception stage, panoramic RGB observations are input and converted into depth maps as a robust intermediate representation. For the policy network, it is lightweight and used to output body-frame velocity commands from depth inputs. Extensive simulation and real-world experiments demonstrate that Fly360 achieves stable omnidirectional obstacle avoidance and outperforms forward-view baselines across all tasks. Our model is available at https://zxkai.github.io/fly360/
Xiangkai Zhang, Dizhe Zhang, WenZhuo Cao +5
Institute of Automation Chinese Academy of Sciences, Beijing, China · Insta360 Research, Shenzhen, China · Wuhan University,Wuhan, China.