Fly360: Omnidirectional Obstacle Avoidance within Drone View

Xiangkai Zhang; Dizhe Zhang; WenZhuo Cao; Zhaoliang Wan; Yingjie Niu; Lu Qi; Xu Yang; Zhiyong Liu

cs.ROMar 6, 2026

Fly360: Omnidirectional Obstacle Avoidance within Drone View

Authors: Xiangkai Zhang, Dizhe Zhang, WenZhuo Cao, Zhaoliang Wan, Yingjie Niu, Lu Qi, Xu Yang, Zhiyong Liu

Organizations: Institute of Automation Chinese Academy of Sciences, Beijing, China · Insta360 Research, Shenzhen, China · Wuhan University,Wuhan, China.

Abstract

Obstacle avoidance in unmanned aerial vehicles (UAVs), as a fundamental capability, has gained increasing attention with the growing focus on spatial intelligence. However, current obstacle-avoidance methods mainly depend on limited field-of-view sensors and are ill-suited for UAV scenarios which require full-spatial awareness when the movement direction differs from the UAV's heading. This limitation motivates us to explore omnidirectional obstacle avoidance for panoramic drones with full-view perception. We first study an under explored problem setting in which a UAV must generate collision-free motion in environments with obstacles from arbitrary directions, and then construct a benchmark that consists of three representative flight tasks. Based on such settings, we propose Fly360, a two-stage perception-decision pipeline with a fixed random-yaw training strategy. At the perception stage, panoramic RGB observations are input and converted into depth maps as a robust intermediate representation. For the policy network, it is lightweight and used to output body-frame velocity commands from depth inputs. Extensive simulation and real-world experiments demonstrate that Fly360 achieves stable omnidirectional obstacle avoidance and outperforms forward-view baselines across all tasks. Our model is available at https://zxkai.github.io/fly360/

Figures & tables

Figure 2: Overview of the experimental setting. Top: The three representative tasks used to evaluate omnidirectional obstacle avoidance:( a ) Hovering maintenance, where the UAV maintains a defined position and orientation while avoiding nearby obstacles; ( b ) Dynamic target following, where the UAV tracks a moving object while reacting to dynamic obstacles; and ( c ) Fixed-trajectory filming, where the UAV follows a predefined path around a target while maintaining camera orientation. Bottom: The four high-fidelity simulation environments used in our evaluation, including ( d ) Park, ( e ) Forest, ( f ) Urban Street, and ( g ) Factory.

Figure 3: The proposed framework unifies panoramic perception and policy learning for omnidirectional UAV obstacle avoidance. Panoramic RGB observations are first processed by a panoramic depth estimation network to produce a depth map, which is then downsampled for policy inference. The policy network fuses low-resolution depth with UAV states to predict body-frame velocity commands. Training is conducted in a differentiable simulator while inference executes the predicted commands through the onboard velocity controller and rotor-level control.

Figure 4: Illustration of the observation vector components used in the policy network. The four components include the relative goal direction $\mathbf{d}_{\text{goal}}$ , current velocity $\mathbf{v}_{t}$ , upward orientation $\mathbf{q}^{\text{up}}_{t}$ , and predefined safety radius $r$ .

Figure 5: Illustration of the proposed fixed random-yaw training strategy. In conventional free-yaw training (top), the UAV’s yaw continuously aligns with the direction of motion, and the onboard camera (yellow cone) only observes a limited forward field of view. In contrast, our training(bottom) randomly samples a yaw angle at the beginning of each rollout and keeps it constant throughout the episode. The panoramic camera (blue region) provides a full $360^{\circ}$ field of view.

Scene	View	Method	#Objs = 3				#Objs = 6
			2.5 m/s		5.0 m/s		2.5 m/s		5.0 m/s
			SR ( $\uparrow$ )	CT ( $\downarrow$ )	SR ( $\uparrow$ )	CT ( $\downarrow$ )	SR ( $\uparrow$ )	CT ( $\downarrow$ )	SR ( $\uparrow$ )	CT ( $\downarrow$ )
Park	Forward-view	Zhang et al. (2025)	0/10	3.48	0/10	3.45	0/10	5.18	0/10	5.11
	Forward-view	Bhattacharya et al. (2025)	0/10	9.58	0/10	7.35	0/10	15.14	0/10	10.46
	Multi-view	Liu et al. (2024)	0/10	12.86	0/10	11.58	0/10	20.16	0/10	19.77
	Multi-view	Liu et al. (2024) ∗	0/10	1.11	0/10	1.33	0/10	1.49	0/10	2.45

Table 1: Quantitative results for hovering maintenance in park and urban street scenes. Each entry reports success rate (SR) and collision time (CT, s) under two obstacle densities (3, 6) and two obstacle speeds (2.5, 5.0 m/s).

Scene	View	Method	Target Speed
			1.5 m/s		3.0 m/s
			SR( $\uparrow$ )	CT ( $\downarrow$ )	SR( $\uparrow$ )	CT( $\downarrow$ )
Forest	Forward-view	Zhang et al. (2025)	1/10	10.19	0/10	6.39
	Forward-view	Bhattacharya et al. (2025)	0/10	38.90	0/10	60.20
	Multi-view	Liu et al. (2024)	0/10	29.60	0/10	27.45
	Multi-view	Liu et al. (2024) ∗	0/10	13.00	1/10	2.46

Table 2: Quantitative results for dynamic target following in forest and factory scenes. Each entry reports the success rate (SR) and collision time (CT, s) under two target speeds (1.5, 3.0 m/s).

Scene	View	Method	Obstacle Speed
			3.0 m/s		6.0 m/s
			SR( $\uparrow$ )	CT ( $\downarrow$ )	SR( $\uparrow$ )	CT( $\downarrow$ )
Park	Forward-view	Zhang et al. (2025)	1/10	52.04	0/10	54.02
	Forward-view	Bhattacharya et al. (2025)	0/10	45.89	0/10	41.68
	Multi-view	Liu et al. (2024)	0/10	57.19	0/10	73.70
	Multi-view	Liu et al. (2024) ∗	0/10	4.98	0/10	6.86

Table 3: Quantitative results for fixed-trajectory filming in park and forest scenes. Each entry reports success rate (SR) and collision time (CT, s) under two obstacle speeds (3.0, 6.0 m/s).

$\sigma$	Filming (Forest)		Hovering (Park)		Following (Factory)
$\sigma$	SR $\uparrow$	CT (s) $\downarrow$	SR $\uparrow$	CT (s) $\downarrow$	SR $\uparrow$	CT (s) $\downarrow$
0	10/10	0.00	1/10	0.90	10/10	0.00
0.05	10/10	0.00	1/10	1.17	10/10	0.00
0.10	9/10	0.07	2/10	1.01	10/10	0.00
0.20	8/10	0.12	2/10	1.80	9/10	0.18

Table 4: Robustness of Fly360 to depth estimation errors under different noise levels. Gaussian noise is added to the depth map.

Method	Params (M)	Latency (ms)	FPS (Hz)
Forward-view ( Zhang et al., 2025 )	2.1	21.0	47.6
Forward-view ( Bhattacharya et al., 2025 )	14.3	105.7	9.5
Multi-view ( Liu et al., 2024 )	9.3	128.1	7.8
Multi-view ( Liu et al., 2024 ) ∗	9.2	130	7.7
Ours (Panoramic)	7.1	22.4	44.6

Table 5: Comparison of model complexity and runtime performance on a desktop GPU (RTX 3090). The parameter count refers only to the policy network, while the latency and FPS reflect the total end-to-end system, including depth estimation and control.

Figure 6: Real-world demonstration of omnidirectional avoidance during hovering. (a) Quadrotor platform equipped with two fisheye cameras whose images are automatically stitched into a panoramic view; due to commercial restrictions, detailed appearances are not shown. (b)-(d) Dynamic obstacle approaching from the rear, front, and side.

Figure 7: A challenging task scenario in which a human continuously pursues the UAV, demonstrating the UAV’s ability to evade a dynamic obstacle consistently.

Simple Hovering				Challenging Chasing
SR $\uparrow$	P (ms)	D (ms)	C (ms)	SR $\uparrow$	P (ms)	D (ms)	C (ms)
5/5	60	12	18	3/5	57	10	21

Table 6: Quantitative results and latency of Fly360 in real-world experiments. P: perception (RGB

\rightarrow

depth), D: decision (policy inference), C: control interface.

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Layer	Operation	Config	Output Size
0	Input	ERP depth (1 ch)	$1\times H\times W$
1	SphereConv2d + LeakyReLU	(32, $3{\times}3$ , s=2)	$32\times H_{1}\times W_{1}$
2	SphereConv2d + LeakyReLU	(64, $3{\times}3$ , s=2)	$64\times H_{2}\times W_{2}$
3	Conv2d + LeakyReLU	(64, $3{\times}3$ , s=1)	$64\times H_{3}\times W_{3}$
4	Conv2d + LeakyReLU	(64, $2{\times}2$ , s=2)	$64\times H_{4}\times W_{4}$
5	Conv2d + LeakyReLU	(128, $3{\times}3$ , s=1)	$128\times H_{5}\times W_{5}$

Appendix

Table 7: Architecture of the Fly360 panoramic policy network. The model receives a

64{\times}128

equirectangular depth map and outputs a 3D body–frame velocity command.

Layer	Operation	Config	Output Size
0	Input	Depth (1 ch)	$1{\times}H{\times}W$
1	Conv2d + LeakyReLU	(32, $2{\times}2$ , s=2)	$32{\times}H_{1}{\times}W_{1}$
2	Conv2d + LeakyReLU	(64, $3{\times}3$ , s=1)	$64{\times}H_{2}{\times}W_{2}$
3	Conv2d + LeakyReLU	(128, $3{\times}3$ , s=1)	$128{\times}H_{3}{\times}W_{3}$
4	Flatten	–	$D_{\text{flat}}$
5	Linear (visual proj)	$(D_{\text{flat}}\rightarrow 192)$	$192$

Appendix

Table 8: Architecture of the forward-view baseline network. The model receives a single-view depth image and outputs the 3D velocity command.

Layer	Operation	Config	Output Size
0	Input + Resize	Depth (1 ch) $\rightarrow 60{\times}90$	$1{\times}60{\times}90$
1	MixTransformer Stage-1	$(1\rightarrow 32)$ , patch=7, stride=4, pad=3, heads=1, layers=2	$32{\times}H_{1}{\times}W_{1}$
2	MixTransformer Stage-2	$(32\rightarrow 64)$ , patch=3, stride=2, pad=1, heads=2, layers=2	$64{\times}H_{2}{\times}W_{2}$
3	Feature Align	PixelShuffle( $\times 2$ ) on Stage-2 + Upsample Stage-1 $\rightarrow 16{\times}24$	$48{\times}16{\times}24$
4	Conv2d (downsample)	$(48\rightarrow 12)$ , $3{\times}3$ , pad=1	$12{\times}16{\times}24$
5	Flatten + Linear (visual proj)	$(4608\rightarrow 512)$	$512$

Appendix

Table 9: Architecture of the transformer-based forward-view baseline ( Bhattacharya et al., 2025 ) . The model receives a single-view depth image and outputs the 3D velocity command.

Layer	Operation	Config	Output Size
0	Input	Depth (6 or 4 ch)	$C{\times}H{\times}W$
1	Conv2d + LeakyReLU	(256, $3{\times}3$ , s=2)	$256{\times}H_{1}{\times}W_{1}$
2	Conv2d + LeakyReLU	(128, $3{\times}3$ , s=1)	$128{\times}H_{2}{\times}W_{2}$
3	Conv2d + LeakyReLU	(128, $3{\times}3$ , s=2)	$128{\times}H_{3}{\times}W_{3}$
4	Conv2d + LeakyReLU	(128, $3{\times}3$ , s=1)	$128{\times}H_{4}{\times}W_{4}$
5	Conv2d + LeakyReLU	(128, $3{\times}3$ , s=2)	$128{\times}H_{5}{\times}W_{5}$

Appendix

Table 10: Architecture of the multi-view baseline network. The model receives six perspective depth maps and outputs the 3D body–frame velocity command.

Category	Term	Coefficient / Parameter	Value
Tracking	$\mathcal{L}_{\mathrm{trk}}$	$\lambda_{\mathrm{trk}}$	$1.0$
Safety	$\mathcal{L}_{\mathrm{avoid}}$	$\lambda_{\mathrm{safe}}$ (avoid)	$1.5$
	$\mathcal{L}_{\mathrm{collide}}$	$\lambda_{\mathrm{safe}}$ (collide)	$2.0$
	—	$\gamma$	$32$
Smoothness	$\mathcal{L}_{\mathrm{acc}}$	$\lambda_{\mathrm{smooth}}$ (acc)	$0.01$
	$\mathcal{L}_{\mathrm{jerk}}$	$\lambda_{\mathrm{smooth}}$ (jerk)	$0.001$

Appendix

Table 11: Loss coefficients and related hyperparameters used in the training script.

Figure 8: Fixed-trajectory filming in a dense forest. Representative frames from forward-view, multi-view, and Fly360 policies. Fly360 follows the predefined filming path while maintaining collision-free motion, whereas the baselines exhibit frequent lateral failures or inconsistent obstacle responses.

Figure 9: Dynamic target following in an industrial environment. Fly360 maintains stable tracking and consistent obstacle clearance, while forward-view and multi-view baselines frequently collide in tight spaces.

Figure 10: Hovering maintenance in a park environment. Fly360 reacts promptly to obstacles approaching from any direction and consistently returns to the target hover point, while the baselines show delayed or incomplete avoidance responses.

Scene	Method	#Objs = 3				#Objs = 6
		2.5 m/s		5.0 m/s		2.5 m/s		5.0 m/s
		SR $\uparrow$	CT $\downarrow$	SR $\uparrow$	CT $\downarrow$	SR $\uparrow$	CT $\downarrow$	SR $\uparrow$	CT $\downarrow$
Park	Ours (joint depth–policy training)	0/10	39.20	0/10	41.33	0/10	44.17	0/10	44.28
	Ours w/o fixed-yaw training	3/10	1.11	1/10	1.60	0/10	3.18	3/10	4.85
	Ours	6/10	0.13	7/10	0.54	1/10	0.90	1/10	1.84
Urban Street	Ours (joint depth–policy training)	0/10	26.30	0/10	31.40	0/10	20.13	0/10	32.17

Appendix

Table 12: Quantitative results for hovering maintenance in park and urban street scenes. Each entry reports success rate (SR) and collision time (CT, s) under two obstacle densities (3, 6) and two obstacle speeds (2.5, 5.0 m/s).

Obstacle Size $r$ (m)	0.01	0.05	0.10	0.20	0.30	0.40	0.50
CT (s) $\downarrow$	4.10	3.25	2.83	2.20	3.43	2.23	2.88

Appendix

Table 13: Robustness of Fly360 under varying obstacle sizes in the hovering maintenance task. All experiments are conducted in the same environment with obstacle speed fixed at 5.0 m/s. Each entry reports the average collision time (CT, s) over ten trials.

UAV Navigation

Unmanned aerial vehicle (UAV) navigation develops methods for locating and guiding drones safely through known, unknown, and changing environments.

See all 272 papers

Omni-IGN: Onboard Image-Goal Navigation for Drones with Omnidirectional Vision
Oct 27, 2026Zichen Yan, Rui Huang, Shenao Wang +3UAV Navigation Image-Goal Navigation
Department of Electrical and Computer Engineering, National University of Singapore, Singapore.
OmniRisk: Omnidirectional Trajectory-Risk Learning for Agile Quadrotor Dynamic Avoidance
Sep 16, 2026Yifan He, Yang Liu, Wenhao Zhao +8Aerial Robotics Safe Motion Planning
Differential Robotics, Hangzhou, China · Zhejiang University, Hangzhou, China
Learning-Based Dynamic Obstacle Avoidance for a UAV Using Only Three Range Sensors
Sep 13, 2026Mohammad Reza Ranjbar Divkoti, A. Pedro AguiarRobot Navigation UAV Navigation
SYSTEC-ARISE Research Center for Systems and Technologies, Faculty of Engineering, University of Porto, 4200-465, Porto, Portugal.

Oct 27, 2026J/K move · Enter open · S save

Omni-IGN: Onboard Image-Goal Navigation for Drones with Omnidirectional Vision

Zichen Yan, Rui Huang, Shenao Wang, Yucong Wang+2

Department of Electrical and Computer Engineering, National University of Singapore, Singapore.

UAV Navigation Image-Goal Navigation Aerial Robotics

Open paper AskPDF arXiv

Image-goal navigation (ImageNav) requires an agent to reach a location specified by a goal image in an unknown environment. For drones, this task demands exploration, goal- directed navigation, and collision avoidance under limited onboard computing resources. Existing aerial ImageNav methods are built on single-view observations, requiring additional rotation and exploration to search for goal-relevant cues outside the field of view. In this paper, we present Omni-IGN, a fully onboard ImageNav system that integrates quad-fisheye perception, an attention-based reinforcement learning (RL) policy, and a collision-aware safety layer. The policy adaptively fuses goal-relevant features across multiple views to provide directional guidance for continuous navigation. The quad-fisheye RGB images support both ImageNav and obstacle avoidance without a dedicated depth sensor. On Habitat benchmarks, Omni-IGN achieves the best performance among continuous-action baselines. Moreover, flight tests in IsaacSim validate the effectiveness of integrating learning-based depth estimation with the action safety layer for collision avoidance. Finally, real-world experiments demonstrate zero-shot sim-to-real transfer without any fine-tuning or external localization. The code and model checkpoints will be released upon acceptance.

Abstract

UAV Navigation

Explore similar work

Omni-IGN: Onboard Image-Goal Navigation for Drones with Omnidirectional Vision

OmniRisk: Omnidirectional Trajectory-Risk Learning for Agile Quadrotor Dynamic Avoidance

Learning-Based Dynamic Obstacle Avoidance for a UAV Using Only Three Range Sensors