DroneDiffusion: Uncertainty-Guided Diffusion Path Planner for UAV Navigation
Authors: Ivan Valuev, Iana Zhura, Valerii Serpiva, Miguel Altamirano Cabrera, Dzmitry Tsetserukou
Organizations: The authors are with the Intelligent Space Robotics Laboratory, Skolkovo Institute of Science and Technology, Moscow, Bolshoy Boulevard 30, bld. 1, 121205, Moscow, Russia.
We present DroneDiffusion, a perception-aware diffusion planner for local 3-D UAV trajectory generation that explicitly models and avoids failure zones caused by lens flare, overexposure, and reflective surfaces. A vision-language perceptual safety estimator produces a pixel-aligned reliability map that highlights regions of degraded sensing. A denoising diffusion probabilistic model (DDPM) is conditioned on this map together with the onboard RGB image, goal point, and initial velocity to generate egocentric 3-D trajectories. To enforce obstacle avoidance at inference time without retraining, we apply training-free CHOMP guidance over a 3-D Euclidean signed distance field. The model is trained entirely in simulation and deployed zero-shot on a real drone. Reliability conditioning raises the mean perceptual reliability along generated trajectories to 0.71, compared with 0.63 for a geometric baseline, while guidance halves the collision rate of the learned prior from 14.7% to 7.8%. Perceptual safety conditioning and geometric guidance are therefore complementary: treating sensor reliability as a first-class signal reduces incursions into unreliable regions while maintaining competitive navigation performance.
Figures & tables
Fig. 1 : Perception failures in indoor UAV flight. Left: RGB observation with severe lens flare. Middle: corresponding depth map with missing and phantom geometry in the corrupted region. Right: our safety map flags the unreliable zone (red), guiding trajectory generation (magenta) away from it.
Fig. 2 : DroneDiffusion pipeline. Left: the corrupted trajectory xt is denoised over n reverse-diffusion steps by a conditional 1-D U-Net, conditioned on the RGB observation, the AnyTraverse reliability map, the initial speed and the goal point, and corrected at each step by training-free obstacle and goal guidance. Right: scene and reliability tokens enter the U-Net bottleneck through two sequential cross-attention blocks, which keeps appearance and perceptual-reliability reasoning separate.
Prompt
Weight ωi
Transparent and specular geometry
a window
1.0
a glass surface
1.0
a transparent surface
1.0
a transparent obstacle
1.0
a mirror
1.0
TABLE I : Perceptual uncertainty prompts and pooling weights ω used in AnyTraverse.
Parameter
Value
Parameter
Value
Cold reverse steps
500
Refinement start t
300
Refinement steps
11
ESDF voxel size
0.05m
UAV radius
0.25m
Safety margin
0.10m
CHOMP η
0.7m
Obstacle coefficient
0.75
Maximum correction
0.08m
Anisotropy A
[0.8,1.0,1.0]
ρ0,ρ2,ρ3
1.0,1.0,0.2
Goal weight λg
0.25
TABLE II : Diffusion and geometric-guidance configuration.
Metric
Full (Ours)
w/o Guid.
w/o Guid. and Reliab.
APF
Collision (%) ↓
7.770±1.150
14.710±2.270
13.320±1.320
6.650±0.000
Min. clearance (m) ↑
0.618±0.013
0.602±0.014
0.610±0.016
0.613±0.000
Unsafe WP (%) ↓
12.890±1.550
14.590±1.780
16.590±1.430
20.520±0.000
Mean reliability ↑
0.714±0.016
0.730±0.013
0.689±0.015
0.626±0.000
Tail reliability ↑
0.522±0.033
0.538±0.005
0.518±0.018
0.390±0.000
Goal error (m) ↓
0.723±0.012
0.610±0.011
0.666±0.014
0.601±0.000
TABLE III : Trajectory-generation results on real drone observations, averaged over five seeds (mean and std across seeds, APF is deterministic). Best per row in bold.
Stage
Mean
Min.
Max.
Share
ESDF
18.38
16.84
21.17
10.13%
AnyTraverse
122.14
117.13
125.23
67.35%
SceneEncoder
5.23
5.06
5.78
2.88%
Generation + guidance (11 steps)
35.61
33.28
38.35
19.63%
Full pipeline (warm start)
181.36
173.87
186.21
100%
TABLE IV : Runtime breakdown on an NVIDIA RTX 4070 Super (12 GB).
Autonomous UAV navigation is conventionally solved by pipelines that separate perception, mapping, and planning into distinct stages, which propagates errors, accumulates latency, and requires environment-specific retuning. End-to-end generative models remove these interfaces by mapping raw observations directly to trajectories, but inherit a subtle failure mode: trained on clean data, they cannot recognise when an observation is unreliable, and treat degraded regions such as glass, mirrors, and overexposed surfaces as valid evidence for planning. We present a reliability-aware diffusion planner for 3D UAV navigation. It conditions trajectory generation on the observation together with a scene-level reliability heatmap that marks where perception cannot be trusted, produced by a lightweight network that distils the open-vocabulary reasoning of a vision-language model within the real-time planning budget. To generalise to unseen environments without retraining, we steer the denoising process with a differentiable two-stage ESDF cost that treats physical obstacles from depth and virtual obstacles from highly unreliable regions on equal footing. In simulation and on a real quadrotor, our planner produces markedly safer trajectories than a state-of-the-art diffusion baseline, reducing the obstacle-violation rate from 40.3% to 9.6% and raising the mean reliability of traversed regions from 0.588 to 0.925. Ablating the reliability term alone drops mean reliability from 0.898 to 0.783, confirming it as the decisive component, while distillation runs the framework up to 2 times faster than the full vision-language model.
Vision-based UAV navigation becomes challenging when navigation targets are distributed across complementary camera views and cannot be reliably observed from a single viewpoint. We propose AgenticDiffusion, an agentic multi-view UAV navigation framework that semantically coordinates first-person-view (FPV) and top-view observations for mission-level navigation. Given a natural-language instruction, AgenticDiffusion identifies the requested targets, selects the most appropriate camera view for each navigation task, determines the corresponding navigation goal, and invokes the appropriate view-conditioned diffusion planner for trajectory generation. The resulting trajectories are executed using Nonlinear Model Predictive Control (NMPC). AgenticDiffusion was evaluated in four real-world indoor scenarios, achieving an overall mission success rate of 80% across 40 physical-flight trials. In mixed-visibility scenarios, where the requested targets were distributed across FPV and top-view observations, coordinated multi-view navigation reduced average mission time by 50.8% relative to FPV-only navigation and by 26.8% relative to Top-only navigation. The semantic view-selection mechanism was also robust to lexical variation in target descriptions, achieving 100% accuracy across 66 test cases, compared with 63.64% for a confidence-based view-selection baseline. In a substantially larger Gazebo environment, AgenticDiffusion achieved a 90% mission success rate and completed the multi-stage mission, whereas the FPV-only and Top-only variants were unable to complete all requested navigation tasks.
Faryal Batool, Muhammad Ahsan Mustafa, Fawad Mehboob +2
Obstacle avoidance in unmanned aerial vehicles (UAVs), as a fundamental capability, has gained increasing attention with the growing focus on spatial intelligence. However, current obstacle-avoidance methods mainly depend on limited field-of-view sensors and are ill-suited for UAV scenarios which require full-spatial awareness when the movement direction differs from the UAV's heading. This limitation motivates us to explore omnidirectional obstacle avoidance for panoramic drones with full-view perception. We first study an under explored problem setting in which a UAV must generate collision-free motion in environments with obstacles from arbitrary directions, and then construct a benchmark that consists of three representative flight tasks. Based on such settings, we propose Fly360, a two-stage perception-decision pipeline with a fixed random-yaw training strategy. At the perception stage, panoramic RGB observations are input and converted into depth maps as a robust intermediate representation. For the policy network, it is lightweight and used to output body-frame velocity commands from depth inputs. Extensive simulation and real-world experiments demonstrate that Fly360 achieves stable omnidirectional obstacle avoidance and outperforms forward-view baselines across all tasks. Our model is available at https://zxkai.github.io/fly360/
Xiangkai Zhang, Dizhe Zhang, WenZhuo Cao +5
Institute of Automation Chinese Academy of Sciences, Beijing, China · Insta360 Research, Shenzhen, China · Wuhan University,Wuhan, China.