Organizations: Institute of Cyber-Systems and Control, College of Control Science and Engineering, Zhejiang University, Hangzhou 310027, China. · Huzhou Institute of Zhejiang University, Huzhou 313000, China. · University of Electronic Science and Technology of China, Chengdu 611731, China. · Differential Robotics Technology Company, Hangzhou 311121, China.
Vision-based Unmanned Aerial Vehicles (UAVs) often suffer from navigation failures in dead ends due to limited sensing accuracy and range. To address this challenge, this paper proposes a systematic solution for efficient dead-end prediction and avoidance. The proposed method introduces a lightweight neural network to predict the relative distance and bearing of potential dead ends within the current field of view using RGB-D inputs. These predictions prune a predefined, compact trajectory library, enabling the planner to proactively avoid dead ends while maintaining navigational smoothness. Notably, our approach transfers across real-world scenarios without manual annotation or fine-tuning on real-world data. The system achieves high-frequency replanning at 50 Hz onboard. Extensive simulation benchmarks demonstrate superior performance in success rate, flight time, and trajectory length, and real-world experiments further validate its effectiveness in complex scenarios.
Figures & tables
Fig. 2 : System overview of DPNet.
Fig. 3 : Examples of automatic ground-truth labeling. (a) Valid: one inner vertex and all outer vertices are visible, so the dead-end geometry is directly identifiable. (b) Valid: the distant dead end is partially clipped in the input depth, but the same geometric visibility criterion still holds. (c) Valid: one inner vertex is visible and the out-of-view outer vertex is not occluded, providing sufficient context for shape inference. (d) Invalid: no inner vertices are visible, making the structure ambiguous (Mask=0).
Fig. 4 : Dead-end-aware collision checking.
Fig. 5 : Dead-end-aware trajectory selection. The two subfigures are consecutive snapshots. (a) The UAV bypasses the dead end. (b) After the dead end leaves the field of view, the smoothness penalty prevents the planner from selecting a trajectory that re-enters the dead end.
Fig. 6 : Planner behavior comparison in a representative simulation scenario.
Diff.
Method
Succ. (%)
Time (s)
Length (m)
Easy
EGO-Planner
80.0
14.54
23.00
Primitive-Planner
86.0
19.91
29.90
Faster
97.0
18.47
25.83
Ours
100.0
13.77
24.35
Hard
EGO-Planner
76.0
11.65
22.13
Primitive-Planner
47.0
9.06
22.34
TABLE I : Benchmark comparison under limited local perception.
Fig. 7 : The real-world experiments. (a)–(c) Key observations of true dead ends. (d) The opened control passage used for comparison.
UAV see-and-reach navigation requires an aerial agent to approach a language-specified target visible in its initial view and stop reliably near it. Existing methods typically map vision-language representations directly to action outputs without explicitly modeling intermediate fine-grained spatial decisions. This direct mapping causes semantic-control misalignment, leading to inconsistent maneuvers and unreliable termination. To address this issue, we propose DBFly, a vision-language waypoint prediction framework that introduces explicit vision-guided spatial deliberation before waypoint generation. Specifically, DBFly introduces a spatial maneuver decision chain that progressively performs target-direction anchoring, spatial diagnosis, and maneuver decision, enabling high-level maneuver intent to explicitly guide continuous waypoint generation. DBFly further constructs an implicit flight corridor by transforming the initial target-direction prior into a persistent geometric reference and deriving an online corridor state from the UAV's current position, thereby providing soft geometric guidance for spatial diagnosis and maneuver correction. In addition, DBFly develops a terminal-convergence-aware stopping strategy that characterizes terminal states through both target proximity and short-horizon motion convergence, enabling more reliable stopping near the target. Extensive experiments across seen, unseen-object, and unseen-scene test sets demonstrate that DBFly improves the success rate over the SOTA baseline by an average of 25.07 percentage points. The project homepage is available at https://xuefanfu.github.io/DBFly-Page.
Fanfu Xue, En Yu, Bohang Liu +4
School of Information Science and Engineering, Shandong University · Faculty of Engineering and Information Technology, University of Technology Sydney · School of Artificial Intelligence, Shandong University +1
End-to-end unmanned aerial vehicle (UAV) navigation can achieve impressive agility in simulation, yet its obstacle-avoidance behavior often degrades after deployment because the policy must tolerate simulator mismatch, sensing irregularity, and variable-rate control. These effects are especially dangerous in cluttered environments, where stale observations or short control irregularities can directly lead to collisions. We present LNN-Fly, a deployment-oriented continuous-time navigation policy for LiDAR-based UAV obstacle avoidance. The policy combines a dynamic-programming-inspired structured recurrent update, explicit conditioning on the elapsed control interval Δt, and an input-driven adaptive forgetting gate that refreshes stale latent state near hazards while preserving consistency during sustained maneuvers. It is trained with differentiable rollouts that incorporate deployment-relevant sensing and timing perturbations. In simulation, LNN-Fly improves obstacle-avoidance performance in the tested settings and shows better tolerance to reduced control frequency, sparse observations, and control-period jitter. It also transfers zero-shot from a simplified differentiable simulator to a physical quadrotor. In indoor cross-frequency real-world tests, the system achieves 100% success over 20 flights, while policy inference has a median latency of 0.514 ms on a desktop graphics processing unit (GPU) and about 2.5 ms on the onboard central processing unit (CPU), with onboard P95 latency below 30 ms.
Yulin Huang, Shaojie Chen, Di Feng +3
Shenzhen Institute for Advanced Study, University of Electronic Science and Technology of China, Shenzhen, Guangdong 518000, China
Vision-based UAV navigation becomes challenging when navigation targets are distributed across complementary camera views and cannot be reliably observed from a single viewpoint. We propose AgenticDiffusion, an agentic multi-view UAV navigation framework that semantically coordinates first-person-view (FPV) and top-view observations for mission-level navigation. Given a natural-language instruction, AgenticDiffusion identifies the requested targets, selects the most appropriate camera view for each navigation task, determines the corresponding navigation goal, and invokes the appropriate view-conditioned diffusion planner for trajectory generation. The resulting trajectories are executed using Nonlinear Model Predictive Control (NMPC). AgenticDiffusion was evaluated in four real-world indoor scenarios, achieving an overall mission success rate of 80% across 40 physical-flight trials. In mixed-visibility scenarios, where the requested targets were distributed across FPV and top-view observations, coordinated multi-view navigation reduced average mission time by 50.8% relative to FPV-only navigation and by 26.8% relative to Top-only navigation. The semantic view-selection mechanism was also robust to lexical variation in target descriptions, achieving 100% accuracy across 66 test cases, compared with 63.64% for a confidence-based view-selection baseline. In a substantially larger Gazebo environment, AgenticDiffusion achieved a 90% mission success rate and completed the multi-stage mission, whereas the FPV-only and Top-only variants were unable to complete all requested navigation tasks.
Faryal Batool, Muhammad Ahsan Mustafa, Fawad Mehboob +2