TransNav: Transformer-Based End-to-End Navigation Method for Static and Dynamic Quadrotor Obstacle Avoidance
Authors: Jianghan Zhu, Zhuo Li, Yanjun Zhang, Jian Sun
Organizations: The authors are with the State Key Laboratory of Autonomous Intelligent Unmanned Systems, School of Automation, Beijing Institute of Technology, Beijing 100081, China
Obstacle avoidance for autonomous quadrotors in cluttered environments remains challenging, especially when both dense static obstacles and moving obstacles are present. Existing methods are often tailored to either static or dynamic settings, which always limit their adaptability across diverse environments. This paper presents TransNav, a transformer-based end-to-end navigation method for quadrotor obstacle avoidance in both static and dynamic environments. TransNav employs a patch-based point-cloud encoding scheme that combines local feature extraction with global contextual modeling, enabling compact yet informative environment representation. The method integrates perception and navigation in a single architecture and directly predicts a finite-horizon action sequence, from which the first action is executed. Moreover, we develop an unsupervised learning strategy by incorporating a lightweight point-mass dynamics model, which directly optimizes trajectory-level costs over the predicted sequence through numerical gradient propagation. Thus, the policy can be efficiently trained without expert demonstrations. Simulation and real-world experiments show that the proposed method achieves strong performance in dense static environments and competitive obstacle-avoidance capability in dynamic scenarios, while maintaining low inference cost for onboard deployment.
Figures & tables
Fig. 1: Hardware platform of the quadrotor in real-world experiments.
Fig. 2: System architecture. Patches of the distance map and sampled states are encoded respectively and then concatenated into a token sequence to be input into the transformer encoder for predicting the acceleration sequence within a time horizon. ESDF map is obtained from the training dataset and is used only during training to evaluate the loss function and compute the gradients. The dataset is sampled from ten randomly generated training environments. In real-world deployment, the state of quadrotor is obtained using the state estimation method (e.g., Fast-LIO2 [ 19 ] , and Point LIO [ 20 ] ).
Fig. 3: Illustration of patch-based LiDAR sensing representation. The distance map is composed of the distances from the nearest point of each partition, which are divided into 100 patches to be input into the subsequent transformer encoder for feature extraction.
Fig. 4: Illustration of backpropagation process. The output action sequence is applied to the simple point-mass model to obtain the state sequence prediction in future horizon. The complete gradient of the loss with respect to the network parameters is obtained directly through backpropagation, and the temporal gradient decay is used to alleviate gradient explosion during the backpropagation process.
Fig. 5: Visualization of flight slices in simulation. The scenarios labeled (a) to (f) correspond to scenarios 1 to 6 in table I , respectively.
Scenario
Method
η(%)
va(m/s)
tp(ms)
Rl
60 static
Ours
100
2.71
6.72
1.12
[ 12 ]
85
3.12
11.73
1.05
[ 16 ]
80
2.63
9.59
1.05
[ 18 ]
100
0.90
23.14
1.05
[ 10 ]
60
2.46
8.43
2.07
105 static
Ours
100
1.97
7.01
1.19
TABLE I: SIMULATED COMPARISON IN DIFFERENT TYPES OF ENVIRONMENTS.
Fig. 6: Real-world experiments. (a) Highlight of a vertical avoidance capability. (b)-(c) Agile flight among static obstacles. (d) Flight through a gate. (e)-(g) Navigation among static and dynamic obstacles. (h) Avoiding fast-moving obstacles. White curves represent the movement paths of dynamic obstacles.
Autonomous quadrotor flight in cluttered environment is an key capability for tasks like inspection and search. Navigation policies trained in simulation and transferred to hardware can fly at high speed in open space, but they can still collide as obstacle clearance shrinks. We attribute most of these collisions to a single mechanism, namely motion in directions the onboard camera does not observe. Heading that is not aligned with the direction of travel and lateral excursion during obstacle avoidance both carry the vehicle toward obstacles outside its field of view. We introduce ClearNav, a navigation policy trained with two reward terms that constrain motion to the observed direction, a forward-looking heading target and a penalty on lateral velocity. A third term penalizes speed near obstacles so that an obstacle entering the field of view remains avoidable. In simulation, across 18 held-out scenarios with 2304 paired flights per policy unseen in training, ClearNav improves success by 6.8 points, cuts the collision rate by 65% relative to the state of the arts, and flies at maximum 5m/s. For the proximity term alone under matched commanded speeds, the collision reduction still holds at 70%. The effect is significant on static scenes and consistent in direction on moving obstacles, though not significant there at the suite level, showing that safer flight in clutter is possible even on the limited sensors and compute of small drones. We also validate our navigation policy in a real-world cluttered environment, showing that it enables the quadrotor to navigate through the cluttered environment successfully and safely.
Aditya Azad, Arthur Huang, Guanrui Li
Worcester Polytechnic Institute, MA, USA · Northeastern University, MA, USA
Navigating dynamic and crowded environments presents significant challenges for quadruped robots due to severe sensor occlusion and unpredictable human motion. Existing approaches face a trade-off: model-based methods, such as Velocity Obstacles (VO), theoretically guarantee safety but rely on accurate obstacle motion estimates that often fail in dense crowds, while end-to-end learning methods offer robustness but lack motion prediction capability of obstacles, leading to collisions or conservative behaviors. To solve this, we propose VOP-Nav, a novel navigation system that combines the geometric safety of VO with the agile adaptability of end-to-end learning. Using only local onboard observations, our system avoids explicit obstacle detection and tracking pipelines. The VOP-Net processes multi-frame LiDAR data to implicitly encode dynamic constraints and predict a safe velocity region derived from Velocity Obstacle theory. Importantly, the VO predictions serve a dual role: they are used as input to the navigation policy during inference and as a reward signal during training to encourage safe motion. Evaluations in Isaac Gym demonstrate that VOP-Nav achieves higher success rates than all baselines while balancing locomotion speed and collision avoidance. Real-world deployment on a Unitree Go2 quadruped robot further validates the system's robustness and efficiency in complex indoor and outdoor dynamic environments.
Shuyu Wu, Zeyu Liu, Tianbao Zhang +6
Shanghai Jiao Tong University, Shanghai 200240, China · State Key Laboratory of High-end Heavy-load Robots, Midea Group, Foshan 528300, China
Safe quadrotor navigation in cluttered and dynamic environments depends not only on instantaneous geometric perception, but more critically on anticipating collision risks induced by relative motion. Conventional modular pipelines frequently suffer from perception latency, while end-to-end learning methods relying on implicit scalar rewards often struggle to extract reliable spatio-temporal features without physics-grounded supervision. To address this, we propose an anticipatory risk-guided reinforcement learning framework. Leveraging privileged simulator states, we construct a directionally aligned future collision risk map based on the Closest Point of Approach (CPA). Through an asymmetric actor-critic architecture, the network is trained to self-predict this structured risk, which explicitly guides the visual policy during deployment. A lightweight spatio-temporal encoder extracts motion cues directly from onboard depth sequences, bypassing explicit object tracking or optical flow estimation. Extensive simulated and real-world experiments demonstrate that our method effectively improves safety margins and flight efficiency in dense dynamic clutters compared to existing baselines. Furthermore, the learned policy achieves robust zero-shot Sim-to-Real transfer on a physical quadrotor, relying purely on abstracted spatio-temporal depth sequences and its self-predicted risk priors, validating the effectiveness of our approach and its robust generalization from simulation to reality.
Yuchao Mei, Guohao Zhang, Luxia Ai +2
National Key Laboratory of Science and Technology on Multi-spectral Information Processing, School of Artificial Intelligence and Automation, Huazhong University of Science and Technology, Wuhan 430074, China