Traversing narrow gaps is challenging for autonomous quadrotors, especially when control commands come directly from high-dimensional visual observations. Existing end-to-end methods often rely on behavior cloning or full-rollout backpropagation through time (BPTT) via differentiable simulation, which can limit policy performance or incur high training costs. We propose a two-stage reinforcement learning framework for more efficient ego-centric visuomotor gap-traversal policy training, leveraging quasi-analytical policy gradients (QPG) via differentiable simulation and critic warm-starting. The framework utilizes QPG to avoid backpropagation through visual rendering, reducing computation and memory costs while improving sample efficiency. In the first stage, an expert actor and critic are trained using privileged observations, including gap geometry. Unlike prior gap-traversal approaches, our training utilizing QPG does not require resetting the agent along optimized reference trajectories. In the second stage, a visual policy is trained using binary gap masks from two ego-centric cameras and low-dimensional observations, while its privileged critic is warm-started from the first stage. This substantially improves training efficiency and traversal success compared with cold-starting the critic or using full-rollout BPTT. Our framework does not require retraining the expert actor when system parameters change, enabling more efficient generalization across drone platforms than state-of-the-art visual gap-traversal methods based on action supervision. The learned visual policy also generalizes to gaps with unseen shapes. Extensive real-world experiments further demonstrate robust gap traversal using binary masks rendered online. Beyond gap traversal, the proposed framework is generic and can be extended to other visuomotor robot learning tasks.
Figures & tables
Fig. 1: Visual agile gap traversal. Top right: the deployed policy maps the binary gate mask Ik from ego-centric cameras, along with low-dimensional observations (the agent’s roll ϕk , pitch θk , accelerometer reading a~kb , goal position pgoal,kb , and past action uk−1 ) to thrust and body-rate commands. Left and bottom right: a quadrotor executes the policy in real flight.
Fig. 2: Left: the bounding box (transparent green) C , representing the quadrotor’s collision. Right: opening dimensions of the gap G . Both are visualized through Isaac Sim.
Fig. 3: Overview of the proposed two-stage training pipeline. In the first stage, a critic Vϕ and an expert policy πθe are trained through QPG in a symmetric actor-critic manner, where the networks accept privileged observations: states (agent’s velocity in its body frame vkb , quaternion qk, and accelerometer readings a~kb ), corner positions gn,kb for n∈[0,3]∩Z , goal position in body frame pgoal,kb , and previous action uk−1 . In the second stage, the critic is copied from the pretraining phase to warm-start the value function. The policy is trained in an asymmetric actor-critic fashion, accepting a binary gate mask from the two ego-centric cameras and low-dimensional observations. The right side shows the policy execution during deployment.
Fig. 4: Comparing the gradient computation time (left) and success rate (right; mean and variation over three seeds) between using a simplified or full model to compute gradients.
Fig. 5: Training curves of the privileged pretraining stage: QPG, full-episode BPTT, PPO, and the ablation without the critic gradient (QPG w/o V-grad).
Fig. 6: Training curves of the second stage: ours with two- or one-camera configuration, full-episode BPTT, and the ablations with cold-start critic and a frozen critic (no critic adaptation).
Fig. 7: Trajectories of a quadrotor gap traversal using our vision-based policy for different gap shapes. (a) rectangular gap, similar to the one used during training, (b) elliptical gap, (c) trapezoidal gap, and (d) triangular gap. The spherical shapes in each image represent the goal region.
Parameter
Nominal
Revised
Mass m [kg]
1.279
1.407 ( +10% )
Body-rate’s ωn [rad/s]
100
70
Body-rate damping ζ
1.00
0.80
Body-rate DC gain K
0.58
0.75
Thrust time constant τ [ms]
12.5
25
TABLE I: Plant parameters of the nominal and revised quadrotor.
Fig. 8: Success rate differences between training with the revised plant (with the old critic or teacher) and the nominal plant. Our two-stage approach is shown in blue, and behavior cloning (BC) is shown in orange.
Fig. 9: Time-lapse images of the real-world experiments. The trained visual policy is executed with a gate binary mask observation rendered online. The gate roll angle varies as follows: (a) 45∘ , (b) 60∘ , (c) 70∘ , and (d) 80∘ .
Precise aggressive maneuvers with lightweight onboard sensors remains a key bottleneck in fully exploiting the maneuverability of drones. Such maneuvers are critical for expanding the systems' accessible area by navigating through narrow openings in the environment. Among the most relevant problems, a representative one is aggressive traversal through narrow gaps with quadrotors under SE(3) constraints, which require the quadrotors to leverage a momentary tilted attitude and the asymmetry of the airframe to navigate through gaps. In this paper, we achieve such maneuvers by developing sensorimotor policies directly mapping onboard vision and proprioception into low-level control commands. The policies are trained using reinforcement learning (RL) with end-to-end policy distillation in simulation. We mitigate the fundamental hardness of model-free RL's exploration on the restricted solution space with an initialization strategy leveraging trajectories generated by a model-based planner. Careful sim-to-real design allows the policy to control a quadrotor through narrow gaps with low clearances and high repeatability. For instance, the proposed method enables a quadrotor to navigate a rectangular gap at a 5 cm clearance, tilted at up to 90-degree orientation, without knowledge of the gap's position or orientation. Without training on dynamic gaps, the policy can reactively servo the quadrotor to traverse through a moving gap. The proposed method is also validated by training and deploying policies on challenging tracks of narrow gaps placed closely. The flexibility of the policy learning method is demonstrated by developing policies for geometrically diverse gaps, without relying on manually defined traversal poses and visual features.
Tianyue Wu, Guangtong Xu, Zihan Wang +6
Institute of Cyber-Systems and Control, College of Control Science and Engineering, Zhejiang University, Hangzhou, China. · Huzhou Institute of Zhejiang University, Huzhou, China. · Differential Robotics Technology Co., Ltd., Hangzhou, China.
-Navigation through narrow and irregular gaps is an essential skill in autonomous drones for applications such as inspection, search-and-rescue, and disaster response. However, traditional planning and control methods rely on explicit gap extraction and measurement, while recent end-to-end approaches often assume regularly shaped gaps, leading to poor generalization and limited practicality. In this work, we present a fully vision-based, end-to-end framework that maps depth images directly to control commands, enabling drones to traverse complex gaps within unseen environments. Operating in the Special Euclidean group SE(3), where position and orientation are tightly coupled, the framework leverages differentiable simulation, a Stop-Gradient operator, and a Bimodal Initialization Distribution to achieve stable traversal through consecutive gaps. Two auxiliary prediction modules-a gap-crossing success classifier and a traversability predictor-further enhance continuous navigation and safety. Extensive simulation and real-world experiments demonstrate the approach's effectiveness, generalization capability, and practical robustness.
Linzuo Zhang, Yu Hu, Feng Yu +3
The authors are with the Shanghai Jiao Tong University, Shanghai 200240, China.
Although quadcopters boast impressive traversal capabilities enabled by their omnidirectional maneuverability, the need for continuous pilot control in complex environments impedes their application in GNSS and telemetry-denied scenarios. To this end, we propose a novel sensorimotor policy that uses stereo-vision depth and visual-inertial odometry (VIO) to autonomously navigate through obstacles in an unknown environment to reach a goal point. The policy is comprised of a pre-trained autoencoder as the perception head followed by a planning and control LSTM network which outputs velocity commands that can be followed by an off-the-shelf commercial drone. We leverage reinforcement and privileged learning paradigms to train the policy in simulation through a two-stage process: 1) initial training with optimal trajectories generated by a global motion planner acting as a supervisory backbone, 2) further fine-tuning in a curriculum environment. To bridge the sim-to-real gap, we employ domain randomization and reward shaping to create a policy that is both robust to noise and domain shift. In outdoor experiments, our approach achieves successful zero-shot transfer to both obstacle environments and a drone platform that were never encountered during training.
Shiladitya Dutta, Aayush Gupta, Varun Saran +1
College of Engineering, Department of Electrical Engineering and Computer Science, University of California Berkeley