Curriculum Reinforcement Learning for Quadrotor Racing with Random Obstacles

Fangyu Sun; Fanxing Li; Yu Hu; Linzuo Zhang; Yueqian Liu; Wenxian Yu; Danping Zou

cs.ROFeb 27, 2026

Curriculum Reinforcement Learning for Quadrotor Racing with Random Obstacles

Authors: Fangyu Sun, Fanxing Li, Yu Hu, Linzuo Zhang, Yueqian Liu, Wenxian Yu, Danping Zou

Organizations: School of Automation and Perception, Shanghai Jiao Tong University, China · Faculty of Aerospace Engineering, Delft University of Technology, The Netherlands

Abstract

Autonomous drone racing has attracted increasing interest as a research topic for exploring the limits of agile flight. However, existing studies primarily focus on obstacle-free racetracks, while the perception and dynamic challenges introduced by obstacles remain underexplored, often resulting in low success rates and limited robustness in real-world flight. To this end, we propose a novel vision-based curriculum reinforcement learning framework for training a robust controller capable of addressing unseen obstacles in drone racing. We combine multi-stage cu rriculum learning, domain randomization, and a multi-scene updating strategy to address the conflicting challenges of obstacle avoidance and gate traversal. Our end-to-end control policy is implemented as a single network, allowing high-speed flight of quadrotors in environments with variable obstacles. Both hardware-in-the-loop and real-world experiments demonstrate that our method achieves faster lap times and higher success rates than existing approaches, effectively advancing drone racing in obstacle-rich environments. The video and code are available at: https://github.com/SJTU-ViSYS-team/CRL-Drone-Racing.

Figures & tables

Fig. 1: Our quadrotor autonomously races with random obstacles in the real world at speeds of up to 8 $m/s$ . (a) The real-world experiment of the S-shaped racetrack and the trajectory. (b) Our onboard drone is equipped with a Raspberry Pi computer and an Intel D435i depth camera. (c) FPV depth images during racing with obstacles.

Fig. 2: The framework of our RL policy training network for the obstacle-rich racing task. The network architecture consists solely of simple CNNs and MLPs without any complex backbones. ”Cat” is the abbreviation for ”Concatenate”.

Fig. 3: Balancing the tasks of gate passing and obstacle avoidance is challenging for training an RL policy. These two tasks are contradictory for reward design, the obstacle-avoidance reward would cause the drone to bypass the gates from the sides rather than passing through them directly.

Fig. 4: Three different racetracks with trajectories flown by our policy in simulation. It can be observed that under our vision-based policy, the drone can successfully race in densely cluttered environments.

Fig. 5: The framework of multi-scene updating. The multi-scene updating approach improves policy training efficiency by customizing the number of scenes in each agent group.

Fig. 6: The reward curves of the ablation study during training. Green, blue, purple, and red represent the cases without GRU, without $r_{t}^{\text{avoid}}$ , one-step learning, and our method, respectively.

	Parameter	Value
Reward	$\lambda_{1}$	0.9
	$\lambda_{2}$	0.05
	$\lambda_{3}$	-0.005
	$\lambda_{4}$	-0.0025
	$\lambda_{5}$	-0.05
	$\lambda_{6}$	-0.01

TABLE I: Key parameters during the experiment.

Method	LT [s]			SR [%]
	S-shaped	3D Circle	J-shaped	S-shaped	3D Circle	J-shaped
Vision-based [ 5 ]	5.4	3.8	3.9	30	40	40
State-based (ours)	3.5	3.8	3.1	30	20	30
Vision-based (ours)	3.4	3.6	2.9	100	100	100

TABLE II: SR and the LT of our method against two baselines among 10 trials in simulation. Three different racetracks are used to evaluate the performance. The best results are bold.

		SR [%]
		S-shaped	3D Circle	J-shaped
Study. 1	w/o GRU	80	70	80
Study. 2	w/o $r^{avoid}$	40	40	30
Study. 3	one-step	0	0	0
	ours	100	100	100

TABLE III: SR comparison for ablation studies among 10 trials.

		SR [%]
		S-shaped	3D Circle	J-shaped
Density [its/2 gates]	2	100	100	100
	3	100	100	100
	4	90	90	90
	5	80	70	80
Gate changing [m]	$\pm 0.3$	100	100	100

TABLE IV: SR of handling gates and obstacles across 10 trials. The results correspond to independent evaluations of two factors: (1) Density, which refers to the number of obstacles affecting flight in the racetrack in every 2 gates; and (2) Gate changing (m), which denotes the gate’s randomization level among the three axes.

Fig. 7: HITL simulation. From left to right, S-shaped, 3D Circle, and J-shaped racetracks.

Fig. 8: Real world experiment results. We validate our algorithm in three racetracks, including (a) S-shaped, (b) 3D circle, (c) J-shaped.

Tracks	S-shaped		3D Circle		J-shaped
	SR [%]	LT [s]	SR [%]	LT [s]	SR [%]	LT [s]
HITL	100	3.5	100	4.1	100	3.2
Real-world	100	4.3	100	5.0	100	3.7

TABLE V: The performance of our policy in HITL simulation and the real world.

Explore similar work

Vision-Guided Outdoor Flight and Obstacle Evasion via Reinforcement Learning
May 23, 2026Shiladitya Dutta, Aayush Gupta, Varun Saran +1RL for Robotics UAV Navigation
College of Engineering, Department of Electrical Engineering and Computer Science, University of California Berkeley
Vector Field Augmented Differentiable Policy Learning for Vision-Based Drone Racing
Mar 9, 2026Yang Su, Feng Yu, Yu Hu +4Policy Learning Visuomotor Policy Learning
The authors are with the Shanghai Jiao Tong University, Shanghai 200240, China
Bridging Performance and Generalization in Reinforcement Learning for Agile Flight
Jun 25, 2026Jonathan Green, Jiaxu Xing, Nico Messikommer +2RL for Robotics Aerial Robotics
Robotics and Perception Group, University of Zurich

May 23, 2026J/K move · Enter open · S save

Vision-Guided Outdoor Flight and Obstacle Evasion via Reinforcement Learning

Shiladitya Dutta, Aayush Gupta, Varun Saran, Avideh Zakhor

College of Engineering, Department of Electrical Engineering and Computer Science, University of California Berkeley

RL for Robotics UAV Navigation Visuomotor Policy Learning Dynamic Obstacle Avoidance Robust RL

Open paper AskPDF arXiv

Although quadcopters boast impressive traversal capabilities enabled by their omnidirectional maneuverability, the need for continuous pilot control in complex environments impedes their application in GNSS and telemetry-denied scenarios. To this end, we propose a novel sensorimotor policy that uses stereo-vision depth and visual-inertial odometry (VIO) to autonomously navigate through obstacles in an unknown environment to reach a goal point. The policy is comprised of a pre-trained autoencoder as the perception head followed by a planning and control LSTM network which outputs velocity commands that can be followed by an off-the-shelf commercial drone. We leverage reinforcement and privileged learning paradigms to train the policy in simulation through a two-stage process: 1) initial training with optimal trajectories generated by a global motion planner acting as a supervisory backbone, 2) further fine-tuning in a curriculum environment. To bridge the sim-to-real gap, we employ domain randomization and reward shaping to create a policy that is both robust to noise and domain shift. In outdoor experiments, our approach achieves successful zero-shot transfer to both obstacle environments and a drone platform that were never encountered during training.

Abstract

Explore similar work

Vision-Guided Outdoor Flight and Obstacle Evasion via Reinforcement Learning

Vector Field Augmented Differentiable Policy Learning for Vision-Based Drone Racing

Bridging Performance and Generalization in Reinforcement Learning for Agile Flight