MAVEN: A Meta-Reinforcement Learning Framework for Varying-Dynamics Expertise in Agile Quadrotor Maneuvers

Jin Zhou; Dongcheng Cao; Xian Wang; Shuo Li

cs.ROMar 11, 2026

MAVEN: A Meta-Reinforcement Learning Framework for Varying-Dynamics Expertise in Agile Quadrotor Maneuvers

Authors: Jin Zhou, Dongcheng Cao, Xian Wang, Shuo Li

Organizations: Authors are with the College of Control Science and Engineering, Zhejiang University, Hangzhou 310027, China

Abstract

Reinforcement learning (RL) has emerged as a powerful paradigm for achieving online agile navigation with quadrotors. Despite this success, policies trained via standard RL typically fail to generalize across significant dynamic variations, exhibiting a critical lack of adaptability. This work introduces MAVEN, a meta-RL framework that enables a single policy to achieve robust end-to-end navigation across a wide range of quadrotor dynamics. Our approach features a novel predictive context encoder, which learns to infer a latent representation of the system dynamics from interaction history. We demonstrate our method in agile waypoint traversal tasks under two challenging scenarios: large variations in quadrotor mass and severe single-rotor thrust loss. We leverage a GPU-vectorized simulator to distribute tasks across thousands of parallel environments, overcoming the long training times of meta-RL to converge in less than an hour. Through extensive experiments in both simulation and the real world, we validate that MAVEN achieves superior adaptation and agility. The policy successfully executes zero-shot sim-to-real transfer, demonstrating robust online adaptation by performing high-speed maneuvers despite mass variations of up to 66.7% and single-rotor thrust losses as severe as 70%.

Figures & tables

Fig. 1 : Demonstration of our policy performing three consecutive flights in the mass variation scenario without landing. (a) We alter the quadrotor mass with a magnet payload. (b) Illustration of a quadrotor with varying masses.

Fig. 2 : Overview of our meta-RL framework for online adaptation to quadrotor dynamics. The policy is trained in parallelized simulation environments on tasks with varying dynamics (mass and thrust loss). A novel predictive context encoder learns to infer a latent variable that conditions the policy network, enabling task-aware adaptation. For deployment, the trained policy performs real-time task inference and navigation on an onboard computer.

Parameter	Value	Parameter	Value
$m_{min},m_{max}$ [kg]	$0.25,0.50$	$\delta_{min},\delta_{max}$ [%]	$0,50$
$W$	$2$	$\boldsymbol{\omega}_{\text{max}}$ [rad/s]	$[12,12,5]$
$\mathbf{k}_{p}$ [m]	$[6,6,1]$	$\mathbf{k}_{v}$ [m/s]	$[15,15,10]$
$\lambda_{1}$	$10$	$\lambda_{2}$	$1\times 10^{-4}$
$\lambda_{3}$	$10$	$r_{\text{collision}}$	$1$
$\omega_{\text{pos}}$	$1.0$	$\omega_{\text{rew}}$	$1.0$

TABLE I : Simulation and Training Parameters

Method	260g			330g			440g			550g
	Avg Vel.	Thr $>$ 0.8	Time	Avg Vel.	Thr. $>$ 0.8	Time	Avg Vel.	Thr. $>$ 0.8	Time	Avg Vel.	Thr. $>$ 0.8	Time
	(m/s)	(%)	(s)	(m/s)	(%)	(s)	(m/s)	(%)	(s)	(m/s)	(%)	(s)
RL-260g	7.82	71.0	1.54	$7.07$	$82.1$	$1.67$	-	-	-	-	-	-
RL-330g	$5.47$	$61.0$	$2.12$	6.04	77.4	1.67	$5.96$	$86.9$	$1.90$	-	-	-
RL-440g	$6.37$	$62.2$	$1.97$	$6.65$	$81.0$	$1.67$	6.06	94.2	1.78	-	-	-
RL-550g	$4.93$	$35.0$	$3.16$	$5.57$	$59.2$	$2.10$	$5.34$	$82.1$	$2.06$	5.23	96.9	2.12

TABLE II : Comparison of Our Method against Baselines Across Various Drone Masses. Metrics include average velocity (Avg Vel.), flight time (Time), and the percentage of flight duration where the normalized throttle exceeds 0.8 (Thr.

>

0.8). Results for the mass-specific RL policy are shown in bold. Crashed trials are denoted by a dash (-).

Fig. 3 : Comparison of trajectories on a switchback track for quadrotors with varying mass (260g, 330g, 440g, and 550g). Shaded areas denote waypoints with a radius of 1.0m. Spots and stars denote the starting and ending waypoints, respectively. Although both RL-DR and our method use a single policy, our approach yields trajectories similar to the mass-specific RL, whereas RL-DR exhibits unnecessary detours at certain masses.

Fig. 4 : Flight trajectories and completion times for our policy and two baselines (mass-specific RL and RL-DR) on two challenging tracks. Our method’s performance closely resembles the mass-specific expert’s, avoiding the inefficient detours or braking of the RL-DR.

Panel of Figure 7 — Fig. 5 : Trajectories of our method against varying degrees of thrust loss, with the result of a standard RL method as a baseline.

Panel of Figure 8 — Fig. 6 : Performance comparison of the proposed method against baselines under varying single-rotor thrust loss. (a) indicates the success rate (%), while (b) shows the average completion time (s).

Fig. 9 : Real-world 3D trajectories and time-series plots of three consecutive flights without landing in the mass variation scenario.

Performance Metric	330g		440g		550g
Performance Metric	Sim	Real	Sim	Real	Sim	Real
Completion Time (s)	$2.50$	$2.68$	$2.98$	$3.22$	$3.47$	$3.52$
Max Velocity (m/s)	$7.29$	$7.69$	$6.77$	$7.11$	$6.07$	$6.48$

TABLE III : Comparison of Simulation and Real-world Flight Metrics Across Mass Variations.

Fig. 10 : Results of the thrust loss scenario. Upper: our real-world setup, where a four-digit label specifies a propeller’s diameter and pitch in tenths of an inch. Bottom: flight trajectories for the ’M’ and ’A’ tracks, annotated with total flight time (T) and maximum velocity (V).

Robot Policy Adaptation

Robot policy adaptation changes or steers learned robot behaviors to handle new tasks, environments, hardware, and user preferences.

See all 173 papers

Adaptive Outer-Loop Control of Quadrotors via Reinforcement Learning
May 15, 2026Vishnu Saj, Sushil Vemuri, Dileep Kalathil +1RL Control Sim-to-Real Transfer
Texas A&M University College Station, TX, USA
Precise Aggressive Aerial Maneuvers with Sensorimotor Policies
Apr 7, 2026Tianyue Wu, Guangtong Xu, Zihan Wang +6UAV Navigation Quadrotor Control
Institute of Cyber-Systems and Control, College of Control Science and Engineering, Zhejiang University, Hangzhou, China. · Huzhou Institute of Zhejiang University, Huzhou, China. · Differential Robotics Technology Co., Ltd., Hangzhou, China.
Bridging Performance and Generalization in Reinforcement Learning for Agile Flight
Jun 25, 2026Jonathan Green, Jiaxu Xing, Nico Messikommer +2RL for Robotics Aerial Robotics
Robotics and Perception Group, University of Zurich

May 15, 2026J/K move · Enter open · S save

Adaptive Outer-Loop Control of Quadrotors via Reinforcement Learning

Vishnu Saj, Sushil Vemuri, Dileep Kalathil, Moble Benedict

Texas A&M University College Station, TX, USA

RL Control Sim-to-Real Transfer Quadrotor Control Adaptive Control

Open paper AskPDF arXiv

Deep Reinforcement Learning (DRL) for quadrotor flight control typically relies on Domain Randomization (DR) for sim-to-real transfer, resulting in overly conservative policies that struggle with dynamic disturbances. To overcome this, we propose a novel adaptive control architecture that actively perceives and reacts to instantaneous perturbations. First, we train an optimal outer-loop policy, then replace its reliance on ground-truth disturbance data with a Residual Dynamics Predictor (RDP). The RDP estimates the external forces and moments acting on the aircraft in flight online using only the history of states and control actions. For seamless hardware transfer, we introduce a data-efficient linear calibration bridge and an online thrust correction mechanism that align the simulated latent space with reality using mere seconds of flight data. Real-world validations on a Crazyflie micro-quadrotor demonstrate that our adaptive controller significantly outperforms baselines, maintaining precise trajectory tracking under severe uncertainties including mass variations, asymmetric payloads, and dynamic slung loads

Abstract

Robot Policy Adaptation

Explore similar work

Adaptive Outer-Loop Control of Quadrotors via Reinforcement Learning

Precise Aggressive Aerial Maneuvers with Sensorimotor Policies

Bridging Performance and Generalization in Reinforcement Learning for Agile Flight