Battery-Aware Reinforcement Learning for Aggressive Quadrotor Flight

Alejandro Sanchez Roncero; Olov Andersson; Petter Ogren

cs.ROSep 29, 2026

Battery-Aware Reinforcement Learning for Aggressive Quadrotor Flight

Authors: Alejandro Sanchez Roncero, Olov Andersson, Petter Ogren

Organizations: Department of Robotics, Perception and Learning, School of Electrical Engineering and Computer Science, Royal Institute of Technology (KTH), SE-100 44 Stockholm, Sweden

Abstract

Agile flight tasks such as drone racing and pursuit-evasion require strong acceleration and precise turns, but the available thrust changes as the battery discharges and voltage drops under load. Conservative command limits make this variation easier to tolerate, at the cost of unused performance. We investigate how learned controllers can use that additional thrust while retaining the flight controller's voltage compensation and rate control. Our training simulator couples an identified load-transient battery model to rotor dynamics and firmware saturation. The feedforward policy receives filtered voltage during both training and deployment. Controlled ablations distinguish the benefit of a larger thrust-command range from that of voltage information. On a 38 g Crazyflie Brushless, the resulting policy reduces circle tracking error by 49% relative to stock-authority RL at 3.84 m/s, while preserving easy-task precision. Mean 20-lap race time decreases from 106.22 s to 95.24 s. Compared with a voltage-blind policy with the same increased authority, hardware error and race time are lower by 15.3% and 4.5%, respectively. In simulation, replacing the policy's voltage input with a recording from a different battery condition worsens hard-circle tracking, with a smaller, voltage-dependent effect in racing. Together, these results show where a simple voltage input complements existing actuator compensation in aggressive learned flight.

Figures & tables

Fig. 1: Hardware flights with Stock (default thrust-command limit, no voltage input) and High-V (higher limit with voltage input). (a,b) Tracking at $3.36$ and $3.84$ m/s, with root-mean-square error (RMSE) averaged over repeated flights. (c) Racing over $20$ laps. The overlays show measured paths on calibrated camera images and virtual gates in racing.

Parameter	Circle	Racing
Actor hidden layers, ELU	$256,256,128$	$256,256,128$
Symmetric critic hidden layers	$256,256,128$	$256,256,128$
Privileged critic: observation	$256,128$	$256,128$
Privileged critic: battery	$64,64$	$64,64$
Privileged critic: value	$256,128,1$	$256,128,1$
No voltage / scalar voltage	$42/43$ inputs	$43/44$ inputs

TABLE I: Learning and command settings

Quantity	Nominal	Randomization
Mass $m$ [g]	$38$	$\pm 5\%$
$\operatorname{diag}\mathbf{J}$ [ $10^{-5}$ kg m 2 ]	$3.3,3.6,5.9$	$\pm 20\%$
Rotor inertia $J_{r}$ [kg m 2 ]	$5\!\times\!10^{-8}$	Fixed
Motor gain multiplier	$1$	$\pm 3/2\%$
Thrust multiplier	$1$	$\pm 5/3\%$
Motor lag $\tau_{m}$ [ms]	$50$	$\pm 15/5\%$

TABLE II: Physical model and training randomization

Metric	Stock	High	High-V
Four-lap race [s]	21.30 $\pm$ 0.10	20.07 $\pm$ 0.16	19.26 $\bm{\pm}$ 0.27
Flying lap, short [s]	5.33 $\pm$ 0.03	5.04 $\pm$ 0.04	4.81 $\bm{\pm}$ 0.08
20-lap race [s]	106.22 $\pm$ 0.26	99.76 $\pm$ 0.84	95.24 $\bm{\pm}$ 0.44
Flying lap, long [s]	5.31 $\pm$ 0.01	4.99 $\pm$ 0.04	4.76 $\bm{\pm}$ 0.02
Mean speed, long [m/s]	3.74	3.89	4.10
Peak speed, long [m/s]	6.08 $\pm$ 0.11	6.74 $\pm$ 0.46	6.95 $\bm{\pm}$ 0.05

TABLE III: Hardware racing performance

Fig. 9: Mean paired percentage reduction in error or lap time with a privileged rather than symmetric critic, where positive values indicate improvement. Parentheses identify alternative voltage filters, and colors are capped at $\pm 25\%$ .

Task	Actor	Symmetric	Privileged
Circle 3.36 m/s	Stock	0.93 $\pm$ 0.01	0.99 $\bm{\pm}$ 0.01
	High	0.94 $\pm$ 0.01	0.97 $\bm{\pm}$ 0.02
	High-V	0.98 $\bm{\pm}$ 0.01	0.98 $\bm{\pm}$ 0.01
Circle 3.84 m/s	Stock	0.96 $\pm$ 0.00	0.97 $\bm{\pm}$ 0.01
	High	0.96 $\pm$ 0.00	0.98 $\bm{\pm}$ 0.00
	High-V	0.99 $\bm{\pm}$ 0.00	0.99 $\bm{\pm}$ 0.00

TABLE IV: Critic prediction accuracy

Explore similar work

AcroRL: Learning Aggressive Quadrotor Inversion using Bidirectional Thrust
May 23, 2026Gabriel Rodriguez, Henri Sayag, Abhishek Rathod +4Quadrotor Bidirectional Thrust
Carnegie Mellon University · Shield AI
Adaptive Outer-Loop Control of Quadrotors via Reinforcement Learning
May 15, 2026Vishnu Saj, Sushil Vemuri, Dileep Kalathil +1Quadrotor Adaptive Control
Texas A&M University College Station, TX, USA
Bridging Performance and Generalization in Reinforcement Learning for Agile Flight
Jun 25, 2026Jonathan Green, Jiaxu Xing, Nico Messikommer +2Autonomous Drones Aerial Robots
Robotics and Perception Group, University of Zurich

May 23, 2026, cs.ROJ/K move · Enter open · S save

AcroRL: Learning Aggressive Quadrotor Inversion using Bidirectional Thrust

Gabriel Rodriguez, Henri Sayag, Abhishek Rathod, John Stecklein+3

Carnegie Mellon University · Shield AI

Quadrotor Bidirectional Thrust Active Flow Control Motion Planning

Open paper AskPDF arXiv

Bidirectional thrust grants quadrotors a second equilibrium condition and increased control authority, expanding the envelope of possible aggressive maneuvers and enabling inverted flight, perching, and sensing. Prior geometric control approaches extend differential flatness through Hopf fibration-based attitude representations to support bidirectional thrust, but struggle with actuator saturation and motor reversal delay during inversions, requiring heuristic thrust posture scheduling and waypoint tuning. We propose a learning-based framework that modulates a constant reference trajectory to perform compact, position-constrained quadrotor inversions while remaining compatible with traditional trajectory generation and tracking across flight regimes. Separate policies are trained via reinforcement learning for nominal-to-inverted and inverted-to-nominal transitions. In JAX-based simulation, the proposed method achieves the lowest position deviation and settling time across all evaluated baselines, reducing position root mean square error (RMSE) by 32% and settling time by 57% relative to the strongest optimization-based baseline. Hardware experiments demonstrate successful inversion across multiple yaw configurations with position RMSE below 0.35m, and compatibility with downstream trajectory generation and control through circular flight in both regimes. Additionally, we provide an open-source implementation of the proposed framework.

Abstract

Explore similar work

AcroRL: Learning Aggressive Quadrotor Inversion using Bidirectional Thrust

Adaptive Outer-Loop Control of Quadrotors via Reinforcement Learning

Bridging Performance and Generalization in Reinforcement Learning for Agile Flight