Learning Robust Control Policies for Inverted Pose on Miniature Blimp Robots

Yuanlin Yang; Lin Hong; Fumin Zhang

cs.ROFeb 27, 2026

Learning Robust Control Policies for Inverted Pose on Miniature Blimp Robots

Authors: Yuanlin Yang, Lin Hong, Fumin Zhang

Organizations: Department of Electronic and Computer Engineering, The Hong Kong University of Science and Technology, Hong Kong, China

Abstract

The ability to achieve and maintain inverted poses is essential for unlocking the full agility of miniature blimp robots (MBRs). However, developing reliable inverted control strategies for MBRs remains challenging due to their complex and underactuated dynamics. To address this challenge, we propose a novel framework that enables robust control policy learning for inverted pose on MBRs. The proposed framework consists of three core stages. First, a high-fidelity three-dimensional (3D) simulation environment is constructed and calibrated using real-world MBR motion data. Second, a robust inverted control policy is trained in simulation using a modified Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm combined with a domain randomization strategy. Third, a mapping layer is designed to bridge the sim-to-real gap and facilitate real-world deployment of the learned policy. Comprehensive evaluations in the simulation environment demonstrate that the learned policy achieves a higher success rate compared to the energy-shaping controller. Furthermore, experimental results confirm that the learned policy with a mapping layer enables an MBR to achieve and maintain a fully inverted pose in real-world settings.

Figures & tables

Fig. 1: Overview of the proposed method for inverted pose on MBRs. The MBR with an upright pose can reach and maintain the inverted pose with the learned robust policy.

Fig. 2: Dynamic model analysis of the MBR: (a) Upright pose; (b) Inverted pose.

Fig. 3: Pipeline of developing and deploying robust policy for inverted control of MBRs.

Fig. 4: 3D simulation environment designed for robust policy learning for inverted control of MBRs.

Fig. 5: The MBR’s structure in the simulation environment.

$g_{\omega_{x}}$	$g_{\omega_{y}}$	$g_{\omega_{z}}$	$g_{a_{x}}$	$g_{a_{y}}$	$g_{a_{z}}$	$g_{\phi}$	$g_{\theta}$	$g_{\psi}$	$\zeta$	$g_{n}$
0.01	0.01	0.01	0.001	0.001	0.001	5.0	5.0	0.5	0.1	10

TABLE I: Parameters in the reward function

$N$	$t_{e}$	$N_{e}$	$\sigma$	$\xi$	$n_{\sigma}$	$d_{p}$	$\gamma$	$\varpi$	$c_{a}$	$c_{b}$	$\lambda$	$\psi_{0}$
10	30s	500	0.15	0.95	100	2	0.98	0.01	0.1	0.1	[0.6, 1]	0.5

TABLE II: Parameters in policy training

Fig. 6: Roll angle variations with changes in $m_{w}$ . Solid lines represent the learned policy, while the dotted line represents the baseline controller.

Method	$m_{w}=5g$	$m_{w}=10g$	$m_{w}=15g$	$m_{w}=20g$	$m_{w}=25g$
Baseline	$\times$	$\times$	$\times$	$\times$	$\checkmark$
Our policy	$\times$	$\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$

TABLE III: Comparison of the baseline controller and the learned policy with varied

m_{w}

Method	$\lambda=0.6$	$\lambda=0.7$	$\lambda=0.8$	$\lambda=0.9$	$\lambda=1.0$
Baseline	$\times$	$\times$	$\times$	$\times$	$\checkmark$
Our policy	$\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$

TABLE IV: Comparison of the baseline controller and the learned policy with varied

\lambda

Fig. 7: Roll angle variations with changes in $\lambda$ .

Method	$g_{m}=0.5$	$g_{m}=1.0$	$g_{m}=1.5$	$g_{m}=2.0$	$g_{m}=2.5$
Baseline	$\times$	$\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$
Our policy	$\times$	$\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$

TABLE V: Comparison of the baseline controller and the learned policy with varied

g_{m}

Fig. 8: Roll angle variations with changes in $g_{m}$ .

Test Case	C1	C2	C3	C4	C5
$m_{w}$ ( $\mathrm{g}$ )	15	15	20	25	25
$\lambda$	0.8	1.0	0.9	0.7	0.8
$g_{m}$	1.7	1.0	1.6	1.5	1.4
Baseline	$\times$	$\times$	$\times$	$\times$	$\times$
Our Policy	$\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$

TABLE VI: Comparison of the baseline controller and the learned policy with varied

m_{w}

\lambda

, and

g_{m}

Fig. 9: Roll angle variations with changes in $m_{w}$ , $\lambda$ and $g_{m}$ .

Fig. 10: Comparison of the average reward across three cases, computed using a moving average with a window size of 19.

Fig. 11: Roll angle variations of the MBR with different $m_{\phi}$ .

Fig. 12: (a) Experimental setup; (b) Visualization of the action sequence of an MBR for achieving an inverted pose.

Weights	$MBR_{1}$	$MBR_{2}$	$MBR_{3}$	$MBR_{4}$	$MBR_{5}$
$m_{w1}(g)$	$25$	$26.07$	$25$	$27.59$	$25$
$m_{w2}(g)$	$0$	$0$	$1.07$	$0$	$2.59$

TABLE VII: Configuration of the extra weights in the physical experiment

Fig. 13: Roll angle variation of the MBR with different $m_{w}$ .

Robotic Control

Robotic control studies how machines use sensing and computation to produce reliable movement and interaction in changing environments.

See all 584 papers

AcroRL: Learning Aggressive Quadrotor Inversion using Bidirectional Thrust
May 23, 2026Gabriel Rodriguez, Henri Sayag, Abhishek Rathod +4RL Control Aerial Robotics
Carnegie Mellon University · Shield AI
ASTER: Attitude-aware Suspended-payload Quadrotor Traversal via Efficient Reinforcement Learning
Mar 11, 2026Dongcheng Cao, Jin Zhou, Shuo LiQuadrotor Control Robotic RL
College of Control Science and Engineering, Zhejiang University, Hangzhou 310027, China.
Bi-Level Reinforcement Learning Control for an Underactuated Blimp via Center-of-Mass Reconfiguration
May 2, 2026Xiaorui Wang, Hongwu Wang, Yue Fan +2RL Control Aerial Robotics

May 23, 2026J/K move · Enter open · S save

AcroRL: Learning Aggressive Quadrotor Inversion using Bidirectional Thrust

Gabriel Rodriguez, Henri Sayag, Abhishek Rathod, John Stecklein+3

Carnegie Mellon University · Shield AI

RL Control Aerial Robotics Quadrotor Control

Open paper AskPDF arXiv

Bidirectional thrust grants quadrotors a second equilibrium condition and increased control authority, expanding the envelope of possible aggressive maneuvers and enabling inverted flight, perching, and sensing. Prior geometric control approaches extend differential flatness through Hopf fibration-based attitude representations to support bidirectional thrust, but struggle with actuator saturation and motor reversal delay during inversions, requiring heuristic thrust posture scheduling and waypoint tuning. We propose a learning-based framework that modulates a constant reference trajectory to perform compact, position-constrained quadrotor inversions while remaining compatible with traditional trajectory generation and tracking across flight regimes. Separate policies are trained via reinforcement learning for nominal-to-inverted and inverted-to-nominal transitions. In JAX-based simulation, the proposed method achieves the lowest position deviation and settling time across all evaluated baselines, reducing position root mean square error (RMSE) by 32% and settling time by 57% relative to the strongest optimization-based baseline. Hardware experiments demonstrate successful inversion across multiple yaw configurations with position RMSE below 0.35m, and compatibility with downstream trajectory generation and control through circular flight in both regimes. Additionally, we provide an open-source implementation of the proposed framework.

Abstract

Robotic Control

Explore similar work

AcroRL: Learning Aggressive Quadrotor Inversion using Bidirectional Thrust

ASTER: Attitude-aware Suspended-payload Quadrotor Traversal via Efficient Reinforcement Learning

Bi-Level Reinforcement Learning Control for an Underactuated Blimp via Center-of-Mass Reconfiguration