ASTER: Attitude-aware Suspended-payload Quadrotor Traversal via Efficient Reinforcement Learning

Dongcheng Cao; Jin Zhou; Shuo Li

cs.ROMar 11, 2026

ASTER: Attitude-aware Suspended-payload Quadrotor Traversal via Efficient Reinforcement Learning

Authors: Dongcheng Cao, Jin Zhou, Shuo Li

Organizations: College of Control Science and Engineering, Zhejiang University, Hangzhou 310027, China.

Abstract

Agile maneuvering of the quadrotor cable-suspended system is significantly hindered by its non-smooth hybrid dynamics. While model-free Reinforcement Learning (RL) circumvents explicit differentiation of complex models, achieving attitude-constrained or inverted flight remains an open challenge due to the extreme reward sparsity under strict orientation requirements. This paper presents ASTER, a robust RL framework that achieves, to our knowledge, the first successful autonomous inverted flight for the cable-suspended system. We propose hybrid-dynamics-informed state seeding (HDSS), an initialization strategy that back-propagates target configurations through physics-consistent kinematic inversions across both taut and slack cable phases. HDSS enables the policy to discover aggressive maneuvers that are unreachable via standard exploration. Extensive simulations and real-world experiments demonstrate remarkable agility, precise attitude alignment, and robust zero-shot sim-to-real transfer across complex trajectories.

Figures & tables

Panel of Figure 1 — Fig. 1: Composite time-lapse illustration of the real-world inverted flight maneuvers, showcasing the continuous motion and attitude transitions.

Fig. 2: Overview of the training pipeline of the proposed framework. We highlight the design of hybrid-dynamics-informed state seeding (HDSS), a critical strategy for state initialization to overcome exploration bottlenecks in reward-sparse tasks. HDSS back-propagates target configurations through hybrid phases to provide physics-consistent initializations. This strategy balances efficient maneuver discovery with global robustness, enabling the policy to master complex flight dynamics.

Fig. 3: Schematic of the reference frames and payload state: including the world frame

\bm{W}

, the quadrotor body frame

\bm{B}

, and the target frame

\bm{T}

(illustrated for a waypoint with a downward-pointing

Z

-axis). The vector

\mathbf{x}_{l}^{\bm{B}}

denotes the payload position described in the quadrotor body frame.

Parameter	Value	Parameter	Value
$T_{\text{max}}$ [m/s 2 ]	$3.5g$	$\bm{\omega}_{\text{max}}$ [rad/s]	$[10,10,3]$
$\mathbf{k}_{q}$ [m]	$[4,4,3]$	$\mathbf{k}_{v}$ [m/s]	$[5,5,5]$
$\mathbf{k}_{l}$ [m]	$[0.5,0.5,0.5]$	$\sigma_{p}$	$3.0$
$\sigma_{\theta}$	$2.0$	$\epsilon_{\theta}$ [ ∘ ]	25
$L$ [m]	$0.75$	$C$	$5$
$\lambda_{1}$	$10$	$\lambda_{2}$	$10$

TABLE I: Simulation and training Parameters

Fig. 4: Agile trajectory visualization on 3 representative tracks. The policy is evaluated on (a) Ribbon , (b) Croissant , and (c) Multi-heading tracks. The quadrotor’s velocity is visualized via heatmaps, with payload trajectories and system keyframes overlaid. The high-speed, attitude-constrained navigation across all tracks validates the robustness and agility of the learned policy. For visual clarity, only inverted waypoints are denoted by circular markers with embedded arrows indicating the target z-axis direction, while standard upright waypoints are hidden.

Fig. 5: Time-lapse visualization of agile flight on the Croissant track in the Genesis simulator. The snapshot sequence demonstrates the policy’s capability to execute consecutive inverted maneuvers while maintaining precise attitude alignment and system stability.

Track	WPs	Time	Avg Vel.	Max Vel.
Track	WPs	(s)	(m/s)	(m/s)
Ribbon	3	$1.61$	$2.77$	$5.03$
Croissant	11	$6.12$	$3.05$	$5.36$
Multi-heading	9	$5.47$	$3.44$	$5.32$

TABLE II: Performance Statistics Across Representative Maneuvering Tracks

Fig. 6: Ablation study of the hybrid-dynamics-informed state seeding (HDSS) strategy. The plots compare the proposed framework against a baseline without HDSS. The baseline remains trapped in a near-zero reward regime, while HDSS enables rapid discovery of high-reward maneuvers.

Variation	Payload Mass ( $m_{l}$ )	Cable Length ( $l$ )
$-40\%$	87.5% / 5.91s	96.5% / 5.97s
$-20\%$	93% / 6.00s	97% / 6.03s
$0\%$ (Nom.)	97.5% / 6.15s	98.5% / 6.15s
$+20\%$	91.5% / 6.28s	82.5% / 6.22s
$+40\%$	82% / 6.47s	56% / 6.31s

TABLE III: Robustness Metrics (SR /

T

) under Independent Parameter Variations

Fig. 7: Cable-suspended payload system in real-world experiments.

Panel of Figure 11 — Fig. 8: Reconstructed 3D trajectories of the quadrotor and payload during the agile inverted maneuvers shown in Fig. 1

Track	Completion Time (s)		Max Quad Vel. (m/s)
Track	Sim	Real	Sim	Real
Single Loop	0.98	1.34	3.83	4.01
Double Loops	2.87	3.14	7.05	6.63

TABLE IV: Comparison of Flight Statistics: Simulation vs. Real-world

Quadrotor Control

Quadrotor control studies how to stabilize, guide, and adapt aerial robots under demanding flight conditions and physical constraints.

See all 127 papers

Agile Fall Recovery for Quadrotors with Bidirectional Thrust via Reinforcement Learning
Jun 15, 2026Anke Zhao, Yuhang Zhong, Kenghou Hoi +5Robot Failure Recovery Robotic RL
Institute of Cyber-Systems and Control, College of Control Science and Engineering, Zhejiang University, Hangzhou 310027, China · Differential Robotics, Hangzhou 311121, China
Learning to Throw: Agile and Accurate Cable-Suspended Payload Delivery with a Quadrotor
Jun 25, 2026Yifan Zhai, Elia Raimondi, Yunfan Ren +4RL for Robotics Robotics Simulation
Robotics and Perception Group, Department of Informatics, University of Zurich, Switzerland
AcroRL: Learning Aggressive Quadrotor Inversion using Bidirectional Thrust
May 23, 2026Gabriel Rodriguez, Henri Sayag, Abhishek Rathod +4RL Control Aerial Robotics
Carnegie Mellon University · Shield AI

Jun 15, 2026J/K move · Enter open · S save

Agile Fall Recovery for Quadrotors with Bidirectional Thrust via Reinforcement Learning

Anke Zhao, Yuhang Zhong, Kenghou Hoi, Junyu Mou+4

Institute of Cyber-Systems and Control, College of Control Science and Engineering, Zhejiang University, Hangzhou 310027, China · Differential Robotics, Hangzhou 311121, China

Robot Failure Recovery Robotic RL Quadrotor Control

Open paper AskPDF arXiv

Autonomous fall recovery is a critical capability for quadrotors operating in real-world environments, where collisions or failures may leave the vehicle resting on the ground in an arbitrary attitude. This problem is challenging because recovery must be achieved under limited onboard sensing, in constrained free space, with ground contact, and in the presence of unknown disturbances. In this letter, we present an RL-based framework for autonomous fall recovery of a quadrotor from arbitrary ground attitudes to stable hover using only lightweight onboard sensors. To address severe partial observability and intermittent sensor invalidity, we train a recurrent policy within an asymmetric actor--critic architecture, leveraging an Incremental Nonlinear Dynamic Inversion (INDI) controller to track the policy output. Combined with high-fidelity simulations of motor response and optical flow, the overall training framework significantly reduces the sim-to-real gap. Simulation ablation studies validate the importance of the main design choices, while real-world experiments demonstrate zero-shot transfer and robust recovery under different initial attitudes, wind disturbances, and additional payloads. These results demonstrate that agile quadrotor fall recovery can be achieved without explicit state estimation using only limited and unreliable onboard sensing.

Abstract

Quadrotor Control

Explore similar work

Agile Fall Recovery for Quadrotors with Bidirectional Thrust via Reinforcement Learning

Learning to Throw: Agile and Accurate Cable-Suspended Payload Delivery with a Quadrotor

AcroRL: Learning Aggressive Quadrotor Inversion using Bidirectional Thrust