SwingRL: Adaptive Observation Reinforcement Learning with World-Model Prediction for Cable-Suspended Hoisting Control
Authors: Guangming Wang, Xiaoyu Zhang, Yucheng Xin, Wanli Ma, Jiucai Liu, Yunxiang Ma, Joe Ingham, Haibing Wu, Yixiong Jing, Olaf Wysocki, Brian Sheil
Organizations: Department of Engineering, University of Cambridge, Cambridge, CB2 1PZ, U.K. · Tsinghua University, Beijing, China · School of Engineering, Cardiff University, Cardiff, U.K. · The University of Hong Kong, Hong Kong, China
Cable-suspended hoisting is widely used to move heavy or bulky payloads that cannot be handled conveniently by rigid pick-and-place systems, for example in crane-assisted construction. Robotic hoisting using flexible cables is challenging because payload motion is underactuated, external disturbances vary, and delayed or lost visual observations can make the perceived payload state stale at control execution. These effects are particularly critical during precise insertion of a suspended payload's sockets onto rebar pins, which is a very common task in construction environments. We present SwingRL, a residual reinforcement-learning (RL) framework that combines an age-aware world model, a classical anti-swing prior, and a recurrent residual policy to address two coupled problems: stale feedback and uncertain dynamics. The world model propagates the newest received payload observation to the current control step using the executed commands, providing a time-aligned state estimate under delayed and lossy sensing. The prior supplies nominal tracking and swing damping. The residual policy learns bounded corrections to the prior rather than the complete control law, compensating for system-parameter variation, external disturbances, and remaining state-estimation errors. We evaluate SwingRL against classical and learning-based baselines across a cumulative difficulty ladder covering system-parameter variation, wind disturbance, degraded sensing, and strong gusts. Under the most difficult setting, SwingRL achieves 69.5% strict and 77.3% broad success, exceeding all baselines by at least 60 percentage points, respectively. World-model ablations support the role of time-aligned state estimation in maintaining insertion success as observation loss increases. Finally, without real-robot fine-tuning, SwingRL achieves 90% success on the physical rig.
Figures & tables
Fig. 2: Swing-RL. Delayed and intermittent payload observations and recorded commands drive a recurrent world model. Its current-state prediction is fused with end-effector proprioception and supplied to both the prior controller and the residual actor. The residual action enters the prior controller’s target before gating, saturation, and smoothing. The asymmetric critic uses privileged state and channel information only during training.
Entries
Dim
Source
1
Swing offset r and rate r˙
4
Camera + proprioception
2
Payload height error Δz and rate z˙pl
2
Camera
3
Payload offset (ppl,xy−pxy∗) , tilt, yaw
4
Camera
4
End-effector offset (pee,xy−pxy∗)
2
Proprioception
5
End-effector velocity vee,xy , z˙ee
3
Proprioception
6
Prior command atbase
3
Controller
TABLE I: Actor observation ot (25-D), normalized by per-entry scales. The world model predicts these entries; current proprioception replaces rows 4–5 and is used to recompute row 1 before the estimation.
Stage 1
Stage 2
Steps
1.2 M
+0.8 M
Wind (m/s)
0.5→10
4 – 8
Cable length (m)
0.50→0.40 – 0.60
0.40 – 0.60
Payload mass (kg)
6.15→5.5 – 7.0
5.5 – 7.0
Initial XY offset range (mm)
10→120
120
Frame delay (periods)
0
{0,1,2} , ramped
TABLE II: Two-stage training schedule. Arrows denote within-stage ramps. Stage 2 adds 0.8 M steps to the Stage 1 checkpoint; delay and loss are introduced through a performance-gated curriculum.
Level
Name
Cable model
Clearance
Start perturbation
Wind
Cable length
Payload mass
Sensing
L0
Simplified
4 links
15 mm
–
–
0.50 m
6.15 kg
exact
L1
Diverse system
10 links
7.5 mm
120 mm, 0.15 m/s, 2 ∘
–
0.40–0.60 m
5.5–7.0 kg
exact
L2
External disturbance
10 links
7.5 mm
120 mm, 0.15 m/s, 2 ∘
2–5 m/s, gusts
0.40–0.60 m
5.5–7.0 kg
exact
L3
Real channels
10 links
7.5 mm
120 mm, 0.15 m/s, 2 ∘
2–5 m/s, gusts
0.40–0.60 m
5.5–7.0 kg
noisy, delayed, lossy
L4
Strong gusts
10 links
7.5 mm
120 mm, 0.15 m/s, 2 ∘
4–8 m/s, gusts
0.40–0.60 m
5.5–7.0 kg
noisy, delayed, lossy
L5
Real case
steel cables
≈ 7.5 mm
> 120 mm
≈ 7 m/s
0.50 m
6.15 kg
real camera
TABLE III: Cumulative Difficulty Ladder Configuration. L0–L4 are incremental: each level retains the factors above and changes only the bold entries. L0 is a separate simplified base used to sanity-check the classical controllers. L5 is for the physical platform. Clearance is defined as the difference between the socket radius and the rebar radius.
Method
Metric
L0
L1
L2
L3
L4
Simplified
Diverse system
External disturbance
Real channels
Strong gusts
PID [ 15 ]
Success rate SRs / SRb (%)
70.7 / 100.0
0.0 / 0.0
0.0 / 0.0
0.0 / 0.8
0.0 / 0.8
Swing angle θs ( ∘ )
0.56
13.94
14.22
10.59
10.60
Swing energy E (mJ)
2.3
1116
1123
1359
1345
MPC [ 11 ]
Success rate SRs / SRb (%)
96.1 / 96.1
0.0 / 0.0
0.0 / 0.0
0.0 / 0.0
0.0 / 0.0
Swing angle θs ( ∘ )
0.38
6.25
6.31
3.47
4.03
TABLE IV: Baseline Comparison. Task performance comparison of the proposed method and baselines across difficulty levels, evaluated on the identical 256 randomized episodes at each difficulty level.
Fig. 3: Swing Energy Comparison. Swing-energy trajectories in simulation across the five difficulty levels in Table III , from left to right: L0 (simplified), L1 (diverse system), L2 (external disturbance), L3 (real channels), and L4 (strong gusts). Markers indicate the end of the episode.
Setting
Variant
SRs (%)
SRb (%)
θs ( ∘ )
(a) Residual and curriculum – 256 paired episodes
L1 (diverse system)
ours
96.9
99.2
1.19
w/o residual
64.8
87.9
0.99
w/o curriculum
96.1
96.5
1.15
L2 (external disturbance)
ours
89.8
95.3
1.48
w/o residual
53.1
78.5
1.30
TABLE V: Ablation Experiments. Ablations of the residual RL, the Stage-1 curriculum, the world model, and the wind band tests. Best value per setting in bold.
Setting
Variant
SR (%)
Successes
(a) Rig channel
p≈0.03 (measured)
ours
90
18/20
w/o WM
80
16/20
(b) World model – frame loss injected
p≈0.33 (0.3 inj.)
ours
85
17/20
w/o WM
70
14/20
TABLE VI: Real-Rig Test. Insertion success rate on the physical rig. Panel (a) fully applies our method; panel (b) injects additional lossy sensing. Best per setting in bold.
Fig. 4: Motion Visualization. Four runs on the physical rig (rows 1-4: PID, MPC, Residual-RL, SwingRL). Each row shows ten uniformly time-sampled frames; the rightmost column overlays the cyan payload outlines to show the swing envelope, with opacity increasing from the first frame to the last.
Quadrotors offer the agility needed to rapidly transport suspended payloads during time-critical applications, including search-and-rescue and medical delivery. While suspended-payload transport and traversal for these missions are well studied, the highly dynamic targeted release of the payload remains comparatively underexplored. State-of-the-art approaches typically rely on model-based trajectory optimization and tracking; however, these methods often yield sub-optimal performance due to conservative feasibility constraints, tracking errors, and the inherent difficulty of analytically modeling flexible rope dynamics. To overcome these limitations, we propose a hybrid simulation framework that couples a high-fidelity analytical quadrotor model with a physics solver for complex rope and payload interactions. By exchanging forces between the two domains at every step, we obtain a physically accurate simulation of the suspended-payload system. Leveraging this environment, we train a deep reinforcement learning (RL) policy that executes agile, accurate payload throws to designated targets. Deployed zero-shot on hardware, our RL policy pushes the boundary of the agility-accuracy trade-off, outperforming the model-based baseline by reducing the landing error by up to 50% and the throw duration by up to 30%. Ablation studies confirm that the coupled simulation is the key enabler of these gains. We further show that the same pipeline trains a policy driven by visual observations rather than an explicit state estimate, achieving accuracy comparable to that of the state-based policy. To accelerate future research in dynamic aerial manipulation, we open-source the simulator to the community upon acceptance.
Yifan Zhai, Elia Raimondi, Yunfan Ren +4
Robotics and Perception Group, Department of Informatics, University of Zurich, Switzerland
Agile maneuvering of the quadrotor cable-suspended system is significantly hindered by its non-smooth hybrid dynamics. While model-free Reinforcement Learning (RL) circumvents explicit differentiation of complex models, achieving attitude-constrained or inverted flight remains an open challenge due to the extreme reward sparsity under strict orientation requirements. This paper presents ASTER, a robust RL framework that achieves, to our knowledge, the first successful autonomous inverted flight for the cable-suspended system. We propose hybrid-dynamics-informed state seeding (HDSS), an initialization strategy that back-propagates target configurations through physics-consistent kinematic inversions across both taut and slack cable phases. HDSS enables the policy to discover aggressive maneuvers that are unreachable via standard exploration. Extensive simulations and real-world experiments demonstrate remarkable agility, precise attitude alignment, and robust zero-shot sim-to-real transfer across complex trajectories.
Dongcheng Cao, Jin Zhou, Shuo Li
College of Control Science and Engineering, Zhejiang University, Hangzhou 310027, China.
Manipulating suspended payloads with humanoid robots is challenging because the robot can only influence an underactuated, oscillatory load through whole-body motion and intermittent contact. Imitation learning provides safe initial behavior but does not directly optimize final placement, while reinforcement learning from scratch is unsafe and sample-inefficient on real humanoids. We present HOIST-Humanoid Optimized with Imitation and Sample-efficient Tuning for manipulating suspended loads. HOIST first finetunes a high-level vision-language-action (VLA) policy from virtual-reality (VR) teleoperation demonstrations and executes its commands through a whole-body controller. It then uses VLA rollouts and iterative batched RL to improve placement accuracy and stopping behavior. Experiments in simulation and on a real humanoid show that HOIST improves over imitation-only and additional-demonstration baselines; compared with pure VLA rollouts, HOIST reduces translational placement error by 19.9 cm and raw angular error by 3.56 degrees, demonstrating the potential of humanoids for underactuated material-handling tasks.
Songyang Liu, Shunyu Yao, Dingyuan Huang +1
Department of Civil and Coastal Engineering University of Florida United States