SwingRL: Adaptive Observation Reinforcement Learning with World-Model Prediction for Cable-Suspended Hoisting Control

Guangming Wang; Xiaoyu Zhang; Yucheng Xin; Wanli Ma; Jiucai Liu; Yunxiang Ma; Joe Ingham; Haibing Wu

cs.ROSep 27, 2026

SwingRL: Adaptive Observation Reinforcement Learning with World-Model Prediction for Cable-Suspended Hoisting Control

Authors: Guangming Wang, Xiaoyu Zhang, Yucheng Xin, Wanli Ma, Jiucai Liu, Yunxiang Ma, Joe Ingham, Haibing Wu,

Organizations: Department of Engineering, University of Cambridge, Cambridge, CB2 1PZ, U.K. · Tsinghua University, Beijing, China · School of Engineering, Cardiff University, Cardiff, U.K. · The University of Hong Kong, Hong Kong, China

Abstract

Cable-suspended hoisting is widely used to move heavy or bulky payloads that cannot be handled conveniently by rigid pick-and-place systems, for example in crane-assisted construction. Robotic hoisting using flexible cables is challenging because payload motion is underactuated, external disturbances vary, and delayed or lost visual observations can make the perceived payload state stale at control execution. These effects are particularly critical during precise insertion of a suspended payload's sockets onto rebar pins, which is a very common task in construction environments. We present SwingRL, a residual reinforcement-learning (RL) framework that combines an age-aware world model, a classical anti-swing prior, and a recurrent residual policy to address two coupled problems: stale feedback and uncertain dynamics. The world model propagates the newest received payload observation to the current control step using the executed commands, providing a time-aligned state estimate under delayed and lossy sensing. The prior supplies nominal tracking and swing damping. The residual policy learns bounded corrections to the prior rather than the complete control law, compensating for system-parameter variation, external disturbances, and remaining state-estimation errors. We evaluate SwingRL against classical and learning-based baselines across a cumulative difficulty ladder covering system-parameter variation, wind disturbance, degraded sensing, and strong gusts. Under the most difficult setting, SwingRL achieves 69.5% strict and 77.3% broad success, exceeding all baselines by at least 60 percentage points, respectively. World-model ablations support the role of time-aligned state estimation in maintaining insertion success as observation loss increases. Finally, without real-robot fine-tuning, SwingRL achieves 90% success on the physical rig.

Figures & tables

Fig. 2: Swing-RL. Delayed and intermittent payload observations and recorded commands drive a recurrent world model. Its current-state prediction is fused with end-effector proprioception and supplied to both the prior controller and the residual actor. The residual action enters the prior controller’s target before gating, saturation, and smoothing. The asymmetric critic uses privileged state and channel information only during training.

	Entries	Dim	Source
1	Swing offset $\mathbf{r}$ and rate $\dot{\mathbf{r}}$	4	Camera + proprioception
2	Payload height error $\Delta z$ and rate $\dot{z}_{\mathrm{pl}}$	2	Camera
3	Payload offset $(\mathbf{p}_{\mathrm{pl},xy}-\mathbf{p}^{*}_{xy})$ , tilt, yaw	4	Camera
4	End-effector offset $(\mathbf{p}_{\mathrm{ee},xy}-\mathbf{p}^{*}_{xy})$	2	Proprioception
5	End-effector velocity $\mathbf{v}_{\mathrm{ee},xy}$ , $\dot{z}_{\mathrm{ee}}$	3	Proprioception
6	Prior command $\mathbf{a}^{\mathrm{base}}_{t}$	3	Controller

TABLE I: Actor observation

\mathbf{o}_{t}

(25-D), normalized by per-entry scales. The world model predicts these entries; current proprioception replaces rows 4–5 and is used to recompute row 1 before the estimation.

	Stage 1	Stage 2
Steps	$1.2$ M	$+0.8$ M
Wind (m/s)	$0.5{\to}10$	$4$ – $8$
Cable length (m)	$0.50{\to}0.40$ – $0.60$	$0.40$ – $0.60$
Payload mass (kg)	$6.15{\to}5.5$ – $7.0$	$5.5$ – $7.0$
Initial XY offset range (mm)	$10{\to}120$	$120$
Frame delay (periods)	$0$	$\{0,1,2\}$ , ramped

TABLE II: Two-stage training schedule. Arrows denote within-stage ramps. Stage 2 adds 0.8 M steps to the Stage 1 checkpoint; delay and loss are introduced through a performance-gated curriculum.

Level	Name	Cable model	Clearance	Start perturbation	Wind	Cable length	Payload mass	Sensing
L0	Simplified	4 links	15 mm	–	–	0.50 m	6.15 kg	exact
L1	Diverse system	10 links	7.5 mm	120 mm, 0.15 m/s, 2 ∘	–	0.40–0.60 m	5.5–7.0 kg	exact
L2	External disturbance	10 links	7.5 mm	120 mm, 0.15 m/s, 2 ∘	2–5 m/s, gusts	0.40–0.60 m	5.5–7.0 kg	exact
L3	Real channels	10 links	7.5 mm	120 mm, 0.15 m/s, 2 ∘	2–5 m/s, gusts	0.40–0.60 m	5.5–7.0 kg	noisy, delayed, lossy
L4	Strong gusts	10 links	7.5 mm	120 mm, 0.15 m/s, 2 ∘	4–8 m/s, gusts	0.40–0.60 m	5.5–7.0 kg	noisy, delayed, lossy
L5	Real case	steel cables	$\approx$ 7.5 mm	$>$ 120 mm	$\approx$ 7 m/s	0.50 m	6.15 kg	real camera

TABLE III: Cumulative Difficulty Ladder Configuration. L0–L4 are incremental: each level retains the factors above and changes only the bold entries. L0 is a separate simplified base used to sanity-check the classical controllers. L5 is for the physical platform. Clearance is defined as the difference between the socket radius and the rebar radius.

Method	Metric	L0	L1	L2	L3	L4
Method	Metric	Simplified	Diverse system	External disturbance	Real channels	Strong gusts
PID [ 15 ]	Success rate $\mathbf{SR}_{s}$ / $\mathbf{SR}_{b}$ (%)	70.7 / 100.0	0.0 / 0.0	0.0 / 0.0	0.0 / 0.8	0.0 / 0.8
	Swing angle $\theta_{s}$ ( ∘ )	0.56	13.94	14.22	10.59	10.60
	Swing energy $\mathcal{E}$ (mJ)	2.3	1116	1123	1359	1345
MPC [ 11 ]	Success rate $\mathbf{SR}_{s}$ / $\mathbf{SR}_{b}$ (%)	96.1 / 96.1	0.0 / 0.0	0.0 / 0.0	0.0 / 0.0	0.0 / 0.0
	Swing angle $\theta_{s}$ ( ∘ )	0.38	6.25	6.31	3.47	4.03

TABLE IV: Baseline Comparison. Task performance comparison of the proposed method and baselines across difficulty levels, evaluated on the identical 256 randomized episodes at each difficulty level.

Fig. 3: Swing Energy Comparison. Swing-energy trajectories in simulation across the five difficulty levels in Table III , from left to right: L0 (simplified), L1 (diverse system), L2 (external disturbance), L3 (real channels), and L4 (strong gusts). Markers indicate the end of the episode.

(a) Residual and curriculum – 256 paired episodes
Setting	Variant	$\mathbf{SR}_{s}$ (%)	$\mathbf{SR}_{b}$ (%)	$\theta_{s}$ ( ∘ )
L1 (diverse system)	ours	96.9	99.2	1.19
	w/o residual	64.8	87.9	0.99
	w/o curriculum	96.1	96.5	1.15
L2 (external disturbance)	ours	89.8	95.3	1.48
	w/o residual	53.1	78.5	1.30

TABLE V: Ablation Experiments. Ablations of the residual RL, the Stage-1 curriculum, the world model, and the wind band tests. Best value per setting in bold.

Setting	Variant	SR (%)	Successes
(a) Rig channel
$p{\approx}0.03$ (measured)	ours	90	18/20
	w/o WM	80	16/20
(b) World model – frame loss injected
$p{\approx}0.33$ (0.3 inj.)	ours	85	17/20
$p{\approx}0.33$ (0.3 inj.)	w/o WM	70	14/20

TABLE VI: Real-Rig Test. Insertion success rate on the physical rig. Panel (a) fully applies our method; panel (b) injects additional lossy sensing. Best per setting in bold.

Fig. 4: Motion Visualization. Four runs on the physical rig (rows 1-4: PID, MPC, Residual-RL, SwingRL). Each row shows ten uniformly time-sampled frames; the rightmost column overlays the cyan payload outlines to show the swing envelope, with opacity increasing from the first frame to the last.

Robotic Control

Robotic control studies how machines use sensing and computation to produce reliable movement and interaction in changing environments.

See all 584 papers

Learning to Throw: Agile and Accurate Cable-Suspended Payload Delivery with a Quadrotor
Jun 25, 2026Yifan Zhai, Elia Raimondi, Yunfan Ren +4RL for Robotics Robotics Simulation
Robotics and Perception Group, Department of Informatics, University of Zurich, Switzerland
ASTER: Attitude-aware Suspended-payload Quadrotor Traversal via Efficient Reinforcement Learning
Mar 11, 2026Dongcheng Cao, Jin Zhou, Shuo LiQuadrotor Control Robotic RL
College of Control Science and Engineering, Zhejiang University, Hangzhou 310027, China.
HOIST: Humanoid Optimization with Imitation and Sample-efficient Tuning for Manipulating Suspended Loads
May 29, 2026Songyang Liu, Shunyu Yao, Dingyuan Huang +1RL for Robotics Humanoid Robot Control
Department of Civil and Coastal Engineering University of Florida United States

Jun 25, 2026J/K move · Enter open · S save

Learning to Throw: Agile and Accurate Cable-Suspended Payload Delivery with a Quadrotor

Yifan Zhai, Elia Raimondi, Yunfan Ren, Ismail Geles+3

Robotics and Perception Group, Department of Informatics, University of Zurich, Switzerland

RL for Robotics Robotics Simulation Robotic Manipulation Quadrotor Control Aerial Manipulation

Open paper AskPDF arXiv

Quadrotors offer the agility needed to rapidly transport suspended payloads during time-critical applications, including search-and-rescue and medical delivery. While suspended-payload transport and traversal for these missions are well studied, the highly dynamic targeted release of the payload remains comparatively underexplored. State-of-the-art approaches typically rely on model-based trajectory optimization and tracking; however, these methods often yield sub-optimal performance due to conservative feasibility constraints, tracking errors, and the inherent difficulty of analytically modeling flexible rope dynamics. To overcome these limitations, we propose a hybrid simulation framework that couples a high-fidelity analytical quadrotor model with a physics solver for complex rope and payload interactions. By exchanging forces between the two domains at every step, we obtain a physically accurate simulation of the suspended-payload system. Leveraging this environment, we train a deep reinforcement learning (RL) policy that executes agile, accurate payload throws to designated targets. Deployed zero-shot on hardware, our RL policy pushes the boundary of the agility-accuracy trade-off, outperforming the model-based baseline by reducing the landing error by up to 50% and the throw duration by up to 30%. Ablation studies confirm that the coupled simulation is the key enabler of these gains. We further show that the same pipeline trains a policy driven by visual observations rather than an explicit state estimate, achieving accuracy comparable to that of the state-based policy. To accelerate future research in dynamic aerial manipulation, we open-source the simulator to the community upon acceptance.

Abstract

Robotic Control

Explore similar work

Learning to Throw: Agile and Accurate Cable-Suspended Payload Delivery with a Quadrotor

ASTER: Attitude-aware Suspended-payload Quadrotor Traversal via Efficient Reinforcement Learning

HOIST: Humanoid Optimization with Imitation and Sample-efficient Tuning for Manipulating Suspended Loads