Learning the Dynamics of Deformable Linear Objects Through Coupled Oscillators for Bimanual Robotic Manipulation
Authors: Iván López-Broceño, Maximilian Stölzle, Daniel Feliu-Talegon, Daniela Rus, Cosimo Della Santina
Organizations: Cognitive Robotics, Delft University of Technology, 2628 Delft, CD, Netherlands · Computer Science and Artificial Intelligence Laboratory, Massachusetts Institute of Technology, Cambridge, MA 02139, USA
Many learning-based approaches for deformable linear objects (DLOs) rely on simplified deformation models that neglect dynamic effects and the mechanical response of the objects, limiting their applicability to lightweight, compliant objects and slow motions. Inspired by recent advances in learning physically structured dynamics with coupled oscillator networks (CONs), we propose Φ-CON for learning force-driven DLO dynamics. The model combines a floating-base coupled oscillator network (FBCON), whose latent potential energy depends only on the DLO's shape, with a learned affine diffeomorphism Φ between observed positions along the DLO and latent coordinates. Its shape dynamics are stable by construction, and its Lagrangian structure yields explicit expressions for kinetic and potential energy. We exploit this structure to design a model-based proportional-derivative (PD) controller that compensates for the learned potential forces, thereby establishing closed-loop stability guarantees and ensuring convergence. We evaluate the approach on four simulated DLOs and a real bimanual robot setup. In simulation, Φ-CON achieves prediction accuracy comparable to neural ordinary differential equation (NODE) models and Lagrangian neural networks (LNNs), with 97.2% and 90.2% fewer parameters, respectively. Relative to a model-free PD baseline, the proposed controller reduces mean endpoint regulation error by 97.5-99.7% in simulation and achieves a mean per-target reduction of 27.9% across both endpoints in real-world experiments, demonstrating the importance of explicitly accounting for the mechanical response of stiff and heavy objects.
Figures & tables
Figure 1: Coordinate transformations of Φ -CON. In observation space (steps 1-3), fixed maps express the marker positions x as a base qb and a shape qr relative to it, and standardize them. A learned affine map acting only on the shape leads to latent space (steps 4-5), where the potential is flat along the base, with equilibria zw,r⋆ at any base position.
Figure 2: Training of Φ -CON. Φ maps observations and inputs from observation space to latent space , where the FBCON is rolled out for H steps and mapped back by Φ−1 . The losses ℓz and ℓx compare the rollout with the data in each space, and ℓss fits the holding forces ga(x) to τ at rest.
Position RMSE [mm] ↓
NODE
LNN
Φ -CON (ours)
Baseline
14.6±0.6
17.6±1.3
14.0±0.6
Heavy
12.5±1.1
13.6±0.7
6.6±0.9
Low damping
15.4±1.6
24.8±1.6
14.8±1.0
Compliant
11.0±1.3
12.9±0.6
4.5±0.5
Table 1: Prediction error in simulation: Position RMSE [mm] for predictions 2s into the future (mean ± std over seeds).
Endpoint error [mm] ↓
PD
PD+FF ( LNN )
PD+FF ( Φ -CON)
Baseline
238.6±4.7
2.5±6.9
1.5±3.5
Heavy
459.4±14.9
2.3±5.5
2.7±5.9
Low damping
238.9±4.6
3.6±5.2
5.9±9.5
Compliant
240.5±10.1
0.7±1.4
0.8±0.6
CPU time [ \SIUnitSymbolMicros ] ↓
7.8±1.1
25.5±2.9
12.6±3.4
GPU time [ \SIUnitSymbolMicros ] ↓
47.2±8.9
114.6±4.1
72.1±8.1
Table 2: Regulation error and computational cost in simulation: Final endpoint regulation error [mm] across the four simulated DLO datasets (mean ± std over test targets and both endpoints), and computation time of one control step for a new target (mean ± std, \Cref app:force-prediction).
Figure 3: Shape regulation in simulation: regulation from a straight rest configuration to the target shape xd ( dashed line ). Left: shape snapshots (axes in meters), with trails showing the preceding 0.6s of motion. Right: shape error over time.
Figure 4: Prediction error in experiments: position (left) and velocity (right) RMSE over 2.5s rollouts on the test trajectories of the electrical cable for NODE , LNN , FBCON , and Φ -CON . The gray region marks the 2s training horizon.
Figure 5: Shape regulation in experiments: regulation to four successive unseen target shapes xd (changes at dotted lines) with PD and PD+FF ( Φ -CON) . Top: shape error. Middle: targets ( green ) and cable configurations at five instants (axes in meters). Bottom: PD+FF photographs, targets (red).
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 6: A DLO represented by N 3D marker positions {x1,…,xN} and actuated by Cartesian forces at both endpoints.
DLO Dataset
E [Pa]
G [Pa]
ρ [kg/m 3 ]
d [N ⋅ m ⋅ s/rad]
Baseline
6×107
1×107
3×103
1.5×10−2
Heavy
6×107
1×107
7.8×103
1.5×10−2
Low damping
6×107
1×107
3×103
3×10−3
Compliant
3×106
1×106
3×103
1.5×10−2
Appendix
Table 3: Physical parameters of the four simulated DLO . Bold values differ from the baseline.
NODE
LNN
FBCON
Φ -CON
Simulation (all four DLOs)
Learning rate
6.5\times10−4
1.0\times10−4 †
2.9\times10−3
2.2\times10−3
Weight decay
0
0
3.9\times10−4
3.8\times10−2
wxvel
1.27
29.0
13.1
31.3
λss
—
0.36
0.65
0.50
Epochs
5000
2000
5000
5000
Appendix
Table 4: Optimization and loss hyperparameters of all models. wxvel : weight of the velocity term in ℓx (position term weighted 1); —: not part of the model.
Figure 7: Prediction error in simulation: position (left) and velocity (right) RMSE over 2s rollouts for NODE , LNN , FBCON , and Φ -CON on the baseline dataset. The gray region marks the 1s training horizon.
Figure 8: Motion prediction in simulation: predictions along a high-velocity trajectory from NODE , LNN , FBCON , and Φ -CON . Thin lines trace motion between snapshots. NODE accumulates error, LNN diverges beyond its 1s training horizon, and FBCON lags behind the ground truth; Φ -CON follows the motion throughout.
Left endpoint
Right endpoint
Time [ \SIUnitSymbolMicros ]
Model
X
Y
Z
X
Y
Z
All
CPU
GPU
Params
MLP
39.77%
12.90%
16.86%
40.78%
6.55%
24.89%
20.47%
15.6±2.4
66.5±7.6
35674
LNN
0.12%
0.23%
0.09%
0.11%
0.27%
0.08%
0.13%
25.4±3.9
105.2±5.0
203584
Φ -CON
0.19%
0.20%
0.15%
0.20%
0.20%
0.14%
0.17%
13.9±3.0
64.8±15.8
20031
Appendix
Table 8: Force prediction error and computational cost in simulation: Median relative endpoint potential-force prediction error, normalized by the endpoint force magnitude, computation time per prediction (mean ± std), and number of parameters.
Kp [N/m]
Kd [N s/m]
PD
PD+FF ( Φ -CON)
Reduction
10
1.0
238.6±4.7
1.5±3.5
99.4%
50
2.24
48.2±1.1
0.44±0.66
99.1%
200
4.47
12.1±0.4
0.15±0.21
98.8%
Appendix
Table 9: Regulation error in simulation for varying control gains: Final endpoint regulation error [mm] on the baseline DLO for increasing PD gains, with Kd∝Kp to keep the damping ratio (mean ± std over the 28 test targets and both endpoints).
Most research on the manipulation of deformable objects focuses on lightweight systems with negligible mechanical response, effectively restricting attention to quasi-static regimes. This assumption excludes a broad class of practically relevant objects, such as hoses, pipes, and wiring harnesses, whose dynamics cannot be ignored during manipulation. In this work, we address this limitation by introducing a closed-loop control architecture that explicitly accounts for object dynamics and recasts manipulation as a shape-regulation problem. Control is achieved by modulating forces and torques applied at multiple fixed points along the object. This approach builds on three methodological contributions: a fully dynamic model of linear deformable objects based on discrete strain parameterizations; an extension of the notion of actuation coordinates to SE(3), yielding a structured and inherently underactuated control architecture; and nonlinear feedback strategies providing explicit conditions for steady-state convergence to desired configurations. Extensive simulations on representative manipulation tasks demonstrate the performance gains enabled by the proposed modelbased formulation. We finally validate the approach experimentally through a real-time closed-loop implementation with online shape estimation, confirming its practical feasibility and effectiveness
Daniel Feliu-Talegon, Cosimo Della Santina
Department of Cognitive Robotics, Delft University of Technology, 2628 CN Delft, The Netherlands · Institute of Robotics and Mechatronics, German Aerospace Center (DLR), 82234 Oberpfaffenhofen, Germany
The manipulation of Deformable Linear Objects (DLOs) such as cables poses a significant challenge for automation due to their infinite degrees of freedom and non-linear dynamics. In this paper we present a machine learning based optimal control approach for the manipulation of DLOs. This approach is divided into two main components: modeling and control. For modeling the dynamics of the DLO, we propose a learning based approach using a bidirectional Long Short-Term Memory (biLSTM) network. The biLSTM network is trained on synthetic data generated by the MuJoCo physics engine. For manipulating the DLO, a model predictive control strategy that employs Model Predictive Path Integral (MPPI) control is selected. The proposed approach is evaluated through simulation and experiments. The results demonstrate the effectiveness of the proposed method in achieving accurate and efficient manipulation of DLOs.
Lukas Zeh, Johannes Meiwaldt, Zexu Zhou +2
Institute for Control Engineering of Machine Tools and Manufacturing Units, University of Stuttgart, Stuttgart, Germany
Ropes, cables, and other deformable linear objects appear in tasks from untangling to cable routing and suturing, yet controlling their shape remains a challenge in robot manipulation. We study model-based shape control in a general setting: the object lies unfixated on a support surface and two arms may grasp and move it anywhere along its length. Because each arm chooses a grasp point, direction, and magnitude, the joint action space is combinatorially large, and the dynamics model's per-prediction cost bounds how much of it a planner can search. We present ForwardDLO, a recurrent latent dynamics model for this unfixated bimanual setting that predicts per-segment displacements grounded in the observed rope state at every step. Our model reaches accuracy comparable to more expensive baselines while containing no explicit segment-to-segment operations, which makes batched evaluation of candidate actions cheap. On open-loop prediction of real rope motion it reaches the lowest error of the learned models we evaluate, 13% below the strongest baseline. Within a fixed time budget it scores 8 to 22 times more candidate actions than models of comparable accuracy while matching them in real-world shape matching; and on a simulated routing task at a 30Hz control rate, this throughput converts into 98% task success versus at most 30% for the baselines at their own budgets. We release the model, code, and a dataset of 2.42 million simulated and 14,107 real rope transitions at https://anonymous.4open.science/r/ForwardDLO/
Tim Missal, Berk Guler, Lucas Domingues +3
Technical University of Darmstadt · Honda Research Institute Europe GmbH · School of Electrical and Computer Engineering, Universidade Estadual de Campinas (UNICAMP), Brazil +4