Learning the Dynamics of Deformable Linear Objects Through Coupled Oscillators for Bimanual Robotic Manipulation
Authors: Iván López-Broceño, Maximilian Stölzle, Daniel Feliu-Talegon, Daniela Rus, Cosimo Della Santina
Organizations: Cognitive Robotics, Delft University of Technology, 2628 Delft, CD, Netherlands · Computer Science and Artificial Intelligence Laboratory, Massachusetts Institute of Technology, Cambridge, MA 02139, USA
Many learning-based approaches for deformable linear objects (DLOs) rely on simplified deformation models that neglect dynamic effects and the mechanical response of the objects, limiting their applicability to lightweight, compliant objects and slow motions. Inspired by recent advances in learning physically structured dynamics with coupled oscillator networks (CONs), we propose Φ-CON for learning force-driven DLO dynamics. The model combines a floating-base coupled oscillator network (FBCON), whose latent potential energy depends only on the DLO's shape, with a learned affine diffeomorphism Φ between observed positions along the DLO and latent coordinates. Its shape dynamics are stable by construction, and its Lagrangian structure yields explicit expressions for kinetic and potential energy. We exploit this structure to design a model-based proportional-derivative (PD) controller that compensates for the learned potential forces, thereby establishing closed-loop stability guarantees and ensuring convergence. We evaluate the approach on four simulated DLOs and a real bimanual robot setup. In simulation, Φ-CON achieves prediction accuracy comparable to neural ordinary differential equation (NODE) models and Lagrangian neural networks (LNNs), with 97.2% and 90.2% fewer parameters, respectively. Relative to a model-free PD baseline, the proposed controller reduces mean endpoint regulation error by 97.5-99.7% in simulation and achieves a mean per-target reduction of 27.9% across both endpoints in real-world experiments, demonstrating the importance of explicitly accounting for the mechanical response of stiff and heavy objects.
Figures & tables
Figure 1: Coordinate transformations of Φ -CON. In observation space (steps 1-3), fixed maps express the marker positions x as a base qb and a shape qr relative to it, and standardize them. A learned affine map acting only on the shape leads to latent space (steps 4-5), where the potential is flat along the base, with equilibria zw,r⋆ at any base position.
Figure 2: Training of Φ -CON. Φ maps observations and inputs from observation space to latent space , where the FBCON is rolled out for H steps and mapped back by Φ−1 . The losses ℓz and ℓx compare the rollout with the data in each space, and ℓss fits the holding forces ga(x) to τ at rest.
Position RMSE [mm] ↓
NODE
LNN
Φ -CON (ours)
Baseline
14.6±0.6
17.6±1.3
14.0±0.6
Heavy
12.5±1.1
13.6±0.7
6.6±0.9
Low damping
15.4±1.6
24.8±1.6
14.8±1.0
Compliant
11.0±1.3
12.9±0.6
4.5±0.5
Table 1: Prediction error in simulation: Position RMSE [mm] for predictions 2s into the future (mean ± std over seeds).
Endpoint error [mm] ↓
PD
PD+FF ( LNN )
PD+FF ( Φ -CON)
Baseline
238.6±4.7
2.5±6.9
1.5±3.5
Heavy
459.4±14.9
2.3±5.5
2.7±5.9
Low damping
238.9±4.6
3.6±5.2
5.9±9.5
Compliant
240.5±10.1
0.7±1.4
0.8±0.6
CPU time [ \SIUnitSymbolMicros ] ↓
7.8±1.1
25.5±2.9
12.6±3.4
GPU time [ \SIUnitSymbolMicros ] ↓
47.2±8.9
114.6±4.1
72.1±8.1
Table 2: Regulation error and computational cost in simulation: Final endpoint regulation error [mm] across the four simulated DLO datasets (mean ± std over test targets and both endpoints), and computation time of one control step for a new target (mean ± std, \Cref app:force-prediction).
Figure 3: Shape regulation in simulation: regulation from a straight rest configuration to the target shape xd ( dashed line ). Left: shape snapshots (axes in meters), with trails showing the preceding 0.6s of motion. Right: shape error over time.
Figure 4: Prediction error in experiments: position (left) and velocity (right) RMSE over 2.5s rollouts on the test trajectories of the electrical cable for NODE , LNN , FBCON , and Φ -CON . The gray region marks the 2s training horizon.
Figure 5: Shape regulation in experiments: regulation to four successive unseen target shapes xd (changes at dotted lines) with PD and PD+FF ( Φ -CON) . Top: shape error. Middle: targets ( green ) and cable configurations at five instants (axes in meters). Bottom: PD+FF photographs, targets (red).
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 6: A DLO represented by N 3D marker positions {x1,…,xN} and actuated by Cartesian forces at both endpoints.
DLO Dataset
E [Pa]
G [Pa]
ρ [kg/m 3 ]
d [N ⋅ m ⋅ s/rad]
Baseline
6×107
1×107
3×103
1.5×10−2
Heavy
6×107
1×107
7.8×103
1.5×10−2
Low damping
6×107
1×107
3×103
3×10−3
Compliant
3×106
1×106
3×103
1.5×10−2
Appendix
Table 3: Physical parameters of the four simulated DLO . Bold values differ from the baseline.
NODE
LNN
FBCON
Φ -CON
Simulation (all four DLOs)
Learning rate
6.5\times10−4
1.0\times10−4 †
2.9\times10−3
2.2\times10−3
Weight decay
0
0
3.9\times10−4
3.8\times10−2
wxvel
1.27
29.0
13.1
31.3
λss
—
0.36
0.65
0.50
Epochs
5000
2000
5000
5000
Appendix
Table 4: Optimization and loss hyperparameters of all models. wxvel : weight of the velocity term in ℓx (position term weighted 1); —: not part of the model.
Figure 7: Prediction error in simulation: position (left) and velocity (right) RMSE over 2s rollouts for NODE , LNN , FBCON , and Φ -CON on the baseline dataset. The gray region marks the 1s training horizon.
Figure 8: Motion prediction in simulation: predictions along a high-velocity trajectory from NODE , LNN , FBCON , and Φ -CON . Thin lines trace motion between snapshots. NODE accumulates error, LNN diverges beyond its 1s training horizon, and FBCON lags behind the ground truth; Φ -CON follows the motion throughout.
Left endpoint
Right endpoint
Time [ \SIUnitSymbolMicros ]
Model
X
Y
Z
X
Y
Z
All
CPU
GPU
Params
MLP
39.77%
12.90%
16.86%
40.78%
6.55%
24.89%
20.47%
15.6±2.4
66.5±7.6
35674
LNN
0.12%
0.23%
0.09%
0.11%
0.27%
0.08%
0.13%
25.4±3.9
105.2±5.0
203584
Φ -CON
0.19%
0.20%
0.15%
0.20%
0.20%
0.14%
0.17%
13.9±3.0
64.8±15.8
20031
Appendix
Table 8: Force prediction error and computational cost in simulation: Median relative endpoint potential-force prediction error, normalized by the endpoint force magnitude, computation time per prediction (mean ± std), and number of parameters.
Kp [N/m]
Kd [N s/m]
PD
PD+FF ( Φ -CON)
Reduction
10
1.0
238.6±4.7
1.5±3.5
99.4%
50
2.24
48.2±1.1
0.44±0.66
99.1%
200
4.47
12.1±0.4
0.15±0.21
98.8%
Appendix
Table 9: Regulation error in simulation for varying control gains: Final endpoint regulation error [mm] on the baseline DLO for increasing PD gains, with Kd∝Kp to keep the damping ratio (mean ± std over the 28 test targets and both endpoints).
Department of Cognitive Robotics, Delft University of Technology, 2628 CN Delft, The Netherlands · Institute of Robotics and Mechatronics, German Aerospace Center (DLR), 82234 Oberpfaffenhofen, Germany
Technical University of Darmstadt · Honda Research Institute Europe GmbH · School of Electrical and Computer Engineering, Universidade Estadual de Campinas (UNICAMP), Brazil +4