Dynamic manipulation of deformable linear objects (DLOs) remains challenging due to underactuated, high-dimensional dynamics, delayed deformation responses, and the presence of multiple feasible motions under obstacle constraints. This paper presents BM2PC, a Bayesian Multimodal MPC framework for agile manipulation of DLOs. Building on the control-as-inference posterior, BM2PC formulates multimodal planning as a set optimization problem that balances control quality and behavioral diversity. An efficient incremental update maintains multiple promising control modes with low additional computational overhead. For real-world deployment, BM2PC incorporates an execution dynamics model that predicts actual end-effector motion and compensates for execution mismatch during fast manipulation. We evaluate BM2PC on rope tracking and obstacle avoidance in simulation and hardware, and on simulated disturbance rejection. In trajectory tracking, BM2PC generalizes better to unseen trajectories than a learning-based baseline without task-specific policy training or fine-tuning. Real-world experiments demonstrate accurate long-horizon tracking, reducing RMSE by 44.0% on the longest task. In obstacle avoidance, BM2PC achieves comparable performance to existing multimodal MPC methods while reducing planning time by 29% to 66%. These results demonstrate accurate dynamic control and efficient multimodal planning within a unified MPC framework. The core implementation of BM2PC and supplementary videos are available at https://github.com/Longyouyuan/BM2PC.
Figures & tables
Figure 1: Overview of the proposed BM2PC framework. Candidate control sequences are sampled around the multimodal warm-start set and evaluated through parallel rollouts using the execution dynamics model, DLO dynamics model, and collision detection. Based on rollout costs and trajectory features, BM2PC performs an incremental multimodal update to maintain a diverse set of high-quality control modes. One mode is selected for execution, while the updated multimodal set is propagated to the next MPC iteration.
Figure 2: The DLO model. The rope is represented by N+1 mass points mj connected by linear spring–damper elements ( kjs,cjs ) and bending spring–damper elements ( kjb,cjb ). Gravity g and air damping cair are also included.
Figure 3: Execution dynamics model. A GRU-based history encoder processes recent measured and desired end-effector states to initialize a latent representation of the current execution dynamics. Conditioned on this latent state and the desired future trajectory, the rollout cell recursively predicts future EE motion. At each rollout step, an MLP predicts the executed acceleration, which is integrated by a physics-informed (PI) head to obtain the next velocity and position.
Parameter
Tracking
Obstacle Avoid.
Dist. Rej.
Prediction horizon H
30
35
35
Number of modes n
1
3
1
Samples / iteration N
402
402
402
Optimization iterations I
10
1
10
Sampling std. σ
1.5
1.5
1.5
Sampling covariance Σ
σ2I
σ2I
σ2I
Table I : BM2PC and controller parameters used in the simulation experiments.
Figure 4: Trajectory tracking results (Sim). Comparison of trajectory tracking performance of the proposed BM2PC method and the SPiD baseline on both smooth parametric trajectories (Sinusoid, Egg, and Lemniscate) and complex hand-drawn trajectories (SpongeBob, PatrickStar, and Flower) under different execution durations. The proposed method achieves accurate tracking across most trajectories, while SPiD fails to reproduce fine geometric details, particularly on complex trajectories with sharp turns and rapid local variations. Thus, SPiD is not shown in the longer cases ( T>12 s) to avoid clutter.
Trajectory (Length)
Sinusoid
Egg
Lemniscate
SpongeBob
PatrickStar
Flower
(5.085 m)
(5.844 m)
(5.540 m)
(18.3 m)
(18.1 m)
(17.5 m)
Duration (s)
4 s
5 s
6 s
4 s
5 s
6 s
4 s
5 s
6 s
12 s
15 s
18 s
12 s
15 s
18 s
12 s
15 s
18 s
Max Velocity (m/s)
2.245
1.797
1.498
2.552
2.043
1.703
2.089
1.672
1.394
1.876
1.501
1.251
1.857
1.486
1.238
1.765
1.412
1.177
SPiD RMSE (cm)
19.26
9.950
10.97
23.32
11.23
9.407
16.17
6.391
11.66
16.12
19.06
30.74
27.94
37.65
36.14
29.47
39.80
30.85
BM2PC RMSE (cm)
9.690
6.230
4.650
6.900
4.850
3.960
6.570
4.800
3.770
7.500
5.670
4.080
8.500
7.000
3.580
5.430
4.120
3.380
Table II : Trajectory tracking simulation task specifications and tracking RMSE comparison.
Figure 5: Obstacle Avoidance (Sim). Comparison of BM2PC, C-MPPI , and SVMPC on the obstacle avoidance task. The top row presents the rope-tip trajectories together with the tracking RMSE and average inference time. The bottom row shows key snapshots of BM2PC, illustrating mode collapse in unimodal regions (A,C) and multimodal emergence near the obstacle (B,D). The rope tip may pass above the obstacle without touching it, as seen in snapshots B and D.
Figure 6: Disturbance Rejection (Sim). The rope tip is subjected to external disturbances at the blue points to evaluate the robustness of the controller. The corresponding rope-tip trajectories and key snapshots of the experiment are presented. BM2PC rapidly suppresses the induced oscillations and drives the rope tip back to the desired trajectory.
Figure 7: Multimodal adaptation under sudden blocking. Comparison of representative trajectories from experiments with τ=0.70 s. (a) Predicted control modes before blocking. (b) The second obstacle appears and blocks the executed plan. (c) Replanning after 25 ms. (d) Predicted modes after 225 ms. (e) Executed rope-tip trajectories. BM2PC switches to an alternative mode on the unblocked side, whereas the ablated variants fail to recover a feasible passage in this example.
Metric
BM2PC
n=1
β=0
Success rate (%)
64.3
45.2
47.0
Collision rate (%)
25.2
39.3
35.7
Mean max. tracking error (cm)
56.2
80.2
76.6
Table III : Ablation of the multimodal update. Results are aggregated over τ∈[0.65,0.90] s.
Figure 8: Sudden obstacle ablation. Success and collision rates versus obstacle lead time τ . Shaded regions indicate 95% Wilson confidence intervals for success rates.
Metric
BM2PC
Exhaustive
Rope-tip RMSE (cm)
6.22 ± 0.48
6.08 ± 0.53
Planning time (ms)
9.63
90.14
Multimodal update (ms)
1.55
81.29
Table IV : Incremental multimodal update versus pool-normalized exhaustive selection.
Figure 9: A test-time example of the execution dynamics model.
Parameter
Tracking
Obstacle Avoid.
Prediction horizon H
50
50
Number of modes n
1
3
Samples / iteration N
81
81
Optimization iterations I
1
2
Sampling std. σ
1.8
1.8
Sampling covariance Σ
σ2I
σ2I
Table V : BM2PC and controller parameters used in the real-world experiments.
Figure 10: Real-world trajectory tracking results. Top: comparison of the desired trajectory and the tracked trajectories generated by BM2PC and SPiD for three trajectories. Bottom: time-lapse snapshots of the real robotic system during the sinusoidal and lemniscate tracking experiments by BM2PC.
Sinusoid
Lemn. ( × 1)
Lemn. ( × 3)
Duration (s)
2.500
4.000
10.800
Length (m)
0.932
1.806
5.420
Max Velocity (m/s)
0.593
0.699
0.680
SPiD RMSE (cm)
4.300
3.451
4.414
BM2PC RMSE (cm)
4.175
2.701
2.474
Table VI : Real-world trajectory tracking task specifications and tracking RMSE comparison.
Figure 11: Real-world obstacle avoidance results. The rope tracks a Lemniscate ( ×3 ) trajectory while a movable obstacle is randomly repositioned along the path to evaluate dynamic obstacle avoidance performance. The blue curve denotes the target trajectory, the orange curve represents the executed trajectory, and the red segments indicate perception failures caused by occlusions, environmental noise, or network instability. Time-lapse snapshots (a)–(h) illustrate the avoidance process under different obstacle configurations. Despite severe perception degradation, including frozen measurements and abrupt trajectory spikes, the controller successfully avoids the obstacle and rapidly returns the rope endpoint to the desired trajectory.
The manipulation of Deformable Linear Objects (DLOs) such as cables poses a significant challenge for automation due to their infinite degrees of freedom and non-linear dynamics. In this paper we present a machine learning based optimal control approach for the manipulation of DLOs. This approach is divided into two main components: modeling and control. For modeling the dynamics of the DLO, we propose a learning based approach using a bidirectional Long Short-Term Memory (biLSTM) network. The biLSTM network is trained on synthetic data generated by the MuJoCo physics engine. For manipulating the DLO, a model predictive control strategy that employs Model Predictive Path Integral (MPPI) control is selected. The proposed approach is evaluated through simulation and experiments. The results demonstrate the effectiveness of the proposed method in achieving accurate and efficient manipulation of DLOs.
Lukas Zeh, Johannes Meiwaldt, Zexu Zhou +2
Institute for Control Engineering of Machine Tools and Manufacturing Units, University of Stuttgart, Stuttgart, Germany
We present CORD-SLS, a real-time control method for safe deformable object manipulation, with a focus on ropes and cloth. At its core is a GPU-parallel differentiable simulator with contact smoothing which enables efficient gradient-based planning through intermittent contact. To robustly satisfy constraints under model and sensing uncertainty, we develop a real-time, GPU-parallel output-feedback robust model predictive control (MPC) algorithm that plans with this simulator. We further show that the simulator accelerates model-based RL for training neural manipulation policies. To improve real-world robustness, we use conformal prediction to calibrate visual-feedback and perception-error bounds for MPC, producing reachable tubes that enable high-probability safe control. We evaluate CORD-SLS on high-dimensional, contact-rich rope and cloth manipulation tasks in simulation and hardware, including obstacle avoidance, routing, folding, and smoothing. Across settings, CORD-SLS achieves millisecond-speed planning, exceeding baselines in safety, speed, and task success.
Wei-Chen Li, Jeffrey Fang, Sasanka Polisetti +2
Georgia Institute of Technology, Atlanta, GA 30308
We address the challenge of enabling robots to manipulate deformable linear objects (DLOs), such as ropes, cables, and rubber bands. Prior work has primarily focused on narrow, task-specific problems, often relying on real-world demonstrations or handcrafted heuristics. Such approaches, however, struggle to scale to the wide variety of materials and tasks encountered in practice, and collecting sufficiently diverse real-world data is often impractical. Additionally, existing simulation environments offer limited support for the broad spectrum of material behaviors necessary for generalizable DLO manipulation. To overcome these limitations, we introduce a differentiable simulator explicitly designed for versatile DLO manipulation. Our simulator models a wide range of material properties-including (in)extensibility, elasticity, bending plasticity, and complex interactions with other objects-providing a robust foundation for learning and evaluating manipulation skills. Building on this simulator, we propose a benchmark suite of representative tasks that highlight the unique challenges of DLO manipulation. The successful execution of these tasks is often hindered by the topological complexity and grasp sensitivity inherent to DLOs. Therefore, we introduce a specialized DLO agent that explicitly manages these challenges by proposing strategic grasping points and decomposing long-horizon tasks to maximize control authority. Finally, we evaluate various policy-learning algorithms using our framework, alongside sim-to-real transfer experiments, demonstrating our platform's potential to advance DLO manipulation.