General-purpose simulators have accelerated the development of robots. Traditional simulators based on first-principles, however, typically require full-state observability or depend on parameter search for system identification. This work presents CableRobotGraphSim, a novel Graph Neural Network (GNN) model for cable-driven robots that aims to address shortcomings of prior simulation solutions. By representing cable-driven robots as graphs, with the rigid-bodies as nodes and the cables and contacts as edges, this model can quickly and accurately match the properties of other simulation models and real robots, while ingesting only partially observable inputs. Accompanying the GNN model is a sim-and-real co-training procedure that promotes generalization and robustness to noisy real data. This model is further integrated with a Model Predictive Path Integral (MPPI) controller for closed-loop navigation, which showcases the model's speed and accuracy.
Figures & tables
Figure 1: Left: Static, open-source 3-bar tensegrity platform. Right: The platform rolling clockwise.
Figure 2: The three stages of a simulation step. Left : Graph and feature generation, where state Xt , rest length ℓtrest , control history Ut−h:t , and future controls Ut+1:t+n form the graph Gt=(Vt,Et) . Middle: Gt is passed through MLP encoders and an LSTM block (with hidden state Ht and memory Ct ) to produce Ht+1,Ct+1 . Ht+1 then undergoes L rounds of message passing, after which the latent node and cable-edge vectors are decoded into n velocity and rest-length changes. Right: The predicted chunk is integrated iteratively to advance ℓtrest to ℓt+nrest and Xt to Xt+n .
Figure 3: Sim-and-real co-training. Multiple simulation datasets are generated by varying system parameters, and are pooled with a real dataset of unknown system parameters. Data points are distinguished by a one-hot encoding node feature of the dataset enumerated index.
Figure 4: Sim2sim evaluation for full trajectories and short horizons, as well as a 3-bar and 6-bar tensegrity.
Full Traj
Full Traj
Short Horizon
Short Horizon
Model
Pos Error (%)
Rot Error (°)
Pos Error (%)
Rot Error (°)
Hybrid GNN - uninformed
Unstable
Unstable
Unstable
Unstable
Hybrid GNN - heuristic
91.12
35.29
N/A
N/A
T-DPE - uninformed
465.59
60.32
9.59
28.76
T-DPE - heuristic
102.76
38.48
N/A
N/A
CableRobotGraphSim (ours)
48.95
26.53
3.95
7.031
Table 1: Real2sim evaluation results on a real 3-bar tensegrity platform.
Figure 5: Running MPPI controller in a loop of model training, task execution, and new data generation for model retraining over three iterations. Positional and rotational errors are evaluated on a pooled test set from all iterations. Success rate and task completion time are measured per iteration.
Figure 6: S-shaped obstacle course for navigating from the red to the green sphere.
Figure 7: Maze map for navigating from the red to the green sphere.
Full Traj
Full Traj
Short Horizon
Short Horizon
Model
Pos Error (%)
Rot Error (°)
Pos Error (%)
Rot Error (°)
(baseline) Hybrid GNN
Unstable
Unstable
Unstable
Unstable
(i) Fully Learned GNN (F-GNN)
20.6±26.14
47.1±27.03
0.11±0.04
4.91±0.99
(ii) F-GNN + Config Feats (FC-GNN)
1.15±1.13
15.43±15.54
0.1±0.04
4.6±1.21
(iii) FC-GNN + Recurrent (FCR-GNN)
1.04±1.3
7.61±4.72
0.06±0.02
3.24±0.88
FCR-GNN + analytical motor
4.49±2.21
31.18±7.22
0.1±0.04
5.14±0.66
Table 2: GNN architecture component ablation study. The first four rows compare GNN components using ground-truth motor outputs. The last two rows compare an analytical vs. a learned motor model.
Figure 8: Sim-and-real co-training ablation study for varying number of simulation datasets.
Figure 9: Prediction chunk size ablation over the number of forward time steps per simulation call.
Linear-deformable manipulation remains challenging due to the complex deformations of objects such as cables and ropes. Prior data-driven approaches, particularly imitation learning, have shown some promise in narrowly defined settings but typically require thousands of demonstrations for specific tasks and cable types, limiting scalability and generalization. We introduce a sim-to-real reinforcement learning (RL) framework for multi-stage cable routing that leverages GPU-parallelized simulation to approximate linear deformable behaviors. Training across thousands of parallel simulations enables the learned policies to generalize across diverse cable geometries and deformation patterns. To bridge the sim-to-real gap, we propose a novel deployment strategy that combines a Simulation In the LOop (SILO) execution framework, localized RL policies, and robust cable state estimation. On real-world cable routing tasks, our approach achieves higher success rates and 2x reduction in cycle times compared to prior state-of-the-art learning methods. To our knowledge, this is the first successful sim-to-real transfer of RL policies for multi-stage cable routing. Videos and additional visualizations are available at https://silo-cable-routing.github.io/
Stone Tao, Jie Xu, Hesam Rabeti +3
1NVIDIA Corporation · University of California, San Diego
Tensegrity robots offer lightweight, compliant mobility over challenging terrain but remain difficult to model and control due to complex contact-rich dynamics and partial observability. This work presents a model predictive path integral (MPPI) controller for a three-bar tensegrity robot driven by a learned graph neural network (GNN) dynamics model. This work first extends prior GNN-based models with a differentiable contact detection module. The extension allows the dynamics model to reason over non-horizontal planar terrains, obstacles, as well as self-collisions. Then, the learned dynamics model and the MPPI controller operate in a closed data-collection loop, iteratively improving model accuracy and control performance. This work further introduces a hybrid MPPI strategy that combines MPPI with turning motion primitives to improve maneuverability. Experiments are performed in MuJoCo across five navigation tasks, which include, wall obstacles, inclines, narrow corridors, low-clearance structures, and a composite 3D obstacle course. The experiments demonstrate that the hybrid MPPI controller operating over the learned GNN dynamics model improves predictive accuracy over a flat-ground baseline model and achieves superior navigation performance compared to A∗-based re-planning and MPPI-only variants. Results show that the contact-aware learned dynamics combined with the sampling-based model predictive control enable robust tensegrity navigation in complex, contact-rich environments.
Nelson Chen, Patrick Meng, Charles Tang +5
Computer Science, Rutgers University, Piscataway, NJ, USA. · Mechanical Engineering, Yale University, New Haven, CT, USA.
Cable routing requires coordinated control of global cable topology and changing local contacts. We present CableVLA, an end-to-end multimodal vision-language-action framework that converts simulation-privileged supervision into deployable cable-topology and tactile representations. TopoHead distills node-level physics and current and future cable-topology information into causal visual context for the action expert. TacSense uses complementary frame and taxel branches to learn contact dynamics from resistive arrays, with simulator-derived kinematics and contact events providing supervision beyond the measured force map. A contact gate activates force-tactile residuals that refine the next 8 arm-and-gripper actions of a frozen topology-conditioned policy. Across 345 MuJoCo evaluations, CableVLA improves success from 62.6% for the π0.5-V visual baseline to 84.9%. TacSense achieves pronounced gains in slip-transition recognition over a CNN-LSTM baseline with a similar parameter count, and this advantage persists under frozen-encoder probes. Topology prediction and 57-task tactile evaluations assess representation quality, while policy adaptation studies evaluate downstream control performance. Cross-simulator and real-robot comparisons further examine zero-shot policy transfer under changes in dynamics and sensing.
Zhifei Teng, Bo Feng, Xiang Zou +4
State Key Laboratory of Intelligent Manufacturing Equipment and Technology, School of Mechanical Science and Engineering, Huazhong University of Science and Technology, Wuhan, China. · School of Mathematics, Harbin Institute of Technology, Harbin, China.