Graph Neural Model Predictive Control for High-Dimensional Systems
Authors: Patrick Benito Eberhard, Luis Pabon, Daniele Gammelli, Hugo Buurmeijer, Amon Lahr, Mark Leone, Andrea Carron, Marco Pavone
Organizations: Institute for Dynamic Systems and Control, ETH Zürich, Zürich CH-8092, Switzerland · Department of Aeronautics and Astronautics, Stanford University, Stanford, CA · NVIDIA Research
The control of high-dimensional systems, such as soft robots, requires models that faithfully capture complex dynamics while remaining computationally tractable. This work presents a framework that integrates Graph Neural Network (GNN)-based dynamics models with structure-exploiting Model Predictive Control to enable real-time control of high-dimensional systems. By representing the system as a graph with localized interactions, the GNN preserves sparsity, while a tailored condensing algorithm eliminates state variables from the control problem, ensuring efficient computation. The complexity of our condensing algorithm scales linearly with the number of system nodes, and leverages Graphics Processing Unit (GPU) parallelization to achieve real-time performance. The proposed approach is validated in simulation and experimentally on a physical soft robotic trunk. Results show that our method scales to systems with up to 1,000 nodes at 100 Hz in closed-loop, and demonstrates real-time reference tracking on hardware with sub-centimeter accuracy, outperforming baselines by 63.6%. Finally, we show the capability of our method to achieve effective full-body obstacle avoidance.
Figures & tables
Fig. 1 : A soft robotic trunk is modeled as a graph with nodes as discrete segments with state xti at time t and edges as physical interactions. A GNN computes the linearized forward dynamics xki , which are condensed into a Quadratic Program (QP) that only depends on the input variables uk . The QP is then solved, and the first input u0 is applied in a receding-horizon fashion.
Model
RMSE [m]
Koopman
(1.16±0.61)×10−3
GNN
(1.38±0.76)×10−3
SSM (Opt.)
(2.88±1.25)×10−3
MLP
(3.59±1.67)×10−3
SSM (Orth.)
(3.67±2.22)×10−3
TABLE I : Average end-effector open-loop RMSE across models, reported as mean ±1σ standard deviation.
Fig. 2 : Scalability of the proposed GNN - MPC framework with respect to the number of subsystems M . The computation times of the relevant stages in Algorithm 1 are shown in log-log scale.
Fig. 3 : Reference tracking of a circle and figure-eight using (a) Koopman Operator modeling, (b) SSM orthogonal reduction and (c) our GNN - MPC approach. The reference is shown in dashed black, while the closed-loop trajectories are shown in red. We additionally provide the MPC plans for each time step in grey.
Fig. 4 : Deflection of the trunk as the obstacle approaches the middle node. The end effector attempts to remain as close as possible to its resting position.
Fig. 5 : Deflection of the trunk as the obstacle approaches the end effector node. The latter moves further away from the obstacle compared to the previous scenario.
We present GPU-SLS, a GPU-parallelized framework for safe, robust nonlinear model predictive control (MPC) that scales to high-dimensional uncertain robotic systems and long planning horizons. Our method jointly optimizes an inequality-constrained, dynamically-feasible nominal trajectory, a tracking controller, and a closed-loop reachable set under disturbance, all in real-time. To efficiently compute nominal trajectories, we develop a sequential quadratic programming procedure with a novel GPU-accelerated quadratic program (QP) solver that uses parallel associative scans and adaptive caching within an alternating direction method of multipliers (ADMM) framework. The same GPU QP backend is used to optimize robust tracking controllers and closed-loop reachable sets via system level synthesis (SLS), enabling reachability-constrained control in both fixed- and receding-horizon settings. We achieve substantial performance gains, reducing nominal trajectory solve times by 97.7% relative to state-of-the-art CPU solvers and 71.8% compared to GPU solvers, while accelerating SLS-based control and reachability by 237x. Despite large problem scales, our method achieves 100% empirical safety, unlike high-dimensional learning-based reachability baselines. We validate our approach on complex nonlinear systems, including whole-body quadrupeds (61D) and humanoids (75D), synthesizing robust control policies online on the GPU in 20 milliseconds on average and scaling to problems with 2 x 10^5 decision variables and 8 x 10^4 constraints. The implementation of our method is available at https://github.com/Jeff300fang/gpu_sls.
Jeffrey Fang, Glen Chou
Georgia Institute of Technology, Atlanta, GA 30308
The multi-robot unlabeled motion planning problem of concurrently assigning robots to goals and generating safe trajectories is central in many collaborative tasks. Recent Graph Neural Network methods offer scalable decentralized solutions but rely on simplified dynamics and simulation environments, overlooking key challenges of real-world deployment such as dynamic feasibility and communication constraints. To address these gaps, we propose a hierarchical framework that combines a Graph ATtention Planner (GATP) with a decentralized Nonlinear Model Predictive Controller (NMPC). GATP provides intermediate subgoals through multi-robot cooperation, and the NMPC enforces safety under nonlinear dynamics and actuation constraints. We evaluate our framework in both simulation and real-world quadrotor experiments. Thanks to attention mechanisms and minimal communication requirements, we demonstrate improved generalization to larger teams, robustness to communication delays up to 200 ms and practical feasibility with decentralized on-board inference.
Manohari Goarin, Yang Zhou, Giuseppe Loianno
New York University, NY 10012, USA · University of California Berkeley, Department of Electrical Engineering and Computer Sciences, Berkeley, CA 94720, USA
Tensegrity robots offer lightweight, compliant mobility over challenging terrain but remain difficult to model and control due to complex contact-rich dynamics and partial observability. This work presents a model predictive path integral (MPPI) controller for a three-bar tensegrity robot driven by a learned graph neural network (GNN) dynamics model. This work first extends prior GNN-based models with a differentiable contact detection module. The extension allows the dynamics model to reason over non-horizontal planar terrains, obstacles, as well as self-collisions. Then, the learned dynamics model and the MPPI controller operate in a closed data-collection loop, iteratively improving model accuracy and control performance. This work further introduces a hybrid MPPI strategy that combines MPPI with turning motion primitives to improve maneuverability. Experiments are performed in MuJoCo across five navigation tasks, which include, wall obstacles, inclines, narrow corridors, low-clearance structures, and a composite 3D obstacle course. The experiments demonstrate that the hybrid MPPI controller operating over the learned GNN dynamics model improves predictive accuracy over a flat-ground baseline model and achieves superior navigation performance compared to A∗-based re-planning and MPPI-only variants. Results show that the contact-aware learned dynamics combined with the sampling-based model predictive control enable robust tensegrity navigation in complex, contact-rich environments.
Nelson Chen, Patrick Meng, Charles Tang +5
Computer Science, Rutgers University, Piscataway, NJ, USA. · Mechanical Engineering, Yale University, New Haven, CT, USA.