Hybrid Feedback Sampling for Sample-Efficient Model Predictive Control
Authors: Chaoyi Pan, Zeji Yi, John Zhang, Zachary Manchester, Guannan Qu, Guanya Shi
Organizations: Department of Electrical and Computer Engineering, Carnegie Mellon University, Pittsburgh, PA 15213, USA · Department of Aeronautics and Astronautics, Massachusetts Institute of Technology, Cambridge, MA 02139, USA · Robotics Institute, Carnegie Mellon University, Pittsburgh, PA 15213, USA
Thanks to its parallelizability and flexibility, sampling-based Model Predictive Control (MPC) has become widely popular for controlling real-world robotic systems. However, for high-dimensional and open-loop unstable dynamical systems, the required number of samples to improve the control sequence will grow exponentially with the horizon, leading to poor sample efficiency and numerical instability. This paper investigates the instability of shooting methods in sampling-based MPC and shows that the optimal sampling proposal distribution can be realized by sampling with an optimized feedback policy. We refer to this algorithm as Feedback Sampling MPC (FS-MPC). FS-MPC involves a hybrid sampling design which balances local and global search based on the system stability and the available computation budget. Our theoretical analysis shows that our hybrid sampling approach achieves faster convergence than standard MPPI and better optimality than standard feedback sampling. Empirically, in diverse contact-rich control tasks like humanoid loco-manipulation and dexterous manipulation, we show that FS-MPC successfully tackles dynamically unstable tasks where standard sample-based approaches struggle, and strictly outperforms feedback policies alone. Finally, we validate our method on humanoid robot locomotion and manipulation tasks in the real world.
Figures & tables
Fig. 1: (a) Motivation of FS-MPC: showing the sampling distributions of FS-MPC (left) and standard MPPI (middle), and the distributions of their samples’ costs (right), where the quadrotor is initialized at the same state. Given the long horizon navigation task with unstable quadrotor dynamics, feedback sampling can stabilize the system while MPPI’s samples diverge. Therefore, the cost distribution of FS-MPC is significantly more optimal. (b) Deploy FS-MPC on the real humanoid robot. When planning over contact-rich and unstable humanoid dynamics, FS-MPC can control the robot with both stability and precision.
Fig. 2: Visualization of shooting control sequence (a), its joint covariance matrix (b) and its linear realization (c). (1) sampling with iLQR feedback K∗ (c1) would lead to stable sampling distribution (b1) which has a rich pattern. (2) Standard MPPI sampling rule (c2) would lead to an isotropic Gaussian distribution (b2), which is not optimal. From the perspective of joint control distribution design, feedback sampling can be viewed as the state space equivalent realization of optimal covariance matrix Σk∗ .
Fig. 3: Hybrid feedback sampling balances local and global search. (1) Hybrid feedback sampling draws from both local and global proposal distributions, enabling escape from local minima. (2) Standard feedback sampling relies solely on the local distribution, often failing to traverse high-cost barriers.
Fig. 4: Task overview over different controllers. (a) Acrobot: the controller needs to swing up the pendulum from the bottom to the top and then balance it at an unstable equilibrium. (b) Quadrotor: the controller needs to track a desired trajectory and fly through a few gates. The quadrotor is unstable and colliding with a gate can lead to a crash. (c) Quadruped Hill: the controller needs to track desired key points to navigate a hill. Due to the uneven terrain, the quadruped would fall and need to get up. (d) H1-2 PNP: the controller needs to do whole-body control for a 23-DoF humanoid robot to pick up and place a box on the target location. The system involves both an unstable system (the humanoid robot) and long-horizon reasoning over the contact. (e) Allegro: a in-hand reorientation task with allegro hand. The robot needs to reorient the object to a random target orientation. In all tasks, the white line is the sampled trajectory, and the pink line is the planned trajectory.
Controllers
iLQR
MPPI
F-MPPI
FS-MPC
Acrobot
104.55±0.63
117.16±0.20
122.12±7.45
76.38±15.24
Quadrotor
8.44±0.37
13.36±9.77
8.78±0.20
7.99±0.40
Quadruped Hill
3.39±0.70
2.79±0.02
2.68±0.17
2.51±0.04
Allegro
16.86±2.40
12.22±4.84
9.27±1.28
6.50±0.38
H1-2 PNP
1.72±0.06
1.55±0.15
2.00±0.43
1.06±0.10
TABLE I: Cost comparison between controllers with iLQR feedback. From top to bottom: tasks are sorted from lower-dimensional to higher-dimensional, and from contact-free to contact-rich. FS-MPC can consistently outperform MPPI and iLQR.
Tasks
humanoidrun
humanoidstandup
iLQR
-
-
PPO
−12.34±0.01
−24.76±0.01
MPPI
−12.50±0.00
−27.75±0.47
F-MPPI (w/ PPO)
−10.07±0.22
−21.14±0.06
FS-MPC (w/ PPO)
−15.65±0.10
−29.41±0.55
TABLE II: Rollout trajectory cumulative cost on the humanoid running/standing up task.
Fig. 5: Tracking error of iLQR and FS-MPC on real robot.
Methods
MPPI
iLQR
FS-MPC
Tracking Error (m)
Failed
0.93±1.00
0.70±0.66
TABLE III: Tracking error comparison over different controllers in the real world. MPPI failed due to the high variance of the sampling procedure. FS-MPC can still stabilize the robot thanks to the feedback design.
Fig. 6: FS-MPC on real robot. Upper: H1 humanoid robot walking from one side of the room to the other. Lower: H1 humanoid robot holding a cylinder and put it into a cart. FS-MPC can finish the task while MPPI failed due to unstable sampling.
Sampling-based model predictive control methods like MPPI and CEM are essential for real-time control of nonlinear robotic systems, particularly where discontinuous dynamics preclude gradient-based optimization. However, these methods derive from information-theoretic objectives that are agnostic to the geometry of the control problem, leading to pathological behaviors such as mode-averaging when the cost landscape is complex. We present OT-MPC, a sampling-based algorithm that overcomes these limitations through an entropy-regularized optimal transport formulation. By computing an optimal coupling between candidate control sequences and low-cost proposals, OT-MPC refines candidates toward nearby promising samples while coordinating updates across the ensemble to maintain coverage of the solution space. We derive closed-form, gradient-free updates via the Sinkhorn algorithm, enabling real-time performance. Experiments on navigation, manipulation, and locomotion tasks demonstrate improved success rates over existing methods.
Vincent Pacelli, Akash Ratheesh, Evangelos A. Theodorou
Autonomous Control and Decision Systems Laboratory Georgia Institute of Technology
Sampling-based Model Predictive Control (SMPC) is a promising strategy for contact-rich robotic manipulation, combining gradient-free optimization with massively parallel GPU simulation. Yet, most prior work relies on simplified dynamics or remains confined to simulation. We present an MPC framework that leverages JAX for large-scale parallelization and efficient computation, coupled with the high-fidelity MuJoCo MJX simulator, and deploy it on a Franka Research 3 executing the Push-T manipulation task through a complete real-to-sim-to-real pipeline. The MTP variant with structured global sampling outperforms unimodal baselines such as CEM, MPPI, and PS across tasks that require mode switching, both in simulation and on hardware. Furthermore, we evaluate online domain randomization within the MPC sample budget, showing that contact-initiation parameters yield interpretable adaptation signals, whereas global physics parameters provide feedback that is too weak for reliable exploitation at typical replanning frequencies. These findings highlight key challenges for sampling-based MPC in contact-rich manipulation-contact sensitivity, tight compute budgets, and the difficulty of obtaining informative domain-randomization signals in real time.
Magnus Dierking, Joao Carvalho, An Thai Le +2
Intelligent Autonomous Systems Lab, TU Darmstadt, Germany · Interactive Robot Perception & Learning Lab, TU Darmstadt, Germany · German Research Center for AI (DFKI) +2
This paper proposes a novel method that extends the Model Predictive Path Integral (MPPI) method with motion primitives for additional structured sampling, which enhances the convergence towards a globally optimal solution. By evaluating motion primitives and perturbed control sequences in a real-time sampling-based optimization loop, this work addresses the limitations of the path planning capabilities of sampling-based controllers. The algorithm is implemented on a quadcopter simulator and tested on an obstacle field navigation task. It is demonstrated that the proposed approach enhances exploration of the control space while maintaining the fast, reactive behavior required for real-time control.
Marlon Mathisen, Aksel Vaaler, Olav Egeland +1
*Norwegian University of Science and Technology, Dept. of Mechanical and Industrial Engineering, Norway