Sampling-based Model Predictive Control as Sequential Monte Carlo Inference
Organizations: Department of Computer Science, Purdue University, USA.
Abstract
Sampling-based model predictive control (MPC) is treated as a family of distinct algorithms: predictive sampling, MPPI, the cross-entropy method, and their biased, colored-noise, and annealed variants all sample control sequences and reweight them by cost. We show a single Sequential Monte Carlo (SMC) inference algorithm implements them all, specified by three independent axes: the proposal, the update rule that reduces cost-weighted particles to a nominal plan, and the effective-sample-size threshold for resampling. We evaluate SMC-based planning on MuJoCo non-linear control, open-loop navigation through obstacle fields, 7-DoF motion planning, and non-prehensile push-T, in simulation and on a Franka FR3. Resampling combats particle degeneracy at long horizons. At a 28-knot horizon in our navigation domain it cut the collision rate by 30% over non-resampling baselines. Programmable proposals improve push-T success over perturbative baselines, improving from 1% to 100% in simulation and 10% to 90% on hardware.
Figures & tables
| MPC Algorithm | Proposal | Resample ( ) | Update |
|---|---|---|---|
| PS [ 45 ] | |||
| MPPI [ 6 ] | |||
| Biased-MPPI [ 29 ] | Any | ||
| DIAL-MPC [ 2 ] | |||
| CEM [ 7 ] | |||
| iCEM [ 36 ] |
| CartPole | DoubleCartPole | HalfCheetah | Walker | Norm. | |||||
| Method | Cost | ms | Cost | ms | Cost | ms | Cost | ms | |
| Hydrax [ 46 ] | |||||||||
| PS | 0.007 0.017 | 22.2 | 0.072 0.076 | 24.6 | 0.017 0.016 | 19.0 | 0.006 0.005 | 19.3 | 2.36 |
| MPPI | 0.006 0.016 | 21.2 | 0.070 0.076 | 24.1 | 0.020 0.017 | 19.0 | 0.004 0.004 | 20.5 | 2.55 |
| CEM | 0.005 0.014 | 20.7 | 0.102 0.073 | 23.4 | 0.043 0.009 | 19.6 | 0.007 0.008 | 22.4 | 4.70 |
| DIAL-MPC | 0.005 0.014 | 99.5 | 0.150 0.038 | 109.6 | 0.041 0.012 | 96.7 | 0.009 0.017 | 93.7 | 4.82 |
Model Predictive Control
Model Predictive Control optimizes future actions with a system model, repeatedly applying the first action as new information arrives.
See all 337 papers