Stein-based Optimization of Sampling Distributions in Model Predictive Path Integral Control

Jace Aldrich; Odest Chadwicke Jenkins

cs.RONov 3, 2025

Stein-based Optimization of Sampling Distributions in Model Predictive Path Integral Control

Authors: Jace Aldrich, Odest Chadwicke Jenkins

Organizations: Laboratory for Perceptive RObotics and Grounded REasoning SystemS and Robotics Department at the University of Michigan, Ann Arbor, USA

Abstract

This paper introduces a method for Model Predictive Path Integral (MPPI) control that optimizes sample generation towards an optimal trajectory through Stein Variational Gradient Descent (SVGD). MPPI relies upon predictive rollout of trajectories sampled from a distribution of possible actions. Traditionally, these action distributions are assumed to be unimodal and represented as Gaussian. This assumption can lead to suboptimal rollout predictions due to sample deprivation and, in the case of differentiable simulation, sensitivity to noise in the cost gradients. By introducing SVGD updates in between MPPI environment steps, we present Stein-Optimized Predictive Path-Integral Inference (SOPPI), an MPPI/SVGD algorithm that can dynamically update noise distributions during rollouts to better capture action sampling distributions without an excessive increase in computational requirements. We demonstrate the efficacy of SOPPI through experiments on a planar cart-pole, 7-DOF robot arm, a planar bipedal walker, and a Unitree G1 29-DOF Robot. Results indicate improved inference of an underlying multi-modal action distribution compared to other Stein-based MPPI algorithms, with improved system performance and similar or improved efficiency across a range of hyper-parameters.

Figures & tables

Fig. 1: A sequence of a 2D Unitree G1 29-DOF walking in a square pattern. SOPPI converges before other algorithms (t=14.0s) followed by SVG-MPPI (18.0s), and eventually MPPI (after 30.4s). Stein-MPC frequently falls after 6.0s, but sometimes converges (approximately 10% of the time)

Fig. 2: Inferred Action Distributions: Cart-pole action distribution at different points in the cart-pole’s swing for all control algorithms evaluated in Section IV-A . An image of the cart-pole of each state is shown in the top right. Visualization code provided by [ 10 ] . All distributions are at similar states (not times) and are for the next planned step in the environment. SOPPI maintains multi-modality where needed, whereas other methods do not.

$K$ : # samples	$f(x,u)$ : Dynamics
$N$ : # time-steps	$k(v,v)$ : Kernel
$M$ : # SVGD updates	$\sigma^{2}$ : Sampling variance
$U_{init}$ : Init. control seq.	$\lambda$ : Temperature

Algorithm 1 SOPPI

Trial	MSE $(x)$	$t_{s,x,0.25m}$	$t_{s,x,0.5m}$	MSE $(\theta)$	$t_{s,\theta,5\%}$	$t_{s,\theta,10\%}$
MPPI H=80, K=500	0.58	6.77 $\pm$ 0.94 †	4.33 $\pm$ 0.02	1.36	2.68 $\pm$ 0.02	1.97 $\pm$ 0.02
SOPPI H=80, K=500	0.53	5.20 $\pm$ 0.80 †	3.46 $\pm$ 0.73	1.29	2.65 $\pm$ 0.02	1.87 $\pm$ 0.04
Stein MPC H=80, K=500	1.26	5.53 $\pm$ 0.13	3.76 $\pm$ 0.13	1.72	4.22 $\pm$ 0.14	3.90 $\pm$ 0.02
SVG-MPPI H=80, K=500	0.78	4.34 $\pm$ 0.18	2.95 $\pm$ 0.03	1.40	2.99 $\pm$ 0.09	2.89 $\pm$ 0.03
MPPI H=80, K=1,000	0.56	6.64 $\pm$ 0.69	4.03 $\pm$ 0.68	1.36	2.69 $\pm$ 0.01	1.98 $\pm$ 0.02
SOPPI H=80, K=1,000	0.53	5.66 $\pm$ 0.51	3.93 $\pm$ 0.66	1.30	2.65 $\pm$ 0.01	1.87 $\pm$ 0.01

TABLE I: Cart Inverted Pendulum System Response for a Horizon (H) of 80 steps or 1.6 seconds, and various particle counts (K). Statistically significant (95%) best results are bolded.

Fig. 3: Median trial pushing distance error compared between algorithms

Algorithm	Mean End Distance Error (mm)	Standard Deviation of End Distance (mm)
MPPI	8.02	5.15
Stein MPC	19.36	16.06
Noisy Stein MPC	37.43	20.94
SVG-MPPI	12.08	5.61
Noisy SVG-MPPI	62.12	8.47
SOPPI (ours)	4.48	4.96

TABLE II: Steady State Distance Errors for the pushing task

Fig. 4: Sample 2D walker gait from initial position.

Algorithm	Mean Walking Time	Standard Deviation	Median Walking Time
MPPI	20.37 s	13.33 s	15.20 s
Stein MPC	3.15 s	3.02 s	1.50 s
SVG-MPPI	9.81 s	15.51 s	3.41 s
SOPPI	44.41 s	24.83 s	39.49 s

TABLE III: Walker Task Stable Gait Time. Longer times indicate further distance traveled upright

Preset	Controller	Succ	Cost of Transport	Execution Time (s)	Path Length (m)	Cross-Track Distance (m)	Speed (m/s)	Control Clock Time (ms)	Total Path Energy (J)
Forward	MPPI	10/10	$2.051\pm 0.043$	$8.7\pm 0.5$	$3.0\pm 0.0$	$0.026\pm 0.007$	$0.34\pm 0.02$	$\mathbf{44.5\pm 0.5}$	$2167\pm 54$
	SOPPI	10/10	$\mathbf{1.695\pm 0.049}$	$\mathbf{3.7\pm 0.0}$	$\mathbf{2.9\pm 0.0}$	$\mathbf{0.014\pm 0.003}$	$\mathbf{0.78\pm 0.01}$	$48.8\pm 0.5$	$\mathbf{1758\pm 50}$
	SVG-MPPI	10/10	$1.980\pm 0.061$	$4.1\pm 0.1$	$\mathbf{2.9\pm 0.0}$	$0.037\pm 0.014$	$0.71\pm 0.02$	$45.0\pm 0.6$	$2061\pm 63$
	Stein MPC	$5/10$	$\textit{2.296}\pm\textit{0.219}$	$\textit{3.6}\pm\textit{0.2}$	$\textit{3.0}\pm\textit{0.0}$	$\textit{0.015}\pm\textit{0.004}$	$\textit{0.82}\pm\textit{0.04}$	$\textit{131.0}\pm\textit{1.8}$	$\textit{2416}\pm\textit{247}$
Square	MPPI	10/10	$2.329\pm 0.027$	$30.5\pm 0.8$	$\mathbf{8.1\pm 0.1}$	$\mathbf{0.048\pm 0.010}$	$0.26\pm 0.01$	$43.8\pm 0.4$	$6674\pm 97$
	SOPPI	10/10	$\mathbf{1.816\pm 0.021}$	$\mathbf{10.2\pm 0.0}$	$8.5\pm 0.0$	$0.165\pm 0.005$	$\mathbf{0.83\pm 0.00}$	$46.9\pm 0.3$	$\mathbf{5463\pm 73}$

TABLE IV: Controller comparison on Unitree G1 29-DOF (10 trials each). Mean

\pm

std over successful trials. Bold = best among controllers with 10/10 success. Italic = Stein MPC metrics, computed over successful trials only.

Fig. 5: Trajectories for the Square and Slalom tasks on the Unitree G1 humanoid robot. Numbered x’s mark waypoint goals in ascending order.

Explore similar work

MP-MPPI: A Motion Primitive Guided Sampling-Based Optimizer for Model Predictive Control
Jul 7, 2026Marlon Mathisen, Aksel Vaaler, Olav Egeland +1Model Predictive Path Integral Model Predictive Control
Model Predictive Path Integral PID Control for Learning-Based Path Following
Mar 31, 2026Teruki Kato, Koshi Oishi, Seigo ItoModel Predictive Path Integral Model Predictive Control
Accelerating Sampling-Based Control via Learned Linear Koopman Dynamics
Mar 5, 2026Wenjian Hao, Yuxuan Fang, Zehui Lu +1Model Predictive Path Integral Koopman Operator

Jul 7, 2026, cs.ROJ/K move · Enter open · S save

MP-MPPI: A Motion Primitive Guided Sampling-Based Optimizer for Model Predictive Control

Marlon Mathisen, Aksel Vaaler, Olav Egeland, Eleni Kelasidi

*Norwegian University of Science and Technology, Dept. of Mechanical and Industrial Engineering, Norway

Model Predictive Path Integral Model Predictive Control Coverage Path Planning Adversarial Motion Priors

Open paper AskPDF arXiv

This paper proposes a novel method that extends the Model Predictive Path Integral (MPPI) method with motion primitives for additional structured sampling, which enhances the convergence towards a globally optimal solution. By evaluating motion primitives and perturbed control sequences in a real-time sampling-based optimization loop, this work addresses the limitations of the path planning capabilities of sampling-based controllers. The algorithm is implemented on a quadcopter simulator and tested on an obstacle field navigation task. It is demonstrated that the proposed approach enhances exploration of the control space while maintaining the fast, reactive behavior required for real-time control.

Abstract

Explore similar work

MP-MPPI: A Motion Primitive Guided Sampling-Based Optimizer for Model Predictive Control

Model Predictive Path Integral PID Control for Learning-Based Path Following

Accelerating Sampling-Based Control via Learned Linear Koopman Dynamics