Stein-based Optimization of Sampling Distributions in Model Predictive Path Integral Control

Jace Aldrich; Odest Chadwicke Jenkins

cs.RONov 3, 2025

Stein-based Optimization of Sampling Distributions in Model Predictive Path Integral Control

Authors: Jace Aldrich, Odest Chadwicke Jenkins

Organizations: Laboratory for Perceptive RObotics and Grounded REasoning SystemS and Robotics Department at the University of Michigan, Ann Arbor, USA

Abstract

This paper introduces a method for Model Predictive Path Integral (MPPI) control that optimizes sample generation towards an optimal trajectory through Stein Variational Gradient Descent (SVGD). MPPI relies upon predictive rollout of trajectories sampled from a distribution of possible actions. Traditionally, these action distributions are assumed to be unimodal and represented as Gaussian. This assumption can lead to suboptimal rollout predictions due to sample deprivation and, in the case of differentiable simulation, sensitivity to noise in the cost gradients. By introducing SVGD updates in between MPPI environment steps, we present Stein-Optimized Predictive Path-Integral Inference (SOPPI), an MPPI/SVGD algorithm that can dynamically update noise distributions during rollouts to better capture action sampling distributions without an excessive increase in computational requirements. We demonstrate the efficacy of SOPPI through experiments on a planar cart-pole, 7-DOF robot arm, a planar bipedal walker, and a Unitree G1 29-DOF Robot. Results indicate improved inference of an underlying multi-modal action distribution compared to other Stein-based MPPI algorithms, with improved system performance and similar or improved efficiency across a range of hyper-parameters.

Figures & tables

Fig. 1: A sequence of a 2D Unitree G1 29-DOF walking in a square pattern. SOPPI converges before other algorithms (t=14.0s) followed by SVG-MPPI (18.0s), and eventually MPPI (after 30.4s). Stein-MPC frequently falls after 6.0s, but sometimes converges (approximately 10% of the time)

Fig. 2: Inferred Action Distributions: Cart-pole action distribution at different points in the cart-pole’s swing for all control algorithms evaluated in Section IV-A . An image of the cart-pole of each state is shown in the top right. Visualization code provided by [ 10 ] . All distributions are at similar states (not times) and are for the next planned step in the environment. SOPPI maintains multi-modality where needed, whereas other methods do not.

$K$ : # samples	$f(x,u)$ : Dynamics
$N$ : # time-steps	$k(v,v)$ : Kernel
$M$ : # SVGD updates	$\sigma^{2}$ : Sampling variance
$U_{init}$ : Init. control seq.	$\lambda$ : Temperature

Algorithm 1 SOPPI

Trial	MSE $(x)$	$t_{s,x,0.25m}$	$t_{s,x,0.5m}$	MSE $(\theta)$	$t_{s,\theta,5\%}$	$t_{s,\theta,10\%}$
MPPI H=80, K=500	0.58	6.77 $\pm$ 0.94 †	4.33 $\pm$ 0.02	1.36	2.68 $\pm$ 0.02	1.97 $\pm$ 0.02
SOPPI H=80, K=500	0.53	5.20 $\pm$ 0.80 †	3.46 $\pm$ 0.73	1.29	2.65 $\pm$ 0.02	1.87 $\pm$ 0.04
Stein MPC H=80, K=500	1.26	5.53 $\pm$ 0.13	3.76 $\pm$ 0.13	1.72	4.22 $\pm$ 0.14	3.90 $\pm$ 0.02
SVG-MPPI H=80, K=500	0.78	4.34 $\pm$ 0.18	2.95 $\pm$ 0.03	1.40	2.99 $\pm$ 0.09	2.89 $\pm$ 0.03
MPPI H=80, K=1,000	0.56	6.64 $\pm$ 0.69	4.03 $\pm$ 0.68	1.36	2.69 $\pm$ 0.01	1.98 $\pm$ 0.02
SOPPI H=80, K=1,000	0.53	5.66 $\pm$ 0.51	3.93 $\pm$ 0.66	1.30	2.65 $\pm$ 0.01	1.87 $\pm$ 0.01

TABLE I: Cart Inverted Pendulum System Response for a Horizon (H) of 80 steps or 1.6 seconds, and various particle counts (K). Statistically significant (95%) best results are bolded.

Fig. 3: Median trial pushing distance error compared between algorithms

Algorithm	Mean End Distance Error (mm)	Standard Deviation of End Distance (mm)
MPPI	8.02	5.15
Stein MPC	19.36	16.06
Noisy Stein MPC	37.43	20.94
SVG-MPPI	12.08	5.61
Noisy SVG-MPPI	62.12	8.47
SOPPI (ours)	4.48	4.96

TABLE II: Steady State Distance Errors for the pushing task

Fig. 4: Sample 2D walker gait from initial position.

Algorithm	Mean Walking Time	Standard Deviation	Median Walking Time
MPPI	20.37 s	13.33 s	15.20 s
Stein MPC	3.15 s	3.02 s	1.50 s
SVG-MPPI	9.81 s	15.51 s	3.41 s
SOPPI	44.41 s	24.83 s	39.49 s

TABLE III: Walker Task Stable Gait Time. Longer times indicate further distance traveled upright

Preset	Controller	Succ	Cost of Transport	Execution Time (s)	Path Length (m)	Cross-Track Distance (m)	Speed (m/s)	Control Clock Time (ms)	Total Path Energy (J)
Forward	MPPI	10/10	$2.051\pm 0.043$	$8.7\pm 0.5$	$3.0\pm 0.0$	$0.026\pm 0.007$	$0.34\pm 0.02$	$\mathbf{44.5\pm 0.5}$	$2167\pm 54$
	SOPPI	10/10	$\mathbf{1.695\pm 0.049}$	$\mathbf{3.7\pm 0.0}$	$\mathbf{2.9\pm 0.0}$	$\mathbf{0.014\pm 0.003}$	$\mathbf{0.78\pm 0.01}$	$48.8\pm 0.5$	$\mathbf{1758\pm 50}$
	SVG-MPPI	10/10	$1.980\pm 0.061$	$4.1\pm 0.1$	$\mathbf{2.9\pm 0.0}$	$0.037\pm 0.014$	$0.71\pm 0.02$	$45.0\pm 0.6$	$2061\pm 63$
	Stein MPC	$5/10$	$\textit{2.296}\pm\textit{0.219}$	$\textit{3.6}\pm\textit{0.2}$	$\textit{3.0}\pm\textit{0.0}$	$\textit{0.015}\pm\textit{0.004}$	$\textit{0.82}\pm\textit{0.04}$	$\textit{131.0}\pm\textit{1.8}$	$\textit{2416}\pm\textit{247}$
Square	MPPI	10/10	$2.329\pm 0.027$	$30.5\pm 0.8$	$\mathbf{8.1\pm 0.1}$	$\mathbf{0.048\pm 0.010}$	$0.26\pm 0.01$	$43.8\pm 0.4$	$6674\pm 97$
	SOPPI	10/10	$\mathbf{1.816\pm 0.021}$	$\mathbf{10.2\pm 0.0}$	$8.5\pm 0.0$	$0.165\pm 0.005$	$\mathbf{0.83\pm 0.00}$	$46.9\pm 0.3$	$\mathbf{5463\pm 73}$

TABLE IV: Controller comparison on Unitree G1 29-DOF (10 trials each). Mean

\pm

std over successful trials. Bold = best among controllers with 10/10 success. Italic = Stein MPC metrics, computed over successful trials only.

Fig. 5: Trajectories for the Square and Slalom tasks on the Unitree G1 humanoid robot. Numbered x’s mark waypoint goals in ascending order.

Explore similar work

Jul 7, 2026cs.RO

MP-MPPI: A Motion Primitive Guided Sampling-Based Optimizer for Model Predictive Control

This paper proposes a novel method that extends the Model Predictive Path Integral (MPPI) method with motion primitives for additional structured sampling, which enhances the convergence towards a globally optimal solution. By evaluating motion primitives and perturbed control sequences in a real-time sampling-based optimization loop, this work addresses the limitations of the path planning capabilities of sampling-based controllers. The algorithm is implemented on a quadcopter simulator and tested on an obstacle field navigation task. It is demonstrated that the proposed approach enhances exploration of the control space while maintaining the fast, reactive behavior required for real-time control.

Marlon Mathisen, Aksel Vaaler, Olav Egeland +1

Model Predictive Path Integral Model Predictive Control Coverage Path Planning Adversarial Motion Priors

Ask PDF

Mar 31, 2026eess.SY

Model Predictive Path Integral PID Control for Learning-Based Path Following

Classical proportional--integral--derivative (PID) control remains widely used in industrial control systems, while model predictive control (MPC) is actively studied to achieve higher performance for systems with nonlinear dynamics. Model predictive path integral (MPPI) control is a sampling-based MPC method that optimizes control inputs without gradient calculations and can handle non-differentiable models and objective functions. However, conventional MPPI directly samples control-input sequences, which can produce large temporal input increments and causes the optimization dimension to grow with the prediction horizon. This study proposes MPPI--PID control, which uses MPPI to optimize PID gains online instead of directly optimizing the control-input sequences. By replacing high-dimensional input-sequence optimization with low-dimensional gain-space optimization while retaining the PID structure, the proposed formulation improves sampling efficiency and promotes smoother control inputs. Theoretical analyses are provided for a unified path-integral update, the relation between optimization dimension and effective sample size, and the temporal correlation of input perturbations induced by the PID structure. The method is evaluated on a learning-based path following of a mini forklift using a residual-learning dynamics model that combines a physical model and a neural network identified from real-machine driving data. Numerical results show that MPPI--PID improves tracking performance over fixed-gain PID, yields smaller input increments than conventional MPPI, and maintains favorable performance under reduced sampling budgets.

Teruki Kato, Koshi Oishi, Seigo Ito

Model Predictive Path Integral Model Predictive Control Feynman-Kac Automatic Differentiation

Ask PDF

Mar 5, 2026cs.RO

Accelerating Sampling-Based Control via Learned Linear Koopman Dynamics

This paper presents an efficient model predictive path integral (MPPI) control framework for systems with complex nonlinear dynamics. To improve the computational efficiency of classic MPPI while preserving control performance, we replace the nonlinear dynamics used for trajectory propagation with a learned linear deep Koopman operator (DKO) model, enabling faster rollout and more efficient trajectory sampling. The DKO dynamics are learned directly from interaction data, eliminating the need for analytical system models. The resulting controller, termed MPPI-DK, is evaluated in simulation on pendulum balancing and surface vehicle navigation tasks, and validated on hardware through reference-tracking experiments on a quadruped robot. Experimental results demonstrate that MPPI-DK achieves control performance close to MPPI with true dynamics while substantially reducing computational cost, enabling efficient real-time control on robotic platforms.

Wenjian Hao, Yuxuan Fang, Zehui Lu +1

Model Predictive Path Integral Koopman Operator

Ask PDF