cs.LGNov 17, 2026

GP--RL--MPC: Uncertainty-Aware Reinforcement Learning-Augmented Model Predictive Control with Gaussian Process Postdecision State-Value Functions

Authors: Jon Li, Jay H. Lee

Organizations: Mork Family Department of Chemical Engineering and Materials Science, University of Southern California, Los Angeles, CA 90089, United States

Abstract

We propose Gaussian Process Reinforcement Learning-Augmented Model Predictive Control (GP--RL--MPC), which uses a learned critic as the MPC terminal penalty to account for costs beyond the prediction horizon. Gaussian process regression (GPR) enables data-efficient critic learning from limited, noisy rollouts within optimistic approximate policy iteration (OAPI), with soft updates mitigating policy chattering. A tunable risk coefficient weights GP epistemic uncertainty in the terminal cost, regulating sensitivity to uncertain critic predictions. The postdecision state formulation shifts disturbance averaging offline to account for aleatoric uncertainty from the terminal transition onward. Under matching objectives and admissible actions, a one-step horizon recovers greedy value-space RL. After fitting GP hyperparameters by exact marginal likelihood, we construct a sparse deterministic training conditional (DTC) posterior using inducing points selected by pivoted Cholesky (PC), reducing online evaluation costs as the training dataset grows. In a continuous stirred-tank reactor (CSTR) subject to exogenous disturbances, GP--RL--MPC with an appropriately selected risk coefficient and policy screening reduces the sum of squared errors (SSE) by an average of 80.52% relative to an otherwise matched baseline MPC across five random seeds (sample standard deviation: 3.48%).

Figures & tables

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. A Tutorial on Gaussian Process Learning-based Model Predictive Control

    Apr 2, 2024Jie Wang, Youmin ZhangModel Predictive ControlRobotic Control

  2. Real-Time Online Learning for Model Predictive Control using a Spatio-Temporal Gaussian Process Approximation

    Mar 18, 2026Lars Bartels, Amon Lahr, Andrea Carron +1Model Predictive ControlGaussian Processes

  3. Uncertainty-Aware Hybrid Offline Reinforcement Learning with Online Action Selection Enabled by Sparse Gaussian-Process Critics

    Nov 10, 2026Marco Chan, Christos Evripidou, Liqiu Dong +2Offline RLOffline-To-Online RL