RL-Guided PAC-NMPC for Probabilistically-Safe Perception-Based Navigation in Unknown Environments

Adam Polevoy; Dillon Capalongo; Katherine Tang; Mark Gonzales; Marin Kobilarov; Joseph Moore

cs.ROSep 30, 2026

RL-Guided PAC-NMPC for Probabilistically-Safe Perception-Based Navigation in Unknown Environments

Authors: Adam Polevoy, Dillon Capalongo, Katherine Tang, Mark Gonzales, Marin Kobilarov, Joseph Moore

Organizations: Johns Hopkins University Applied Physics Laboratory, Laurel, MD 20723, USA. · Department of Mechanical Engineering, Johns Hopkins University, Baltimore, MD 21218, USA.

Abstract

In this paper, we present an approach for combining stochastic nonlinear model predictive control (SNMPC) and reinforcement learning (RL) to enable probabilistically-safe perception-based navigation in unknown environments. Our method first uses RL to train probabilistic actor-critic and sensor prediction models. We then leverage these probabilistic models in a sampling-based SNMPC framework known as Probably Approximately Correct (PAC)-NMPC, which uses hard constraints to enforce finite-time statistical guarantees on the probability of collision and value function improvement. By ensuring that our finite-horizon SNMPC policies decrease the value function in expectation, we can approach the long-horizon performance of the RL approach while satisfying probabilistic safety constraints. Through simulation experiments, we show that our approach can improve the safety of perception-based RL navigation policies and scale to high dimensional systems with large sensor input spaces and complex nonlinear dynamics. We also demonstrate our approach through hardware experiments, showing improved performance for vision-based navigation with an agile fixed-wing aerial vehicle in unknown environments.

Figures & tables

Fig. 1 : Actor-Critic PAC-NMPC for navigation and obstacle avoidance on a fixed-wing with an on-board depth camera.

Approach	Stochastic MPC	Stochastic Value Func.	Safety Guarantees	Perception (max dim)	Hardware Demo
DMPC [ 38 ] , CACTO [ 39 ]	✗	✗	✗	✗	✗
PETS [ 35 ] , MQP [ 37 ]	✓	✗	✗	✗	✗
DiffStack [ 44 ]	✗	✗	✗	✓ (5 $\times$ a) ∗	✗
AC-MPC [ 46 ]	✗	✗	✗	✓ (12 $\times$ 2)	✓
POLO [ 36 ]	✓	✓	✗	✗	✗
LOOP [ 42 ] , TD-MPC [ 43 ]	✓	✗	✗	✓ (16, 84 $\times$ 84 $\times$ 9)	✗

TABLE I : Comparison of MPC & RL hybrid approaches.

Approach		Success	Stuck	Violation
RL Actor Network		$90\%$	$\mathbf{3\%}$	$7\%$
MPPI w/ Learned $V^{\boldsymbol{\phi}\boldsymbol{\psi}}$		$70\%$	$7\%$	$23\%$
PAC-NMPC	Quad. Term. Cost	$76\%$	$24\%$	$\mathbf{0\%}$
	Map & A*	89%	4%	7%
	Learned $V^{\boldsymbol{\phi}\boldsymbol{\psi}}$	$\mathbf{97\%}$	$\mathbf{3\%}$	$\mathbf{0\%}$

TABLE II : Simulated cluttered environments results.

Fig. 5 : Cluttered and trap environment examples. Slices of the learned value functions are visualized as heatmaps.

Fig. 6 : PAC Bounds & Monte Carlo estimates.

Approach		Success	Stuck	Violation
RL Actor Network		$82\%$	$\mathbf{2\%}$	$16\%$
MPPI w/ Learned $V^{\boldsymbol{\phi}\boldsymbol{\psi}}$		$55\%$	$9\%$	$36\%$
PAC-NMPC	Quad. Term. Cost	$47\%$	$53\%$	$\mathbf{0\%}$
	Map & A*	91%	4%	5%
	Learned $V^{\boldsymbol{\phi}\boldsymbol{\psi}}$	$\mathbf{93\%}$	$7\%$	$\mathbf{0\%}$

TABLE III : Simulated trap environments results.

Approach	Success	Stuck	Violation
Actor Policy	$85\%$	$\mathbf{0\%}$	$15\%$
PAC-NMPC w/ Learned $V^{\boldsymbol{\phi}\boldsymbol{\psi}}$	$\mathbf{95\%}$	$5\%$	$\mathbf{0\%}$
Actor Policy w/ Mismatch	$70\%$	$\mathbf{0\%}$	$30\%$
PAC-NMPC w/ Learned $V^{\boldsymbol{\phi}\boldsymbol{\psi}}$ w/ Mismatch	$80\%$	$20\%$	$\mathbf{0\%}$

TABLE IV : Hardware environments results.

Fig. 8 : Edge540 aerial vehicle with RealSense D450 depth camera.

Fig. 9 : Histogram of fixed-wing aerial vehicle model acceleration errors. Fitted Gaussian distribution shown in orange.

Panel of Figure 13 — Fig. 10 : Image prediction on decimated RealSense D450 depth image collected during fixed-wing flight. Depth is visualized as blue (near) to yellow (far).

Model	FLOPs (M)	$\delta_{1}$	$\delta_{2}$	$\delta_{3}$	REL	RMSE
Geometric (Nearest)	N/A	0.133	0.217	0.306	2.838	0.634
Geometric (Splat)	N/A	0.186	0.309	0.419	2.240	0.548
Fully Connected	36.798	0.848	0.935	0.963	0.170	0.081
U-Net	42.283	0.923	0.965	0.983	0.087	0.058

TABLE V : Performance of depth image prediction models on simulated data.

\delta_{i}

is the percentage of pixels satisfying

\max(\frac{\hat{d}}{d},\frac{d}{\hat{d}})<1.25^{i}

where

\hat{d}

and

d

are predicted and ground-truth depths. REL and RMSE are the mean absolute relative error and root mean squared error respectively.

Fig. 11 : Simulation environment example. Ceiling/floor not shown. RL actor in yellow, PAC-NMPC with global planner in red, AC-PAC-NMPC in blue.

Approach		Success	Cost
RL Actor Network		81%	468.1
PAC-NMPC	Depth Image History	52%	757.7
	Map	63%	810.2
	Map & A*	74%	738.8
	Map & A* Unknown	85%	718.14
	Actor-Critic (ours)	90%	513.3

TABLE VI : Baseline Comparison Results

Panel of Table 17 — TABLE VII : Ablation Study Results

Fig. 12 : Out-of-distribution simulation environment example. Ceiling/floor not shown. RL actor (collision) in yellow, PAC-NMPC with global planner in red, AC-PAC-NMPC in blue.

Approach		Success	Cost
RL Actor Network		46%	481.8
PAC-NMPC	Map & A* Unknown	76%	732.6
PAC-NMPC	Actor-Critic (ours)	76%	512.1

TABLE VIII : Out Of Distribution Experiment Results

Panel of Figure 20 — Fig. 13 : Example hardware environments with time-lapse trajectories of the aerial vehicle controlled by our approach.

Fig. 14 : Optimized PAC bounds compared to Monte Carlo estimates at each planning interval for one trial.

Panel of Figure 22 — Fig. 15 : Spurious noise in consecutive depth camera measurements, which caused a spike in $\mathcal{C}^{+}_{\alpha}$ .

Approach		Success	Cost
RL Actor Network		40%	403.8
PAC-NMPC	Map & A* Unknown	40%	325.4
PAC-NMPC	Actor-Critic (ours)	80%	252.2

TABLE IX : Fixed-wing aerial vehicle hardware results.

Fig. 16 : Flight paths from one hardware trial.

Explore similar work

Online, Reachability-Aware, Sampling-Based Motion Planning
Sep 8, 2026Brendan Gould, Zhiyuan Zhang, Panagiotis Tsiotras +1Model Predictive Control Motion Planning
Probabilistic Recursively Feasible Motion Planning Under Uncertain Environments
May 18, 2026Hyeontae Sung, Hyeongchan Ham, Junyoung Park +2Model Predictive Control Motion Planning
School of Electrical Engineering, Korea Advanced Institute of Science and Technology, Daejeon, Korea · The SYCAMORE Lab, ´Ecole Polytechnique F´ed´erale de Lausanne (EPFL), Switzerland
Interactive Trajectory Planning with Learning-based Distributionally Robust Model Predictive Control and Markov Systems
May 8, 2026Erik Börve, Nikolce Murgovski, Morteza Haghir Chehreghani +1Model Predictive Control Robust Trajectory
Chalmers University of Technology, Chalmersgatan 4, 412 96 Göteborg, Sweden · Volvo Group Trucks Technology, Gropegårdsgatan 2, 417 15 Göteborg, Sweden

Sep 8, 2026, cs.ROJ/K move · Enter open · S save

Online, Reachability-Aware, Sampling-Based Motion Planning

Brendan Gould, Zhiyuan Zhang, Panagiotis Tsiotras, Samuel Coogan

Model Predictive Control Motion Planning Robotic Systems Autonomous Racing Motion

Open paper AskPDF arXiv

Sampling-Based Model-Predictive Control (MPC) algorithms are a flexible class of controllers used for navigation on a wide range of robotic systems. Historically, such approaches have lacked hard safety guarantees, a shortcoming which we remedy in this work by computing guaranteed reachable-set overapproximations online with a fast, interval-based pipeline. We show that our method achieves similar performance to a state-of-the-art reachability-based planner without the need for the expensive pre-computation step, and can be scaled to systems that are infeasible using existing approaches. Finally, we demonstrate that our technique reduces safety violations by over 99% in a racing simulation and successfully controls a model racecar on real hardware experiments without crashes.

Abstract

Explore similar work

Online, Reachability-Aware, Sampling-Based Motion Planning

Probabilistic Recursively Feasible Motion Planning Under Uncertain Environments

Interactive Trajectory Planning with Learning-based Distributionally Robust Model Predictive Control and Markov Systems