SRL-MPC: Shape-Aware Reinforcement Learned Model Predictive Control

Ruihua Han; Rui Gao; Zhe Liu; Xinyi Wang; Chang Chen; Shuai Wang; Qi Hao; Jia Pan

cs.ROAug 21, 2026

SRL-MPC: Shape-Aware Reinforcement Learned Model Predictive Control

Authors: Ruihua Han, Rui Gao, Zhe Liu, Xinyi Wang, Chang Chen, Shuai Wang, Qi Hao, Jia Pan, +1 more

Organizations: The University of Hong Kong · Southern University of Science and Technology · University of Michigan · Shenzhen Institutes of Advanced Technology

Abstract

Safe and efficient shape-aware navigation in heterogeneous crowds and robot fleets remains challenging. Traditional approaches often assume homogeneous robots, sparse workspaces, simplified geometry, offline computation, or handcrafted parameters to make the problem tractable, which limits their deployment in dense crowd scenarios. Toward this end, we propose Shape-Aware Reinforcement Learned Model Predictive Control (SRL-MPC), a method for safe, efficient, and adaptive navigation in crowds with heterogeneous shapes without geometry simplification. To encode shape-aware safety, we formulate high-order control barrier function (HOCBF) constraints from geometric separation features (GSFs) based on support function transformation. A reinforcement learning (RL) framework then learns a neural policy that reads GSFs and outputs real-time MPC parameter updates, enabling the MPC solver to adapt to neighboring crowd geometries. The key advantage of SRL-MPC is that it preserves the safety structure and generalizability of MPC while integrating the adaptability and intelligence of RL. Experiments in randomized crowd scenarios with arbitrary shaped robot fleets demonstrate the effectiveness, scalability, and robustness of SRL-MPC. The results show that SRL-MPC substantially outperforms representative baselines in safety and adaptability. Project website: https://hanruihua.github.io/srl_mpc_project/

Figures & tables

Figure 1: Overview of SRL-MPC. Key component: 1. Shape aware HOCBF condition is formulated based on GSFs and work as residual penalty in the optimization to improve safety; 2. the reinforcement learned policy adapts MPC parameters based on neighboring GSFs to handle dense scenarios.

Figure 2: Neural network architecture. The GSFs are encoded by CNNs and then concatenated with the self-observation to form a fused feature, which is shared by the actor and critic.

Panel of Figure 3 — Figure 3: Training scenario examples. Each episode randomly samples start positions, goal positions, and convex polygon footprints; arrived robots park as static obstacles.

Panel of Figure 4 — Figure 4: Evaluation rollout with $25$ randomly generated polygonal robots and qualitative held-out challenging scenarios, including cross geometry, nonconvex union, circular polygon, and through traffic. Full rollouts are shown in Appendix A.4 .

Figure 5: Episode outcomes for baseline methods in the random convex polygon scenario.

Robots	Method	Success $\uparrow$	Collision $\downarrow$	Timeout $\downarrow$	Time (s) $\downarrow$	Speed (m/s) $\uparrow$	Path (m) $\downarrow$
10	ORCA [ 32 ]	86.0	8.0	6.0	$5.09\pm 2.58$	$0.878\pm 0.137$	$4.14\pm 2.25$
	AVOCADO [ 25 ]	95.0	0.0	5.0	$5.61\pm 2.63$	$0.785\pm 0.137$	$4.58\pm 2.28$
	VO-polytope [ 19 ]	65.0	35.0	0.0	$7.46\pm 3.91$	$0.536\pm 0.067$	$4.14\pm 2.25$
	SARL [ 4 ]	89.0	8.0	3.0	$8.90\pm 5.24$	$0.664\pm 0.164$	$5.78\pm 3.11$
	RL-RVO [ 14 ]	79.0	14.0	7.0	$6.99\pm 4.36$	$0.780\pm 0.136$	$5.42\pm 2.96$
	SRL-MPC (ours)	100.0	0.0	0.0	$6.11\pm 3.65$	$0.981\pm 0.091$	$6.02\pm 3.55$

Table 1: Baseline comparison in the random convex polygon scenario.

Robots	Method	Success $\uparrow$	Collision $\downarrow$	Timeout $\downarrow$	Time (s) $\downarrow$	Speed (m/s) $\uparrow$	Path (m) $\downarrow$
10	dist_mpc (w/o HOCBF+RL)	77.0	21.0	2.0	$4.99\pm 2.49$	$0.997\pm 0.107$	$5.00\pm 2.51$
10	manual_mpc (w/o RL)	96.0	1.0	3.0	$6.69\pm 5.32$	$0.983\pm 0.091$	$6.60\pm 5.24$
10	rule_mpc (rule, w/o RL)	94.0	0.0	6.0	$6.87\pm 5.55$	$0.980\pm 0.094$	$6.76\pm 5.43$
10	SRL-MPC (ours, with HOCBF+RL)	100.0	0.0	0.0	$6.11\pm 3.65$	$0.981\pm 0.091$	$6.02\pm 3.55$
15	dist_mpc (w/o HOCBF+RL)	51.0	46.0	3.0	$4.81\pm 2.73$	$0.997\pm 0.106$	$4.81\pm 2.74$
15	manual_mpc (w/o RL)	93.0	2.0	5.0	$6.85\pm 4.74$	$0.969\pm 0.101$	$6.64\pm 4.53$

Table 2: Ablation study in the random convex polygon scenario.

Perception noise				Action delay				Parameter delay
$\sigma$ (m)	Succ.	Coll.	T.out	$K$ (step)	Succ.	Coll.	T.out	$K$ (step)	Succ.	Coll.	T.out
0.00	98.0	1.0	1.0	0	98.0	1.0	1.0	0	98.0	1.0	1.0
0.02	100.0	0.0	0.0	1	99.0	1.0	0.0	1	98.0	2.0	0.0
0.05	83.0	17.0	0.0	2	82.0	17.0	1.0	2	100.0	0.0	0.0
0.10	40.0	60.0	0.0	3	2.0	98.0	0.0	3	99.0	1.0	0.0
0.20	1.0	99.0	0.0	5	0.0	100.0	0.0	5	100.0	0.0	0.0

Table 3: Robustness analysis under perception noise and delay perturbations for

N=15

Figure 6: Real-world platform deployment.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 7 — Figure 7: Additional challenging scenario rollouts. Rows show cross geometry, nonconvex union, circular polygon, and through traffic scenarios sampled from held-out seeds.

Figure 8: Reactive pair example in the random polygon scenario. The upper row shows the interaction between R6 and R16 at representative time steps, and the lower row shows the corresponding changes in inter-pair distance, $w_{p}$ , $w_{u}$ , and $d_{\mathrm{safe}}$ , indicating that the learned policy adapts the MPC parameters to the current neighbor geometry.

Group	Symbol or item	Value
Prediction model	$T$ , $\Delta t$	$5$ , $0.1\,\mathrm{s}$
HOCBF-MPC	$K$ , $\|\mathcal{N}_{i}^{\mathrm{cbf}}\|$ , $H_{c}$ , distance horizon, $\gamma$ , $\rho_{\mathrm{obs}}$	$5$ , $3$ , $3$ , $1$ , $0.9$ , $100$
Adaptive parameters	$\lambda_{i}^{t}$	Initial: $(1.0,1.0,0.2\,\mathrm{m})$ ; $\Lambda=[0.01,1.0]\times[0.1,10.0]\times[0.2,1.0]$ ; $w_{\theta}=0.01$ fixed
Solver and geometry	$Q_{1}$ , $Q_{2}$	Shapely , Splitting Conic Solver ( SCS ), one outer iteration; distance cost disabled for SRL-MPC
Training world	$N_{\mathrm{train}}$ , workspace, $\mathbb{C}_{i}$	$15$ robots; $10\,\mathrm{m}\times 10\,\mathrm{m}$ ; convex polygon radius $[0.1,0.4]\,\mathrm{m}$ ; irregularity $[0.1,1.0]$ ; goal threshold $0.3\,\mathrm{m}$

Appendix

Table 4: Controller and scenario parameters used by SRL-MPC .

Group	Symbol or item	Value
Observation	$\mathbf{o}_{i}^{t}$ , $\mathbf{o}_{i,\mathrm{neighbor}}^{t}$	$79=4+3\times 5\times 5$
Action	$\Delta\lambda_{i}^{t}$	$[-0.5,0.5]$ , $[-0.5,0.5]$ , $[-0.1,0.1]\,\mathrm{m}$ ; absolute values clipped to $\Lambda$
Network	CNN, actor, critic	Convolution $3\!\rightarrow\!16\!\rightarrow\!32$ ; actor $36\rightarrow 64\rightarrow 6$ ; critic $36\rightarrow 64\rightarrow 1$ ; log-scale clamp $[-5,0]$
PPO schedule	Frames, batch, mini-batch, epochs	$10^{7}$ , $1000$ , $500$ , $3$
PPO and generalized advantage estimation (GAE) parameters	$\gamma_{\mathrm{PPO}}$ , $\lambda_{\mathrm{GAE}}$	$0.99$ , $0.95$ ; clip $0.2$ ; entropy $0.01$ ; learning rate $3\times 10^{-4}$ ; max grad norm $1.0$
Training reward	$r_{\mathrm{arr}}$ , $r_{\mathrm{col}}$ , $r_{\mathrm{safe}}$ , $r_{\mathrm{step}}$	$40$ , $80$ , $1$ , $0.05$

Appendix

Table 5: Reinforcement learning and neural-network parameters.

Group	Symbol or item	Value
Evaluation sweep	$N$ , $n_{\mathrm{ep}}$ , $E_{\max}$	$N\in\{10,15,20,25\}$ ; $n_{\mathrm{ep}}=100$ ; $E_{\max}=500$ steps ( $50\,\mathrm{s}$ )
Randomization	Train/test seeds	Training seed $0$ ; evaluation seed $100$ ; disjoint scenario sequences
Scenario	random_polygon	Random start-goal pairs; random convex polygon footprints; collision mode stop
Ablation methods	dist_mpc , manual_mpc , rule_mpc	Fixed $w_{p}=0.01$ , $w_{\theta}=0.01$ , $w_{u}=10.0$ ; rule_mpc uses $d_{\mathrm{safe}}=0.30$ by default and switches to $(w_{p},w_{u},d_{\mathrm{safe}})=(1.0,0.1,0.20)$ after $10$ stuck steps
VO baselines	ORCA , AVOCADO , VO-polytope	Neighbor distance $15\,\mathrm{m}$ ; time horizon $0.5\,\mathrm{s}$ ; safety buffer $0.1\,\mathrm{m}$ ; VO-polytope neighbor distance $3\,\mathrm{m}$
RL baselines	RL-RVO , SARL	Pretrained policies; desired speed $1.0\,\mathrm{m/s}$ ; $\Delta t=0.1\,\mathrm{s}$

Appendix

Table 6: Evaluation and baseline settings.

Method	$N=10$	$N=15$	$N=20$	$N=25$
manual_mpc	$5.482\pm 1.585$	$6.586\pm 1.715$	$8.488\pm 1.418$	$9.955\pm 1.803$
dist_mpc	$5.202\pm 1.449$	$7.313\pm 1.125$	$8.693\pm 1.688$	$11.260\pm 1.358$
ORCA	$0.021\pm 0.003$	$0.022\pm 0.001$	$0.024\pm 0.002$	$0.026\pm 0.002$
AVOCADO	$0.022\pm 0.002$	$0.026\pm 0.002$	$0.028\pm 0.002$	$0.028\pm 0.001$
SARL	$15.612\pm 0.904$	$16.353\pm 1.810$	$17.346\pm 1.670$	$18.561\pm 1.737$
RL-RVO	$0.357\pm 0.048$	$0.366\pm 0.037$	$0.370\pm 0.033$	$0.377\pm 0.041$

Appendix

Table 7: Per-robot per-step controller compute time in milliseconds.

Explore similar work

Apr 6, 2026

FORMULA: FORmation MPC with neUral barrier Learning for safety Assurance

Multi-robot systems (MRS) are essential for large-scale applications such as disaster response, material transport, and warehouse logistics, yet ensuring robust, safety-aware formation control in cluttered and dynamic environments remains a major challenge. Existing model predictive control (MPC) approaches suffer from limitations in scalability and provable safety, while control barrier functions (CBFs), though principled for safety enforcement, are difficult to handcraft for large-scale nonlinear systems. This paper presents FORMULA, a safe distributed, learning-enhanced predictive control framework that integrates MPC with Control Lyapunov Functions (CLFs) for stability and neural network-based CBFs for decentralized safety, eliminating manual safety constraint design. This scheme maintains formation integrity during obstacle avoidance, resolves deadlocks in dense configurations, and reduces online computational load. Simulation results demonstrate that FORMULA enables scalable, safety-aware, formation-preserving navigation for multi-robot teams in complex environments.

Qintong Xie, Weishu Zhan, Peter Chin

Thayer School of Engineering, Dartmouth College, NH 03784. · Department of Computer Science, The University of Manchester, Manchester M13 9PL, United Kingdom.

Multi-Robot Formation Control Model Predictive Control Multi-Robot Motion Planning Control Barrier Functions

Ask PDF

Jul 11, 2026

Navigating the Crowd: Non-linear MPC with Social Forces Dynamics for Human-Aware Robot Navigation

Safe and socially compliant navigation remains a fundamental challenge for autonomous robots operating in human-populated environments. Beyond collision avoidance, robots must anticipate human motion and respect personal space to ensure human comfort. Model Predictive Control (MPC) offers a robust alternative to classical and data-driven methods, although its effectiveness strongly depends on accurate human motion prediction and efficient computation. This paper introduces SFM-NMPC, a Social Force Model-based Non-linear Model Predictive Control framework that embeds human motion prediction directly within the optimization loop. By incorporating the Social Force Model into the dynamic model of surrounding agents, the controller jointly predicts the trajectories of humans and robots over the prediction horizon, thereby enabling socially-aware planning. A tailored set of social cost functions guides the optimization toward human-compliant behaviors. Despite the increased model complexity, the proposed formulation runs in real time at 20 Hz. Extensive simulated testing in crowded environments demonstrates that SFM-NMPC outperforms state-of-the-art baselines in social compliance metrics while maintaining efficient and smooth navigation. Visual trajectory analysis and an ablation study further highlight the contribution of the embedded SFM dynamics and social cost terms, confirming the effectiveness of the proposed approach for real-world social navigation.

Stefano Trepella, Andrea Ostuni, Mauro Martini +5

Department of Electronics and Telecommunications, Politecnico di Torino, 10129, Torino, Italy. · School of Engineering, Pablo de Olavide University, Crta. Utr-era km 1, Seville, Spain

Nonlinear MPC Safe Robot Navigation Social Robot Navigation

Ask PDF

May 20, 2026

DiffCVaR: Reinforcement Learning for Risk Adaptation via Differentiable CVaR Barrier Functions

Planning through crowded environments under uncertain obstacle motions remains difficult, as stochastic interactions often induce overly conservative behavior or reduced efficiency. To address this challenge, we propose an end-to-end risk adaptation framework for crowd navigation under obstacle-motion uncertainty modeled by a Gaussian mixture model. The framework combines reinforcement learning~(RL) with a differentiable quadratic-program safety layer based on Conditional Value-at-Risk~(CVaR) barrier functions, jointly learning nominal control input, risk level, and safety margin and enforcing explicit probabilistic safety constraints. This design enables context-aware adaptation, promoting efficient behavior while invoking caution only when necessary. We conduct extensive evaluations in dynamic, uncertain, and crowded environments across varying obstacle densities and robot models, and further assess generalization under three out-of-distribution cases. Comparisons across optimization-based, RL-based, and integrated RL and optimization methods are provided, and the proposed method is shown to deliver the strongest overall performance in safety, efficiency, and generalization under uncertainty.

Xinyi Wang, Taekyung Kim, Bardh Hoxha +2

Department of Robotics, University of Michigan, Ann Arbor, MI, 48109, USA · Toyota Motor North America, Research & Development, Ann Arbor, MI, 48105, USA · Department of Aerospace Engineering, University of Michigan, Ann Arbor, MI, 48109, USA

Control Barrier Functions Risk-Sensitive RL Safe RL Conditional Value-at-Risk Robot Collision Avoidance

Ask PDF