SRL-MPC: Shape-Aware Reinforcement Learned Model Predictive Control

Ruihua Han; Rui Gao; Zhe Liu; Xinyi Wang; Chang Chen; Shuai Wang; Qi Hao; Jia Pan

cs.ROAug 21, 2026

SRL-MPC: Shape-Aware Reinforcement Learned Model Predictive Control

Authors: Ruihua Han, Rui Gao, Zhe Liu, Xinyi Wang, Chang Chen, Shuai Wang, Qi Hao, Jia Pan, +1 more

Organizations: The University of Hong Kong · Southern University of Science and Technology · University of Michigan · Shenzhen Institutes of Advanced Technology

Abstract

Safe and efficient shape-aware navigation in heterogeneous crowds and robot fleets remains challenging. Traditional approaches often assume homogeneous robots, sparse workspaces, simplified geometry, offline computation, or handcrafted parameters to make the problem tractable, which limits their deployment in dense crowd scenarios. Toward this end, we propose Shape-Aware Reinforcement Learned Model Predictive Control (SRL-MPC), a method for safe, efficient, and adaptive navigation in crowds with heterogeneous shapes without geometry simplification. To encode shape-aware safety, we formulate high-order control barrier function (HOCBF) constraints from geometric separation features (GSFs) based on support function transformation. A reinforcement learning (RL) framework then learns a neural policy that reads GSFs and outputs real-time MPC parameter updates, enabling the MPC solver to adapt to neighboring crowd geometries. The key advantage of SRL-MPC is that it preserves the safety structure and generalizability of MPC while integrating the adaptability and intelligence of RL. Experiments in randomized crowd scenarios with arbitrary shaped robot fleets demonstrate the effectiveness, scalability, and robustness of SRL-MPC. The results show that SRL-MPC substantially outperforms representative baselines in safety and adaptability. Project website: https://hanruihua.github.io/srl_mpc_project/

Figures & tables

Figure 1: Overview of SRL-MPC. Key component: 1. Shape aware HOCBF condition is formulated based on GSFs and work as residual penalty in the optimization to improve safety; 2. the reinforcement learned policy adapts MPC parameters based on neighboring GSFs to handle dense scenarios.

Figure 2: Neural network architecture. The GSFs are encoded by CNNs and then concatenated with the self-observation to form a fused feature, which is shared by the actor and critic.

Panel of Figure 3 — Figure 3: Training scenario examples. Each episode randomly samples start positions, goal positions, and convex polygon footprints; arrived robots park as static obstacles.

Panel of Figure 4 — Figure 4: Evaluation rollout with $25$ randomly generated polygonal robots and qualitative held-out challenging scenarios, including cross geometry, nonconvex union, circular polygon, and through traffic. Full rollouts are shown in Appendix A.4 .

Figure 5: Episode outcomes for baseline methods in the random convex polygon scenario.

Robots	Method	Success $\uparrow$	Collision $\downarrow$	Timeout $\downarrow$	Time (s) $\downarrow$	Speed (m/s) $\uparrow$	Path (m) $\downarrow$
10	ORCA [ 32 ]	86.0	8.0	6.0	$5.09\pm 2.58$	$0.878\pm 0.137$	$4.14\pm 2.25$
	AVOCADO [ 25 ]	95.0	0.0	5.0	$5.61\pm 2.63$	$0.785\pm 0.137$	$4.58\pm 2.28$
	VO-polytope [ 19 ]	65.0	35.0	0.0	$7.46\pm 3.91$	$0.536\pm 0.067$	$4.14\pm 2.25$
	SARL [ 4 ]	89.0	8.0	3.0	$8.90\pm 5.24$	$0.664\pm 0.164$	$5.78\pm 3.11$
	RL-RVO [ 14 ]	79.0	14.0	7.0	$6.99\pm 4.36$	$0.780\pm 0.136$	$5.42\pm 2.96$
	SRL-MPC (ours)	100.0	0.0	0.0	$6.11\pm 3.65$	$0.981\pm 0.091$	$6.02\pm 3.55$

Table 1: Baseline comparison in the random convex polygon scenario.

Robots	Method	Success $\uparrow$	Collision $\downarrow$	Timeout $\downarrow$	Time (s) $\downarrow$	Speed (m/s) $\uparrow$	Path (m) $\downarrow$
10	dist_mpc (w/o HOCBF+RL)	77.0	21.0	2.0	$4.99\pm 2.49$	$0.997\pm 0.107$	$5.00\pm 2.51$
10	manual_mpc (w/o RL)	96.0	1.0	3.0	$6.69\pm 5.32$	$0.983\pm 0.091$	$6.60\pm 5.24$
10	rule_mpc (rule, w/o RL)	94.0	0.0	6.0	$6.87\pm 5.55$	$0.980\pm 0.094$	$6.76\pm 5.43$
10	SRL-MPC (ours, with HOCBF+RL)	100.0	0.0	0.0	$6.11\pm 3.65$	$0.981\pm 0.091$	$6.02\pm 3.55$
15	dist_mpc (w/o HOCBF+RL)	51.0	46.0	3.0	$4.81\pm 2.73$	$0.997\pm 0.106$	$4.81\pm 2.74$
15	manual_mpc (w/o RL)	93.0	2.0	5.0	$6.85\pm 4.74$	$0.969\pm 0.101$	$6.64\pm 4.53$

Table 2: Ablation study in the random convex polygon scenario.

Perception noise				Action delay				Parameter delay
$\sigma$ (m)	Succ.	Coll.	T.out	$K$ (step)	Succ.	Coll.	T.out	$K$ (step)	Succ.	Coll.	T.out
0.00	98.0	1.0	1.0	0	98.0	1.0	1.0	0	98.0	1.0	1.0
0.02	100.0	0.0	0.0	1	99.0	1.0	0.0	1	98.0	2.0	0.0
0.05	83.0	17.0	0.0	2	82.0	17.0	1.0	2	100.0	0.0	0.0
0.10	40.0	60.0	0.0	3	2.0	98.0	0.0	3	99.0	1.0	0.0
0.20	1.0	99.0	0.0	5	0.0	100.0	0.0	5	100.0	0.0	0.0

Table 3: Robustness analysis under perception noise and delay perturbations for

N=15

Figure 6: Real-world platform deployment.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 7 — Figure 7: Additional challenging scenario rollouts. Rows show cross geometry, nonconvex union, circular polygon, and through traffic scenarios sampled from held-out seeds.

Figure 8: Reactive pair example in the random polygon scenario. The upper row shows the interaction between R6 and R16 at representative time steps, and the lower row shows the corresponding changes in inter-pair distance, $w_{p}$ , $w_{u}$ , and $d_{\mathrm{safe}}$ , indicating that the learned policy adapts the MPC parameters to the current neighbor geometry.

Group	Symbol or item	Value
Prediction model	$T$ , $\Delta t$	$5$ , $0.1\,\mathrm{s}$
HOCBF-MPC	$K$ , $\|\mathcal{N}_{i}^{\mathrm{cbf}}\|$ , $H_{c}$ , distance horizon, $\gamma$ , $\rho_{\mathrm{obs}}$	$5$ , $3$ , $3$ , $1$ , $0.9$ , $100$
Adaptive parameters	$\lambda_{i}^{t}$	Initial: $(1.0,1.0,0.2\,\mathrm{m})$ ; $\Lambda=[0.01,1.0]\times[0.1,10.0]\times[0.2,1.0]$ ; $w_{\theta}=0.01$ fixed
Solver and geometry	$Q_{1}$ , $Q_{2}$	Shapely , Splitting Conic Solver ( SCS ), one outer iteration; distance cost disabled for SRL-MPC
Training world	$N_{\mathrm{train}}$ , workspace, $\mathbb{C}_{i}$	$15$ robots; $10\,\mathrm{m}\times 10\,\mathrm{m}$ ; convex polygon radius $[0.1,0.4]\,\mathrm{m}$ ; irregularity $[0.1,1.0]$ ; goal threshold $0.3\,\mathrm{m}$

Appendix

Table 4: Controller and scenario parameters used by SRL-MPC .

Group	Symbol or item	Value
Observation	$\mathbf{o}_{i}^{t}$ , $\mathbf{o}_{i,\mathrm{neighbor}}^{t}$	$79=4+3\times 5\times 5$
Action	$\Delta\lambda_{i}^{t}$	$[-0.5,0.5]$ , $[-0.5,0.5]$ , $[-0.1,0.1]\,\mathrm{m}$ ; absolute values clipped to $\Lambda$
Network	CNN, actor, critic	Convolution $3\!\rightarrow\!16\!\rightarrow\!32$ ; actor $36\rightarrow 64\rightarrow 6$ ; critic $36\rightarrow 64\rightarrow 1$ ; log-scale clamp $[-5,0]$
PPO schedule	Frames, batch, mini-batch, epochs	$10^{7}$ , $1000$ , $500$ , $3$
PPO and generalized advantage estimation (GAE) parameters	$\gamma_{\mathrm{PPO}}$ , $\lambda_{\mathrm{GAE}}$	$0.99$ , $0.95$ ; clip $0.2$ ; entropy $0.01$ ; learning rate $3\times 10^{-4}$ ; max grad norm $1.0$
Training reward	$r_{\mathrm{arr}}$ , $r_{\mathrm{col}}$ , $r_{\mathrm{safe}}$ , $r_{\mathrm{step}}$	$40$ , $80$ , $1$ , $0.05$

Appendix

Table 5: Reinforcement learning and neural-network parameters.

Group	Symbol or item	Value
Evaluation sweep	$N$ , $n_{\mathrm{ep}}$ , $E_{\max}$	$N\in\{10,15,20,25\}$ ; $n_{\mathrm{ep}}=100$ ; $E_{\max}=500$ steps ( $50\,\mathrm{s}$ )
Randomization	Train/test seeds	Training seed $0$ ; evaluation seed $100$ ; disjoint scenario sequences
Scenario	random_polygon	Random start-goal pairs; random convex polygon footprints; collision mode stop
Ablation methods	dist_mpc , manual_mpc , rule_mpc	Fixed $w_{p}=0.01$ , $w_{\theta}=0.01$ , $w_{u}=10.0$ ; rule_mpc uses $d_{\mathrm{safe}}=0.30$ by default and switches to $(w_{p},w_{u},d_{\mathrm{safe}})=(1.0,0.1,0.20)$ after $10$ stuck steps
VO baselines	ORCA , AVOCADO , VO-polytope	Neighbor distance $15\,\mathrm{m}$ ; time horizon $0.5\,\mathrm{s}$ ; safety buffer $0.1\,\mathrm{m}$ ; VO-polytope neighbor distance $3\,\mathrm{m}$
RL baselines	RL-RVO , SARL	Pretrained policies; desired speed $1.0\,\mathrm{m/s}$ ; $\Delta t=0.1\,\mathrm{s}$

Appendix

Table 6: Evaluation and baseline settings.

Method	$N=10$	$N=15$	$N=20$	$N=25$
manual_mpc	$5.482\pm 1.585$	$6.586\pm 1.715$	$8.488\pm 1.418$	$9.955\pm 1.803$
dist_mpc	$5.202\pm 1.449$	$7.313\pm 1.125$	$8.693\pm 1.688$	$11.260\pm 1.358$
ORCA	$0.021\pm 0.003$	$0.022\pm 0.001$	$0.024\pm 0.002$	$0.026\pm 0.002$
AVOCADO	$0.022\pm 0.002$	$0.026\pm 0.002$	$0.028\pm 0.002$	$0.028\pm 0.001$
SARL	$15.612\pm 0.904$	$16.353\pm 1.810$	$17.346\pm 1.670$	$18.561\pm 1.737$
RL-RVO	$0.357\pm 0.048$	$0.366\pm 0.037$	$0.370\pm 0.033$	$0.377\pm 0.041$

Appendix

Table 7: Per-robot per-step controller compute time in milliseconds.

Explore similar work

FORMULA: FORmation MPC with neUral barrier Learning for safety Assurance
Apr 6, 2026Qintong Xie, Weishu Zhan, Peter ChinMulti-Robot Formation Control Model Predictive Control
Thayer School of Engineering, Dartmouth College, NH 03784. · Department of Computer Science, The University of Manchester, Manchester M13 9PL, United Kingdom.
Navigating the Crowd: Non-linear MPC with Social Forces Dynamics for Human-Aware Robot Navigation
Jul 11, 2026Stefano Trepella, Andrea Ostuni, Mauro Martini +5Nonlinear MPC Safe Robot Navigation
Department of Electronics and Telecommunications, Politecnico di Torino, 10129, Torino, Italy. · School of Engineering, Pablo de Olavide University, Crta. Utr-era km 1, Seville, Spain
DiffCVaR: Reinforcement Learning for Risk Adaptation via Differentiable CVaR Barrier Functions
May 20, 2026Xinyi Wang, Taekyung Kim, Bardh Hoxha +2Control Barrier Functions Risk-Sensitive RL
Department of Robotics, University of Michigan, Ann Arbor, MI, 48109, USA · Toyota Motor North America, Research & Development, Ann Arbor, MI, 48105, USA · Department of Aerospace Engineering, University of Michigan, Ann Arbor, MI, 48109, USA

Apr 6, 2026J/K move · Enter open · S save

FORMULA: FORmation MPC with neUral barrier Learning for safety Assurance

Qintong Xie, Weishu Zhan, Peter Chin

Thayer School of Engineering, Dartmouth College, NH 03784. · Department of Computer Science, The University of Manchester, Manchester M13 9PL, United Kingdom.

Multi-Robot Formation Control Model Predictive Control Multi-Robot Motion Planning Control Barrier Functions

Open paper AskPDF arXiv

Multi-robot systems (MRS) are essential for large-scale applications such as disaster response, material transport, and warehouse logistics, yet ensuring robust, safety-aware formation control in cluttered and dynamic environments remains a major challenge. Existing model predictive control (MPC) approaches suffer from limitations in scalability and provable safety, while control barrier functions (CBFs), though principled for safety enforcement, are difficult to handcraft for large-scale nonlinear systems. This paper presents FORMULA, a safe distributed, learning-enhanced predictive control framework that integrates MPC with Control Lyapunov Functions (CLFs) for stability and neural network-based CBFs for decentralized safety, eliminating manual safety constraint design. This scheme maintains formation integrity during obstacle avoidance, resolves deadlocks in dense configurations, and reduces online computational load. Simulation results demonstrate that FORMULA enables scalable, safety-aware, formation-preserving navigation for multi-robot teams in complex environments.

Abstract

Explore similar work

FORMULA: FORmation MPC with neUral barrier Learning for safety Assurance

Navigating the Crowd: Non-linear MPC with Social Forces Dynamics for Human-Aware Robot Navigation

DiffCVaR: Reinforcement Learning for Risk Adaptation via Differentiable CVaR Barrier Functions