Prescribed Performance Guided Safe Reinforcement Learning for Roundabout Navigation of Autonomous Vehicles

Wenhan Li; Yangrui Zhang; Hang Zhou; Peiran Liu; Yiding Ji

cs.ROOct 20, 2026

Prescribed Performance Guided Safe Reinforcement Learning for Roundabout Navigation of Autonomous Vehicles

Authors: Wenhan Li, Yangrui Zhang, Hang Zhou, Peiran Liu, Yiding Ji

Organizations: Robotics and Autonomous Systems Thrust, Systems Hub, the Hong Kong University of Science and Technology (Guangzhou), Guangzhou, Guangdong Province, China

Abstract

Connected and Autonomous Vehicles (CAVs) face significant challenges in roundabout navigation due to dense traffic interactions and unpredictable agent behaviors, necessitating high levels of adaptability and safety. While the combination of Reinforcement Learning (RL) and Model Predictive Control (MPC) offers a promising approach, existing frameworks suffer from a structural mismatch where pointwise RL decisions fail to align with MPC's trajectory level optimization, and often lack rigorous safety guarantees. This work proposes RLTrajGen-PPMPC-CBF, a hierarchical framework for safe, smooth and comfort roundabout navigation, which regulates the coupling between RL-based planning and MPC-based control. We employ Prescribed Performance Constraints (PPC) to enforce a gradual, stable convergence between reference trajectories and control outputs, ensuring MPC tracking performance remains consistent with intended behavior from RL. Additionally, a Control Barrier Function (CBF) layer provides real-time safety corrections that are fed back into the RL process. Simulation results in the MetaDrive environment demonstrate that our method achieves a 6.25% increase in average success rate and a 65.87% reduction in Motion Sickness Dose Value (MSDV) compared to the optimal baseline, meanwhile accelerating training convergence.

Figures & tables

Fig. 1: Comparison between the proposed method ( bottom left ) and baseline approaches ( upper left ). We constrain the errors between RL and MPC outputs in prescribed performance bounds to improve trajectory planning performance.

Fig. 2: Overview of RLTrajGen-PPMPC-CBF framework. The RL module (top left) maps observations from the environment (top right) to actions

(\bm{q}_{t},\bm{w}_{t})

. The action

\bm{q}_{t}

is processed by the trajectory generator (bottom left) to construct Trajectory Generator

\mathcal{G}_{t}

and produce a prediction sequence

\bm{z}_{t}^{*}

of length

n_{p}

, which is then used to solve an MPC problem yielding

u^{\mathrm{MPC}_{t}}

. This control input is refined by the CBF module (bottom right) to obtain

u^{\mathrm{CBF}_{t}}

, which is applied to the environment to generate rewards for RL training.

Reward	Explanation	Definition
$\bm{r_{1}}$
$r_{\mathrm{dest}}$	Progress to destination	$\\|\bm{l}_{t}-\bm{l}_{t-1}\\|$
$r_{\mathrm{speed}}$	Speed regulation	$v_{t}/v_{\max}$
$r_{\mathrm{steer}}$	Steering change penalty	$-\\|\theta_{t}-\theta_{t-1}\\|$
$r_{\mathrm{terminal}}$	Terminal reward	$10$ if arrived, else $0$
$\bm{r_{2}}$

TABLE I: Rewards Composition

Fig. 3: ( left ) Discrete-time performance verification for five cases: continuous-time baseline, constant weight(

\mathrm{e}^{-\iota\Delta t}

), and random weights from normal distributions

\mathcal{N}\sim(\mathrm{e}^{-\iota\Delta t},\sigma^{2})

where

\sigma=0.1/0.2/0.3

. ( right ) Schematic of the reference Trajectory Generator parameterized by

(\nu_{t}^{*},\kappa_{t}^{*})

Fig. 4: The establishment of collision avoidance diagrams.

Fig. 5: ( left ) Simulation environment diagram. ( right ) Training Hyperparameter Table.

Index	Name	Description
Benchmark
M1	PPO-only	The RL policy directly outputs control commands $[a,\delta]$ to the simulator, following the official MetaDrive tutorial [ 31 ] and serving as a standard baseline.
M2	PPO-MPC	RL provides high-level decisions, while MPC acts as the low-level controller, without additional coupling modules.
M3	CBFG-PPO	A safety-augmented variant of M1 that incorporates a CBF layer to enforce safety constraints and guide subsequent updates in the learning process [ 8 ] .
Ablation
M4	RLTrajGen-MPC-CBF	Replaces the RL module in M3 with RLTrajGen to generate a reference trajectory for MPC tracking.

TABLE II: Methods Overview

Fig. 6: AR evolution curves of all methods during training.

Fig. 7: ASR evolution curves of all methods during training.

Fig. 8: Velocity, heading angle, acceleration, and steering angle profiles compared with baselines and ablation studies.

Fig. 9: Visualized real-time trajectories for various methods.

Fig. 10: Comparison of

\mathrm{MSDV}

and

\mathrm{JERK}

Method	Avg. Rwd	Avg. Succ. Rate	$\mathrm{MSDV}_{V}$	$\mathrm{MSDV}_{L}$	$\mathrm{JERK}_{V}$	$\mathrm{JERK}_{L}$
M1	156.74	67.50%	0.643	2.216	1.714	49.97
M2	159.18	63.75%	0.997	3.077	4.005	67.49
M3	161.59	77.50%	0.991	2.863	4.555	62.76
M4	138.68	38.75%	0.992	2.697	2.915	66.55
M5	170.02	90.00%	0.458	2.229	2.229	14.81
M6	178.05	96.25%	0.457	0.690	2.483	8.815

TABLE III: Indicator Comparison

Autonomous Driving Planning

Autonomous driving planning selects safe, feasible vehicle actions and trajectories amid traffic, uncertainty, road rules, and changing goals.

See all 384 papers

Reinforcement Learning-Based Control of CAV Platoon Joining Maneuvers in Mixed Traffic
Aug 27, 2026Biao Yin, Abderrahmane Kasmi, Nadir FarhiRL Control Intelligent Transportation Systems
Université Gustave Eiffel, SATIE, Gif-sur-Yvette, 91190, France. · Université Paris Dauphine-PSL, Paris, 75016, France. · Cosys-Grettia, Univ Gustave Eiffel, F-77454 Marne-la-Vallee, France.
Beyond Conservative Automated Driving in Multi-Agent Scenarios via Coupled Model Predictive Control and Deep Reinforcement Learning
Apr 15, 2026Saeed Rahmani, Gözde Körpe, Zhenlin +4Autonomous Driving Planning Multi-Agent Control
Gavin · TU Delft, Faculty of Civil Engineering and Geosciences, Department of Transport and Planning, Delft, The Netherlands · NVIDIA, San Jose, CA, USA
ROSA-RL: Uncertainty-Aware Roundabout Optimized Speed Advisory with Reinforcement Learning
Jun 15, 2026Anna-Lena Schlamp, Jeremias Gerner, Klaus Bogenberger +2Autonomous Driving Planning RL for Autonomous Driving
Institute AImotion Bavaria Technische Hochschule Ingolstadt Ingolstadt, Germany · School of Engineering and Design Technical University of Munich Munich, Germany · CARISSMA Institute of Automated Driving (C-IAD) Technische Hochschule Ingolstadt Ingolstadt, Germany

Aug 27, 2026J/K move · Enter open · S save

Reinforcement Learning-Based Control of CAV Platoon Joining Maneuvers in Mixed Traffic

Biao Yin, Abderrahmane Kasmi, Nadir Farhi

Université Gustave Eiffel, SATIE, Gif-sur-Yvette, 91190, France. · Université Paris Dauphine-PSL, Paris, 75016, France. · Cosys-Grettia, Univ Gustave Eiffel, F-77454 Marne-la-Vallee, France.

RL Control Intelligent Transportation Systems RL for Autonomous Driving Safe RL

Open paper AskPDF arXiv

Connected and automated vehicle (CAV) platooning offers a promising approach to improving road safety and traffic capacity. However, platoon control in real-world traffic is challenging due to uncertainty and heterogeneous driving behaviors. Reinforcement learning (RL) has strong potential for addressing such control problems, but its practical deployment raises challenges related to safety and learning efficiency. This paper proposes a generic modeling and simulation framework for investigating CAV platoon joining maneuvers and comparing deep reinforcement learning (DRL)-based control algorithms. The problem is particularly challenging in mixed-traffic environments, where CAVs coexist with human-driven vehicles exhibiting heterogeneous longitudinal and lateral behaviors. The objective is to achieve safe and efficient joining maneuvers by either incorporating penalties for risky behaviors into the learning process or using an external safety controller to constrain the learned policy. An agent-based modeling framework coupled with the Simulation of Urban MObility (SUMO) simulator is used to evaluate Deep Q-Network (DQN), Double Deep Q-Network (DDQN), and Proximal Policy Optimization (PPO). Results show that PPO outperforms DQN and DDQN, achieving a joining success rate of approximately 98 % and a collision rate below 1 %, largely due to risk-related penalties incorporated into the reward function. However, this improved performance requires more decision steps to complete the maneuver, revealing a trade-off between safety, joining effectiveness, and decision efficiency. An external safety controller effectively prevents collisions, although its interventions may reduce joining efficiency. The results highlight the importance of jointly considering safety and efficiency when designing RL-based controllers for CAV platoon joining in mixed traffic.

Abstract

Autonomous Driving Planning

Explore similar work

Reinforcement Learning-Based Control of CAV Platoon Joining Maneuvers in Mixed Traffic

Beyond Conservative Automated Driving in Multi-Agent Scenarios via Coupled Model Predictive Control and Deep Reinforcement Learning

ROSA-RL: Uncertainty-Aware Roundabout Optimized Speed Advisory with Reinforcement Learning