cs.LGNov 11, 2026

Epigraph Form Reinforcement Learning-based Safe Trajectory Tracking Control for Connected and Autonomous Vehicles

Authors: Xiaoke Yang, Jin Chen, Yafei Wang, Qi Zhang, Ming Li, Haiwen Wu, Chengliang Yin, Yong Zhang

Organizations: School of Mechanical Engineering, Shanghai Jiao Tong University, Shanghai 200240, China · KTH Royal Institute of Technology, Stockholm SE-10044, Sweden · Department of Mechanical and Automation Engineering, The Chinese University of Hong Kong, Hong Kong 999077, China

Abstract

Safe trajectory tracking (STT) aims to enable autonomous vehicles to follow a reference trajectory accurately while avoiding collisions. This problem is particularly challenging for Connected and Autonomous Vehicles (CAVs) in highly dynamic environments. Existing non-learning-based STT methods often use simplified vehicle and interaction models to meet real-time requirements, which may result in suboptimal tracking accuracy. Although Deep Reinforcement Learning (DRL) can help address this limitation, widely used Lagrangian DRL methods often suffer from unstable training because increasing Lagrange multipliers amplify policy-gradient magnitudes. To address these issues, we propose a novel reinforcement learning framework for STT, called STT Control Epigraph Form RL (STTC-EFRL). In this framework, the STT problem is formulated as an epigraph-form min-max optimization problem. Based on this formulation, we develop a learning mechanism that uses a decreasing scalar related to the epigraph to adjust the policy gradient descent direction without increasing the gradient magnitude, thereby improving training stability. We also develop a computationally efficient ray-casting method to compute a polyhedron scaling factor for safety quantification. Experimental results, including Vehicle-in-the-Loop tests, show that STTC-EFRL achieves higher tracking accuracy than the considered non-learning-based baseline and more stable training than widely used Lagrangian DRL methods, while satisfying safety requirements in connected environments.

Figures & tables

Explore similar work

CardsList
  1. Expert Knowledge-driven Reinforcement Learning for Autonomous Racing via Trajectory Guidance and Dynamics Constraints

    Mar 6, 2026Bo Leng, Weiqi Zhang, Zhuoren Li +4Constrained RLAutonomous Racing

  2. Safe Reinforcement Learning of Autonomous Highway Driving: A Unified Framework for Safety and Efficiency

    Jun 12, 2026Chufei Yan, Zhihao Cui, Yiyan Lv +3Reward MachinesReinforcement Learning

  3. Knowledge-Data-Dual-Driven Reinforcement Learning for Autonomous Vehicle Control in Mixed Traffic

    Aug 14, 2026Jie Fang, Wei Zheng, Mengyun Xu +1Hybrid Action Space RLRL for Autonomous Driving