Macroscopic Characteristics of Mixed Traffic Flow with Deep Reinforcement Learning Based Automated and Human-Driven Vehicles

Pankaj Kumar; Pranamesh Chakraborty; Subrahmanya Swamy Peruru

cs.AIMar 26, 2026

Macroscopic Characteristics of Mixed Traffic Flow with Deep Reinforcement Learning Based Automated and Human-Driven Vehicles

Authors: Pankaj Kumar, Pranamesh Chakraborty, Subrahmanya Swamy Peruru

Organizations: Department of Civil Engineering, Indian Institute of Technology Kanpur, Kanpur-208016, U.P., India · Department of Electrical Engineering Indian Institute of Technology Kanpur, Kanpur-208016, U.P., India

Abstract

Automated Vehicle (AV) control in mixed traffic, where AVs coexist with human-driven vehicles, poses significant challenges in balancing safety, efficiency, comfort, fuel efficiency, and compliance with traffic rules while capturing heterogeneous driver behavior. Traditional car-following models, such as the Intelligent Driver Model (IDM), often struggle to generalize across diverse traffic scenarios and typically do not account for fuel efficiency, motivating the use of learning-based approaches. Although Deep Reinforcement Learning (DRL) has shown strong microscopic performance in car-following conditions, its macroscopic traffic flow characteristics remain underexplored. This study focuses on analyzing the macroscopic traffic flow characteristics and fuel efficiency of DRL-based models in mixed traffic. A Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm is implemented for AVs' control and trained using the NGSIM highway dataset, enabling realistic interaction with human-driven vehicles. Traffic performance is evaluated using the Fundamental Diagram (FD) under varying driver heterogeneity, heterogeneous time-gap penetration levels, and different shares of RL-controlled vehicles. A macroscopic level comparison of fuel efficiency between the RL-based AV model and the IDM is also conducted. Results show that traffic performance is sensitive to the distribution of safe time gaps and the proportion of RL vehicles. Transitioning from fully human-driven to fully RL-controlled traffic can increase road capacity by approximately 7.52%. Further, RL-based AVs also improve average fuel efficiency by about 28.98% at higher speeds (above 50 km/h), and by 1.86% at lower speeds (below 50 km/h) compared to the IDM. Overall, the DRL framework enhances traffic capacity and fuel efficiency without compromising safety.

Figures & tables

Fig. 1 : General schematic representation of (a) time-space region to measure traffic flow characteristics, and (b) parallelogram region with trajectory points inside

Fig. 2 : Example leader vehicle trajectories produced by the Ornstein–Uhlenbeck process

Fig. 3 : Normalized rolling rewards of the TD3 agent during training

Fig. 4 : Visualisation of traffic flow analysis: (a) time–space trajectory data with generated parallelogram, (b) magnified section of region A, and (c) empirical fundamental diagrams under varying levels of driver heterogeneity

Penetration rate of driver heterogeneity (%)	Optimal traffic density (veh/km)	Traffic capacity (veh/h)
$\left[1.0T\ (60\%),1.5T\ (20\%),2T\ (20\%)\right]$	51	1773
$\left[1.0T\ (20\%),1.5T\ (60\%),2T\ (20\%)\right]$	49	1683
$\left[1.0T\ (34\%),1.5T\ (33\%),2T\ (33\%)\right]$	47	1632
$\left[1.0T\ (20\%),1.5T\ (20\%),2T\ (60\%)\right]$	43	1508

Table 1: Estimated FD parameters corresponding to different penetration rates of driver heterogeneity

Penetration rate of RL (%)	Optimal traffic density (veh/km)	Traffic capacity (veh/h)	Increase in capacity (%)
100	50	1716	7.52
75	47	1632	2.25
50	46	1614	1.13
25	45	1609	0.82
0	45	1596	0

Table 2: Estimated FD parameters for different penetration rates of RL-based vehicles with a fixed safe time gap of

T=1.5

sec

Fig. 5 : Fuel efficiency performance of RL and IDM models: (a) average fuel efficiency, and (b) % of time with positive acceleration across speed ranges

Explore similar work

Aug 14, 2026

Knowledge-Data-Dual-Driven Reinforcement Learning for Autonomous Vehicle Control in Mixed Traffic

In mixed traffic, decision-making for autonomous vehicles (AVs) confronts three interrelated challenges. First, physics-based priors incorporated into reinforcement learning (RL) models fail to capture latent interactive vehicle intentions and diverse driver behaviors, limiting the proactive reasoning capabilities. Second, abrupt maneuvers by surrounding vehicles cause non-stationarity, leaving long-tail safety events under-explored. Third, hybrid action spaces destabilize unified RL training due to the different temporal scales of continuous car-following and discrete lane-changing maneuvers. To address these issues, we propose Knowledge-Data Dual-driven Reinforcement Learning (KDDRL). First, a conditional deep generative model synthesizes intention-aware future trajectories, converting passive perception into proactive predictive states. Second, a knowledge-data dual-driven paradigm operates on these predictive states, fusing probabilistic data-driven insights with physical constraints to guide safe exploration through safety-critical scenarios. Third, a coupling module compresses both intention-aware trajectories and physical constraints into compact shared embeddings. This unified representation enables asynchronous multi-timescale optimization of continuous car-following and discrete lane-changing while preserving mutual information. Evaluations on dataset-calibrated simulations demonstrate that KDDRL effectively handles intention uncertainty, accelerates training convergence, and outperforms conventional baseline methods in terms of safety, efficiency, and comfort.

Jie Fang, Wei Zheng, Mengyun Xu +1

department of Civil Engineering, Fuzhou University, Fuzhou, 350108, China

RL for Autonomous Driving Reinforcement Learning Autonomous Driving Planning

Ask PDF

Jun 7, 2026

Platooning Connected, Autonomous, and Human-Driven Vehicles: A Deep Reinforcement Learning-based Approach

Conventionally, existing vehicle platooning approaches are designed for connected vehicles, typically including connected autonomous vehicles and connected human-driven vehicles. Non-connected vehicles, such as non-connected autonomous or human-driven vehicles, are not incorporated. As a result, these platooning approaches may not properly reflect real-world mixed traffic conditions at the current stage. To address this limitation, this study proposes a hybrid platooning pattern that conditionally permits non-connected vehicles to join platoons, thereby enhancing platooning diversity and flexibility. However, it was found that the unregulated integration of non-connected vehicles can trigger rapid platoon expansion, significantly amplifying the risk of disturbance propagation in traffic flow. This, in turn, exacerbates the inherent conflict between traffic throughput and stability. To mitigate these challenges, this paper further develops a hybrid platooning control strategy based on deep reinforcement learning (DRL). This strategy integrates vehicle dynamics, platoon topology, and traffic flow states through a multi-level state representation network, enabling a dynamic trade-off between traffic capacity and stability. Numerical simulations demonstrate that the proposed strategy effectively suppresses velocity disturbance propagation by dynamically optimizing platoon structures, thereby significantly enhancing the stability and safety of mixed traffic while reducing fuel consumption and emissions.

Zhen Qina, Dong-Fan Xie, Heng Ma +2

aSchool of Systems Science, Beijing Jiaotong University, Beijing 100044, China · bFaculty of Science and Engineering, University of Nottingham Ningbo China

Reinforcement Learning Intelligent Transportation Systems

Ask PDF

Jun 12, 2026

Safe Reinforcement Learning of Autonomous Highway Driving: A Unified Framework for Safety and Efficiency

Deep reinforcement learning (DRL) offers a compelling route to decision-making for advanced autonomous vehicles (AVs), yet its trial-and-error nature makes it difficult to guarantee safety during training and to achieve both safety and efficiency at deployment. We propose a unified safe reinforcement learning (SRL) framework that integrates safe distance (SD), reward machines (RM), and mixture-of-experts (MoE), termed MoE-RM-SRL. For deployment, SD and RM jointly shape a rule-aware reward that encodes highway traffic regulations and stage-wise objectives, enabling safe and reliable behavior without sacrificing efficiency. For training, we introduce a sparsely gated MoE layer comprising up to 11 deep Q-networks (DQNs); an SD-based gating rule activates a minimal set of experts for lane-keeping and lane-changing, mitigating the instability, discontinuities, and impulsive transients commonly induced by switching between heterogeneous controllers (e.g., MPC/rule-based modules and learned policies). We implement the proposed architecture in CARLA and integrate it with a 6-DoF driver-in-the-loop virtual-reality (DiL-VR) platform. Experiments in stochastic two-lane traffic show that MoE-RM-SRL substantially improves safety and efficiency over state-of-the-art baselines, and the framework naturally extends to multi-lane driving as well as on-ramp merging and exiting scenarios.

Chufei Yan, Zhihao Cui, Yiyan Lv +3

School of Physics, Northeast Normal University, Changchun, 130024, China. · 2Clean Energy Automotive Engineering Center, School of Automotive Studies, Tongji University, Shanghai, 201804, China. · 3Mengshi Automobile Technology Company, Dongfeng Motor Corporation, Wuhan, 430010, China.

Reinforcement Learning Mixture of Experts Autonomous Driving RL for Autonomous Driving Safe RL

Ask PDF