IntNet: A Communication-Driven Multi-Agent Reinforcement Learning Framework for Cooperative Autonomous Driving

Leandro Parada; Kevin Yu; Panagiotis Angeloudis

cs.ROOct 25, 2026

IntNet: A Communication-Driven Multi-Agent Reinforcement Learning Framework for Cooperative Autonomous Driving

Authors: Leandro Parada, Kevin Yu, Panagiotis Angeloudis

Organizations: L. Parada, K. Yu, and P. Angeloudis are with the Centre for Transport Studies, Department of Civil and Environmental Engineering, Imperial College London, UK

Abstract

Achieving safety in autonomous driving through Multi-Agent Reinforcement Learning (MARL) is a critical yet challenging task due to non-stationarity, partial observability, and the need for effective coordination among agents. Although earlier cooperative MARL methods have aimed to improve coordination by sharing encoded state observations, we find these strategies are insufficient in safety-critical scenarios. To address this gap, we present IntNet, a novel MARL framework that enhances safety by incorporating the transmission of vehicle intent and adaptive communication scheduling into a unified end-to-end learning paradigm, jointly optimising all components for safe coordination. Central to our approach is an observation prediction module that enables agents to forecast subsequent policy outputs, which they can then share across a vehicle-to-vehicle network. Our model-free architecture employs a Graph Attention Network to process incoming messages and a scheduler component to dynamically optimise the communication graph for bandwidth efficiency. We compare IntNet against state-of-the-art communicative MARL methods in complex urban environments with both autonomous and human-operated vehicles, achieving improved coordination, lower collision rates, and reducing communication efforts by up to 60%. Through extensive experiments, we assess the framework's learning efficiency, scalability, and the balance between information sharing and bandwidth usage for collision-free trajectories.

Figures & tables

Fig. 1 : Illustration of our MARL framework in an urban traffic context, showcasing the strategic exchange of state information and vehicle intent, coordinated by a scheduler to optimise bandwidth and enable real-time decision-making for autonomous vehicles.

Fig. 2 : This figure depicts IntNet, our MARL framework for safe, efficient CAV navigation, highlighting the sharing of encoded observations and vehicle intentions. The scheduler optimises the communication graph, while the GAT module processes incoming messages with attention to prioritise information for decision-making.

Panel of Figure 3 — Fig. 3 : Scenarios used for the experiments.

Panel of Figure 4 — Fig. 4 : Team reward of the different methods during training.

Panel of Figure 5 — Fig. 5 : Success rate of the different methods during training.

Method	Highway	Intersection	Lane Merging
No Comm	$62.6\pm 3.6$	$65.3\pm 8.4$	$10.3\pm 6.4$
IC3Net	$59.5\pm 6.1$	$70.7\pm 6.3$	$10.5\pm 8.3$
CommNet	$64.1\pm 8.4$	$41.0\pm 8.2$	$26.3\pm 7.6$
MAGIC	$77.6\pm 8.1$	$75.4\pm 6.7$	$66.1\pm 10.1$
IntNet (Ours)	$97.1\pm 2.6$	$89.7\pm 5.9$	$93.2\pm 9.9$

TABLE I : Test arrival rate (%) of different Communicative MARL methods.

Panel of Figure 7 — Fig. 6 : (a) Intersection scenario with ten agents. (b) Effect of increasing the number of agents on the collision rate.

Agent Count	IntNet	IC3Net
2	$0.0\pm 0.12$	$0.0\pm 0.05$
4	$1.1\pm 1.5$	$2.2\pm 1.8$
6	$1.7\pm 1.6$	$8.6\pm 3.5$
8	$3.5\pm 4.5$	$12.3\pm 5.12$
10	$5.2\pm 5.1$	$15.4\pm 7.4$

TABLE II : Test collision rates (%) for IntNet and IC3Net across various agent counts.

	Graph Density	Success rate (%)
No Scheduler	$1.0\pm 0.0$	$86.3\pm 5.1$
Scheduler	$0.41\pm 0.4$	$88.2\pm 5.8$

TABLE III : Effect of the communication scheduler on graph density and success rate.

Panel of Figure 10 — Fig. 7 : Heatmaps of the communication graphs when entering and leaving the intersection.

Parameter	Highway	Intersection	Lane Merging
Lanes Count	4	4	3
N° Agents	4	4	3
N° HDVs	4	2	2
Max Steps	40	15	15
Speed Range	[20, 30]	[0, 9]	[20, 30]
Simulation Frequency	8	10	15

TABLE IV : Simulation Parameters for the different scenarios. 0 0 footnotetext: Reward components are listed in the order:

r_{\text{speed}},r_{\text{coll}},r_{\text{arr}},r_{\text{merge}}

. Values represent the weights assigned to each component in the respective scenarios.

Hyper-parameter	Value
batch size	200
enc_hid_size	128
gat_num_heads	4
gat_num_heads_out	1
gat_hid_size	32
gat_encoder_out_size	32

TABLE V : IntNet set of hyperparameters.

Fig. 8 : Average test success rate with varying prediction horizons

H

in the Lane Merging Scenario.

RL for Autonomous Driving

Reinforcement learning for autonomous driving trains vehicle policies through interaction to plan, control, and coordinate safely in traffic.

See all 120 papers

COIN: Collaborative Interaction-Aware Multi-Agent Reinforcement Learning for Self-Driving Systems
Mar 26, 2026Yifeng Zhang, Jieming Chen, Tingguang Zhou +4Centralized Training with Decentralized Execution Multi-Agent Coordination
Department of Mechanical Engineering, College of Design and Engineering, National University of Singapore, 21 Lower Kent Ridge Rd, 119077, Singapore · Department of Electrical and Electronic Engineering, The Hong Kong Polytechnic University, Hong Kong, 100872, China · School of Vehicle and Mobility, Tsinghua University, Beijing, 100084, China
TSC: Topology-Conditioned Stackelberg Coordination for Multi-Agent Reinforcement Learning in Interactive Driving
Feb 27, 2026Xiaotong Zhang, Gang Xiong, Yuanjing Wang +3Multi-Agent Coordination Decentralized MARL
State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China · School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing 100049, China · Department of Natural Sciences, University of Durham, Durham DH1 3LE, United Kingdom +4
Compact Latent Coordination for Autonomous Vehicles at Unsignalized Intersections
Jul 23, 2026Gil Lifshits, Igal Bilik, Gilad KatzMulti-Agent Coordination Hierarchical RL
Ben-Gurion University of the Negev

Mar 26, 2026J/K move · Enter open · S save

COIN: Collaborative Interaction-Aware Multi-Agent Reinforcement Learning for Self-Driving Systems

Yifeng Zhang, Jieming Chen, Tingguang Zhou, Tanishq Duhan+3

Department of Mechanical Engineering, College of Design and Engineering, National University of Singapore, 21 Lower Kent Ridge Rd, 119077, Singapore · Department of Electrical and Electronic Engineering, The Hong Kong Polytechnic University, Hong Kong, 100872, China · School of Vehicle and Mobility, Tsinghua University, Beijing, 100084, China

Centralized Training with Decentralized Execution Multi-Agent Coordination RL for Autonomous Driving Cooperative MARL Multi-Agent Reinforcement Learning

Open paper AskPDF arXiv

Multi-Agent Self-Driving (MASD) systems provide an effective solution for coordinating autonomous vehicles to reduce congestion and enhance both safety and operational efficiency in future intelligent transportation systems. Multi-Agent Reinforcement Learning (MARL) has emerged as a promising approach for developing advanced end-to-end MASD systems. However, achieving efficient and safe collaboration in dynamic MASD systems remains a significant challenge in dense scenarios with complex agent interactions. To address this challenge, we propose a novel collaborative(CO-) interaction-aware(-IN) MARL framework, named COIN. Specifically, we develop a new counterfactual individual-global twin delayed deep deterministic policy gradient (CIG-TD3) algorithm, crafted in a "centralized training, decentralized execution" (CTDE) manner, which aims to jointly optimize the individual objectives (navigation) and the global objectives (collaboration) of agents. We further introduce a dual-level interaction-aware centralized critic architecture that captures both local pairwise interactions and global system-level dependencies, enabling more accurate global value estimation and improved credit assignment for collaborative policy learning. We conduct extensive simulation experiments in dense urban traffic environments, which demonstrate that COIN consistently outperforms other advanced baseline methods in both safety and efficiency across various system sizes. These results highlight its superiority in complex and dynamic MASD scenarios, as further validated through real-world robot demonstrations. Supplementary videos are available at https://marmotlab.github.io/COIN/

Abstract

RL for Autonomous Driving

Explore similar work

COIN: Collaborative Interaction-Aware Multi-Agent Reinforcement Learning for Self-Driving Systems

TSC: Topology-Conditioned Stackelberg Coordination for Multi-Agent Reinforcement Learning in Interactive Driving

Compact Latent Coordination for Autonomous Vehicles at Unsignalized Intersections