SIGN-MAPPO: State-Intent-Aware Graph Multi-Agent Reinforcement Learning for Cooperative Driving under Imperfect V2X

Zide Fan; Jiahui Yuan; Kaixin Zhang; Junhao Fan; Xiaohe Li; Yiru Wang

cs.ROOct 11, 2026

SIGN-MAPPO: State-Intent-Aware Graph Multi-Agent Reinforcement Learning for Cooperative Driving under Imperfect V2X

Authors: Zide Fan, Jiahui Yuan, Kaixin Zhang, Junhao Fan, Xiaohe Li, Yiru Wang

Organizations: Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100190, China · Key Laboratory of Target Cognition and Application Technology, Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing, China · University of Chinese Academy of Sciences and the School of Electronic, Electrical and Communication Engineering, University of Chinese Academy of Sciences

Abstract

Cooperative driving at mixed-traffic unsignalized intersections is challenged by imperfect vehicle-to-everything (V2X) communication, human-driven vehicle (HV) intentions that cannot be directly communicated, and the potential incompatibility of independently generated crossing decisions. This paper proposes State-Intent-Aware Graph Multi-Agent Reinforcement Learning for Cooperative Driving under Imperfect V2X (SIGN-MAPPO) to address these challenges through three coordinated mechanisms. Each decision-eligible connected and automated vehicle (CAV) first reconstructs a receiver-specific traffic graph from its ego state and the peer-state records available to it, from which a shared graph-attention policy generates a raw Stop/Go intention augmented with predicted HV-intent features. CAV intentions are then exchanged through timestamped vehicle-to-vehicle (V2V) messages, and a local conflict-aware arbitration layer converts each CAV's intention into a safe executable action despite potentially inconsistent peer information. In the principal Simulation of Urban MObility (SUMO) evaluation, SIGN-MAPPO achieves a 97.53% completion rate, a 0.10% collision rate, and a mean intersection traversal time of 35.28 s. Ablation studies further confirm the complementary contributions of HV-intent prediction and CAV-intent exchange to this performance. The trained policy also remains robust under the evaluated communication-delay and packet-loss ranges, and cross-scenario tests show complete traffic clearance across the tested settings, though safety margins narrow under compound unpredictable HV behavior.

Figures & tables

Fig. 1: Mixed-traffic unsignalized-intersection scenario, V2V and RSU-assisted information flows, and the changing set of decision-eligible CAVs.

Symbol	Definition
$t$ , $T$ , and $\Delta t$	Decision-step index, finite horizon, and decision interval
$i$ and $j$	Index of a decision-eligible CAV $i\in\mathcal{I}_{t}$ and index of any vehicle $j$ represented in the traffic state, respectively
$R_{\mathrm{ctrl}}$	Longitudinal length of the control region
$\mathcal{I}$ and $\mathcal{I}_{t}$	CAVs considered in an episode and decision-eligible lane-head CAVs at step $t$
$\mathcal{S}$ and $s_{t}$	Complete traffic-state space and its realized state at step $t$
$\mathcal{P}$ and $\mathcal{R}$	State-transition function and reward function

TABLE I: Principal Notation Used in the Problem Formulation

Component	Activation and effect
Passage	Positive when a CAV exits the conflict area; decreases as its accumulated waiting time increases
Progress	Positive when a CAV moves toward the stop line and negative when it moves away; the magnitude is bounded
Delay	Nonpositive before conflict-area entry; increases in magnitude with elapsed time, near-stop blocking, excessive waiting, new stops, and low-speed motion
Yielding	Positive when a CAV yields to a longer conflicting queue; includes an adjustment for the kinematic risk of conflicting HVs
Arbitration	Positive when a raw Go intention is retained and negative when it is changed to Stop
Risk	Negative when a raw Go intention violates the waiting condition or temporally overlaps a conflicting route

TABLE II: Environmental Reward Components

Fig. 2: Online decision process of SIGN-MAPPO: (a) receiver-specific state reconstruction and graph construction, (b) HV-intent prediction and graph-attention policy, and (c) timestamped CAV-intent exchange and local conflict-aware arbitration.

Fig. 3: Shared-parameter multi-agent PPO training and receiver-specific execution in SIGN-MAPPO.

1	Input: shared actor parameters $\theta$ , critic parameters $\phi$ , predictor parameters $\psi$ , and rollout horizon $T$ .
2	for each training iteration do
3	Reset the traffic environment and receiver-side communication caches.
4	for $t=0,\ldots,T-1$ do
5	Determine the decision-eligible CAV set $\mathcal{I}_{t}$ .
6	for each $i\in\mathcal{I}_{t}$ do
7	Reconstruct the traffic-state observation $\bm{o}_{i,t}$ .

Table 6

Item	Setting
Intersection	Four-leg unsignalized intersection
Inbound lanes	One left-turn and two through lanes per approach
Control-region length	55 m upstream of each stop line
Simulation step / episode limit	0.1 s / 6000 steps
Main demand	90 arriving vehicles
CAV penetration	50%

TABLE III: Principal Simulation and Evaluation Settings

Parameter	Value
GAT layers / heads / hidden size	2 / 4 / 64
Base / HV-intent feature size	10 / 6
Maximum HV history / GRU hidden size	$30\times 10$ / 64
Actor / critic learning rate	$3\times 10^{-4}$ / $10^{-3}$
Discount $\gamma$ / GAE $\lambda$	0.99 / 0.85
PPO clip / epochs / minibatch	0.20 / 4 / 128

TABLE IV: Principal Model and Training Settings

Fig. 4: Training dynamics of SIGN-MAPPO over 300 episodes. Panels (a)–(c) show episode-level values and their 20-episode moving averages (MA20), while panel (d) reports the MA20 values of the HV-intent prediction and policy- execution indicators.

Method	Comp.	Coll.	Trav.
	(%) $\uparrow$	(%) $\downarrow$	(s) $\downarrow$
Mixed-Traffic Controller [ 7 ]	92.56	0.00	52.89
Conflict-TTC [ 9 , 39 ]	100.00	0.20	97.47
FCFS-Priority [ 8 ]	92.30	1.17	58.18
MLP-MAPPO [ 20 ]	74.64	3.89	31.13
Attn-MAPPO [ 6 ]	73.04	6.68	30.00

TABLE V: Main Comparison Over 45 Test Episodes

Fig. 5: CAV-intent exchange ablation. Error bars denote 95% CIs.

Fig. 6: HV-intent prediction ablation under the hesitant-HV stress condition. Error bars denote 95% CIs.

Metric	Value
Behavior and movement/aggressive outputs ( $n=1{,}954$ )
Overall accuracy (%)	89.87
Balanced accuracy (%)	79.26
Aggressive-class recall (%)	66.47
Non-aggressive-class recall (%)	92.05
Movement-intent accuracy (%)	98.62

TABLE VI: Held-Out Performance of the HV-Intent Predictor

Fig. 7: State-channel robustness under delay (left column) and packet loss (right column). Points and error bars denote standardized paired mean changes and 95% CIs relative to the corresponding zero-impairment reference.

Fig. 8: Response to additional CAV-intent delay (left column) and packet loss (right column) under fixed nonideal state sharing. Points and error bars denote standardized paired mean changes and 95% CIs relative to the corresponding reference with the same state-channel setting.

Case	Comp.	Coll.	Trav.	Wait
	(%) $\uparrow$	(%) $\downarrow$	(s) $\downarrow$	(s) $\downarrow$
G0 Reference	100.00	0.000	21.92	4.06
G1 Two lanes	100.00	0.267	22.01	3.93
G2 Six lanes	100.00	0.087	18.05	4.43
G3 Low demand	100.00	0.185	15.19	1.75
G4 High demand	100.00	0.099	44.70	12.59

TABLE VII: Fixed-Policy Cross-Scenario Results

Fig. 9: Fixed-policy performance in the nominal high-demand scenario G4 and the compound HV-behavior stress scenario G9. Error bars denote 95% CIs.

Multi-Agent Coordination

Multi-agent coordination studies how autonomous agents align actions, share information, and divide work to achieve individual or collective goals.

See all 516 papers

IntNet: A Communication-Driven Multi-Agent Reinforcement Learning Framework for Cooperative Autonomous Driving
Oct 25, 2026Leandro Parada, Kevin Yu, Panagiotis AngeloudisRL for Autonomous Driving Cooperative MARL
L. Parada, K. Yu, and P. Angeloudis are with the Centre for Transport Studies, Department of Civil and Environmental Engineering, Imperial College London, UK
Compact Latent Coordination for Autonomous Vehicles at Unsignalized Intersections
Jul 23, 2026Gil Lifshits, Igal Bilik, Gilad KatzMulti-Agent Coordination Hierarchical RL
Ben-Gurion University of the Negev
V2X-WAM: A Cooperative World Action Model for End-to-End Autonomous Driving
Sep 29, 2026Junwei You, Weizhe Tang, Can Wang +6End-to-End Autonomous Driving Action-Conditioned World Models
ITS Center, Research Institute of Highway Ministry of Transport, Beijing, 100088, China · State Key Lab of Intelligent Transportation System, Research Institute of Highway Ministry of Transport, Beijing, 100088, China · Department of Civil and Environmental Engineering, University of Wisconsin–Madison, Madison, WI, 53706, USA +4

Oct 25, 2026J/K move · Enter open · S save

IntNet: A Communication-Driven Multi-Agent Reinforcement Learning Framework for Cooperative Autonomous Driving

Leandro Parada, Kevin Yu, Panagiotis Angeloudis

L. Parada, K. Yu, and P. Angeloudis are with the Centre for Transport Studies, Department of Civil and Environmental Engineering, Imperial College London, UK

RL for Autonomous Driving Cooperative MARL Sequential MARL Autonomous Driving Safety Evaluation

Open paper AskPDF arXiv

Achieving safety in autonomous driving through Multi-Agent Reinforcement Learning (MARL) is a critical yet challenging task due to non-stationarity, partial observability, and the need for effective coordination among agents. Although earlier cooperative MARL methods have aimed to improve coordination by sharing encoded state observations, we find these strategies are insufficient in safety-critical scenarios. To address this gap, we present IntNet, a novel MARL framework that enhances safety by incorporating the transmission of vehicle intent and adaptive communication scheduling into a unified end-to-end learning paradigm, jointly optimising all components for safe coordination. Central to our approach is an observation prediction module that enables agents to forecast subsequent policy outputs, which they can then share across a vehicle-to-vehicle network. Our model-free architecture employs a Graph Attention Network to process incoming messages and a scheduler component to dynamically optimise the communication graph for bandwidth efficiency. We compare IntNet against state-of-the-art communicative MARL methods in complex urban environments with both autonomous and human-operated vehicles, achieving improved coordination, lower collision rates, and reducing communication efforts by up to 60%. Through extensive experiments, we assess the framework's learning efficiency, scalability, and the balance between information sharing and bandwidth usage for collision-free trajectories.

Abstract

Multi-Agent Coordination

Explore similar work

IntNet: A Communication-Driven Multi-Agent Reinforcement Learning Framework for Cooperative Autonomous Driving

Compact Latent Coordination for Autonomous Vehicles at Unsignalized Intersections

V2X-WAM: A Cooperative World Action Model for End-to-End Autonomous Driving