Equivariant Multi-agent Reinforcement Learning for Multimodal Vehicle-to-Infrastructure Systems

Charbel Bou Chaaya; Mehdi Bennis

cs.LGApr 8, 2026

Equivariant Multi-agent Reinforcement Learning for Multimodal Vehicle-to-Infrastructure Systems

Authors: Charbel Bou Chaaya, Mehdi Bennis

Organizations: The authors are with the Centre for Wireless Communications, University of Oulu, Finland

Abstract

In this paper, we study a vehicle-to-infrastructure (V2I) system where distributed base stations (BSs) acting as road-side units (RSUs) collect multimodal (wireless and visual) data from moving vehicles. We consider a decentralized rate maximization problem, where each RSU relies on its local observations to optimize its resources, while all RSUs must collaborate to guarantee favorable network performance. We recast this problem as a distributed multi-agent reinforcement learning (MARL) problem, by incorporating rotation symmetries in terms of vehicles' locations. To exploit these symmetries, we propose a novel self-supervised learning framework where each BS agent aligns the latent features of its multimodal observation to extract the positions of the vehicles in its local region. Equipped with this sensing data at each RSU, we train an equivariant policy network using a graph neural network (GNN) with message passing layers, such that each agent computes its policy locally, while all agents coordinate their policies via a signaling scheme that overcomes partial observability and guarantees the equivariance of the global policy. We present numerical results carried out in a simulation environment, where ray-tracing and computer graphics are used to collect wireless and visual data. Results show the generalizability of our self-supervised and multimodal sensing approach, achieving more than two-fold accuracy gains over baselines, and the efficiency of our equivariant MARL training, attaining more than 50% performance gains over standard approaches.

Figures & tables

Figure 1: System model showcasing multiple BS agents, each equipped with a camera, deployed in a symmetric V2I environment. Each agent aligns latent features of its multimodal observations to retrieve the locations of the vehicles in its local region, which are used to communicate with its neighbors and train an equivariant MARL policy.

Symbol	Definition	Symbol	Definition
$\mathcal{A},\mathcal{K}$	Sets of BS - RSU agents and users	$a,k$	Agent and user index
$\mathcal{K}_{a}$	Set of users served by agent $a$	$\beta,N$	Bandwidth and number of antennas
$\mathcal{B}_{a}$	Set of beams / actions of agent $a$	$\mathbf{b}_{a}$	Beam / action of agent $a$
$\mathbf{h}_{a,k}$	Wireless channel between BS $a$ and user $k$	$\psi_{k},\varrho_{k}$	Downlink SINR and rate of user $k$
$\mathcal{C}_{a},c$	Set of cameras of agent $a$ and camera index	$\mathbf{i}_{a,c}$	Image taken by camera $c$ of agent $a$
$h_{c}$	Camera height	$\varphi^{\text{LoS}}_{c},\phi^{\text{LoS}}_{c}$	Camera line-of-sight viewing angles

Table I: Definition of symbols used throughout this paper

Figure 2: A global symmetry: when the vehicles’ positions rotate, the optimal policy is permuted between and within agents.

Panel of Figure 3 — Figure 3: Two equivalent states: a global state transformation is equivalent to a rotation of local states, followed by a permutation of local states and communication graph edges.

Figure 4: Proposed self-supervised multimodal sensing framework.

Panel of Figure 5 — Figure 5: Proposed image-based localization.

Figure 6: Equivariant layer: when the input is transformed by a group action, the output permutes over the group channels.

Figure 7: Proposed GNN for MARL equivariant policy training. Each agent collects multimodal observations which are processed as detailed in section IV and algorithm 1 to extract its users’ locations which serve as the GNN input. Each agent encodes its state estimate, which is used along an emergent signaling scheme over the GNN ’s message passing layers to update its state embedding, and finally compute its policy. All networks are implemented as equivariant layers followed by point-wise nonlinearities.

Figure 8: Simulation setting showing the top view of the considered V2I network. A Blender scene is processed by Sionna to render wireless CSI using ray tracing.

		Latent Space Metrics			Localization error [m]
Method	Figure	CT $\left(\uparrow\right)$	TW $\left(\uparrow\right)$	KS $\left(\downarrow\right)$	Mean $\left(\downarrow\right)$	95 th percentile $\left(\downarrow\right)$
Supervised	—	0.999935	0.999939	0.013794	0.368543	0.978475
Baseline	Fig. 9b	0.993567	0.995233	0.118514	3.916458	9.258683
Proposed	Fig. 9c	0.999407	0.999515	0.032431	1.442052	3.366604
Proposed (Partial)	Fig. 10	0.996655	0.996944	0.073794	2.092158	5.983181

Table II: Latent space and localization metrics for different methods

Panel of Figure 9 — Figure 9: Comparison between different sensing frameworks.

Figure 10: Localization performance of the proposed method with partial alignment.

Panel of Figure 18 — Figure 18: Convergence of MARL algorithms for different system parameters affecting the state and action sets.

Figure 19: Impact of different MARL training schemes using our proposed equivariant policy network and a standard non-equivariant network

\left(\left\lvert\mathcal{B}_{a}\right\rvert=64,\;\;\left\lvert\mathcal{K}_{a}\right\rvert=4\right)

Panel of Figure 20 — Figure 20: Comparison between different MARL algorithms under partial symmetry ( $\left\lvert\mathcal{B}_{a}\right\rvert=256,\;\;\left\lvert\mathcal{K}_{a}\right\rvert=4$ ).

Figure 21: Impact of localization error on the performance of different MARL algorithms (

\left\lvert\mathcal{B}_{a}\right\rvert=256,\;\;\left\lvert\mathcal{K}_{a}\right\rvert=4

). We inject such errors by randomly flipping some of the rows of the optimized matching matrix. Note the different scaling on the two vertical axes.

Figure 22: Energy efficiency of our multimodal system design compared to a separate design.

	CSI model	Image model
Number of parameters	34.2 M	36.9 M
FLOPs	18.18 G	104.7 G

Table III: Computing costs for different modalities

Equivariant Policy Learning

See all 8 papers

Multi-Agent Reinforcement Learning for V2X Resource Allocation: Disentangling MARL Challenges Through Benchmarking
Feb 18, 2026Siyuan Wang, Lei Lei, Pranav Maheshwari +2RL Benchmarks Reinforcement Learning
College of Engineering, University of Guelph, Guelph, Ontario, Canada · College of Electrical Engineering and Computer Sciences, Ningbo University, Ningbo, 315211, China
Multi-Agent Reinforcement Learning for C-V2X RAT Selection
Jul 13, 2026Moritz Schaffenroth, Uwe Kölbel, Heike Lepke +2Wireless Communications V2X Communication
Department of Electrical Engineering, Media and Computer Science Ostbayerische Technische Hochschule Amberg-Weiden Amberg, Germany
Compact Latent Coordination for Autonomous Vehicles at Unsignalized Intersections
Jul 23, 2026Gil Lifshits, Igal Bilik, Gilad KatzMulti-Agent Coordination Hierarchical RL
Ben-Gurion University of the Negev

Feb 18, 2026J/K move · Enter open · S save

Multi-Agent Reinforcement Learning for V2X Resource Allocation: Disentangling MARL Challenges Through Benchmarking

Siyuan Wang, Lei Lei, Pranav Maheshwari, Sam Bellefeuille+1

College of Engineering, University of Guelph, Guelph, Ontario, Canada · College of Electrical Engineering and Computer Sciences, Ningbo University, Ningbo, 315211, China

RL Benchmarks Reinforcement Learning V2X Communication Wireless Resource Allocation Robust RL Generalization in RL+1

Open paper AskPDF arXiv

Radio resource allocation (RRA) is a critical function in cellular vehicle-to-everything (C-V2X) networks, where vehicles must share limited wireless resources to support safety-critical communications. Multi-agent reinforcement learning (MARL) has emerged as a promising approach for this problem. However, key MARL challenges, including non-stationarity, coordination difficulty, large action space, partial observability, and limited robustness and generalization, are often intertwined, making it difficult to assess their individual impact on performance in vehicular environments. Moreover, existing studies primarily focus on developing new algorithms, while systematic benchmarking and comparative analyses remain limited. To address this gap, we formulate C-V2X RRA as a hierarchy of multi-agent interference games that progressively introduce key MARL challenges. Based on this framework, we develop a suite of benchmark learning tasks and construct training and testing datasets from SUMO-generated highway traces with diverse vehicular topologies and interference conditions. Using the proposed benchmark, we evaluate representative MARL algorithms spanning value-based, actor-critic, Independent Learning (IL), and Centralized Training with Decentralized Execution (CTDE) paradigms. The results identify robustness and generalization across diverse vehicular topologies as the dominant challenge among those considered in this work, reducing average normalized return by up to 59 percentage points, and show that, on the most challenging task, the best actor-critic method outperforms the best value-based method by 42%. By revealing the relative strengths and limitations of different MARL paradigms and open-sourcing the code, datasets, and benchmark suite, this work provides a systematic and reproducible foundation for evaluating and advancing MARL algorithms in vehicular networks.

Abstract

Equivariant Policy Learning

Explore similar work

Multi-Agent Reinforcement Learning for V2X Resource Allocation: Disentangling MARL Challenges Through Benchmarking

Multi-Agent Reinforcement Learning for C-V2X RAT Selection

Compact Latent Coordination for Autonomous Vehicles at Unsignalized Intersections