Partial Attention in Deep Reinforcement Learning for Safe Multi-Agent Control

Turki Bin Mohaya; Peter Seiler

eess.SYMar 23, 2026

Partial Attention in Deep Reinforcement Learning for Safe Multi-Agent Control

Authors: Turki Bin Mohaya, Peter Seiler

Organizations: Department of Electrical Engineering and Computer Science, University of Michigan, Ann Arbor, MI 48109, USA.

Abstract

Attention mechanisms excel at learning sequential patterns by discriminating data based on relevance and importance. This provides state-of-the-art performance in advanced generative artificial intelligence models. This paper applies this concept of an attention mechanism for multi-agent safe control. We specifically consider the design of a neural network to control autonomous vehicles in a highway merging scenario. The environment is modeled as a Decentralized Partially Observable Markov Decision Process (Dec-POMDP). Within a QMIX framework, we include partial attention for each autonomous vehicle, thus allowing each ego vehicle to focus on the most relevant neighboring vehicles. Moreover, we propose a comprehensive reward signal that considers the global objectives of the environment (e.g., safety and vehicle flow) and the individual interests of each agent. Simulations are conducted in the Simulation of Urban Mobility (SUMO). The results show better performance compared to other driving algorithms in terms of safety, driving speed, and reward.

Figures & tables

Fig. 1: Left: The highway merging problem. Middle and Right: Our contribution is deploying partial attention to the most critical interactions for safe highway merging.

Fig. 2: Performance during the training phase.

Fig. 3: Comparison of the proposed method against SUMO IDM in the evaluation phase.

Parameter	Value
Episodes	$1000$
Maximum time steps per episode	$1000$
Optimizer	AdamW [ 13 ]
$B$ (batch size)	$256$
$\gamma$ (discount factor)	$0.99$
Learning rate	$0.0001$

Table I: Training Hyperparameters

Parameter	Value
Number of agents	$16$
Initial velocity for highway agents ( $\mathrm{m}/\mathrm{s}$ )	$\sim\text{Uniform}([7,10])$
Initial velocity merging agents ( $\mathrm{m}/\mathrm{s}$ )	$\sim\text{Uniform}([4,8])$
Departure time ( $\mathrm{s}$ )	$\sim\text{Uniform}([0,100])$
Route	$\sim\text{Uniform}\left(\{HW,M\}\right)$
Vehicle length ( $\mathrm{m}$ )	$5.0$

Table II: Vehicle Generation Parameters

Coefficient	$c_{1}$	$c_{2}$	$c_{3}$	$c_{4}$	$c_{5}$	$c_{6}$	$c_{7}$	$c_{8}$
Value	40	0.5	0.9	1.0	1.0	3.0	0.00001	0.01

Table III: Reward Coefficients

RL for Autonomous Driving

Reinforcement learning for autonomous driving trains vehicle policies through interaction to plan, control, and coordinate safely in traffic.

See all 120 papers

Fully Decentralized and Safety-Aware Multi-Agent Reinforcement Learning for Control on Networks
Sep 28, 2026Theodore Rogalski, Shirantha WelikalaMulti-Agent Control Decentralized MARL
Electrical & Computer Engineering Department Stevens Institute of Technology Hoboken, USA
Reward-Conditioned Attention: How Reward Design Shapes What Autonomous Driving Agents See
Jun 23, 2026Mohamed Benabdelouahad, Ahmed Djalal Hacini, Nadir Farhi +1Reward Shaping RL for Autonomous Driving
National School of Artificial Intelligence (ENSIA), Sidi Abdellah Campus, Algiers, Algeria. · Cosys-Grettia, Univ Gustave Eiffel, F-77454 Marne-la-Vallee, France.
Beyond Conservative Automated Driving in Multi-Agent Scenarios via Coupled Model Predictive Control and Deep Reinforcement Learning
Apr 15, 2026Saeed Rahmani, Gözde Körpe, Zhenlin +4Autonomous Driving Planning Multi-Agent Control
Gavin · TU Delft, Faculty of Civil Engineering and Geosciences, Department of Transport and Planning, Delft, The Netherlands · NVIDIA, San Jose, CA, USA

Sep 28, 2026J/K move · Enter open · S save

Fully Decentralized and Safety-Aware Multi-Agent Reinforcement Learning for Control on Networks

Theodore Rogalski, Shirantha Welikala

Electrical & Computer Engineering Department Stevens Institute of Technology Hoboken, USA

Multi-Agent Control Decentralized MARL Multi-Agent Reinforcement Learning

Open paper AskPDF arXiv

This paper develops a safe and fully decentralized multi-agent reinforcement learning (MARL) algorithm to solve a class of discrete-time control problems on networks, including the persistent monitoring problem. Fully decentralized control of agents, while offering numerous benefits, faces issues such as exponentially increasing sample complexity, lack of global information about the system, and challenges in coordinating between agents. To address these issues, this paper introduces a fully decentralized multi-agent reinforcement learning algorithm that integrates deep reinforcement learning with safety considerations. This method feeds a history of local observations of the network's state into two parallel neural-network branches: the graph encoder, which adds structural information and correlations among nodes, and a state estimator, which predicts the uncertainty at each node in the graph. Additionally, the result of feeding that input into an actor-critic network is passed through a discrete-time control barrier heuristic to reduce the likelihood that any node will be neglected. This approach enables teams of fully decentralized agents to solve challenging problems by increasing system awareness and incorporating built-in safety measures to prevent the adoption of potentially harmful control policies. Numerical results from a custom simulation environment demonstrate that the proposed algorithm achieves 26.3 percent lower average uncertainty than a centralized control policy and is within 1 percent of the uncertainty performance of a more computationally complex algorithm with added attention layers.

Abstract

RL for Autonomous Driving

Explore similar work

Fully Decentralized and Safety-Aware Multi-Agent Reinforcement Learning for Control on Networks

Reward-Conditioned Attention: How Reward Design Shapes What Autonomous Driving Agents See

Beyond Conservative Automated Driving in Multi-Agent Scenarios via Coupled Model Predictive Control and Deep Reinforcement Learning