Sequential MARL

MARL: Multi-Agent Reinforcement Learning

Sequential multi-agent reinforcement learning studies settings where agents act in turn or asynchronously, often with partial observations and limited communication. Approaches include temporal mean-field models, Stackelberg leader-follower policies, hierarchical controllers, and sequential policy-gradient or trust-region updates; counterfactual critics and world models help assign credit for team outcomes. These methods are tested in cooperative games, dense traffic, aerial networks, and competitive tasks, with attention to scaling and robustness. A central tension is how to coordinate and generalize across agents and action orders without relying on costly communication or centralized access to joint trajectories.

Latest papers

22 papers4 papers in the last four weeks, against 2 the four weeks before. 0.0% of all new papers.

IntNet: A Communication-Driven Multi-Agent Reinforcement Learning Framework for Cooperative Autonomous Driving
Oct 25, 2026Leandro Parada, Kevin Yu, Panagiotis AngeloudisRL for Autonomous Driving Cooperative MARL
L. Parada, K. Yu, and P. Angeloudis are with the Centre for Transport Studies, Department of Civil and Environmental Engineering, Imperial College London, UK
MARL-BattleGrounds: A Novel JAX-native Benchmark for Heterogeneous and Competitive Multi-agent RL
Oct 10, 2026Ulixes Tariq Hawili, Subramanian Ramamoorthy, Yoann Altmann +1RL Benchmarks Multi-Agent Reinforcement Learning
School of Engineering, University of Edinburgh, Edinburgh, United Kingdom · School of Informatics, University of Edinburgh, Edinburgh, United Kingdom · School of Engineering and Physical Sciences, Heriot-Watt University, Edinburgh, United Kingdom
Independent Multi-Agent Reinforcement Learning with Counterfactual Semantic-Social World Models
Oct 6, 2026Fernando Martinez, Tao Li, Yingdong Lu +1Sequential MARL Action-Conditioned World Models
Department of Computer and Information Sciences, Fordham University, New York, NY, 10023, USA · Department of Systems Engineering, City University of Hong Kong, Hong Kong SAR, 999077, China · IBM Research, Yorktown Heights, NY, 10598, USA
Emergence, Not Bandwidth: Physical Coupling and the Limits of Learned Multi-Agent Communication
Sep 28, 2026Mihir Chauhan, Aniket BeraSequential MARL Partially Observable Markov Decision Processes
IDEAS Lab, Department of Computer Science Purdue University
Turning Safety into Competence: Minimally Exploitable Robot Policies via Safety-Filtered Reinforcement Learning
Sep 23, 2026Ruihan Wu, Rui Yang, Donggeon David Oh +2Sequential MARL RL for Robotics
Department of Computer Science, Johns Hopkins University, Baltimore, MD 21218, USA · Department of Electrical and Computer Engineering, Princeton University, Princeton, NJ 08544, USA
Robust and Efficient Communication for Multi-Agent Learning
Sep 14, 2026Rafael Pina, Varuna De Silva, Corentin ArtaudSequential MARL Multi-Agent Communication
Institute for Digital Technologies · Loughborough University London, United Kingdom
Predictive Lightweight MARL for Resilient Coverage in Sparse-Signaling Aerial Networks
Jul 24, 2026Chuan-Chi Lai, Ang-Hsun TsaiSequential MARL Multi-Agent Control
Department of Communications Engineering, National Chung Cheng University, Minxiong Township, Chiayi County 621301, Taiwan · Advanced Institute of Manufacturing with High-tech Innovations (AIM-HI), National Chung Cheng University, Minxiong Township, Chiayi County 621301, Taiwan · Department of Communications Engineering, Feng Chia University, Taichung 407102, Taiwan
Agon: Competitive Cross-Model RL with Implicit Rival Grading of Reasoning
Jul 8, 2026Vladislav BeliaevSequential MARL RL for Language Model Reasoning
Independent Researcher
Low Variance Trust Region Optimization with Independent Actors and Sequential Updates in Cooperative Multi-agent Reinforcement Learning
Jun 24, 2026Bang Giang Le, Viet Cuong TaSequential MARL Trust Region Policy Optimization
Human Machine Interaction Laboratory, VNU University of Engineering and Technology, 144 Xuan Thuy, Cau Giay, 100000, Hanoi, Vietnam.
Hierarchical Control in Multi-Agent Games: LLM-based Planning and RL Execution
Jun 18, 2026Jannik Hösch, Alessandro Sestini, Florian Fuchs +6Sequential MARL Multi-Agent Coordination
1Electronic Arts (EA) · 2KTH Royal Institute of Technology, Stockholm, Sweden
Learning Multi-Agent Communication Protocol: Study on Information Entropy Efficiency in MARL
Jun 5, 2026Xinren Zhang, Zixin Zhong, Jiadong YuSequential MARL Agent Communication Protocols
The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China
Learning to Hand Off: Provably Convergent Workflow Learning under Interface Constraints
May 18, 2026Jiayu Li, Enpei Zhang, Dawei Zhou +2Sequential MARL Sequential Decision Making
Stern School of Business · New York University · Department of Computer Science +2
DLM: Unified Decision Language Models for Offline Multi-Agent Sequential Decision Making
Apr 26, 2026Zhuohui Zhang, Bin Cheng, Bin HeSequential MARL Offline RL
Department of Control Science and Engineering, Tongji University, Shanghai 201804, China · Shanghai Institute of Intelligent Science and Technology; Shanghai Research Institute for Intelligent Autonomous Systems; State Key Laboratory of Autonomous Intelligent Unmanned Systems, Shanghai 200120, China
COSAC: Counterfactual Credit Assignment in Sequential Cooperative Teams
Apr 20, 2026Shripad Deshmukh, Jayakumar Subramanian, Raghavendra Addanki +1Sequential MARL Credit Assignment in RL
College of Information and Computer Sciences University of Massachusetts Amherst · Adobe Research San Jose, CA
Investigating the Impact of Subgraph Social Structure Preference on the Strategic Behavior of Networked Mixed-Motive Learning Agents
Apr 4, 2026Xinqi Gao, Mario VentrescaMulti-Agent Systems Multi-Agent Reinforcement Learning
Xinqi Gao is with the Department of Industrial Engineering, Purdue University, West Lafayette, IN 47906, USA · Mario Ventresca is with the Department of Industrial Engineering, Purdue University, West Lafayette, IN 47906, USA
CoMaTrack: Competitive Multi-Agent Game-Theoretic Tracking with Vision-Language-Action Models
Mar 24, 2026Youzhi Liu, Li Gao, Liu Liu +2Game-Theoretic RL Reinforcement Learning
Amap, Alibaba Group
Spatio-Temporal Attention Enhanced Multi-Agent DRL for UAV-Assisted Wireless Networks with Limited Communications
Mar 23, 2026Che Chen, Lanhua Li, Shimin Gong +3Spatio-Temporal Attention Multi-Agent Reinforcement Learning
School of Intelligent Systems Engineering, Shenzhen Campus of Sun Yat-sen University, Shenzhen 518000, China, and Southern Marine Science and Engineering Guangdong Laboratory (Zhuhai), Zhuhai 519082, China · Department of Equipment Management and Unmanned Aerial Vehicle Engineering, Air Force Engineering University, Xi’an · School of Computing and Artificial Intelligence, Jiangxi University of Finance and Economics, Nanchang 330032, China +1
ImageEdit-R1: Boosting Multi-Agent Image Editing via Reinforcement Learning
Mar 9, 2026Yiran Zhao, Yaoqi Ye, Xiang Liu +2Image Editing RL for Image Editing
National University of Singapore · Adobe Research
Competitive Multi-Operator Reinforcement Learning for Joint Pricing and Fleet Rebalancing in AMoD Systems
Mar 5, 2026Emil Kragh Toft, Carolin Schmidt, Daniele Gammelli +1Dynamic Pricing Multi-Agent Reinforcement Learning
Technical University of Denmark, Denmark · Technical University of Munich, Germany · Stanford University, USA
TSC: Topology-Conditioned Stackelberg Coordination for Multi-Agent Reinforcement Learning in Interactive Driving
Feb 27, 2026Xiaotong Zhang, Gang Xiong, Yuanjing Wang +3Multi-Agent Coordination Decentralized MARL
State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China · School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing 100049, China · Department of Natural Sciences, University of Durham, Durham DH1 3LE, United Kingdom +4
Hierarchical Lead Critic based Multi-Agent Reinforcement Learning
Feb 25, 2026David Eckel, Henri MeeßMulti-Agent Reinforcement Learning Cooperative MARL
Fraunhofer Institute IVI, Ingolstadt, Germany
Mean-Field Reinforcement Learning without Synchrony
Feb 20, 2026Shan YangMean-Field MARL Reinforcement Learning
Department of Industrial Systems Engineering and Management, National University of Singapore, Singapore.

Oct 25, 2026J/K move · Enter open · S save

IntNet: A Communication-Driven Multi-Agent Reinforcement Learning Framework for Cooperative Autonomous Driving

Leandro Parada, Kevin Yu, Panagiotis Angeloudis

L. Parada, K. Yu, and P. Angeloudis are with the Centre for Transport Studies, Department of Civil and Environmental Engineering, Imperial College London, UK

RL for Autonomous Driving Cooperative MARL Sequential MARL Autonomous Driving Safety Evaluation

Open paper AskPDF arXiv

Achieving safety in autonomous driving through Multi-Agent Reinforcement Learning (MARL) is a critical yet challenging task due to non-stationarity, partial observability, and the need for effective coordination among agents. Although earlier cooperative MARL methods have aimed to improve coordination by sharing encoded state observations, we find these strategies are insufficient in safety-critical scenarios. To address this gap, we present IntNet, a novel MARL framework that enhances safety by incorporating the transmission of vehicle intent and adaptive communication scheduling into a unified end-to-end learning paradigm, jointly optimising all components for safe coordination. Central to our approach is an observation prediction module that enables agents to forecast subsequent policy outputs, which they can then share across a vehicle-to-vehicle network. Our model-free architecture employs a Graph Attention Network to process incoming messages and a scheduler component to dynamically optimise the communication graph for bandwidth efficiency. We compare IntNet against state-of-the-art communicative MARL methods in complex urban environments with both autonomous and human-operated vehicles, achieving improved coordination, lower collision rates, and reducing communication efforts by up to 60%. Through extensive experiments, we assess the framework's learning efficiency, scalability, and the balance between information sharing and bandwidth usage for collision-free trajectories.