Multi-Agent Reinforcement Learning

Also known as MARL

Multi-agent reinforcement learning studies how multiple decision-makers learn when each agent’s actions affect both the environment and other agents, across cooperative, competitive, and mixed-motive settings. Common approaches include centralized training with decentralized execution, independent or shared actor-critic policies, value factorization such as QMIX, self-play and game-solving methods, and graph-based communication or hierarchical policies. Researchers test these methods in benchmarks such as SMAC, MPE, Overcooked, and multi-agent MuJoCo, and apply them to robotics, traffic, wireless networks, energy systems, and language-agent workflows. A central challenge is scaling coordination and credit assignment while keeping learning stable under non-stationarity and policies robust, safe, and fair with unfamiliar partners and conditions.

Latest papers

371 papers47 papers in the last four weeks, up 81% on the four weeks before. 0.3% of all new papers.

Policy Gradient with Self-Attention for Model-Free Distributed Nonlinear Multi-Agent Games
Sep 22, 2025Eduardo Sebastián, Maitrayee Keskar, Eeman Iqbal +3Policy Gradient Self-Attention
Department of Computer Science and Technology, University of Cambridge, UK · Department of Electrical and Computer Engineering, University of California San Diego, USA · RoPeRt group, at DIIS - I3A, Universidad de Zaragoza, Spain
Distributionally Robust Markov Games with Average Reward
Aug 5, 2025Zachary Roch, Yue WangNash Equilibrium Computation Multi-Agent System Optimization
Department of Electrical and Computer Engineering, University of Central Florida, Orlando, Florida, USA. · Department of Computer Science, University of Central Florida, Orlando, Florida, USA.
Hierarchical Message-Passing Policies for Multi-Agent Reinforcement Learning
Jul 31, 2025Tommaso Marzi, Cesare Alippi, Andrea CiniMulti-Agent Coordination Hierarchical RL
IDSIA USI-SUPSI, Università della Svizzera italiana, Lugano, Switzerland · Politecnico di Milano, Milan, Italy · EPFL, Lausanne, Switzerland
Chasing Moving Targets with Online Self-Play Reinforcement Learning for Safer Language Models
Jun 9, 2025Mickel Liu, Liwei Jiang, Yancheng Liang +4Self-Play RL Adversarial Attacks on LLMs
Paul G. Allen School of Computer Science & Engineering, University of Washington, Seattle, WA, USA · Department of Computer Science, Stanford University, Stanford, CA, USA
Ensemble-MIX: Enhancing Sample-Efficiency in Multi-Agent RL via Selective Ensemble-Based Exploration
Jun 3, 2025Tom Danino, Nahum ShimkinMulti-Agent Reinforcement Learning Sample-Efficient RL
The Andrew and Erna Viterbi Faculty of Electrical & Computer Engineering, Technion- Israel Institute of Technology
Learning Large-Scale Competitive Team Behaviors with Mean-Field Interactions
Apr 29, 2025Bhavini Jeloka, Yue Guan, Panagiotis TsiotrasMean-Field MARL Multi-Agent Reinforcement Learning
School of Aerospace Engineering, Georgia Institute of Technology, Atlanta, GA, USA
MARFT: Multi-Agent Reinforcement Fine-Tuning
Apr 21, 2025Junwei Liao, Muning Wen, Jun Wang +1Multi-Agent LLM Systems RL Fine-Tuning
Shanghai Jiao Tong University · Shanghai Innovation Institute · OPPO Research Institute
SUB-PLAY: Adversarial Policies against Partially Observed Multi-Agent Reinforcement Learning Systems
Feb 6, 2024Oubo Ma, Yuwen Pu, Linkang Du +5Adversarial Attacks Partially Observable RL
Zhejiang University · Xi’an Jiaotong University · Laboratory for Big Data and Decision +2
JaxMARL: Multi-Agent RL Environments and Algorithms in JAX
Nov 16, 2023Alexander Rutherford, Benjamin Ellis, Matteo Gallici +18RL Benchmarks Reinforcement Learning
University of Oxford · Universitat Politècnica de Catalunya · University College London +2
Multi-Agent Reinforcement Learning-Based UAV Pathfinding for Obstacle Avoidance in Stochastic Environment
Oct 25, 2023Qizhen Wu, Kexin Liu, Lei Chen +1Mean-Field MARL Multi-Agent Path Finding
School of Automation Science and Electrical Engineering, Beihang University, Beijing 100191, China. · Advanced Research Institute of Multidisciplinary Sciences and State Key Laboratory of CNS/ATM, Beijing Institute of Technology, Beijing 100081, China.
Stackelberg POMDP: Learning to Lead via Reinforcement Learning
Oct 7, 2022Matthias Gerstgrasser, Gianluca Brero, Alon Eden +6Game-Theoretic RL Reinforcement Learning
John A. Paulson School of Engineering and Applied Sciences, Harvard University · Data Science Initiative, Brown University · School of Computer Science and Engineering, Hebrew University of Jerusalem +2

Sep 22, 2025J/K move · Enter open · S save

Policy Gradient with Self-Attention for Model-Free Distributed Nonlinear Multi-Agent Games

Eduardo Sebastián, Maitrayee Keskar, Eeman Iqbal, Eduardo Montijano+2

Department of Computer Science and Technology, University of Cambridge, UK · Department of Electrical and Computer Engineering, University of California San Diego, USA · RoPeRt group, at DIIS - I3A, Universidad de Zaragoza, Spain

Policy Gradient Self-Attention Multi-Agent Control Model-Free RL Multi-Agent Communication Multi-Agent Reinforcement Learning

Open paper AskPDF arXiv

Multi-agent games in dynamic nonlinear settings are challenging due to the time-varying interactions among the agents and the non-stationarity of the (potential) Nash equilibria. In this paper we consider model-free games, where agent transitions and costs are observed without knowledge of the transition and cost functions that generate them. We propose a novel distributed policy structure that follows the communication constraints in multi-team games, with multiple agents per team, and learned through policy gradients. Our formulation is inspired by the structure of distributed policies in linear quadratic games, which take the form of time-varying linear feedback gains. In the nonlinear case, we model the policies as nonlinear feedback gains, parameterized by self-attention layers to account for the time-varying multi-agent communication topology. We demonstrate that our approach achieves strong performance in several settings, including distributed linear and nonlinear regulation, and simulated and real multi-robot pursuit-and-evasion games.