Multi-Agent Reinforcement Learning

Also known as MARL

Multi-agent reinforcement learning studies how multiple decision-makers learn when each agent’s actions affect both the environment and other agents, across cooperative, competitive, and mixed-motive settings. Common approaches include centralized training with decentralized execution, independent or shared actor-critic policies, value factorization such as QMIX, self-play and game-solving methods, and graph-based communication or hierarchical policies. Researchers test these methods in benchmarks such as SMAC, MPE, Overcooked, and multi-agent MuJoCo, and apply them to robotics, traffic, wireless networks, energy systems, and language-agent workflows. A central challenge is scaling coordination and credit assignment while keeping learning stable under non-stationarity and policies robust, safe, and fair with unfamiliar partners and conditions.

Latest papers

373 papers47 papers in the last four weeks, up 81% on the four weeks before. 0.3% of all new papers.

CANTANTE: Optimizing Agentic Systems via Contrastive Credit Attribution
May 13, 2026Tom ZehleMulti-Agent LLM Systems Multi-Agent System Optimization
University of Freiburg · ELLIS Institute Tübingen
Finding the Weakest Link: Adversarial Attack against Multi-Agent Communications
May 13, 2026Maxwell Standen, Junae Kim, Claudia SzaboAdversarial Attacks Adversarial Perturbations
The University of Adelaide · DST Group
Offline Two-Player Zero-Sum Markov Games with KL Regularization
May 13, 2026Claire Chen, Yuheng Zhang, Xinyu Liu +3Nash Equilibrium Computation Reinforcement Learning
1California Institute of Technology. · University of Illinois Urbana-Champaign. · University of Virginia. +2
ChipMATE: Multi-Agent Training via Reinforcement Learning for Enhanced RTL Generation
May 13, 2026Zhongkai Yu, Yichen Lin, Chenyang Zhou +12RL for Code Generation Code Generation
UCSD · Columbia University
Events as Triggers for Behavioral Diversity in Multi-Agent Reinforcement Learning
May 12, 2026Hannes Büchi, Manon Flageat, Eduardo Sebastián +1Behavioral Diversity in RL Cooperative MARL
Department of Computer Science and Technology, University of Cambridge, UK
Metric-Gradient Projection for Stable Multi-Agent Policy Learning
May 12, 2026Zuyuan Zhang, Sizhe Tang, Mahdi Imani +1Hodge Decomposition Multi-Agent Systems
The George Washington University · Northeastern University
Information and Contract Design for Repeated Interactions between Agents with Misaligned Incentives
May 11, 2026Nanda Kishore Sreenivas, Kate LarsonImperfect-Information Games Multi-Agent Communication
University of Waterloo
DelAC: A Multi-agent Reinforcement Learning of Team-Symmetric Stochastic Games
May 11, 2026Duan-Shin Lee, Yu-Hsiu HungNash Equilibrium Game Theory
Department of Computer Science, National Tsing Hua University · MediaTek Inc.
PC3D: Zero-Shot Cooperation Across Variable Rosters via Personalized Context Distillation
May 11, 2026Ahmet Onur Akman, Rafał KucharskiZero-Shot Learning Decentralized MARL
Doctoral School of Exact and Natural Sciences, Jagiellonian University, Kraków, Poland · Faculty of Mathematics and Computer Science, Jagiellonian University, Kraków, Poland
Towards Autonomous Railway Operations: A Semi-Hierarchical Deep Reinforcement Learning Approach to the Vehicle Rescheduling Problem
May 11, 2026Alberto Castagna, Stefan Zahlner, Adrian Egli +4Multi-Agent Coordination Intelligent Transportation Systems
∗enliteAI, AT · †SBB CFF FFS, CH · ‡Flatland Association, CH
Rethinking Ratio-Based Trust Regions for Policy Optimization in Multi-Agent Reinforcement Learning
May 9, 2026Chulabhaya Wijesundara, Andrea Baisero, Zhongheng Li +3Multi-Agent System Optimization Policy Optimization
Northeastern University · STR · University of California, Irvine
Beyond Self-Play: Hierarchical Reasoning for Continuous Motion in Closed-Loop Traffic Simulation
May 9, 2026Weifan Zhang, Xiaofeng Zhao, Adel Bazzi +3Human Behavior Simulation Hierarchical RL
Isuzu Technical Center of America, Plymouth, MI, USA · School of Aeronautics and Astronautics, Purdue University, West Lafayette, IN 47907, USA
SceneFactory: GPU-Accelerated Multi-Agent Driving Simulation with Physics-Based Vehicle Dynamics
May 8, 2026Yicheng Zhu, Yang Chen, Tao Li +1GPU Acceleration Vehicle Dynamics
Rochester Institute of Technology · City University of Hong Kong
Experience Sharing in Mutual Reinforcement Learning for Heterogeneous Language Models
May 8, 2026Xiaoze Liu, Dhananjay Ram, Yuting Zhang +3RL for Language Models Reinforcement Learning
1Purdue University · *Work done during an internship at Amazon. · 2AWS Agentic AI
Rethinking Priority Scheduling for Sequential Multi-Agent Decision Making in Stackelberg Games
May 8, 2026Xiangyu Liu, Liang Zhang, Bo Jin +1Sequential Decision Making Hierarchical RL
Dalian University of Technology
Decentralized Diffusion Policy Learning for Enhanced Exploration in Cooperative Multi-agent Reinforcement Learning
May 8, 2026Yuyang Zhang, Haldun Balim, Na LiDiffusion Policy Decentralized MARL
SEAS, Harvard University
Randomness is sometimes necessary for coordination
May 7, 2026Rohan Patil, Jai Malegaonkar, Henrik I. ChristensenMulti-Agent Coordination Multi-Agent Systems
Cross-Modal Navigation with Multi-Agent Reinforcement Learning
May 7, 2026Shuo Liu, Xinzichen Li, Christopher AmatoCross-Modal Learning Multi-Agent Collaboration
Khoury College of Computer Sciences Northeastern University Boston, MA, 02120
Graph-SND: Sparse Aggregation for Behavioral Diversity in Multi-Agent Reinforcement Learning
May 6, 2026Shawn RayMulti-Agent Reinforcement Learning Behavioral Diversity in RL
Carnegie Mellon University
Strat-Reasoner: Reinforcing Strategic Reasoning of LLMs in Multi-Agent Games
May 6, 2026Yidong He, Yutao Lai, Pengxu Yang +4Game-Playing Agents RL for Language Model Reasoning
School of Software Engineering, South China University of Technology · Department of Computer Science, University of Oxford
Tree-based Credit Assignment for Multi-Agent Memory System
May 6, 2026Marina Mao, Alexandr Liu, Pengbo Li +3Credit Assignment Long-Horizon LLM Agents
University of Science and Technology of China · 2LLM Department, Tencent · 3The Hong Kong University of Science and Technology +1
Hierarchical Multiagent Reinforcement Learning for Multi-Group Tax Game
May 6, 2026Honglei Guo, Yexin Li, Chiyuan Wang +1Game Theory Hierarchical RL
College of Computer Science and Technology Zhejiang University State Key Laboratory of General Artificial Intelligence, BIGAI
Structural Equivalence and Learning Dynamics in Delayed MARL
May 5, 2026Jules Sintes, Ana Bušić, Jiamin ZhuPartially Observable Markov Decision Processes Zero-Shot Policy Transfer
Inria · DIENS, École Normale Supérieure, PSL University · Paris, France +2
MARS-DA: A Hierarchical Reinforcement Learning Framework for Risk-Aware Multi-Agent Bidding in Power Grids
May 4, 2026Jiayi Chen, Xuan Zhang, Guiling WangHierarchical RL Risk-Sensitive RL
Department of Computer Science, New Jersey Institute of Technology
Taming the Curses of Multiagency in Robust Markov Games with Large State Space through Linear Function Approximation
May 4, 2026Jingchu Gai, Laixi ShiMulti-Agent System Optimization Markov Models
Machine Learning Department, Carnegie Mellon University. · Department of Electrical and Computer Engineering, Johns Hopkins University, MD 21286, USA.
A Structural Threshold in Decision Capacity Governs Collapse in Self-Play Reinforcement Learning
May 4, 2026Arahan KujurGame-Playing Agents Self-Play RL
Independent Researcher
Experience Constrained Hierarchical Federated Reinforcement Learning for Large-scale UAV Teams in Hazardous Environments
May 4, 2026Qinwei Huang, Rui Zuo, Simon Khan +1Hierarchical Federated Learning Federated RL
EECS Syracuse University Syracuse, USA · AFRL Air Force Research Laboratory Rome, USA
Coopetition-Gym v1: A Formally Grounded Platform for Mixed-Motive Multi-Agent Reinforcement Learning under Strategic Coopetition
May 3, 2026Vik Pant, Eric YuRL Benchmarks Game Theory
Faculty of Information University of Toronto · Faculty of Information and Department of Computer Science University of Toronto
MAGIC: Multi-Step Advantage-Gated Causal Influence for Multi-agent Reinforcement Learning
May 3, 2026Haohan Yu, Jinmiao Cong, Shengzhi Wang +2Multi-Agent Coordination Intrinsic Reward Methods
1Dalian University of Technology · 2Microsoft, Beijing, China
CoFlow: Coordinated Few-Step Flow for Offline Multi-Agent Decision Making
May 2, 2026Guowei Zou, Haitao Wang, Beiwen Zhang +2Flow Matching Multi-Agent Coordination
Sun Yat-sen University

May 13, 2026J/K move · Enter open · S save

CANTANTE: Optimizing Agentic Systems via Contrastive Credit Attribution

Tom Zehle

University of Freiburg · ELLIS Institute Tübingen

Multi-Agent LLM Systems Multi-Agent System Optimization Large Language Model-Guided Optimization LLM Agent Workflow Optimization Prompt Optimization Multi-Agent Reinforcement Learning

Open paper AskPDF arXiv

LLM-based multi-agent systems have demonstrated strong performance across complex real-world tasks, such as software engineering, predictive modeling, and retrieval-augmented generation. Yet automating their configuration remains a structural challenge, as scores are available only at the system level, whereas the parameters governing agent behavior are local. We argue that optimizing these systems is fundamentally a credit-assignment problem. We therefore introduce CANTANTE, a framework that decomposes system-level rewards into per-agent update signals by contrasting rollouts of multiple joint configurations on the same query. We instantiate it for prompt optimization, treating agent prompts as learnable system parameters. We evaluate CANTANTE against GEPA and MIPROv2 on programming (MBPP), mathematical reasoning (GSM8K), and multi-hop question answering (HotpotQA). Across these benchmarks, CANTANTE achieves the best average rank among all evaluated optimizers and consistently outperforms unoptimized prompts. It improves over the strongest baseline by +18.9 percentage points on MBPP and +12.5 percentage points on GSM8K, while incurring a lower inference cost. It remains within one standard deviation of the strongest baseline on HotpotQA. Crucially, our credit correlation analysis confirms that the attributer produces meaningful per-agent signals rather than echoing the global system score.