Decentralized MARL

MARL: Multi-Agent Reinforcement Learning

Decentralized multi-agent reinforcement learning (MARL) studies how agents can coordinate when each acts on local observations rather than relying on a central controller with complete system information. Common approaches include independent policy or value learning, centralized training with decentralized execution, parameter sharing, graph-based message passing, and distillation of centralized experts; theory also uses potential games, consensus, and regret analysis to establish learning guarantees. Work spans benchmarks such as SMAC and MPE as well as robotics, traffic control, energy networks, and wireless resource allocation, with increasing attention to safety, changing team sizes, and limited communication. A central tension is that local policies can scale and tolerate failures, yet coordination, credit assignment, and global safety are difficult without richer communication or centralized training—and those supports can undermine the autonomy decentralization is meant to provide.

Latest papers

122 papers16 papers in the last four weeks, up 14% on the four weeks before. 0.1% of all new papers.

Hierarchical Message-Passing Policies for Multi-Agent Reinforcement Learning
Jul 31, 2025Tommaso Marzi, Cesare Alippi, Andrea CiniMulti-Agent Coordination Hierarchical RL
IDSIA USI-SUPSI, Università della Svizzera italiana, Lugano, Switzerland · Politecnico di Milano, Milan, Italy · EPFL, Lausanne, Switzerland
Decentralized Best-Response-Based Learning in Two-Player Zero-Sum Stochastic Games: A Finite-Sample Analysis
Sep 2, 2024Zaiwei Chen, Kaiqing Zhang, Eric Mazumdar +2Nash Equilibrium Game Theory
IE, Purdue University. · ECE, University of Maryland, College Park. · CMS, Caltech. +1

Jul 31, 2025J/K move · Enter open · S save

Hierarchical Message-Passing Policies for Multi-Agent Reinforcement Learning

Tommaso Marzi, Cesare Alippi, Andrea Cini

IDSIA USI-SUPSI, Università della Svizzera italiana, Lugano, Switzerland · Politecnico di Milano, Milan, Italy · EPFL, Lausanne, Switzerland

Multi-Agent Coordination Hierarchical RL Decentralized MARL Multi-Agent Communication Multi-Agent Reinforcement Learning

Open paper AskPDF arXiv

Decentralized Multi-Agent Reinforcement Learning (MARL) methods allow for learning scalable multi-agent policies, but suffer from partial observability and induced non-stationarity. These challenges can be addressed by introducing mechanisms that facilitate coordination and high-level planning. Specifically, coordination and temporal abstraction can be achieved through communication (e.g., message passing) and Hierarchical Reinforcement Learning (HRL) approaches to decision-making. However, optimization issues limit the applicability of hierarchical policies to multi-agent systems. As such, the combination of these approaches has not been fully explored. To fill this void, we propose a novel and effective methodology for learning multi-agent hierarchies of message-passing policies. We adopt the feudal HRL framework and rely on a hierarchical graph structure for planning and coordination among agents. Agents at lower levels in the hierarchy receive goals from the upper levels and exchange messages with neighboring agents at the same level. To learn hierarchical multi-agent policies, we design a novel reward-assignment method based on training the lower-level policies to maximize the advantage function associated with the upper levels. Results on relevant benchmarks show that our method performs favorably compared to the state of the art.

Latest papers

Hierarchical Message-Passing Policies for Multi-Agent Reinforcement Learning

Decentralized Best-Response-Based Learning in Two-Player Zero-Sum Stochastic Games: A Finite-Sample Analysis