Decentralized MARL

MARL: Multi-Agent Reinforcement Learning

Decentralized multi-agent reinforcement learning (MARL) studies how agents can coordinate when each acts on local observations rather than relying on a central controller with complete system information. Common approaches include independent policy or value learning, centralized training with decentralized execution, parameter sharing, graph-based message passing, and distillation of centralized experts; theory also uses potential games, consensus, and regret analysis to establish learning guarantees. Work spans benchmarks such as SMAC and MPE as well as robotics, traffic control, energy networks, and wireless resource allocation, with increasing attention to safety, changing team sizes, and limited communication. A central tension is that local policies can scale and tolerate failures, yet coordination, credit assignment, and global safety are difficult without richer communication or centralized training—and those supports can undermine the autonomy decentralization is meant to provide.

Latest papers

122 papers16 papers in the last four weeks, up 14% on the four weeks before. 0.1% of all new papers.

Belief-Conditioned Multi-Agent Actor Critic for Decentralized LLM Collaboration
Oct 26, 2026Tianle Chen, Shuo Liu, Christopher AmatoMulti-Agent LLM Systems Multi-Agent Reinforcement Learning
Khoury College of Computer Sciences, Northeastern University, Boston, MA, 02215, USA
HA-DiPO: Diffusion Policy Optimization for Heterogeneous Cooperative Transport
Oct 24, 2026Xuebin Li, Jinpeng Xiao, Zong Chen +4Multi-Robot Cargo Transportation Cooperative MARL
School of Mathematics, Harbin Institute of Technology, Harbin, China · State Key Laboratory of Intelligent Manufacturing Equipment and Technology, School of Mechanical Science and Engineering, Huazhong University of Science and Technology, Wuhan, China · Department of Electrical Engineering and Information Technology, University of Naples Federico II, Naples, Italy
Probabilistic Energy Modeling Unifies Versatile Swarm Control
Oct 22, 2026Xiang Zhou, Xiangyu Zhu, Xintong Wang +5Swarm Robotics Robotic Control
School of Aeronautics and Astronautics, Sun Yat-sen University, Shenzhen 518107, China.
Learning Self-Orchestration in Decentralized Agent Swarms
Oct 15, 2026Guowei Xu, Zhenting Qi, Amani Kiruga +2Multi-Agent Orchestration Multi-Agent Coordination
Harvard University
CALM: Decentralized Control of Cooperative Aerial Manipulation with Multi-Agent Reinforcement Learning
Oct 13, 2026Jiaxi Huo, Petre Ricioppo, Fausto Francesco LizzioAerial Manipulation Decentralized MARL
Department of Mechanical and Aerospace Engineering, Politecnico di Torino, Torino, Italy
Reinforcement Learning Based Heterogeneous Task Allocation Under Spatio-Temporal Environmental Uncertainty for Multi-Robot Teams
Oct 11, 2026Xi Lin, Yaxuan Li, Paul Szenher +1Multi-Robot Task Allocation Decentralized MARL
Department of Mechanical Engineering, Stevens Institute of Technology, 1 Castle Point Terrace, Hoboken, NJ 07030, USA
Before We Act: Learning Action-Conditioned Team-Response Beliefs for Decentralized Multi-Robot Manipulation
Oct 10, 2026Zihao Li, Yanan Zhou, Ranpeng Qiu +2Multi-Agent Coordination Multi-Agent Collaboration
The University of Sydney · Zeno AI
Multi-Agent Coordination via Support-Preserving Distillation
Oct 7, 2026Sangmin Lee, Youngju Na, Chanmi Lee +1Multi-Agent Coordination Offline MARL
School of Computing, KAIST
Independent Multi-Agent Reinforcement Learning with Counterfactual Semantic-Social World Models
Oct 6, 2026Fernando Martinez, Tao Li, Yingdong Lu +1Sequential MARL Action-Conditioned World Models
Department of Computer and Information Sciences, Fordham University, New York, NY, 10023, USA · Department of Systems Engineering, City University of Hong Kong, Hong Kong SAR, 999077, China · IBM Research, Yorktown Heights, NY, 10598, USA
Foundation Model-Aided Multi-Agent Reinforcement Learning for Wireless Random Access Network Optimization
Oct 6, 2026Myeung Suk Oh, Zhiyao Zhang, Alvaro Velasquez +2Wireless Resource Allocation Decentralized MARL
The Ohio State University Columbus, Ohio, USA · University of Colorado Boulder Boulder, Colorado, USA · Pathfinder Applied Research, LLC Ashburn, Virginia, USA
Visual Swarm Navigation via Deep Reinforcement Learning and Evolutionary Hybrid Design
Oct 5, 2026Álvaro Díez, Fidel AznarSwarm Robotics Multi-Robot Exploration
Department of Computer Science and Artificial Intelligence, University of Alicante
Taylor Representations for Model-Free RL in Networked MDPs
Oct 4, 2026Salah Chikhi, Abdelhaq Chaoui, Asuman Ozdaglar +1Markov Decision Processes Representation Learning for RL
Massachusetts Institute of Technology
Fully Online Decentralized Learning in Stochastic Games with Unknown Independent Chains
Oct 1, 2026S. Rasoul EtesamiGame Theory Online Mirror Descent
Department of Industrial and Systems Engineering, Coordinated Science Laboratory University of Illinois Urbana-Champaign, Urbana, IL, USA 61801
MASkillBlender: Decentralized Whole-Body Coordination for Multi-Humanoid Loco-Manipulation via Skill Blending
Oct 1, 2026Yifan Hu, Luhang Hong, Mingkang Long +5Humanoid Loco-Manipulation Reinforcement Learning
Nanyang Technological University · Southeast University · Purple Mountain Laboratories
VERA: Verifiable Feasibility Representations with Counterfactual Credit for Constrained Multi-Agent Control
Sep 30, 2026Bo Yin, Dongbo Li, Hongkai Chen +2Multi-Agent Control Constrained RL
Harbin Institute of Technology, Harbin, China · The Chinese University of Hong Kong, Hong Kong SAR, China
Physics-Informed Multi-Agent Coordination for Hospital Patient Flow Optimization
Sep 29, 2026Guoqing Zhang, Rafik Hadfi, Takayuki ItoMulti-Agent Coordination Healthcare
Graduate School of Informatics, Kyoto University, Kyoto, Japan
Fully Decentralized and Safety-Aware Multi-Agent Reinforcement Learning for Control on Networks
Sep 28, 2026Theodore Rogalski, Shirantha WelikalaMulti-Agent Control Decentralized MARL
Electrical & Computer Engineering Department Stevens Institute of Technology Hoboken, USA
ICMAPE: In-Context Multiagent Pure Exploration
Sep 27, 2026Xinyi Hu, Alessio Russo, Aldo PacchianoRL Exploration Decentralized MARL
Boston University · Broad Institute of MIT and Harvard
Reinforcement Learning of Communication in a Mesh of Small Language Models
Sep 24, 2026Mehmet Kerem TurkcanMulti-Agent Communication RL for LLM Agents
Department of Civil Engineering, Columbia University
A Decentralized Partially Observable Team Decision Methodology with Delayed Information Sharing
Sep 22, 2026Xiaoxing Ren, Thomas Parisini, Andreas A. MalikopoulosMarkov Decision Processes Linear Markov Decision Processes
School of Civil and Environmental Engineering, Cornell University, Ithaca, NY, USA · Dept. of Electrical and Electronic Engineering, Imperial College London, London SW7 2AZ, UK · Dept. of Electronic Systems, Aalborg University, Denmark +2
Fully Byzantine-Resilient Multi-Agent Reinforcement Learning
Sep 22, 2026Haejoon Lee, Dimitra PanagouDecentralized MARL Actor-Critic Methods
Robotics Department, University of Michigan, Ann Arbor, MI, USA
Latent Telepathy: Multi-Robot Communication with Self-Supervised Perceptual Latents
Sep 20, 2026Howard Wang, Han Zheng, Cathy WuMulti-Robot Systems Decentralized MARL
Columbia University, New York, NY, USA · Laboratory for Information and Decision Systems, Massachusetts Institute of Technology, Cambridge, MA, USA
CoRe-MARL: Cooperative Redistribution Under Unknown Dynamics Using Recurrent Multi-Agent Reinforcement Learning
Sep 16, 2026Naimur Rahman Chowdhury, Shatabdi Sen Prapti, Md. Salehin Seyam +1Partially Observable RL Multi-Agent Resource Allocation
Industrial and Systems Engineering, North Carolina State University, Raleigh, NC, United States · Department of Industrial and Production Engineering, Bangladesh University of Engineering and Technology, Dhaka, 1000, Bangladesh
High-Probability Nash Regret for Decentralized Learning in Markov $α$ -Potential Games: Episodic and Fully Online Asynchronous Algorithms with Applications to Markov Congestion Games
Sep 14, 2026S. Rasoul EtesamiPolicy Gradient Asynchronous RL
Department of Industrial and Systems Engineering Coordinated Science Laboratory University of Illinois Urbana-Champaign, Urbana, IL, USA
AssemblyGrid v1: A Benchmark for Multi-Robot Production with Temporary Coalitions, Local Information, and Geometric Constraints
Sep 13, 2026Fouad Bahrpeyma, David Heik, Dirk ReicheltMulti-Robot Task Allocation Cooperative MARL
Faculty of Computer Science/Mathematics Hochschule für Technik und Wirtschaft Dresden – University of Applied Sciences Dresden, Germany
Learning Multi-Agent Task Assignment and Navigation in the Factory: from Simulation to Real Robots
Sep 13, 2026Abdalwhab Bakheet Mohamed Abdalwhab, Giovanni Beltrame, David St-OngeDecentralized MARL Multi-Robot Task Allocation
INIT Robots Lab, École de technologie supérieure, 1100 Notre-Dame W., Canada · MISTLab, Department of Computer and Software Engineering, Polytechnique Montréal, Canada
Learning Communication-Conditioned Generative Policies for Decentralized Multi-Agent Collision Avoidance
Sep 13, 2026Prajwal Koirala, Mark CampbellCollision Avoidance Decentralized MARL
Sibley School of Mechanical and Aerospace Engineering, Cornell University
Decentralized Safe Multi-Agent Reinforcement Learning via Predictive Shielding
Sep 7, 2026Yacine El Yamani, Hanna Krasowski, Elena VanneauxMulti-Robot Coordination Robot Safety
ENSTA, IP Paris, Palaiseau, France · UC Berkeley, USA
SMaRT-Tug: Structured Multi-Agent Reinforcement Learning for Physics-Based Tugboat-Barge Collaborative Manipulation
Sep 7, 2026Junkai Lu, Jiadong Zhao, Jiacheng Zhang +8Multi-Agent Coordination Multi-Agent Control
National University of Singapore Singapore, Singapore · ST Engineering Unmanned & Integrated Systems Pte. Ltd. Singapore, Singapore
A Schema Bounded Language Model for Refining Robot Policies Without Destabilizing Local Learning
Sep 4, 2026Chongwen Dong, Mithun Paul Saint-Germain, Pinjari Asif +1Robot Policy Learning Robot Policy Adaptation
Department of Mechanical Engineering Northern Arizona University 15600 S. McConnell Dr., Flagstaff, AZ 86011 · School of Informatics, Computing, and Cyber Systems Northern Arizona University 1295 Knoles Dr., Flagstaff, AZ 86011 · Helen and John C. Hartmann Department of Electrical Engineering and School of Applied Engineering and Technology New Jersey Institute of Technology 323 Martin Luther King Jr. Blvd., Newark, NJ 07102

Oct 26, 2026J/K move · Enter open · S save

Belief-Conditioned Multi-Agent Actor Critic for Decentralized LLM Collaboration

Tianle Chen, Shuo Liu, Christopher Amato

Khoury College of Computer Sciences, Northeastern University, Boston, MA, 02215, USA

Multi-Agent LLM Systems Multi-Agent Reinforcement Learning Decentralized MARL Cooperative MARL

Open paper AskPDF arXiv

Recent work has explored optimizing LLM collaboration through Multi-Agent Reinforcement Learning (MARL), but many approaches still rely on predefined workflows, limiting efficiency and scalability. Decentralized LLM collaboration is challenging because agents execute independently with limited communication, share a joint objective, and must infer useful information from communication traces. In this paper, we propose BCMAAC, a Belief-Conditioned Multi-Agent Actor-Critic method that maintains beliefs over contextual information and uses them to facilitate training. Experiments on Minecraft game-playing, web searching and tool-using tasks show that BCMAAC improves task performance and collaborative behavior compared to naive actor-critic baselines. The improvements are particularly pronounced in multi-turn tasks with richer contextual information, where belief modeling provides a clear advantage.