Multi-Agent Coordination

Ask about this topic

Overview

Multi-agent coordination studies how agents with partial information, different capabilities, or competing incentives can act together in settings such as robot teams, traffic networks, resource allocation, and language-model workflows. Common approaches include decentralized and centralized multi-agent reinforcement learning, game-theoretic planning, role and task assignment, and communication through messages, shared memory, or learned coordination policies. Recent work also develops orchestration protocols and benchmarks for language-model agents, measuring not just task success but information use, role adherence, safety, and coordination failures. A central tension is that communication and additional agents can improve complex teamwork but also introduce overhead, propagate errors, or make performance worse; individual reasoning ability alone does not guarantee effective cooperation.

Latest papers

CardsList

520 papers84 papers in the last four weeks, up 91% on the four weeks before. 0.5% of all new papers.

Mar 3, 2026

MA-CoNav: A Master-Slave Multi-Agent Framework with Hierarchical Collaboration and Dual-Level Reflection for Long-Horizon Embodied VLN

Vision-Language Navigation (VLN) aims to empower robots with the ability to perform long-horizon navigation in unfamiliar environments based on complex linguistic instructions. Its success critically hinges on establishing an efficient language-understanding -- visual-perception -- embodied-execution'' closed loop. Existing methods often suffer from perceptual distortion and decision drift in complex, long-distance tasks due to the cognitive overload of a single agent. Inspired by distributed cognition theory, this paper proposes MA-CoNav, a Multi-Agent Collaborative Navigation framework. This framework adopts a Master-Slave'' hierarchical agent collaboration architecture, decoupling and distributing the perception, planning, execution, and memory functions required for navigation tasks to specialized agents. Specifically, the Master Agent is responsible for global orchestration, while the Subordinate Agent group collaborates through a clear division of labor: an Observation Agent generates environment descriptions, a Planning Agent performs task decomposition and dynamic verification, an Execution Agent handles simultaneous mapping and action, and a Memory Agent manages structured experiences. Furthermore, the framework introduces a ``Local-Global'' dual-stage reflection mechanism to dynamically optimize the entire navigation pipeline. Empirical experiments were conducted using a real-world indoor dataset collected by a Limo Pro robot, with no scene-specific fine-tuning performed on the models throughout the process. The results demonstrate that MA-CoNav comprehensively outperforms existing mainstream VLN methods across multiple metrics.

Ling Luo, Qianqian Bai

Southwestern University of Finance and Economics, Chengdu, 611130, China

Vision-Language NavigationMulti-Agent CoordinationMulti-Agent Planning
AskPDF
Mar 3, 2026

Generalized Per-Agent Advantage Estimation for Multi-Agent Policy Optimization

In this paper, we propose a novel framework for multi-agent reinforcement learning that enhances sample efficiency and coordination through accurate per-agent advantage estimation. The core of our approach is Generalized Per-Agent Advantage Estimator (GPAE), which employs a per-agent value iteration operator to compute precise per-agent advantages. This operator enables stable off-policy learning by indirectly estimating values via action probabilities, eliminating the need for direct Q-function estimation. To further refine estimation, we introduce a double-truncated importance sampling ratio scheme. This scheme improves credit assignment for off-policy trajectories by balancing sensitivity to the agent's own policy changes with robustness to non-stationarity from other agents. Experiments on benchmarks demonstrate that our approach outperforms existing approaches, excelling in coordination and sample efficiency for complex scenarios.

Seongmin Kim, Giseung Park, Woojun Kim +3

KAIST Daejeon, Korea · University of Toronto Toronto, Ontario, Canada · Carnegie Mellon University Pittsburgh, Pennsylvania, USA +1

Multi-Agent Reinforcement LearningSample-Efficient RLMulti-Agent Coordination
AskPDF
Mar 1, 2026

Silo-Bench: A Scalable Environment for Evaluating Distributed Coordination in Multi-Agent LLM Systems

Large language models are increasingly deployed in multi-agent systems to overcome context limitations by distributing information across agents. Yet whether agents can reliably compute with distributed information, rather than merely exchange it, remains an open question. We introduce SILO-BENCH, a role-agnostic benchmark of 30 algorithmic tasks across three communication complexity levels, evaluating 54 configurations over 1,620 experiments. Our experiments expose a fundamental Communication-Reasoning Gap: agents spontaneously form task-appropriate coordination topologies and exchange information actively, yet systematically fail to synthesize distributed state into correct answers. The failure is localized to the reasoning-integration stage where agents often acquire sufficient information but cannot integrate it. This coordination overhead compounds with scale, eventually eliminating parallelization gains entirely. These findings demonstrate that naively scaling agent count cannot circumvent context limitations, and SILO-BENCH provides a foundation for tracking progress toward genuinely collaborative multi-agent systems. The code is available at https://github.com/jwyjohn/acl26-silo-bench .

Yuzhe Zhang, Feiran Liu, Yi Shan +8

Beijing University of Technology, Beijing, China · Zhejiang University, Hangzhou, China · Meituan LongCat Interaction Team +2

Multi-Agent LLM SystemsMulti-Agent CoordinationMulti-Agent ReasoningMulti-Agent Communication
AskPDF
Feb 27, 2026

COOP2^2: Defining, Observing, and Repairing Cooperation in LLM Multi-Agent Systems

Many complex tasks require extended effort, diverse capabilities, or coordinated actions beyond what a single agent can provide. However, simply adding more agents does not guarantee better performance, as effective cooperation depends on how agents interact with each other and with task structure to satisfy evolving constraints over time. This challenge is amplified for LLM-based multi-agent systems (LLM-MAS): plans, messages, and revisions occur in natural language, whereas task progress depends on grounded environment actions. Current evaluations mostly treat cooperation as an implicit ingredient of final task success, leaving both cooperation and the effect of multi-agent interaction on task dynamics difficult to study. We introduce COOP2^2, an evaluation framework that grounds high-level agent cooperation dynamics in LLM-MAS within task progress in the environment. COOP2^2 then defines cooperative tasks with verifiable cooperative requirements, allowing us to analyze how cooperation unfolds over time with respect to task progress, as well as where and why cooperation breaks down. Building on this framework, we develop COOP2^2-Repair, which predicts constraint failures from group plans and opens targeted repair channels for guided revisions. Across two environments and three communication structures, COOP2^2-Repair improves task success and constraint satisfaction while exposing the additional decision overhead and communication load required for repair. The project web page can be found at: https://happyeureka.github.io/coop2.

Hanqing Yang, Narjes Nourzad, Shiyu Chen +3

CMU · USC · SUTD +1

Multi-Agent CollaborationMulti-Agent CoordinationLLM Agent EvaluationMulti-Agent PlanningMulti-Agent CommunicationLLM Planning
AskPDF
Feb 27, 2026

DIG to Heal: Scaling General-purpose Agent Collaboration via Explainable Dynamic Decision Paths

The increasingly popular agentic AI paradigm promises to harness the power of multiple, general-purpose large language model (LLM) agents to collaboratively complete complex tasks. While many agentic AI systems reduce complexity through predefined workflows or fixed agent roles, the ideal is to support truly autonomous agents capable of emergent collaboration across many interacting agents. Yet in practice, such unstructured interactions often lead to redundant work and cascading failures that are difficult to interpret or correct. In this work, we study multi-agent systems composed of general-purpose LLM agents that solve problems through emergent collaboration, without relying on predefined roles, control flows, or communication constraints. We introduce the Dynamic Interaction Graph (DIG), which captures emergent collaboration as a time-evolving causal network of agent activations and interactions. DIG makes emergent collaboration observable and explainable for the first time, enabling real-time identification, explanation, and correction of collaboration-induced error patterns directly from agents' collaboration paths. Thus, DIG fills a critical gap in understanding how general LLM agents solve problems together in truly agentic multi-agent systems. The project webpage can be found at: https://happyeureka.github.io/dig.

Hanqing Yang, Hyungwoo Lee, Yuhang Yao +4

CMU · Zoom · Salesforce +2

Multi-Agent LLM SystemsMulti-Agent SystemsMulti-Agent CoordinationEmergent Behavior in Multi-Agent Systems
AskPDF
Feb 27, 2026

TSC: Topology-Conditioned Stackelberg Coordination for Multi-Agent Reinforcement Learning in Interactive Driving

Safe and efficient autonomous driving in dense traffic is fundamentally a decentralized multi-agent coordination problem, where interactions at conflict points such as merging and weaving must be resolved reliably under partial observability. With only local and incomplete cues, interaction patterns can change rapidly, often causing unstable behaviors such as oscillatory yielding or unsafe commitments. Existing multi-agent reinforcement learning (MARL) approaches either adopt synchronous decision-making, which exacerbate non-stationarity, or depend on centralized sequencing mechanisms that scale poorly as traffic density increases. To address these limitations, we propose Topology-conditioned Stackelberg Coordination (TSC), a learning framework for decentralized interactive driving under communication-free execution, which extracts a time-varying directed priority graph from braid-inspired weaving relations between trajectories, thereby defining local leader-follower dependencies without constructing a global order of play. Conditioned on this graph, TSC endogenously factorizes dense interactions into graph-local Stackelberg subgames and, under centralized training and decentralized execution (CTDE), learns a sequential coordination policy that anticipates leaders via action prediction and trains followers through action-conditioned value learning to approximate local best responses, improving training stability and safety in dense traffic. Experiments across four dense traffic scenarios show that TSC achieves superior performance over representative MARL baselines across key metrics, most notably reducing collisions while maintaining competitive traffic efficiency and control smoothness.

Xiaotong Zhang, Gang Xiong, Yuanjing Wang +3

State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China · School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing 100049, China · Department of Natural Sciences, University of Durham, Durham DH1 3LE, United Kingdom +4

Multi-Agent CoordinationDecentralized MARLRL for Autonomous DrivingMulti-Agent Reinforcement LearningSequential MARL
AskPDF
Feb 27, 2026

RUMAD: Reinforcement-Unifying Multi-Agent Debate

Multi-agent debate (MAD) systems leverage collective intelligence to enhance reasoning capabilities, yet existing approaches struggle to simultaneously optimize accuracy, consensus formation, and computational efficiency. Static topology methods lack adaptability to task complexity variations, while external LLM-based coordination risks introducing privileged knowledge that compromises debate neutrality. This work presents RUMAD (Reinforcement-Unifying Multi-Agent Debate), a novel framework that formulates dynamic communication topology control in MAD as a reinforcement learning (RL) problem. RUMAD employs a content-agnostic observation scheme that captures high-level debate dynamics avoiding access to raw agent reasoning content. RUMAD uses a multi-objective reward to model solution quality, cohesion and efficiency. A PPO-trained controller dynamically adjusts edge weights in the communication graph, while a dual-threshold mechanism enables fine-grained control over both agent activation and information visibility. Experimental evaluation across MMLU, GSM8K, and GPQA benchmarks demonstrates that RUMAD achieves substantial efficiency gains, reducing token costs by over 80%, while still improving reasoning accuracy compared to single LLM model and multiple MAD baselines. Notably, RUMAD trained exclusively on MMLU exhibits robust zero-shot generalization to out-of-domain (OOD) tasks, indicating that the learned communication strategies capture task-independent principles of effective multi-agent coordination. These results establish RUMAD as a efficient and robust approach for deploying multi-agent reasoning application with practical resource constraints.

Chao Wang, Han Lin, Huaze Tang +2

Tsinghua University, Shenzhen, China · Zhejiang University, Hangzhou, China

Communication Topology OptimizationMulti-Agent DebateMulti-Agent CoordinationMulti-Agent CommunicationRL for LLM Agents
AskPDF
Feb 26, 2026

Robust Information Design for Multi-Agent Systems with Complementarities: Smallest-Equilibrium Threshold Policies

We study information design in multi-agent systems (MAS) with binary actions and strategic complementarities, where an external designer influences behavior only through signals. Agents play the smallest-equilibrium of the induced Bayesian game, reflecting conservative, coordination-averse behavior typical in distributed systems. We show that when utilities admit a convex potential and welfare is convex, the robustly implementable optimum has a remarkably simple form: perfect coordination at each state: either everyone acts or no one does. We provide a constructive threshold rule: compute a one-dimensional score for each state, sort states, and pick a single threshold (with a knife-edge lottery for at most one state). This rule is an explicit optimal vertex of a linear program (LP) characterized by feasibility and sequential obedience constraints. Empirically, in both vaccination and technology-adoption domains, our constructive policy matches LP optima, scales as O(∣Θ∣log⁡∣Θ∣)O(|Θ|\log|Θ|), and avoids the inflated welfare predicted by obedience-only designs that assume the designer can dictate the (best) equilibrium. The result is a general, scalable recipe for robust coordination in MAS with complementarities.

Farzaneh Farhadi, Maria Chli

Aston University, Birmingham, United Kingdom

Information DesignMulti-Agent CoordinationGame TheoryImperfect-Information Games
AskPDF
Feb 26, 2026

Agentic AI for Intent-driven Optimization in Cell-free O-RAN

Agentic artificial intelligence (AI) is emerging as a key enabler for autonomous radio access networks (RANs), where multiple large language model (LLM)-based agents reason and collaborate to achieve operator-defined intents. The open RAN (O-RAN) architecture enables the deployment and coordination of such agents. However, most existing works consider simple intents handled by independent agents, while complex intents that require coordination among agents remain unexplored. In this paper, we propose an agentic AI framework for intent translation and optimization in cell-free O-RAN. A supervisor agent translates the operator intents into an optimization objective and minimum rate requirements. Based on this information, a user weighting agent retrieves relevant prior experience from a memory module to determine the user priority weights for precoding. If the intent includes an energy-saving objective, then an open radio unit (O-RU) management agent will also be activated to determine the set of active O-RUs by using a deep reinforcement learning (DRL) algorithm. A monitoring agent measures and monitors the user data rates and coordinates with other agents to guarantee the minimum rate requirements are satisfied. To enhance scalability, we adopt a parameter-efficient fine-tuning (PEFT) method that enables the same underlying LLM to be used for different agents. Simulation results show that the proposed agentic AI framework reduces the number of active O-RUs by 41.93% when compared with three baseline schemes in energy-saving mode. Using the PEFT method, the proposed framework reduces the memory usage by 92% when compared with deploying separate LLM agents.

Mohammad Hossein Shokouhi, Vincent W. S. Wong

Department of Electrical and Computer Engineering, The University of British Columbia, Vancouver, Canada

Wireless CommunicationsIntent-Based NetworkingLLM Agent OrchestrationWireless Resource AllocationOpen Radio Access NetworkMulti-Agent Coordination
AskPDF
Feb 25, 2026

Training Generalizable Collaborative Agents via Strategic Risk Aversion

Many emerging agentic paradigms require agents to collaborate with one another (or people) to achieve shared goals. Unfortunately, existing approaches to learning policies for such collaborative problems produce brittle solutions that fail when paired with new partners. We attribute these failures to a combination of free-riding during training and a lack of strategic robustness. To address these problems, we study the concept of strategic risk aversion and interpret it as a principled inductive bias for generalizable cooperation with unseen partners. While strategically risk-averse players are robust to deviations in their partner's behavior by design, we show that, in collaborative games, they also (1) can have better equilibrium outcomes than those at classical game-theoretic concepts like Nash, and (2) exhibit less or no free-riding. Inspired by these insights, we develop a multi-agent reinforcement learning (MARL) algorithm that integrates strategic risk aversion into standard policy optimization methods. Our empirical results across collaborative benchmarks (including an LLM collaboration task) validate our theory and demonstrate that our approach consistently achieves reliable collaboration with heterogeneous and previously unseen partners across collaborative tasks.

Chengrui Qu, Yizhou Zhang, Nicolas Lanzetti +1

Department of Computing and Mathematical Sciences, California Institute of Technology, Pasadena, CA, USA.

Zero-Shot CoordinationCooperative MARLMulti-Agent Coordination
AskPDF
Feb 24, 2026

Training-Free Agentic AI: Probabilistic Control and Coordination in Multi-Agent LLM Systems

Multi-agent large language model (LLM) systems enable complex, long-horizon reasoning by composing specialized agents, but practical deployment remains hindered by inefficient routing, noisy feedback, and high interaction cost. We introduce REDEREF, a lightweight and training-free controller for multi-agent LLM collaboration that improves routing efficiency during recursive delegation. REDEREF integrates (i) belief-guided delegation via Thompson sampling to prioritize agents with historically positive marginal contributions, (ii) reflection-driven re-routing using a calibrated LLM or programmatic judge, (iii) evidence-based selection rather than output averaging, and (iv) memory-aware priors to reduce cold-start inefficiency. Across multi-agent split-knowledge tasks, we show that while recursive retry alone saturates task success, belief-guided routing reduces token usage by 28%, agent calls by 17%, and time-to-success by 19% compared to random recursive delegation, and adapts gracefully under agent or judge degradation. These results demonstrate that simple, interpretable probabilistic control can meaningfully improve the efficiency and robustness of multi-agent LLM systems without training or fine-tuning.

Mohammad Parsa Hosseini, Ankit Shah, Saiyra Qureshi +3

Accenture.

Multi-Agent LLM SystemsLLM Agent OrchestrationLLM Agent RoutingMulti-Agent Coordination
AskPDF
Feb 24, 2026

Probing Dec-POMDP Reasoning in Cooperative MARL

Cooperative multi-agent reinforcement learning (MARL) is typically framed as a decentralised partially observable Markov decision process (Dec-POMDP), a setting whose hardness stems from two key challenges: partial observability and decentralised coordination. Genuinely solving such tasks requires Dec-POMDP reasoning, where agents use history to infer hidden states and coordinate based on local information. Yet it remains unclear whether popular benchmarks actually demand this reasoning or permit success via simpler strategies. We introduce a diagnostic suite combining statistically grounded performance comparisons and information-theoretic probes to audit the behavioural complexity of baseline policies (IPPO and MAPPO) across 37 scenarios spanning MPE, SMAX, Overcooked, Hanabi, and MaBrax. Our diagnostics reveal that success on these benchmarks rarely requires genuine Dec-POMDP reasoning. Reactive policies match the performance of memory-based agents in over half the scenarios, and emergent coordination frequently relies on brittle, synchronous action coupling rather than robust temporal influence. These findings suggest that some widely used benchmarks may not adequately test core Dec-POMDP assumptions under current training paradigms, potentially leading to over-optimistic assessments of progress. We release our diagnostic tooling to support more rigorous environment design and evaluation in cooperative MARL.

Kale-ab Tessera, Leonard Hinckeldey, Riccardo Zamboni +2

University of Edinburgh, Edinburgh, United Kingdom · Politecnico di Milano, Milan, Italy

Cooperative MARLMulti-Agent ReasoningMulti-Agent CoordinationPartially Observable Markov Decision ProcessesAI Agent Benchmarks
AskPDF
Feb 23, 2026

A General Equilibrium Theory of Orchestrated AI Agent Systems

We establish a general equilibrium theory for systems of large language model (LLM) agents operating under centralized orchestration. The framework is a production economy in the sense of Arrow-Debreu (1954), extended to infinite-dimensional commodity spaces following Bewley (1972). Each LLM agent is modeled as a firm whose production set Y a ⊂\subset H = L 2 ([0, T ], R R ) represents the feasible metric trajectories determined by its frozen model weights. The orchestrator is the consumer, choosing a routing policy over the agent DAG to maximize system welfare subject to a budget constraint evaluated at functional prices p ∈\in H A . These prices-elements of the Hilbert dual of the commodity space-assign a shadow value to each metric of each agent at each instant. We prove, via Brouwer's theorem applied to a finitedimensional approximation V K ⊂\subset H, that every such economy admits at least one general equilibrium (p * , y * , ππ * ). A functional Walras' law holds as a theorem: the value of functional excess demand is zero for all prices, as a consequence of the consumer's budget constraint-not by construction. We further establish Pareto optimality (First Welfare Theorem), decentralizability of Pareto optima (Second Welfare Theorem), and uniqueness with geometric convergence under a contraction condition (Banach). The orchestration dynamics constitute a Walrasian t{â}tonnement that converges globally under the contraction condition, unlike classical t{â}tonnement (Scarf, 1960). The framework admits a DSGE interpretation with SLO parameters as policy rates.

Jean-Philippe Garnier

Br.AI.K

Multi-Agent LLM SystemsMulti-Agent CoordinationLLM Agent Orchestration
AskPDF
Feb 22, 2026

Self-Configurable Mesh-Networks for Scalable Distributed Submodular Bandit Optimization

We study how to scale distributed bandit submodular coordination under realistic communication constraints in bandwidth, data rate, and connectivity. We are motivated by multi-agent tasks of active situational awareness in unknown, partially-observable, and resource-limited environments, where the agents must coordinate through agent-to-agent communication. Our approach enables scalability by (i) limiting information relays to only one-hop communication and (ii) keeping inter-agent messages small, having each agent transmit only its own action information. Despite these information-access restrictions, our approach enables near-optimal action coordination by optimizing the agents' communication neighborhoods over time, through distributed online bandit optimization, subject to the agents' bandwidth constraints. Particularly, our approach enjoys an anytime suboptimality bound that is also strictly positive for arbitrary network topologies, even disconnected. To prove the bound, we define the Value of Coordination (VoC), an information-theoretic metric that quantifies for each agent the benefit of information access to its neighbors. We validate in simulations the scalability and near-optimality of our approach: it is observed to converge faster, outperform benchmarks for bandit submodular coordination, and can even outperform benchmarks that are privileged with a priori knowledge of the environment.

Zirui Xu, Vasileios Tzoumas

Department of Aerospace Engineering, University of Michigan, Ann Arbor, MI 48109 USA

Multi-Agent CoordinationSubmodular OptimizationCommunication Topology OptimizationGraph BanditsCombinatorial Multi-Armed Bandits
AskPDF
Feb 21, 2026

(Perlin) Noise as AI coordinator

Large scale control of nonplayer agents is central to modern games, while production systems still struggle to balance several competing goals: locally smooth, natural behavior, and globally coordinated variety across space and time. Prior approaches rely on handcrafted rules or purely stochastic triggers, which either converge to mechanical synchrony or devolve into uncorrelated noise that is hard to tune. Continuous noise signals such as Perlin noise are well suited to this gap because they provide spatially and temporally coherent randomness, and they are already widely used for terrain, biomes, and other procedural assets. We adapt these signals for the first time to large scale AI control and present a general framework that treats continuous noise fields as an AI coordinator. The framework combines three layers of control: behavior parameterization for movement at the agent level, action time scheduling for when behaviors start and stop, and spawn or event type and feature generation for what appears and where. We instantiate the framework reproducibly and evaluate Perlin noise as a representative coordinator across multiple maps, scales, and seeds against random, filtered, deterministic, neighborhood constrained, and physics inspired baselines. Experiments show that coordinated noise fields provide stable activation statistics without lockstep, strong spatial coverage and regional balance, better diversity with controllable polarization, and competitive runtime. We hope this work motivates a broader exploration of coordinated noise in game AI as a practical path to combine efficiency, controllability, and quality.

Kaijie Xu, Clark Verbrugge

McGill University, Montreal, Quebec, Canada

Multi-Agent ControlMulti-Agent CoordinationGame-Playing Agents
AskPDF
Feb 21, 2026

When Coordination Is Avoidable: A Monotonicity Analysis of Organizational Tasks

Organizations devote substantial resources to coordination, yet which tasks actually require it for correctness remains unclear. The problem is acute in multi-agent AI systems, where coordination cost is directly measurable and can exceed the cost of the work itself. Distributed systems theory provides a precise criterion: coordination is required when a task specification is non-monotonic, meaning that as histories grow, new information can invalidate prior conclusions. Here we show that Thompson's classic taxonomy of interdependence maps to that criterion, yielding a decision rule for when coordination is required for correctness. We formalize the correspondence in a bridge theorem, apply the rule to 65 workflows from the American Productivity & Quality Center (APQC) and, with a calibrated large language model (LLM), 13,417 Occupational Information Network (ONET) tasks, and illustrate it in multi-agent AI simulations. Under our decompositions, 74% of workflows and 42% of ONET tasks are monotonic, implying that up to 24-57% of coordination spending is unnecessary for correctness.

Harang Ju

Carey Business School, Johns Hopkins University, Baltimore, MD 21202

Multi-Agent Coordination
AskPDF
Feb 20, 2026

Diffusing to Coordinate: Efficient Online Multi-Agent Diffusion Policies

Online Multi-Agent Reinforcement Learning (MARL) is a prominent framework for efficient agent coordination. Crucially, enhancing policy expressiveness is pivotal for achieving superior performance. Diffusion-based generative models are well-positioned to meet this demand, having demonstrated remarkable expressiveness and multimodal representation in image generation and offline settings. Yet, their potential in online MARL remains largely under-explored. A major obstacle is that the intractable likelihoods of diffusion models impede entropy-based exploration and coordination. To tackle this challenge, we propose among the first Online off-policy MARL framework using Diffusion policies (OMAD) to orchestrate coordination. Our key innovation is a relaxed policy objective that maximizes scaled joint entropy, facilitating effective exploration without relying on tractable likelihood. Complementing this, within the centralized training with decentralized execution (CTDE) paradigm, we employ a joint distributional value function to optimize decentralized diffusion policies. It leverages tractable entropy-augmented targets to guide the simultaneous updates of diffusion policies, thereby ensuring stable coordination. Extensive evaluations on MPE and MAMuJoCo establish our method as the new state-of-the-art across 1010 diverse tasks, demonstrating a remarkable 2.5×2.5\times to 5×5\times improvement in sample efficiency.

Zhuoran Li, Hai Zhong, Xun Wang +3

Institute for Interdisciplinary Information Sciences, Tsinghua University · ByteDance

Multi-Agent Reinforcement LearningMulti-Agent CoordinationDiffusion PolicyOff-Policy RLMaximum Entropy RL
AskPDF
Feb 19, 2026

Action-Graph Policies: Learning Action Co-dependencies in Multi-Agent Reinforcement Learning

Coordinating actions is the most fundamental form of cooperation in multi-agent reinforcement learning (MARL). Successful decentralized decision-making often depends not only on good individual actions, but on selecting compatible actions across agents to synchronize behavior, avoid conflicts, and satisfy global constraints. In this paper, we propose Action Graph Policies (AGP), that model dependencies among agents' available action choices. It constructs, what we call, coordination contexts, that enable agents to condition their decisions on global action dependencies. Theoretically, we show that AGPs induce a strictly more expressive joint policy compared to fully independent policies and can realize coordinated joint actions that are provably more optimal than greedy execution even from centralized value-decomposition methods. Empirically, we show that AGP achieves 80-95% success on canonical coordination tasks with partial observability and anti-coordination penalties, where other MARL methods reach only 10-25%. We further demonstrate that AGP consistently outperforms these baselines in diverse multi-agent environments.

Nikunj Gupta, James Zachary Hare, Jesse Milzman +2

University of Southern California, Los Angeles, CA, USA · DEVCOM Army Research Laboratory, Adelphi, MD, USA · DEVCOM ARL Army Research Office, Los Angeles, CA, USA.

Multi-Agent CoordinationDecentralized MARL
AskPDF
Feb 18, 2026

Multi-Agent Lipschitz Bandits

We study the decentralized multi-player stochastic bandit problem over a continuous, Lipschitz-structured action space where hard collisions yield zero reward. Our objective is to design a communication-free policy that maximizes collective reward, while separating coordination costs from learning costs. We propose a modular protocol that first solves the multi-agent coordination problem by identifying and seating players on distinct, high-value regions via a novel maxima-directed search and then decouples the problem into NN independent single-player Lipschitz bandits. In the consensus regime, we obtain an end-to-end regret bound whose dominant learning term is O~(T(d+1)/(d+2))\tilde{O}(T^{(d+1)/(d+2)}), matching the single-player Lipschitz rate; the upfront coordination cost is horizon-independent at fixed confidence and only polylogarithmic in TT in the expected-regret form. Under an additional public coverage/scheduling assumption for the epochic extension, we also obtain a gap-free O~(T(d+1)/(d+2))\tilde{O}(T^{(d+1)/(d+2)}) guarantee. We further derive a matching lower bound for the dominant learning term and extend the framework to general distance-threshold collision models.

Sourav Chakraborty, Amit Kiran Rege, Claire Monteleoni +1

University of Colorado Boulder · INRIA Paris

Lipschitz BanditsMulti-Agent BanditsMulti-Armed BanditsMulti-Agent Coordination
AskPDF
Feb 18, 2026

NestRL: A Nested Training Regime for Mutual Adaptation in Human-AI Teaming

Mutual adaptation is a central challenge in human-AI teaming, as humans naturally adjust their strategies in response to an AI agent's behavior. Existing approaches attempt to approximate human behavior by diversifying training partners; however, these partners are typically static and fail to capture the adaptive nature of human teammates. When agents are trained jointly in standard multi-agent settings, they often converge to opaque coordination strategies that work only with their co-trained partners, leading to poor generalization. To model adaptive human behavior, we formulate human-AI teaming as an Interactive Partially Observable Markov Decision Process (I-POMDP). We propose NestRL, a nested training regime that learns the solution to a finite-level I-POMDP by training agents at each level against adaptive agents from the level below. This exposes agents to adaptive behavior while preventing emergence of opaque coordination strategies. We provide theoretical analysis showing that NestRL agents avoid convergence to partner-specific strategies, and validate this empirically in the Overcooked domain against state-of-the-art baselines. NestRL achieves higher task performance with both unseen adaptive agents and real human teammates, while exhibiting significantly greater adaptability over the course of interaction.

Upasana Biswas, Durgesh Kalwar, Subbarao Kambhampati +1

School of Computing and AI, Arizona State University, Tempe, AZ, USA · Department of Computer Science, Colorado State University, Fort Collins, CO, USA

Human-AI CollaborationMulti-Agent CoordinationCooperative MARLAd Hoc Teamwork
AskPDF
Feb 18, 2026

Graphon Mean-Field Subsampling for Cooperative Heterogeneous Multi-Agent Reinforcement Learning

Coordinating large populations of interacting agents is a central challenge in multi-agent reinforcement learning (MARL), where the size of the joint state-action space scales exponentially with the number of agents. Mean-field methods alleviate this burden by aggregating agent interactions, but these approaches assume homogeneous interactions. Recent graphon-based frameworks capture heterogeneity, but are computationally expensive as the number of agents grows. Therefore, we introduce GMFS\texttt{GMFS}, a G\textbf{G}raphon M\textbf{M}ean-F\textbf{F}ield S\textbf{S}ubsampling framework for scalable cooperative MARL with heterogeneous agent interactions. By subsampling κκ agents according to interaction strength, we approximate the graphon-weighted mean-field and learn a policy with sample complexity poly(κ)\mathrm{poly}(κ) and optimality gap O(1/κ)O(1/\sqrtκ). We verify our theory with numerical simulations in robotic coordination, showing that GMFS\texttt{GMFS} achieves near-optimal performance.

Emile Anand, Richard Hoffmann, Sarah Liaw +1

School of Computer Science, Georgia Institute of Technology, Atlanta, GA, USA · Computing and Mathematical Sciences, California Institute of Technology, Pasadena, CA, USA · Kempner Institute, Harvard University, Cambridge, MA, USA

Reinforcement LearningCooperative MARLMean-Field MARLHeterogeneous MARLMulti-Agent Coordination
AskPDF
Feb 17, 2026

MARLEM: A Multi-Agent Reinforcement Learning Simulation Framework for Implicit Cooperation in Decentralized Local Energy Markets

This paper introduces a novel, open-source MARL simulation framework for studying implicit cooperation in LEMs, modeled as a decentralized partially observable Markov decision process and implemented as a Gymnasium environment for MARL. Our framework features a modular market platform with plug-and-play clearing mechanisms, physically constrained agent models (including battery storage), a realistic grid network, and a comprehensive analytics suite to evaluate emergent coordination. The main contribution is a novel method to foster implicit cooperation, where agents' observations and rewards are enhanced with system-level key performance indicators to enable them to independently learn strategies that benefit the entire system and aim for collectively beneficial outcomes without explicit communication. Through representative case studies (available in a dedicated GitHub repository in https://github.com/salazarna/marlem, we show the framework's ability to analyze how different market configurations (such as varying storage deployment) impact system performance. This illustrates its potential to facilitate emergent coordination, improve market efficiency, and strengthen grid stability. The proposed simulation framework is a flexible, extensible, and reproducible tool for researchers and practitioners to design, test, and validate strategies for future intelligent, decentralized energy systems.

Nelson Salazar-Pena, Alejandra Tabares, Andres Gonzalez-Mancera

Department of Mechanical Engineering, Universidad de los Andes, 111711, Bogotá D.C., Colombia · Department of Industrial Engineering, Universidad de los Andes, 111711, Bogotá D.C., Colombia

Multi-Agent CoordinationPower SystemsDecentralized MARLEmergent Behavior in Multi-Agent SystemsReward Shaping
AskPDF
Feb 17, 2026

Harnessing Implicit Cooperation: A Multi-Agent Reinforcement Learning Approach Towards Decentralized Local Energy Markets

This paper proposes implicit cooperation, a framework enabling decentralized agents to approximate optimal coordination in local energy markets without explicit peer-to-peer communication. We formulate the problem as a decentralized partially observable Markov decision problem that is solved through a multi-agent reinforcement learning task in which agents use stigmergic signals (key performance indicators at the system level) to infer and react to global states. Through a 3x3 factorial design on an IEEE 34-node topology, we evaluated three training paradigms (CTCE, CTDE, DTDE) and three algorithms (PPO, APPO, SAC). Results identify APPO-DTDE as the optimal configuration, achieving a coordination score of 91.7% relative to the theoretical centralized benchmark (CTCE). However, a critical trade-off emerges between efficiency and stability: while the centralized benchmark maximizes allocative efficiency with a peer-to-peer trade ratio of 0.6, the fully decentralized approach (DTDE) demonstrates superior physical stability. Specifically, DTDE reduces the variance of grid balance by 31% compared to hybrid architectures, establishing a highly predictable, import-biased load profile that simplifies grid regulation. Furthermore, topological analysis reveals emergent spatial clustering, where decentralized agents self-organize into stable trading communities to minimize congestion penalties. While SAC excelled in hybrid settings, it failed in decentralized environments due to entropy-driven instability. This research proves that stigmergic signaling provides sufficient context for complex grid coordination, offering a robust, privacy-preserving alternative to expensive centralized communication infrastructure.

Nelson Salazar-Pena, Alejandra Tabares, Andres Gonzalez-Mancera

Department of Mechanical Engineering, Universidad de los Andes, 111711, Bogotá D.C., Colombia · Department of Industrial Engineering, Universidad de los Andes, 111711, Bogotá D.C., Colombia

Multi-Agent CoordinationDecentralized MARLPower Systems
AskPDF
Feb 16, 2026

Fluid-Agent Reinforcement Learning

The primary focus of multi-agent reinforcement learning (MARL) has been to study interactions among a fixed number of agents embedded in an environment. However, in the real world, the number of agents is neither fixed nor known a priori. Moreover, an agent can decide to create other agents (for example, a cell may divide, or a company may spin off a division). In this paper, we propose a framework that allows agents to create other agents; we call this a fluid-agent environment. We present game-theoretic solution concepts for fluid-agent games and empirically evaluate the performance of several MARL algorithms within this framework. Our experiments include fluid variants of established benchmarks such as Predator-Prey and Level-Based Foraging, where agents can dynamically spawn, as well as a new environment we introduce that highlights how fluidity can unlock novel solution strategies beyond those observed in fixed-population settings. We demonstrate that this framework yields agent teams that adjust their size dynamically to match environmental demands.

Shishir Sharma, Doina Precup, Theodore J. Perkins

Mila – Quebec Artificial Intelligence Institute, Montreal, Canada · McGill University, Montreal, Canada · Ottawa Hospital Research Institute, University of Ottawa, Ottawa, Canada

Multi-Agent Reinforcement LearningMulti-Agent CoordinationGame Theory
AskPDF
Feb 16, 2026

Socially-Weighted Alignment: A Game-Theoretic Framework for Multi-Agent LLM Systems

Deploying large language model (LLM) agents in shared environments introduces a fundamental tension between individual alignment and collective stability: locally rational decisions can impose negative externalities that degrade system-level performance. We propose Socially-Weighted Alignment (SWA), a game-theoretic framework that modifies inference-time decision making by interpolating between an agent's private objective and an estimate of group welfare via a social weight λ∈[0,1]λ\in[0,1]. In a shared-resource congestion game with nn agents and congestion severity ββ, we show that SWA induces a critical threshold λ∗=(n−β)/(n−1)λ^*=(n-β)/(n-1) above which agents no longer have marginal incentive to increase demand under overload, yielding a phase transition from persistent congestion to stable operation near capacity. We further provide an inference-time algorithmic instantiation of SWA that does not require parameter updates or multi-agent reinforcement learning, and use a multi-agent simulation to empirically validate the predicted threshold behavior.

Furkan Mumcu, Yasin Yilmaz

University of South Florida

Congestion GamesMulti-Agent LLM SystemsLLM AlignmentMulti-Agent CoordinationGame Theory
AskPDF
Feb 16, 2026

Noncooperative Virtual Queue Coordination via Uncertainty-Aware Correlated Equilibria

Collaborative virtual queueing has been proposed as a mechanism to mitigate airport surface congestion while preserving airline autonomy over aircraft-level pushback decisions. A central coordinator can regulate aggregate pushback capacity but cannot directly control which specific aircraft are released, limiting its ability to steer system-level performance. We propose a noncooperative coordination mechanism for collaborative virtual queueing based on the correlated equilibrium concept, which enables the coordinator to provide incentive-compatible recommendations on aircraft-level pushback decisions without overriding airline autonomy. To account for uncertainty in airlines' internal cost assessments, we introduce chance constraints into the correlated equilibrium formulation. This formulation provides explicit probabilistic guarantees on incentive compatibility, allowing the coordinator to adjust the confidence level with which airlines are expected to follow the recommended actions. We further propose a scalable algorithm for computing chance-constrained correlated equilibria by exploiting a reduced-rank structure. Numerical experiments demonstrate that the proposed method scales to realistic traffic levels up to 210 eligible pushbacks per hour, reduces accumulated delay by up to approximately 8.9% compared to current first-come-first-served schemes, and reveals a trade-off between confidence level, deviation robustness, and achievable cost efficiency.

Jaehan Im, David Fridovich-Keil, Ufuk Topcu

Department of Aerospace Engineering, The University of Texas at Austin, Austin, Texas, USA

Air Traffic ManagementMulti-Agent CoordinationQueueing TheoryGame TheoryChance-Constrained Optimization
AskPDF
Feb 12, 2026

Grounded Laplacians of Directed Signed Matrix-Weighted Networks: Spectral Properties and Applications to Non-Trivial Consensus

Grounded Laplacians provide the spectral link between external information and network convergence. This paper establishes positive-stability results for grounded Laplacians in directed signed matrix-weighted networks, where directionality, antagonism, and singular edge weight matrices coexist. First, under in-degree dominance and positive-negative reachability, we derive explicit local thresholds for the grounding gains. Second, a scaled, kernel-based certificate replaces the unscaled degree condition with a signed matrix-weighted Dirichlet decomposition and a joint-kernel test for the scaled symmetric part. The computable margin γpγ_p lower-bounds the minimum real part of the spectrum and certifies exponential contraction in the PP-norm. Under absolute generalized balance, the kernel-intersection test is given; the balanced and definite-edge unbalanced undirected cases follow. As an application, non-trivial consensus (NTC) on signed matrix-weighted networks is studied. Informed agents, external signals and coupling terms are designed to steer all agents to any prescribed nonzero state without requiring structural balance. Switching topology case retains non-trivial consensus result under certain conditions. Realizing NTC on signed matrix-weighted networks demonstrates that groups with both cooperative and antagonistic multi-dimensional interactions can achieve consensus, which was previously deemed exclusive to fully cooperative groups.

Tianmu Niu, Bing Mao, Hao Liao +2

Multi-Agent CoordinationOpinion Dynamics
AskPDF
Jan 24, 2026

Does Centralized Learning Help Embodied Agents Coordinate, and When Can a Tabular Study Tell?

Tabular studies are a common way to compare independent and centralized multi-agent learners, because they avoid function approximation. A natural expectation is that centralized learning, which learns joint actions, helps agents coordinate, especially when their bodies make solo success hard. Earlier work tests this only in parts: it measures coordination for one learning method, compares methods with the agents' bodies fixed, or varies speed but reports only how well the team did. We test it in a predator-prey environment, where teammates cooperatively hunt together while two teams compete. We compare independent Q-learning (IQL) with central Q-learning (CQL) in all four predator and prey pairings, under limits on speed and stamina, on a 5x5 and an 8x8 grid trained for 200,000 episodes each, and check that the agents learn on both. We measure coordination directly, with predictions for the main measurement fixed in advance. In this task, where one predator can capture prey alone, we find three things. First, CQL costs its agents in both roles on both grids, and its predators coordinate no more than IQL's. Second, its one benefit comes from acting on the best joint action: this cut capture time by 42% on the small grid with faster prey and unlimited stamina, but helped nowhere on the larger grid, where the best joint action had mostly never been tried. Third, on the larger grid a tie-breaking default decided which method won, even though the agents had passed a learning check. A tabular comparison can therefore be trusted only if it also checks how much of the state space the agents covered and implements the methods alike. We propose simple checks for both.

Muhammad Ahmed Atif, Nehal Naeem Haji, Muhammad Ebad Atif +1

Dhanani School of Science and Technology, Habib University, Karachi, Pakistan

Multi-Agent Reinforcement LearningCooperative MARLMulti-Agent Coordination
AskPDF
Jan 14, 2026

MACRO-LLM: LLM-Empowered Multi-Agent Collaborative Reasoning under Spatiotemporal Partial Observability

Large Language Model (LLM) agents deployed in complex real-world scenarios increasingly operate as spatially distributed entities. However, this physical dispersion constrains agents to limited local perception and finite temporal horizons. We characterize this bottleneck as spatiotemporal partial observability, where spatial and temporal limitations are fundamentally coupled: resolving spatial conflicts requires temporal reasoning about neighbors' future actions, while temporal planning requires spatial context beyond local perception. To bridge this gap, we introduce MACRO-LLM, LLM-empowered multi-agent collaborative reasoning under spatiotemporal partial observability. The architecture interleaves spatial and temporal reasoning within each decision cycle via three interdependent modules: (1) the CoProposer mitigates temporal uncertainty by verifying candidate actions via predictive rollouts; (2) the Negotiator overcomes spatial myopia by resolving conflicts through mean-field statistical aggregation, grounded in the CoProposer's rollout rewards; and (3) the Introspector closes the reasoning loop by analyzing environmental drift and attributing performance changes to refine strategies. Extensive evaluations on two complex long-horizon tasks, cooperative platoon planning and pandemic control, demonstrate that our framework enables robust coordination under spatiotemporal partial observability.

Handi Chen, Running Zhao, Xiuzhe Wu +2

The University of Hong Kong

Multi-Agent LLM SystemsMulti-Agent CoordinationMulti-Agent ReasoningLong-Horizon Agent Tasks
AskPDF
Jan 10, 2026

The Axiom of Consent: Authorization, Friction, and Multi-Agent Coordination

Coordination research collapses four objects: operative control, authorization, a model-derived friction score, and observed outcomes. The Axiom of Consent is a stake-weighted unanimity principle; majority and supermajority thresholds are explicit relaxations, not versions of the axiom. Decision loci are structural facts, whereas authorization and legitimacy require normative and measurement premises. Alignment, calibrated stakes, and information deficit are candidate coordinates, and F = sigma(1 + epsilon)/(1 + alpha) is a phenomenological ansatz. The Replicator-Optimization Mechanism supplies a conditional persistence interface: irreducibility suffices for its finite, static, positive-fitness continuous-time Perron result; primitivity is required only for the corresponding discrete-time power convergence, and the componentwise ranking is narrower. Neither persistence result derives authorization. A resource-allocation instantiation specifies an identification contract but observes no authorization acts or effective voice. Its exploratory MARL companion uses target-vector correlation and observation noise as narrow proxy treatments, not measures of general alignment or information deficit. Under that proxy and reward-gap design, the composite loses to an independent-effects model and a feasible-centred frozen crossing yields the opposite interaction direction. Cooperative target correlation lowers the gap under shared-state contention; separable IQL is structurally invariant and separable VDN is a non-detection. Paired partial sharing modulates the gradient without establishing an exact dose law or endpoint equivalence. Target support changes opposition and residual-policy conclusions. The surviving contribution is an authorization architecture and measurement discipline, not a universal friction law.

Murad Farzulla

Dissensus, London, UK · King’s College London, London, UK

Multi-Agent CoordinationReplicator Dynamics
AskPDF