Organizations: The Key Laboratory of Cognition and Decision Intelligence for Complex Systems, Institute of Automation, Chinese Academy of Sciences · Tsinghua University · Moonshot AI · Faculty of Computing, Harbin Institute of Technology · Nankai University
In the realm of multi-agent systems, the challenge of partial observability is a critical barrier to effective coordination and decision-making. Existing approaches, such as belief state estimation and inter-agent communication, often fall short. Belief-based methods are limited by their focus on past experiences without fully leveraging global information, while communication methods often lack a robust model to effectively utilize the auxiliary information they provide. To solve this issue, we propose Global State Diffusion Algorithm~(GlobeDiff) to infer the global state based on the local observations. By formulating the state inference process as a multi-modal diffusion process, GlobeDiff overcomes ambiguities in state estimation while simultaneously inferring the global state with high fidelity. We prove that the estimation error of GlobeDiff under both unimodal and multi-modal distributions can be bounded. Extensive experimental results demonstrate that GlobeDiff achieves superior performance and is capable of accurately inferring the global state.
Figures & tables
Figure 1: The overall framework of GlobeDiff. During the execution phase, we first construct auxiliary local observations x and then infer the global state s^ using GlobeDiff. Agents make decisions based on the inferred global state s^ .
Figure 2: The training process of Globediff is divided into two parts: minimizing the difference between the prior network pϕ and the posterior network qψ , and then training the diffusion model based on the forward and backward process.
Figure 3: Comparison results with global state inference baselines in SMAC-v1 (PO) tasks with win rate over three random seeds.
Figure 4: Comparison results with global state inference baselines in SMAC-v2 (PO) tasks with win rate over three random seeds.
Figure 5: Visualization of global states generated by GlobeDiff, VAE and MLP. The first plot displays true states and subsequent plots show inferred states per agent. White points denote individual states with polygons highlighting local neighborhoods. Gradient shading (light green to purple) indicates training progression. The similarity between the polygon structures of the inferred and true states reflects the predicted quality.
Figure 6: Comparison results with generative model baselines in SMAC-v1 (PO) tasks with win rate over three random seeds.
Figure 7: Comparison results with generative model baselines in SMAC-v2 (PO) tasks with win rate over three random seeds.
Figure 8: Win rate with various sight range in original SMAC tasks.
Figure 9: Ablation of prior network.
Figure 10: Ablation study for diffusion step and residual blocks on the zerg 5v5 task.
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 11: Experiments of the qualitative analysis.
Vanilla MAPPO
Vanilla MAPPO (Large)
GlobeDiff
zerg 5v5
0.22 ± 0.01
0.23 ± 0.00
0.33 ± 0.02
protoss 5v5
0.21 ± 0.02
0.24 ± 0.01
0.38 ± 0.01
terran 5v5
0.16 ± 0.01
0.17 ± 0.00
0.24 ± 0.01
zerg 10v10
0.13 ± 0.01
0.15 ± 0.01
0.25 ± 0.01
zerg 10v11
0.06 ± 0.01
0.07 ± 0.01
0.12 ± 0.01
terran 10v11
0.02 ± 0.00
0.03 ± 0.00
0.07 ± 0.01
Appendix
Table 1: Comparison results between Vanilla MAPPO, Vanilla MAPPO (Large), and GlobeDiff
Figure 12: U-Net architecture in Diffusion model.
Hyper-parameters
Value
Hyper-parameters
Value
Hyper-parameters
Value
Critic Learning Rate
5e-4
Actor Learning Rate
5e-4
Use GAE
True
Gain
0.01
Optim
1e-5
Batch Size
64
Training Threads
4
Num Mini-Batch
1
Rollout Threads
8
Entropy Coef
0.01
Max Grad Norm
10
Episode Length
400
Optimizer
Adam
Hidden Layer Dim
64
GAE λ
0.95
Activation Function
Relu
PPO Epoch
15
γ
0.99
Appendix
Table 2: Hyper-parameters for MAPPO.
Prior Network pϕ
Value
Posterior Network qψ
Value
z dim
16
z dim
16
Hidden Layer Dim
1024
Hidden Layer Dim
1024
Hidden Layer Num
3
Hidden Layer Num
3
Activation Function
Relu
Activation Function
Relu
Learning Rate
2e-4
Learning Rate
2e-4
Weight Decay
1e-4
Weight Decay
1e-4
Appendix
Table 3: Hyper-parameters for prior network pϕ and posterior network qψ .
AI agents can each make locally valid decisions yet jointly produce an invalid result. We call this the global coherence problem: a failure of shared state, not merely of intelligence. Our Observation-Aliasing Impossibility Theorem gives the exact boundary. A policy can guarantee a valid action exactly when all worlds producing the same observation share an admissible action. If k indistinguishable worlds require pairwise-disjoint actions, the best randomized worst-case success is 1/k; more reasoning, roles, messages, or samples cannot recover the missing distinction. A stronger model can reason better within its context, but it cannot see beyond it. The smallest sufficient context is NP-hard to find, yet counterexample-guided refinement finds a sufficient one and recertifies it as rules change. We give local-to-global runtime semantics X = (H, C, G, F; D): topology H records overlapping scopes; category C governs state-changing actions; groupoid G keeps reversible translations; sheaf F tests whether local views glue into one world; and history D keeps only what alters legal futures. Each preserves the needed information as state is split, translated, or changed. Models propose; the harness owns shared state and governs commit. Ten studies test the boundary. On a revision benchmark, a frontier model scores 40/40 when the deciding event is visible; hidden, every arm is consistent with chance (1/3); one restored fact returns 40/40. On TeamBench, ordinary teams exceed a shared budget in 5/5 runs, a visible live count leaves 4/5, and commit enforcement leaves 0/5. In tau2-bench Telecom, current-state checks score 0.07 after silent reverts; the harness scores 1.00. Refinement keeps a sufficient context through rule changes at 28% of the all-field cost. Solvers that already own the state tie it, as predicted. Local intelligence cannot substitute for missing global state.
Communication is a key component in multi-agent reinforcement learning (MARL) for mitigating partial observability, yet prior approaches often rely on inefficient information exchange or fail to transmit sufficient state information. To address this, we propose LLM-driven Multi-Agent Communication (LMAC), which leverages an LLM's reasoning capability to design a communication protocol that enables all agents to reconstruct the underlying state as accurately and uniformly as possible. LMAC iteratively refines the protocol using an explicit state-awareness criterion, improving state recovery while narrowing differences in agents' knowledge. Experiments on diverse MARL benchmarks show that LMAC improves state reconstruction across agents and yields substantial performance gains over prior communication baselines.
Sangjun Bae, Yisak Park, Sanghyeon Lee +1
Graduate School of Artificial Intelligence, UNIST, Ulsan, South Korea.
Decentralized multi-robot coordination under partial observability remains challenging, especially in communication-free settings where agents must act solely from local sensor observations. Privileged policy distillation provides a promising approach by transferring knowledge from a globally informed oracle to sensor-constrained students. However, in multi-agent systems, the same local observation may correspond to multiple global configurations requiring qualitatively different cooperative actions, making the conditional action distribution inherently multi-modal. Standard deterministic distillation collapses these modes to their mean, often yielding invalid or hesitant actions. To address this issue, we propose CoDiMAD, a three-stage framework that trains a privileged oracle with MAPPO, constructs an offline dataset of local-observation-oracle-action pairs, and distills the oracle into decentralized students parameterized as conditional denoising diffusion probabilistic models. By approximating the conditional oracle-action distribution through the diffusion reverse process, CoDiMAD samples decisive actions from coherent coordination modes rather than averaging across them. Theoretical analysis characterizes the mode-averaging failure of deterministic distillation and the distributional recovery property of diffusion-based distillation. Experiments on three cooperative tasks show that CoDiMAD consistently outperforms direct local MARL and deterministic distillation baselines. The source code will be made publicly available upon acceptance.
Jiyue Tao, Shunheng Xin, Tongsheng Shen +2
Robotics and Control Laboratory, School of Advanced Manufacturing and Robotics, and the State Key Laboratory of Turbulence and Complex Systems, Peking University, Beijing, 100871, China · National Innovation Institute of Defense Technology, Beijing 100071, China