cs.AIAug 27, 2026

SIGMA: Structured Noise-Effect-Aware Grouped Multi-Agent Aggregation

Authors: Li Mingqian

Organizations: School of Computer Science and technology, Tongji University Shanghai, China

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Cooperative multi-agent reinforcement learning (MARL) faces significant challenges in maintaining robust coordination under noisy observations. Although observation disturbances are often introduced independently across agents, their downstream effects on cooperative decision-making can become structured through underlying cooperation structures. We characterize this phenomenon as structured noise effects, where noise-induced decision effects exhibit local correlation among agents with stronger task-related dependencies while remaining globally heterogeneous across different agents and local structures. Existing robust MARL methods, however, rarely explicitly characterize or exploit such structure-dependent noise effects. To address this limitation, we propose SIGMA, a hierarchical collaboration framework that exploits cooperation structures to learn robust representations under noisy observations. SIGMA first organizes agents into adaptive local structures through density-based grouping and performs intra-group consensus aggregation to preserve shared task-relevant information while smoothing agent-specific representation deviations. Inter-group attention then adaptively integrates information across different groups to preserve global coordination while accommodating their heterogeneous contributions. Experiments on noisy-observation tasks in StarCraft II empirically validate the structured noise effects and demonstrate that SIGMA consistently improves robustness under observation noise while maintaining competitive performance in noise-free environments.

Figures & tables

Figure 1: Overall framework of SIGMA for robust cooperative representation learning under noisy observations.
ScenarioMethodσ=0\sigma=0σ=1.5\sigma=1.5σ=2.5\sigma=2.5σ=5.0\sigma=5.0
5m_vs_6mQMIX79.87%35.64%40.00%11.87%
HYGMA96.88%88.75%83.13%0.25%
SIGMA (Ours)95.63%94.37%90.00%66.25%
8m_vs_9mQMIX83.13%82.26%65.89%16.25%
HYGMA98.45%88.12%77.50%23.75%
SIGMA (Ours)97.24%95.00%91.87%94.37%
Table 1: Performance comparison under different observation-noise levels on SMAC.
Figure 2: Training performance of different methods under strong observation noise ( σ=5.0\sigma=5.0 ) on two SMAC scenarios.
ScenarioWithinBetweenΔρ\Delta\rhoRandom Δρ\Delta\rho
5m_vs_6m0.308±0.0120.308\pm 0.0120.076±0.0040.076\pm 0.0040.232±0.009\mathbf{0.232\pm 0.009}−0.006±0.001-0.006\pm 0.001
8m_vs_9m0.346±0.0010.346\pm 0.0010.265±0.0030.265\pm 0.0030.082±0.001\mathbf{0.082\pm 0.001}0.002±0.0000.002\pm 0.000
Table 2: Local correlation of relative Q-effects across SMAC scenarios.
ScenarioAdaptive HHRandom HHΔH\Delta HPositive ΔH\Delta H (%)
5m_vs_6m1.023±0.0011.023\pm 0.0010.771±0.0000.771\pm 0.0000.251±0.001\mathbf{0.251\pm 0.001}92%±0.00392\%\pm 0.003
8m_vs_9m1.094±0.0021.094\pm 0.0020.804±0.0000.804\pm 0.0000.290±0.002\mathbf{0.290\pm 0.002}93%±0.00293\%\pm 0.002
Table 3: Global heterogeneity of centered Q-effects across SMAC scenarios.
Figure 3: Performance comparison of SIGMA and its ablated variants under noisy observations.
More in this topic

Cooperative MARL

Cooperative multi-agent reinforcement learning (MARL) trains agents to coordinate toward shared goals under partial information and decentralized control.

See all 247 papers

Explore similar work

CardsList
Feb 25, 2026

Hierarchical Lead Critic based Multi-Agent Reinforcement Learning

Cooperative Multi-Agent Reinforcement Learning (MARL) solves complex tasks that require coordination from multiple agents, but is often limited to either local (independent learning) or global (centralized learning) perspectives. In this paper, we introduce a novel sequential training scheme and MARL architecture, which learns from multiple perspectives on different hierarchy levels. We propose the Hierarchical Lead Critic (HLC) - inspired by natural emerging distributions in team structures, where following high-level objectives combines with low-level execution. HLC demonstrates that introducing multiple hierarchies, leveraging local and global perspectives, can lead to improved performance with high sample efficiency and robust policies. Experimental results conducted on cooperative, non-communicative, and partially observable MARL benchmarks demonstrate that HLC outperforms single hierarchy baselines and scales robustly with increasing amounts of agents and difficulty.

David Eckel, Henri Meeß

Fraunhofer Institute IVI, Ingolstadt, Germany

Multi-Agent Reinforcement LearningCooperative MARLHierarchical RLSequential MARL
AskPDF
Jun 28, 2026

HiComm: Hierarchical Communication for Multi-agent Reinforcement Learning

Cooperative multi-agent reinforcement learning (MARL) often relies on communication to mitigate partial observability, yet most existing protocols treat messages as flat dense vectors detached from the structure of the observations they summarize. This design overlooks an important source of inductive bias in many cooperative environments, where observations naturally follow a hierarchy such as groups and entities. We propose HiComm, a plug-in communication module that grounds messages in the sender's hierarchical observation. HiComm is receiver-driven: the receiver issues a query, and the hierarchy is resolved through a three-stage decoding process that first selects a group, then a sender, and then an entity within that group, returning the corresponding feature slice as the message. This converts communication from unstructured vector transmission into structured information retrieval over the sender's observation hierarchy. We instantiate this mechanism with Straight-Through Gumbel-Softmax for differentiable discrete selection and a lightweight shared projection design that attaches to standard MARL pipelines. Experiments across cooperative MARL tasks with different observation structures and coordination demands show that HiComm matches or outperforms representative learned communication baselines while reducing communication volume by up to 23×23\times per receiver per episode.

Runze Zhao, Dongruo Zhou, Sumit Kumar Jha +2

Luddy School of Informatics, Computing, and Engineering Indiana University Bloomington Bloomington, IN 47408 · Department of Computer & Information Science & Engineering University of Florida Gainesville, FL 32611 · Department of Electrical Engineering and Computer Science United States Military Academy West Point, NY 10996 +1

Multi-Agent CommunicationCooperative MARL
AskPDF
Feb 23, 2026

Descent-Guided Policy Gradient for Scalable Cooperative Multi-Agent Learning

Scaling cooperative multi-agent reinforcement learning (MARL) is fundamentally limited by cross-agent noise. When agents share a common reward, each agent's learning signal is computed from a shared return that depends on all agents, so the stochasticity of the other agents enters the signal as cross-agent noise that grows with NN. Many engineering systems, such as cloud computing and power systems, have differentiable analytical models that prescribe efficient system states, providing a new reference beyond noisy shared returns. In this work, we propose Descent-Guided Policy Gradient (DG-PG), a framework that augments policy-gradient updates with a low-noise descent signal toward this reference. We prove that DG-PG reduces policy-gradient estimator variance from O(N)\mathcal{O}(N) to O(1)\mathcal{O}(1), preserves the stationary points of the cooperative objective, and achieves agent-independent sample complexity O(1/ε3)\mathcal{O}(1/ε^3). We evaluate DG-PG on cloud resource scheduling and power dispatch tasks, which involve discrete and continuous decisions, respectively. In both tasks, DG-PG converges within 10 episodes on average at every scale up to 1500 heterogeneous agents.

Shan Yang, Yang Liu

National University of Singapore

Cooperative MARLMulti-Agent Reinforcement LearningPolicy Gradient Methods
AskPDF