cs.LGMay 6, 2026

Graph-SND: Sparse Aggregation for Behavioral Diversity in Multi-Agent Reinforcement Learning

Authors: Shawn Ray

Organizations: Carnegie Mellon University

Abstract

System Neural Diversity (SND) measures behavioral heterogeneity in multi-agent reinforcement learning by averaging pairwise distances over all (n2)\binom{n}{2} agent pairs, making each call quadratic in team size. We introduce Graph-SND, which replaces this complete-graph average with a weighted average over the edges of an arbitrary graph GG. Three regimes follow: G=KnG=K_n recovers SND exactly; a fixed sparse GG defines a localized diversity measure at O(E)O(|E|) cost; and random edge samples yield an unbiased Horvitz-Thompson estimator and a normalized sample mean with O(1/m)O(1/\sqrt{m}) concentration in the sampled edge count mm. For fixed sparse graphs we prove forwarding-index distortion bounds for expanders and a spectral refinement under low-rank distance structure; for random dd-regular graphs we prove an unconditional probabilistic O~(Dmax/n)\widetilde{\mathcal{O}}(D_{\max}/\sqrt{n}) bound. On VMAS we verify recovery, unbiasedness, concentration, and wall-clock scaling, with a PettingZoo TVD panel checking non-Gaussian transfer. In a 500-iteration n=100n=100 PPO run, Bernoulli-0.10.1 Graph-SND tracks full SND while reducing per-call metric time by about 10×10\times, and frozen-policy GPU timing up to n=500n=500 follows the predicted (n2)/E\binom{n}{2}/|E| speedup. Random dd-regular expanders empirically achieve SNDGu/SND[0.9987,1.0013]\mathrm{SND}_{G}^{\mathrm{u}}/\mathrm{SND} \in [0.9987, 1.0013] at Θ(nlogn)Θ(n \log n) edges. In DiCo diversity control at n=50n=50, Bernoulli-0.10.1 Graph-SND preserves set-point tracking with paired reward differences indistinguishable from zero across nine matched cells while cutting per-call metric cost by 9.5×{\sim}9.5\times. Together, these results show that the SND aggregation bottleneck can be removed without changing the metric's semantics, yielding a drop-in sparse alternative that scales beyond complete-graph SND and supports both passive measurement and closed-loop diversity control.

Explore similar work

CardsList