When Is Collective Intelligence a Lottery? Multi-Agent Scaling Laws for Memetic Drift in LLMs

Hidenori Tanaka

cs.AIMar 25, 2026

When Is Collective Intelligence a Lottery? Multi-Agent Scaling Laws for Memetic Drift in LLMs

Authors: Hidenori Tanaka

Organizations: CBS–NTT Program in Physics of Intelligence, Center for Brain Science, Harvard University · Physics of Artificial Intelligence Group, NTT Research, Inc.

Abstract

Multi-agent systems powered by large language models (LLMs) are increasingly deployed in settings that shape consequential decisions, both directly and indirectly. Yet it remains unclear whether their outcomes reflect collective reasoning, systematic bias, or mere chance. Recent work has sharpened this question with naming games, showing that even when no individual agent favors any label a priori, populations rapidly break symmetry and reach consensus. Here, we reveal the mechanism by introducing a minimal model, Quantized Simplex Gossip (QSG), and trace the microscopic origin of this agreement to mutual in-context learning. In QSG, agents maintain internal belief states but learn from one another's sampled outputs, so one agent's arbitrary choice becomes the next agent's evidence and can compound toward agreement. By analogy with neutral evolution, we call this sampling-driven regime memetic drift. QSG predicts a crossover from a drift-dominated regime, where consensus is effectively a lottery, to a selection regime, where weak biases are amplified and shape the outcome. We derive scaling laws for drift-induced polarization as a function of population size, communication bandwidth, in-context adaptation rate, and agents' internal uncertainty, and we validate them in both QSG simulations and naming-game experiments with LLM populations. Together, these results provide a framework for studying the collective mechanisms of social representation formation in multi-agent systems.

Figures & tables

Panel of Figure 1 — Figure 1: Symmetry breaking in an LLM naming game (GPT-4o, $N=24$ , $K=3$ ). (a) Mean coordination across trials (95% CI). (b) Representative label-frequency trajectories from three trials, one for each eventual winner. (c) Mean simplex trajectories conditioned on the eventual winning label.

Figure 2: Individual vs. mutual in-context learning. Top: standard in-context learning, where a single agent updates from i.i.d. tokens drawn from a stationary external distribution. Bottom: mutual in-context learning, where agents update from one another’s sampled outputs, so the population becomes its own evolving data source.

Panel of Figure 3 — Figure 3: Drift–selection crossover in multi-agent naming from GPT-4o experiments and QSG theory. Panels (a)–(c) show two-label (Dog, Cat) naming-game experiments with GPT-4o using random ordered speaker–listener pairs for a fixed referent $r$ . (a) At $N=8$ , runs show substantial run-to-run variability, and the inset shows winner counts across trials. (b) At $N=800$ , a weak asymmetry consistently selects the same winner (Cat), again reflected in the inset counts. (c) Fixation probability versus $N$ shows a finite-size crossover. (d) The corresponding tempered-sampling crossover diagram derived from QSG theory in $(T,mN/\alpha)$ ; the black curve marks $\Gamma_{T}=1$ , where $\Gamma_{T}=\frac{mN}{\alpha}\left|\frac{1}{T}-1\right|$ .

Figure 4: Quantized Simplex Gossip (QSG): interaction protocol. 1. A population of $N$ agents, each with an internal state $x_{i}\in\Delta^{K-1}$ . 2. An ordered speaker–listener pair is selected uniformly at random. 3. For a fixed referent, the speaker samples and sends a discrete message $y$ from $x_{S}$ in the quantized regimes (Hard or Top- $m$ ); Soft corresponds to transmitting the full distribution. 4. The listener updates toward the received message with adaptation rate $\alpha$ , $x_{L}\leftarrow(1-\alpha)x_{L}+\alpha y$ .

Panel of Figure 5 — Figure 5: Probabilistic naming and sampling-noise geometry. (a) A referent is represented internally as a distribution over candidate labels. (b) Agent states lie on the simplex: near-uniform, high-entropy states generate larger sampling noise under quantization, whereas peaked, low-entropy states generate less. (c) Sampling-driven drift strength on the simplex, proportional to $1-\|x\|_{2}^{2}$ , is maximal near the center and vanishes at the vertices.

Panel of Figure 6 — Figure 6: QSG simulations validate the mean-field approximation and drift identities. (a) Hard-sampling trajectories and their ensemble mean track the mean-field solution in Eq. ( 12 ) ( $N=24$ , $K=10$ , $\alpha=0.2$ ). (b) One-step excess drift from shared snapshot states $X$ , plotted against the predicted variance-injection term; dashed reference line $y=x$ ( $N=24$ , $K=10$ , $\alpha=0.5$ ). (c) Symmetric $1/m$ corollary. Under $x_{i}=\mathbf{1}/K$ , the normalized excess drift follows the theoretical line $(1-1/K)(1/m)$ ( $N=24$ , $K=10$ , $\alpha=0.5$ ).

Figure 7: Sampling-driven drift from uncertainty and bandwidth. For fixed $\alpha$ and $N$ , the drift term in Eq. ( 15 ) scales as $(1-\|x_{S}\|_{2}^{2})/m$ , increasing with uncertainty and decreasing with bandwidth $m$ .

Panel of Figure 8 — Figure 8: LLM scaling tests of QSG predictions. Top row: GPT-4o; bottom row: Claude Haiku 4.5. Panels (a,d) show $K=3$ polarization trajectories well captured by a single effective- $\alpha$ mean-field fit shared across $N$ ; panels (b,e) show early drift $\Delta U/\mathrm{step}$ decreasing near the predicted $1/N^{2}$ law; and panels (c,f) show time to consensus increasing close to dashed $cN^{2}$ fits.

Figure 9: Top-

m

scaling in GPT-4o. Early-round drift follows the predicted

1/m

law.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 10 — Figure 10: Example speaker prompt (Hard, $m=1$ , $K=3$ , $H=10$ ). Labels are synthetic 5-character strings; memory is zero-padded with <PAD> tokens. Under the delayed-reveal protocol used in all experiments, the listener prompt follows the same basic structure. The listener does not observe the speaker’s current output but updates its memory with the speaker’s label after responding.

Multi-Agent Systems

Multi-Agent Systems studies how autonomous agents coordinate, compete, communicate, and learn in shared or interdependent tasks.

See all 675 papers

Apr 3, 2026

Do Agent Societies Develop Intellectual Elites? The Hidden Power Laws of Collective Cognition in LLM Multi-Agent Systems

Large Language Model (LLM) multi-agent systems are increasingly deployed as interacting agent societies, yet scaling these systems often yields diminishing or unstable returns, the causes of which remain poorly understood. We present the first large-scale empirical study of coordination dynamics in LLM-based multi-agent systems, introducing an atomic event-level formulation that reconstructs reasoning as cascades of coordination. Analyzing over 1.5 Million interactions across tasks, topologies, and scales, we uncover three coupled laws: coordination follows heavy-tailed cascades, concentrates via preferential attachment into intellectual elites, and produces increasingly frequent extreme events as system size grows. We show that these effects are coupled through a single structural mechanism: an integration bottleneck, in which coordination expansion scales with system size while consolidation does not, producing large but weakly integrated reasoning processes. To test this mechanism, we introduce Deficit-Triggered Integration (DTI), which selectively increases integration under imbalance. DTI improves performance precisely where coordination fails, without suppressing large-scale reasoning. Together, our results establish quantitative laws of collective cognition and identify coordination structure as a fundamental, previously unmeasured axis for understanding and improving scalable multi-agent intelligence.

Kavana Venkatesh, Jiaming Cui

Department of Computer Science, Virginia Tech, Blacksburg, VA

Multi-Agent Coordination Multi-Agent LLMs Emergent Behavior in Multi-Agent Systems Heavy-Tailed Distributions

Ask PDF

Oct 14, 2026

Drivers of LLM Consensus Formation Under Competing Incentives: A Simulation Study

Large Language Models (LLMs) are increasingly deployed in multi-agent environments that produce unanimous agreement, both in intended interactions that need coordination and in unintended emergent interactions. Consequently, understanding how LLMs with competing incentives reach consensus is crucial to better interpret and design LLM collectives. We introduce an evaluation, based on the Network Consensus game, to assess the mechanisms driving this process. Here, a network of agents must agree on one of two colours to receive a payoff, but agents are offered different payoffs for the agreed colour. We vary components such as communication access and network structure and assess six open-source models and one closed model (ChatGPT 5.6-sol). We find that open-source LLMs reach consensus less often than humans in comparable experiments and only if messages are communicated across the global network. ChatGPT 5.6-sol, however, is able to reach consensus with communication limited to its neighbours and with no communication ability. This is achieved through lower stubbornness and higher majority-following tendencies, and by providing more information about its neighbours' states in messages. However, situating agents in a highly clustered graph reduces consensus, regardless of these traits, demonstrating that both system design and individual model traits drive collective outcomes.

Isabel Jones, Matthew Edwards, Janina A. Hoffmann

University of Bristol Bristol, United Kingdom · University of Bath Bath, United Kingdom

Multi-Agent Consensus Multi-Agent LLM Systems Large Language Model-Based Social Simulation Multi-Agent Communication Normative Conformity in Language Models

Ask PDF

Aug 3, 2026

Emergence of Biased Consensus in Multi-Agent LLM Debates

Multi-agent LLM debates achieve strong performance on decision-making tasks as well as problem-solving benchmarks, yet their safety and fairness risks remain poorly understood. Notably, interaction can amplify the biases of single LLMs, raising concerns for real-world deployment. We identify the emergence of collective (often biased) norms in multi-agent LLM debates and show that noise (e.g., LLM sampling temperature) is a key driver. To explain this, we propose an analytical framework drawing on physics-inspired theoretical models of social dynamics. We predict a phase transition to collective bias when conformity surpasses a critical threshold given the LLMs' initial bias and debate noise. We test the theoretical predictions through controlled experiments and observe a finite-size crossover consistent with an underlying phase transition. We further find that agent heterogeneity suppresses emergence by smoothing (rounding) this transition. Finally, we show that these insights generalize to realistic decision-making tasks, including investment decisions and LLM-as-a-judge evaluation.

Maya Okawa

CBS-NTT Program in Physics of Intelligence, Harvard University · Physics of Artificial Intelligence Group, NTT Research, Inc.

Opinion Dynamics Social Bias in Language Models Multi-Agent Debate Multi-Agent Systems Multi-Agent Consensus Emergent Behavior in Multi-Agent Systems

Ask PDF

Abstract

Multi-Agent Systems

Explore similar work

Do Agent Societies Develop Intellectual Elites? The Hidden Power Laws of Collective Cognition in LLM Multi-Agent Systems

Drivers of LLM Consensus Formation Under Competing Incentives: A Simulation Study

Emergence of Biased Consensus in Multi-Agent LLM Debates