Existing large language model (LLM) based multi-agent systems have been shown to scale poorly, and we identify three problems: reading burden that grows with the group, useful information diluted by shared work of unknown quality, and fragility to failed agents. We argue that the problems are largely due to agent-centric designs: the coordination structure is wired into fixed workflows, central coordinators, or peer-to-peer messaging, and recent blackboard-style methods that move the coordination into a shared medium relieve only the last problem. To close the gaps, we build ScaleMAS, where asynchronous agents interact with a shared datastore of linked messages via three operations: publishing messages with optional references to earlier ones that they build on, searching for messages by content, and visiting messages by address. Search returns at most k messages, so an agent's read stays bounded at any group size; results are ranked by relevance and by how often they are cited, which we find to be an effective proxy for quality; and messages persist after their producers leave, so the group carries on when individual agents fail. Across four agentic tasks, ScaleMAS outperforms agent-centric and blackboard-style baselines at lower inference compute, continues to scale up to 128 agents, and tolerates losing agents midway.
Figures & tables
Figure 1: ScaleMAS shows stronger scalability. Task performance v.s. inference compute per example (log scale) with Qwen3.5-9B. Each point is one group size, scaled from 1 to 2, 4, 8, and 16 agents; AOrchestra, whose coordinator decides how many sub-agents to launch, is a single point. At matched compute, ScaleMAS reaches higher performance than Multi-Agent Debate (all-to-all message-passing), AOrchestra (coordinator-based), DeLM (blackboard-style shared context), and parallel ReAct agents without communication, and it improves with group size up to eight agents on every benchmark.
Figure 2: Two design principles for multi-agent coordination. Left: agent-centric designs wire coordination structure into the agents. A workflow-based system fixes agents’ successors, a coordinator-based system routes results through one main agent, and a message-passing system lets agents address others. Right: substrate-centric designs place the coordination in a shared medium. Messages are pushed to a datastore, organized with certain structures; agents interact only with the medium by writing to and reading from it.
Figure 3: A minimal example of two agents working through the substrate on a deep research query. Each row is one agent’s trajectory, with the three operations in blue and the task’s own tools in gray. A dashed stem marks the step that published a message, and a dotted arrow points from a referenced message to the message that cites it.
modules/solvers/solveset.rst
modules/sets.rst (edited)
… Set objects conform to Pythonic conventions when possible, i.e., x in A and for i in A both work when they can be computed .
… By contrast, x in y always gives TypeError for SymPy sets, even when the membership could easily be computed.
Table 1: Example of a cross-file contradiction from the SymPy documentation corpus. The injected text is in red, and the clashing claims are underlined.
Figure 5Figure 6Figure 7Figure 8
Table 2: Ablating cross-agent communication. Performance (%) with four Qwen3.5-9B agents. Private substrate partitions the datastore by author, so each agent reads only its own messages; w/o substrate runs independent ReAct agents.
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Project
Files
Snippets
Project
Files
Snippets
CPython
507
2,806
pandas
78
392
Docker
930
2,074
PyTorch
245
597
Godot
1,586
4,512
Ray
534
1,347
JAX
233
920
SciPy
339
635
Kubernetes
1,630
3,115
scikit-learn
102
475
Linux kernel
3,433
8,319
SQLAlchemy
152
583
Appendix
Table 3: The sixteen documentation corpora of Contradiction Audit, with the number of files kept after filtering and the number of snippets in the full corpus.
Unedited snippet
Edited snippet
Why they conflict
CPython library/math.rst
… atan2(y, x) : atan(y / x) , in radians …
… atan2(y, x) : Return atan(y / x) , in degrees. The result is between -180 and 180 . …
The function summary gives the result of atan2 in radians; its own entry now gives it in degrees.
--oidc-issuer-url : URL of the provider that allows the API server to discover public signing keys. Only URLs that use the https:// scheme are accepted.
… The URL scheme may be either http or https ; plain http issuers are accepted.
An http issuer URL is rejected in one and accepted in the other.
LLVM ProjectGovernance.md
… An individual cannot serve on two area teams . Members of an area team are elected for 1 year terms.
… Elections for area teams occur in January every third year, and members elected in these elections serve three-year terms.
Area team members serve one-year terms in one and three-year terms in the other.
Appendix
Table 4: Contradiction Audit examples within one file , one per corpus. Both snippets come from the file named after the project. The injected text is in red, and … marks omitted text.
Unedited snippet
Edited snippet
Why they conflict
Docker
guides/testcontainers-java-lifecycle.md … Even without explicitly stopping the container in @AfterAll , Testcontainers uses the Ryuk container to clean up containers automatically when the JVM exits.
guides/testcontainers-java-getting-started.md … Note that Testcontainers never removes containers on its own, so you must always stop the container explicitly in @AfterAll ; otherwise it keeps running even after the JVM exits.
Testcontainers removes containers automatically when the JVM exits in one, and never removes them in the other.
Godot
community/asset_library/submitting_to_assetlib.rst … No submodules , or any submodules must be non-essential. GitHub does not include submodules in the downloaded ZIP file, so if the asset needs the contents of the submodule, your asset won’t work.
community/asset_store/submitting_to_asset_store.rst … Submodules are allowed, even essential ones. GitHub includes submodules in the downloaded ZIP file, so if the asset needs the contents of the submodule, it will still work.
GitHub’s ZIP download leaves out submodules in one and includes them in the other.
JAX
export/export.md … Note that the rehydrated function does not support any other transformations, e.g., forward-mode AD (jvp), or jax.vmap .
new_docs/501/export.md … Note that the rehydrated function also supports all other transformations, e.g., forward-mode AD (jvp), and jax.vmap .
The rehydrated function supports no other transformations in one, and all of them in the other.
Appendix
Table 5: Contradiction Audit examples across two files , one per corpus. Each excerpt is headed by the file its snippet comes from. The injected text is in red, and … marks omitted text.
Setting
BrowseComp-Plus
Contradiction Audit
FATE
SWE-bench Verified
Messages returned per search ( k )
8
Messages per visit call
8
Retrieval function, BM25 k1 and b
BM25, 1.5 and 0.75
Citation weight α , γ , s0
0.3 , 0.98 , 0
References per message (maximum)
32
Tokens per message (sub-task and artifact)
256
256
1,024
1,024
Appendix
Table 6: Settings of ScaleMAS in all experiments unless stated otherwise. The same temperature, generation limit, and random seeds are shared by every other baseline.
BrowseComp-Plus
FATE-H
Contradiction Audit
Form
Case 1
Case 2
Case 3
Case 4
Case 5
Case 6
Case 7
Case 8
Relay
∙
∙
Correction
∙
∙
∙
Selection by citation weight
∙
∙
Division of a proof
∙
∙
Matching across agents
∙
∙
Appendix
Table 7: Forms of coordination in the eight cases.
Large Language Model (LLM) multi-agent systems are increasingly deployed as interacting agent societies, yet scaling these systems often yields diminishing or unstable returns, the causes of which remain poorly understood. We present the first large-scale empirical study of coordination dynamics in LLM-based multi-agent systems, introducing an atomic event-level formulation that reconstructs reasoning as cascades of coordination. Analyzing over 1.5 Million interactions across tasks, topologies, and scales, we uncover three coupled laws: coordination follows heavy-tailed cascades, concentrates via preferential attachment into intellectual elites, and produces increasingly frequent extreme events as system size grows. We show that these effects are coupled through a single structural mechanism: an integration bottleneck, in which coordination expansion scales with system size while consolidation does not, producing large but weakly integrated reasoning processes. To test this mechanism, we introduce Deficit-Triggered Integration (DTI), which selectively increases integration under imbalance. DTI improves performance precisely where coordination fails, without suppressing large-scale reasoning. Together, our results establish quantitative laws of collective cognition and identify coordination structure as a fundamental, previously unmeasured axis for understanding and improving scalable multi-agent intelligence.
Kavana Venkatesh, Jiaming Cui
Department of Computer Science, Virginia Tech, Blacksburg, VA
Large language model (LLM) multi-agent systems can scale along two distinct dimensions: by increasing the number of agents and by improving through accumulated experience over time. Although prior work has studied these dimensions separately, their interaction under realistic cost constraints remains unclear. In this paper, we introduce a conceptual scaling view of multi-agent systems that jointly considers team size and lifelong learning ability, and we study how memory design shares this landscape. To this end, we propose LLMA-Mem, a lifelong memory framework for LLM multi-agent systems under flexible memory topologies. We evaluate LLMA-Mem on MultiAgentBench across coding, research, and database environments. Empirically, LLMA-Mem consistently improves long-horizon performance over baselines while reducing cost. Our analysis further reveals a non-monotonic scaling landscape: larger teams do not always produce better long-term performance, and smaller teams can outperform larger ones when memory better supports the reuse of experience. These findings position memory design as a practical path for scaling multi-agent systems more effectively and more efficiently over time.
Shanglin Wu, Yuyang Luo, Yueqing Liang +4
Emory University · Illinois Institute of Technology · University of Notre Dame +1
The burgeoning field of LLM-based Multi-Agent Systems (MAS) promises to tackle complex tasks through collaborative intelligence, yet fundamental questions regarding their scaling behavior and intrinsic collective dynamics remain underexplored. This paper systematically investigates how the performance of a homogeneous MAS evolves as the number of agents increases, isolating the variable of collaboration from model or knowledge heterogeneity. We propose the Sequential Iterative Multi-Agent System (SIMAS) framework, a minimalist architecture centered on sequential inter-agent communication, to clearly observe scaling effects. Through extensive experiments across diverse tasks and model scales, we establish that MAS performance does not scale monotonically with agent count but follows a pattern of diminishing returns, governed by a trade-off between collaborative synergy and coordination overhead. Our findings reveal that effective MAS requires a sufficiently capable base LLM, that task type critically modulates the optimal agent count, and that collective intelligence is an emergent property contingent on strategic interaction design rather than a guaranteed outcome of agent plurality. The performance degradation stems coordination overhead rather than merely long-context failure, and the scaling tendency generalizes across interaction architectures like structured debate topologies. This work provides a foundational understanding of MAS scaling laws, offering practical guidance for designing efficient collaborative systems and challenging the prevailing assumption that more agents invariably lead to better performance.