cs.AINov 13, 2026

More Agents, More Samples, or More Thinking? Untangling Coordination, Ensembling, and Test-Time Compute in Multi-Agent LLMs

Authors: Seyed Mahmoud Sajjadi Mohammadabadi

Organizations: Department of Computer Science and Engineering, University of Nevada, Reno, NV 89557, USA

Abstract

Evaluations of multi-agent language models often compare communicating architectures against a single greedy pass. This practice confounds coordination with sample multiplicity and inference compute. We synthesize nine primary empirical evaluations comprising 516 unique multi-agent configurations organized along a resource vector R with six dimensions and a baseline ladder with four categories. Against baselines with a single pass (Category A), multi-agent accuracy differences are negative overall and near zero when omitting the largest contributing study. When evaluated against baselines matched on sample count (Category B) or resource budget (Category C), multi-agent systems average -0.042 in accuracy, and multilevel random effects modeling on the log odds scale remains negative across sensitivity checks. Multi-agent accuracy equals or falls below the baseline in 74% of these comparable contrasts, and study means are nonpositive in almost all evaluated studies. Across studies reporting both baselines, moving from a greedy pass to a sampled baseline shifts accuracy differences by -0.038 on average. Because communication adds no information when agents share identical evidence, positive returns require partitioned evidence, external verification, or error diversity across models. We provide a reviewer checklist and accounting utilities.

Figures & tables

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Benchmarking Open-Ended Multi-Agent Coordination in Language Agents

    Jun 6, 2026Kale-ab Abebe Tessera, Andras Szecsenyi, Cameron Barker +7Multi-Agent CoordinationLLM Agent Evaluation

  2. TeamTR: Trust-Region Fine-Tuning for Multi-Agent LLM Coordination

    May 1, 2026Yi Xie, Siao Liu, Falong Fan +3Multi-Agent System OptimizationMulti-Agent Coordination

  3. Silo-Bench: A Scalable Environment for Evaluating Distributed Coordination in Multi-Agent LLM Systems

    Mar 1, 2026Yuzhe Zhang, Feiran Liu, Yi Shan +8Multi-Agent LLM SystemsMulti-Agent Coordination