Predicting the scale limits of social mechanisms in agent societies

Zengqing Wu; Chuan Xiao

cs.MAAug 24, 2026

Predicting the scale limits of social mechanisms in agent societies

Authors: Zengqing Wu, Chuan Xiao

Organizations: University of Osaka Osaka, Japan

Abstract

Societies of interacting language-model agents offer a controllable and repeatable way to study collective behaviour at scales that would be difficult to test with people. Their scientific value, however, depends on whether a social mechanism that works in a small group still operates when thousands of agents interact, and testing this directly requires costly large-scale runs. Here we introduce an audit that predicts a mechanism's fate as a population grows. It asks how often the mechanism can act, whether agents use the information it supplies, and whether the measurement itself creates apparent scale effects. Controlled experiments show that a single structural term can decide whether reciprocity, consensus or punishment survives scaling. For gossip, the population at which the mechanism fails is set by the reach and lifetime of its messages. In language-model societies, agents respond not only to social information but to how it is expressed: counts and percentages led to different scale behaviour. Predictions made before execution held on third-party code and a second model family, while a failed prediction exposed the boundary of the finding. The audit provides a prospective way to decide which social mechanisms can be interpreted across population scales.

Figures & tables

Figure 1: Prospective audit and adjudication workflow. Before any scale outcome exists, the mechanism implementation yields structural quantities

q_{j}(N)

, interface probes identify the effective inputs and per-encounter gains

\Delta_{j}

, and protocol checks screen the observation plan. The completed path table supports per-path (Level 1) predictions, and an aggregate (Level 2) prediction is issued only when all active paths are enumerated, their

q_{j}

and

\Delta_{j}

identified, the gains stable, and interactions specified or negligible; otherwise the aggregate is declared unidentified. Predictions are logged with curve class, applicable range and failure assumption before execution. After execution, a nonzero composition residual diagnoses an interaction or missing path, and any explanation outside the frozen checklist is recorded as an audit failure. The path table shows example entries with their checklist codes.

Figure 2: Single structural terms set the scale class in rule-based societies. (A) Direct reciprocity collapses where measured re-encounter falls below threshold, while network reciprocity on fixed degree stays flat. (B) Consensus under global sampling holds at every size, while the same update rule under local sampling on a ring freezes from roughly

N=30

upward. (C) Independent aggregation improves with population, while sequential social influence pins accuracy near the analytic plateau of

0.692

. (D) A reputation mechanism whose update step carries a

1/N

dilution factor loses its effect as the population grows, while the per-event variant holds. (E) Replacing organic transmission with a public record flips the scale class of the same gossip mechanism with individual responses held fixed. Points are means over seeds (Methods).

Figure 3: Failure scale against the product of fan-out and mark lifetime

f\tau

. (A) In the first design, which used memory bounded at twenty marks and a relative crossing threshold, the exponent fitted against the raw lifetime is

0.245

, outside the registered band. The bound can truncate the effective lifetime

\tau_{\mathrm{eff}}

below

\tau

. (B) In the amended memory-unbounded design, which uses fixed absolute crossing thresholds and has

\tau_{\mathrm{eff}}=\tau

, the exponent is

1.00

with bootstrap intervals inside

[0.97,1.04]

over ten seeds at three registered thresholds.

Figure 4: Single-decision probe rates by cell and engine. Points are give rates with

n

between 24 and 48 decisions per cell, and horizontal bars are 95% Wilson intervals. A targeted report about the current partner produces sanctioning, an identically worded report about a third party does not, and count-format reports act without detected dependence on the stated population size. The panel shows the six primary probe cells on the three engines with per-cell reruns. The remaining engines and cells are tabulated in Supplementary Information. Engine labels and run dates are listed in the Supplementary Information model manifest.

Figure 5: Count and percentage formats lead to different stated-size dependence. (A) Dose lattice in the count format on the primary engine, give rates with $n=24$ decisions per cell. Response tracks the number of reports, while no stated-size dependence was detected in this count-format lattice. Anti-diagonal cells state identical proportions. (B, C) The same lattice with explicit percentages on two model families. The stated size now acts, through a sharp threshold on one engine and a graded curve on the other. (D) Baseline-adjusted report $\times\log N$ interactions with 95% confidence intervals across all six lattices, a post hoc analysis specified after the registered per-lattice fits (Methods). Count-format intervals contain zero on every engine, while both percentage-format interactions are negative and exclude zero. F marks Firth fits under separation.

Figure 6: Predictions filed in advance and their outcomes. (A) Third-party cooperation library on a frozen parameter grid. Lines are the filed predictions and open markers the observed values, which agree exactly on this deterministic system. The shaded band is the filed sign-flip window. (B) Cross-model-family test filed before execution, with the report count fixed at three. Curves are observed give rates for the graded and threshold arms, and the blue ticks with bands are the two interior predictions filed in advance with their

\pm 0.20

bands, both met by the graded arm. The threshold arm failed its registered flatness criteria.

Appendix figures & tables23 assets

Supplementary material from the paper’s appendix.

Appendix

Code	Name	Rule and diagnostic	Use in predictions
C1	Scale-comparable observable	The observable must have an $N$ -comparable interpretation. Extensive counts and whole-population conjunctions, such as $P(\text{all agree})=p^{N}$ , carry mechanical size dependence and require normalisation or a separate null model	prediction undefined if violated
C2	Activatability	Mechanism must activate at the reference scale or under exogenous seeding. Zero ignition only at large $N$ is a positive A2-class result, zero ignition everywhere is an invalid test	validity gate
C3	Causal neighbourhood	Neighbourhood within the measurement horizon must clear the boundary and stay locally isomorphic across $N$ . On rings this reduces to $N\gtrsim 2rT$ . Globally coupled systems need separate argument	validity gate for small $N$
C4	Endgame isolation	Final rounds of known horizons excluded or reported separately	measurement window
C5	Dynamic range	No ceilings or floors. Saturated cells are retuned and excluded from adjudication	measurement window
E1	Identity-indexed input	Decision function indexed by the identity of the object	routes to A2 coverage audit

Appendix

Table S1: The closed checklist, part one: protocol checks (C1–C5) and input identity rules (E1–E4). Every prediction cites codes from this list, and an explanation outside it is an audit failure.

Code	Name	Rule and diagnostic	Use in predictions
A1	Re-encounter probability	Pair re-interaction probability, under random matching $\sim 1/(N{-}1)$ . Diagnostic: count the pair generator	decay or transition of $q_{j}$
A2	Targeted coverage	Probability that a decision point holds information about the current object, $\sim\lambda\,\tau_{\mathrm{eff}}/N$ . Diagnostic: measure the informed fraction	decay or transition of $q_{j}$
A3	Monitoring reach	Probability that behaviour is observed by a potential sanctioner. Diagnostic: count observation edges	decay or transition of $q_{j}$
A4	Time budget	Convergence time $\tau(N)$ against a fixed horizon $T$ . Diagnostic: measure $\tau(N)$ , discriminate by extending $T$	decay or transition of $q_{j}$
A5	Update-step dilution	A $1/N$ factor inside the update step of a global aggregate. Diagnostic: read the update rule	decay or transition of $q_{j}$
I1	Memory-flux lifetime	$\tau_{\mathrm{eff}}=(1/\tau+\lambda/M)^{-1}$ , harmonic mean-field approximation under independent hazards	shifts turning scale only

Appendix

Table S2: The closed checklist, part two: degradation classes (A1–A5), correction terms (I1–I3), and compensation channels (B1–B4). Correction terms shift the absolute position of a turning scale and never justify a decay prediction on their own. Compensation channels require the stated quantitative diagnostic at the time of invocation.

System	Path	Input (E)	$q_{j}(N)$ (source)	$\Delta_{j}$ (source)	Codes	Filed class	Outcome
Pair A, direct (anchor)	remembered re-encounter	partner identity (E1)	$\sim 1/N$ under random matching (code)	full conditional defection (code)	A1	transition	collapse at $N{=}20$ – $40$ at threshold
Pair A, network	neighbour clustering	neighbour payoffs	$N$ -independent local structure (code)	imitation of best neighbour (code)	–	flat	share stable near $0.1$ to $10^{5}$
Pair B, local (anchor)	convergence within budget	neighbour opinions	$\tau(N)$ outruns fixed $T$ (measured)	majority adoption (code)	A4	transition	frozen from $N\approx 30$
Pair B, global	mean-field sampling	sampled opinions	$N$ -independent (code)	same (code)	–	flat	flat to $10^{5}$
Pair C, sequential (anchor)	informative decisions	public record	saturates near four (measured)	cascade adoption (code)	–	flat below analytic anchor	pinned near $0.689$ vs $0.692$
Pair C, independent	private signals	own signal	grows with $N$ (code)	majority count (code)	–	growth	$0.75\to 1.00$

Appendix

Table S3: Path tables, rule-based systems. Gains derive from code. Sources in parentheses. Classes are flat, decay, or transition (decay crossing a registered threshold).

System	Path	Input (E)	$q_{j}(N)$ (source)	$\Delta_{j}$ (source)	Codes	Filed class	Outcome
Count channel	targeted report	report count (E3, probed)	stated $N$ not consumed, no dilution path (probes)	give $1.00\to 0.00$ on targeted report. Dose $+0.48$ per $\log k$ (probes)	E3	flat in stated $N$	no detected stated-size dependence, four engines
Percentage channel	stated ratio	explicit percentage (E3, probed)	report always delivered in probes; its stated ratio $k/N$ falls with $N$	threshold (Firth $+4.8/-4.1$ ) or graded ( $+0.66/-0.48$ ) response to the ratio (probes)	E3	decay in stated $N$	negative interactions, both model families
Composed mechanism	ignition, transmission, sanction	identity marks (E1)	fired at $N{=}8$ . Zero ignition at $N{=}64$ over $945$ decisions	probe values above	A2, C2, C4	window-qualified composition	residual zero in window. Whole-run verdicts failed on final round
Third-party library	within-match re-encounter	partner identity (E1)	repeat turns per opponent fall under per-capita budget (code)	strategy payoff structure (code)	A1, A4	sign flip in $N{=}24$ – $48$	$20/20$ filed quantities exact

Appendix

Table S4: Path tables, language model and external systems. Gains are probe-measured on the stated engines. Give rates with

n

per cell in the result tables.

Engine (as reported)	Resolved identity	Access	Notes and run dates
Primary engine, early probes	claude-sonnet-4-6	vendor CLI	targeted, placebo, count and proportion probes and wording robustness (2026-07-12). Envelopes not logged in this early batch
Primary engine, later runs	claude-sonnet-5	vendor CLI	confirmatory rerun, count lattice, factor separations, cross-model-family test, percentage lattice (2026-07-13 to 07-20). Full envelopes. Provider fixes sampling temperature
Opus family	claude-opus-4-8	vendor CLI	cross-model probes (2026-07-12) and factor-separation reruns (2026-07-19)
GPT-5.5	gpt-5.5 via the codex agent CLI	vendor CLI	cross-model probes (2026-07-12) and count lattice (2026-07-22). Reasoning-capable configuration
Gemini	gemini-3.5-flash	public API	cross-family probes, count lattice, temperature sweep at $t\in\{0,0.7,1.0\}$ (2026-07-19 to 07-22)
DeepSeek	deepseek-v4-flash	public API	accessed with reasoning mode on (reasoner) and off (chat) over fixed weights. Percentage and count lattices, cross-model-family test (2026-07-19 to 07-22)

Appendix

Table S5: Engines, engine labels with resolved identities where logged, access routes, and run dates. Every level-type quantity in the paper carries the most specific of these labels and a run date in the released ledger. Fully resolved provider identities are available for the envelope-logged runs.

	$N=40$	$N=200$	$N=1000$
$k=0$	0.708 (24)	0.500 (24)	0.583 (24)
$k=1$	0.958 (24)	0.917 (24)	1.000 (24)
$k=5$	0.875 (24)	0.625 (24)	0.792 (24)
$k=25$	0.750 (24)	0.708 (24)	0.792 (24)

Appendix

Table S6: Count-format dose lattice, primary engine (claude-sonnet-5, wording W3). Give rates with

n

in parentheses.

	$N=40$	$N=200$	$N=1000$
$k=0$	0.667 (24)	0.750 (24)	0.875 (24)
$k=1$	0.333 (24)	0.500 (24)	0.292 (24)
$k=5$	0.167 (24)	0.250 (24)	0.174 (23)
$k=25$	0.208 (24)	0.500 (24)	0.167 (24)

Appendix

Table S7: Count-format lattice, deepseek-reasoner (calibrated wording W3). Give rates with

n

in parentheses.

	$N=40$	$N=200$	$N=1000$
$k=0$	1.000 (24)	1.000 (24)	1.000 (24)
$k=1$	0.917 (24)	1.000 (24)	0.708 (24)
$k=5$	0.292 (24)	0.458 (24)	0.250 (24)
$k=25$	0.083 (24)	0.208 (24)	0.208 (24)

Appendix

Table S8: Count-format lattice, gemini-3.5-flash (calibrated wording W3). Give rates with

n

in parentheses.

	$N=40$	$N=200$	$N=1000$
$k=0$	0.958 (24)	1.000 (24)	0.917 (24)
$k=1$	0.833 (24)	0.792 (24)	0.833 (24)
$k=5$	0.667 (24)	0.625 (24)	0.792 (24)
$k=25$	0.583 (24)	0.458 (24)	0.708 (24)

Appendix

Table S9: Count-format lattice, GPT-5.5 (calibrated wording W1). Give rates with

n

in parentheses.

	$N=40$	$N=200$	$N=1000$
$k=0$	0.833 (24)	0.708 (24)	0.583 (24)
$k=1$	1.000 (24)	1.000 (24)	1.000 (24)
$k=5$	1.000 (24)	1.000 (24)	1.000 (24)
$k=25$	0.000 (24)	0.958 (24)	1.000 (24)

Appendix

Table S10: Percentage-format lattice, claude-sonnet-5. Give rates with

n

in parentheses.

	$N=40$	$N=200$	$N=1000$
$k=0$	0.833 (24)	0.458 (24)	0.583 (24)
$k=1$	0.667 (24)	0.958 (24)	0.625 (24)
$k=5$	0.304 (23)	0.750 (24)	0.792 (24)
$k=25$	0.042 (24)	0.333 (24)	0.542 (24)

Appendix

Table S11: Percentage-format lattice, deepseek-reasoner. Give rates with

n

in parentheses.

	$N=8$	$N=40$	$N=200$	$N=1000$
No information	0.833 (24)	0.833 (24)	0.792 (24)	0.750 (24)
Proportion-format report ( $k=3$ )	0.000 (24)	0.500 (24)	0.708 (24)	0.875 (24)

Appendix

Table S12: Cross-model-family test filed before execution, deepseek-reasoner (graded arm). Give rates by stated population.

	$N=8$	$N=40$	$N=200$	$N=1000$
No information	1.000 (24)	0.958 (24)	0.792 (24)	0.792 (24)
Proportion-format report ( $k=3$ )	0.583 (24)	1.000 (24)	1.000 (24)	1.000 (24)

Appendix

Table S13: Cross-model-family test filed before execution, claude-sonnet-5 (threshold arm). Give rates by stated population.

	$t=0$	$t=0.7$	$t=1.0$
No information	1.000 (24)	1.000 (24)	1.000 (24)
Targeted report	0.000 (24)	0.000 (24)	0.000 (24)
Count report, $N=1000$	0.000 (24)	0.000 (24)	0.000 (24)

Appendix

Table S14: Temperature sweep, gemini-3.5-flash. Give rates at three sampling temperatures.

Figure S1: Peer review at forty-fold scale. All series are descriptive. (A) Share of on-time reviews arriving in the final

72

hours of the review window, against the registered threshold of twice the uniform-rate benchmark. (B) Share of reviews in the sharpest calendar and assignment-relative windows for ICLR 2025 and TMLR:

72.9

versus

1.24

percent in the calendar window and

35.4

versus

6.8

percent in the per-paper window. (C) Active formal coordination devices coded from ten years of policy documents. Their composition shifts from feedback and monitoring toward admission control and sanctions as submissions grow.

Half	Area	Submissions	Reviews	Clust.
D	alignment, fairness, safety, privacy, and societal co…	1048	4225	0.586
D	applications to computer vision, audio, language, and…	1373	5528	0.594
D	applications to physical sciences (physics, chemistry…	543	2204	0.592
D	applications to robotics, autonomy, planning	244	973	0.624
D	generative models	1063	4278	0.603
D	learning theory	365	1460	0.609

Appendix

Table S15: Deadline clustering by primary area, ICLR 2025. D = development half, V = validation half. Clust. = share of on-time reviews arriving in the final 72 hours.

Venue	Roots	Reviews	Official figure	Note
ICLR 2024	7,404	28,028	7,262 submissions reported	reviewed forums equal official count exactly
ICLR 2025	11,672	46,748	11,672 / 11,603	roots minus desk = 11,602, Fact Sheet 11,603
ICLR 2026	19,814	75,859	19,525 valid; 779 desk	API desk count 908 at extraction; enforcement continued
TMLR	7,576	21,454	about 4,026 searchable	invitation enumeration more complete

Appendix

Table S16: Extraction reconciliation. Roots are unique submission forums enumerated by invitation; reviews are public official reviews with trusted timestamps.

Format	Engine	Fit	$n$	$R$	$R\log k$	$\log N$	$R\times\log N$
count	sonnet (primary)	GLM	288	$-1.18$ $[-3.75,1.39]$	$+0.48$ $[0.18,0.77]$	$+0.16$ $[-0.20,0.52]$	$-0.16$ $[-0.62,0.29]$
count	reasoner	GLM	287	$-0.22$ $[-2.80,2.35]$	$+0.13$ $[-0.10,0.35]$	$-0.37$ $[-0.81,0.07]$	$+0.41$ $[-0.08,0.90]$
count	gemini	Firth	288	$+2.38$ $[-6.06,10.81]$	$+1.09$ $[0.80,1.38]$	$+0.00$ $[-1.52,1.52]$	$+0.08$ $[-1.47,1.62]$
count	gpt-5.5	GLM	288	$+4.23$ $[-1.50,9.96]$	$+0.36$ $[0.13,0.59]$	$+0.34$ $[-0.60,1.28]$	$-0.47$ $[-1.44,0.50]$
pct	reasoner	GLM	287	$+3.54$ $[1.08,6.01]$	$+0.66$ $[0.41,0.90]$	$+0.34$ $[-0.04,0.72]$	$-0.82$ $[-1.27,-0.38]$
pct	sonnet	Firth	288	$+6.43$ $[-1.70,14.56]$	$+4.83$ $[2.33,7.33]$	$+0.37$ $[-0.03,0.78]$	$-4.50$ $[-6.58,-2.41]$

Appendix

Table S17: Baseline-adjusted logistic models

\mathrm{keep}\sim 1+R+R\log k+\log N+R\log N

over all lattice decisions including

k=0

baselines. Estimates with Wald 95% intervals. The fit column reports a generalized linear model (GLM) or, under separation, Firth penalised likelihood. The

R\times\log N

column is the quantity reported in the main text.

Pool	Arm	$N$	Payoff advantage per turn
II Grudger	per-capita budget	6	$+0.550$
II Grudger	per-capita budget	12	$+0.364$
II Grudger	per-capita budget	24	$+0.087$
II Grudger	per-capita budget	48	$-1.575$
II Grudger	per-capita budget	96	$-1.537$
II Grudger	fixed match length	6	$+0.300$

Appendix

Table S18: The twenty registered quantities of the prospective third-party code test. Predictions for all twenty were filed in the time-ordered ledger before the experiment ran and all twenty observed values agreed exactly with the filed values on this deterministic system, so filed and observed columns coincide and are shown once.

Registered criterion	Graded arm	Flatness arm
Give rate strictly monotone in $N$	pass	–
Anchor: give $\leq 0.2$ at $N=8$	pass	–
Anchor: give $\geq 0.8$ at $N=1000$	pass	–
Interior: $0.396\pm 0.20$ at $N=40$	pass	–
Interior: $0.682\pm 0.20$ at $N=200$	pass	–
Give-rate rise of at least $0.4$ across the tested range	pass	–

Appendix

Table S19: Per-criterion adjudication of the cross-model-family test filed before execution. The graded arm ran on deepseek-reasoner and the companion flatness arm on claude-sonnet; criteria are quoted from the ledger entry. Criteria not applicable to an arm are marked –.

Registered criterion	gemini-3.5-flash	gpt-5.5	deepseek-reasoner
(i) dose effect $\beta_{\log k}>0$ , $\|z\|>1.96$	pass	pass	fail
(ii) 95% CI for $\beta_{\log N}$ contains $0$	pass	pass	pass
(iii) anti-diagonal give-rate spread $>0.15$	pass	pass	pass

Appendix

Table S20: Registered criteria for the count-format lattice on further engines, filed before data collection. Criterion (iii) requires the anti-diagonal cells, which state identical proportions, to differ, ruling out ratio reading. The engine failing (i) responded to the presence of reports rather than their number and is recorded as consistent rather than confirmed in the main text.

Engine	B	T	X	Count	Pct 8	Pct 1000
claude-sonnet-4-6	1.00	0.00	1.00	0.00	0.96	–
claude-opus-4-8	1.00	0.50	1.00	0.75	0.96	–
gpt-5.5	0.92	0.00	0.96	0.00	0.00	0.17
gemini-3.5-flash	1.00	0.00	1.00	0.00	0.00	0.04
deepseek-chat	1.00	1.00	0.96	0.25	0.83	0.96
deepseek-reasoner	0.92	0.00	0.75	0.00	0.17	0.88

Appendix

Table S21: Cross-engine give rates for the core probe cells (wording V1,

n=24

decisions per cell). B, no information. T, targeted report about the current partner. X, identical report about a third party. Count, count-format report at stated

N{=}1000

. Pct 8 and Pct 1000 , percentage-format reports at stated

N{=}8

and

N{=}1000

. The directional structure, sanction on a targeted report with a placebo third-party report, appears on five of six engines, with a partial response on one and no response on deepseek-chat, which shares weights with deepseek-reasoner and differs only in whether the reasoning mode is active. Cells not run on an engine are marked –.

Outcome	What followed
Exponent test failed under bounded memory (slope $0.245$ )	memory turnover identified as a candidate violated assumption. A separately registered amendment using unbounded memory and an absolute threshold passed with slope $1.00$ , without isolating which change accounted for the difference
Aggregate verdict inconclusive in the routing pair	mandatory component metrics and the path decomposition step
Flatness failed at the smallest population	causal neighbourhood condition. Registered retest above threshold passed with spread $0.014$
Registered sign prediction failed in the early gossip design	sign treated as implementation property, demonstrated by the refusal variant
Two composed-system verdicts failed on the final round	endgame isolation rule for live runs
A registered interpretation was refuted by its own control cell	the format account of proportion framing

Appendix

Table S22: Failed, inconclusive, or criterion-inadequate registered outcomes.

Large Language Model-Based Social Simulation

Large language model-based social simulation studies how language-model agents represent people and interact to produce social behavior.

See all 257 papers

Superminds Test: Actively Evaluating Collective Intelligence of Agent Society via Probing Agents
Apr 24, 2026Xirui Li, Ming Li, Yunze Xiao +4Multi-Agent Coordination AI Agent Evaluation
University of Maryland · 2Mohamed bin Zayed University of Artificial Intelligence · 3Carnegie Mellon University
Benchmarking large language model agent societies against human behavioural distributions
Aug 28, 2026Raad Bin TareafLarge Language Model-Based Social Simulation AI Agent Benchmarks
AI Cluster, XU Exponential University of Applied Sciences, Marlene-Dietrich-Allee 12B, 14482, Potsdam, Germany.
Population Physics, Population Problems: Safety and Emergence in LLM Societies
Sep 27, 2026Adrian de WynterMulti-Agent LLM Systems Large Language Model-Based Social Simulation
Microsoft & The University of York

Apr 24, 2026J/K move · Enter open · S save

Superminds Test: Actively Evaluating Collective Intelligence of Agent Society via Probing Agents

Xirui Li, Ming Li, Yunze Xiao, Ryan Wong+3

University of Maryland · 2Mohamed bin Zayed University of Artificial Intelligence · 3Carnegie Mellon University

Multi-Agent Coordination AI Agent Evaluation AI Agent Benchmarks Multi-Agent Communication Emergent Behavior in Multi-Agent Systems

Open paper AskPDF arXiv

Collective intelligence refers to the ability of a group to achieve outcomes beyond what any individual member can accomplish alone. As large language model agents scale to populations of millions, a key question arises: Does collective intelligence emerge spontaneously from scale? We present the first empirical evaluation of this question in a large-scale autonomous agent society. Studying MoltBook, a platform hosting over two million agents, we introduce Superminds Test, a hierarchical framework that probes society-level intelligence using controlled Probing Agents across three tiers: joint reasoning, information synthesis, and basic interaction. Our experiments reveal a stark absence of collective intelligence. The society fails to outperform individual frontier models on complex reasoning tasks, rarely synthesizes distributed information, and often fails even trivial coordination tasks. Platform-wide analysis further shows that interactions remain shallow, with threads rarely extending beyond a single reply and most responses being generic or off-topic. These results suggest that collective intelligence does not emerge from scale alone. Instead, the dominant limitation of current agent societies is extremely sparse and shallow interaction, which prevents agents from exchanging information and building on each other's outputs.

Abstract

Large Language Model-Based Social Simulation

Explore similar work

Superminds Test: Actively Evaluating Collective Intelligence of Agent Society via Probing Agents

Benchmarking large language model agent societies against human behavioural distributions

Population Physics, Population Problems: Safety and Emergence in LLM Societies