Safety evaluations typically focus on individual agents, but interacting agents can spread harmful information and influence the environment in which later decisions are made. We study how collective failure changes with harmful-agent fraction and society size in a controlled financial agent society, where agents communicate over a social network and trade in a shared market. In the primary financial scenario, collective failure requires broad harmful diffusion together with severe price dislocation or liquidity stress. Across all tested society sizes, failure remains rare at low harmful fractions but rises sharply over a narrow range. As society size grows from N=100 to N=2000, the harmful fraction associated with a 50% failure probability decreases from 4.7% to 2.2%, while the corresponding number of harmful agents increases from approximately 5 to 44. In contrast, when the number of harmful agents is held fixed, their impact becomes weaker as the society grows. Controlled interventions further show that broader network reach shifts the collapse boundary toward lower harmful fractions, whereas stronger conformity alone has little effect. To characterize these effects, we introduce Agent Society Dynamics, a finite-size framework for relating harmful-agent fraction, society size, and interaction structure to collective failure. Overall, our results reveal a nonlinear, size-dependent collapse transition in financial agent societies, showing that collective failure depends not only on the prevalence of harmful agents but also on the size and interaction structure of the surrounding society. Code is available at https://github.com/SAIL-Research-Lab/WolfSociety.
Figures & tables
Figure 1
Figure 1: Overview of the financial agent society and its main scaling pattern. Harmful agents influence others through communication and trading, while the resulting actions change market and social conditions observed by later agents. As society size grows, the harmful fraction associated with collapse decreases while the corresponding number of harmful agents increases. Agent Society Dynamics relates this pattern to direct harmful influence and feedback through the shared environment.
Scenario
Perturbed channel
Primary readout
Social-information manipulation
S1 Pump-and-dump
Social demand and diffusion
Price–liquidity dislocation
S2 Scalping
High-reach promotion
Price–liquidity dislocation
Market-microstructure manipulation
S3 Spoofing/layering
Displayed market depth
Cancellation and liquidity stress
S4 Wash trading
Artificial trading volume
Fake liquidity and withdrawal loss
Table 1: Manipulation scenarios inspired by public cases and their primary readouts. S1 is the primary analysis scenario.
Figure 2: Nonlinear collapse transition in the primary scenario. Collapse probability remains low at small harmful fractions, rises sharply over a narrow range, and saturates at higher fractions. The underlying episode severity provides a continuous view of the same transition.
Figure 3: Finite-size scaling of the collapse boundary. As society size grows, the harmful fraction associated with 50% collapse probability decreases, while the corresponding harmful count increases sublinearly.
ℓ
bN [95% CI]
bK [95% CI]
0
−0.764[−0.818,−0.717]
1.046[0.902,1.195]
0.5
−0.811[−0.860,−0.759]
1.022[0.897,1.150]
1
−0.822[−0.865,−0.780]
1.103[0.977,1.232]
Table 2: Fixed-count severity under three directly implemented liquidity rules. Each rule is evaluated using a separate set of simulated trajectories.
Figure 4: Intervention effects on the collapse boundary. Points show the mean boundary shift across N∈{300,1000} , with 95% CIs from paired bootstrap resampling. Negative values indicate that collapse occurs at lower harmful fractions, while positive values indicate that higher harmful fractions are required. The joint-decrease and joint-increase conditions change response precision, conformity, attention capacity, and graph degree together.
Condition
Exponent [95% CI]
Boundary evidence
All-rule
0.164[−0.009,0.251]
0.0475→0.0300
Standard quota
0.181[−0.063,0.289]
0.0500→0.0300
Increased quota
0.190[−0.081,0.276]
0.0475→0.0280
Table 3: Controller robustness over the common resolved range of five society sizes, N≤1000 .
Backbone
αc(100)
αc(1000)
ν2pt
GPT-4.1
0.0525
0.0260
0.305
Qwen3-235B
0.0550
0.0260
0.325
Llama-3.3-70B
0.0550
0.0260
0.325
GLM-4.5
0.0550
0.0260
0.325
Kimi K2.6
0.0525
0.0280
0.273
Table 4: Cross-backbone robustness in the primary S1 setting. Each backbone is evaluated under the same protocol at N=100 and N=1000 .
Appendix figures & tables18 assets
Supplementary material from the paper’s appendix.
Appendix
ID
Mechanism abstraction
Public record
Preserved structure
S1
Pump, social amplification, exit
SEC social-media manipulation enforcement [ U.S. Securities and Exchange Commission, 2022a ]
Promote–recruit–sell ordering; no named asset or amount
S2
High-reach promotion and scalping
SEC social-influencer and touting enforcement [ U.S. Securities and Exchange Commission, 2022a , U.S. Securities and Exchange Commission, 2022b ]
Promoter reach and private-position conflict
S3
Spoofing/layering
DOJ JPMorgan spoofing enforcement [ U.S. Department of Justice, 2020 ]
Non-bona-fide displayed depth and rapid cancellation
S4
Wash trading/fake liquidity
CFTC Coinbase wash-trading order [ U.S. Commodity Futures Trading Commission, 2021 ]
Self-trading, false volume, and later withdrawal
Appendix
Table 5: Public-case provenance and scenario abstraction. Sources motivate mechanism classes only; numerical simulation settings are not case estimates.
ID
Case-grounded mechanism
Plain-language episode
Primary failure event
S1
Microcap pump-and-dump
Harmful accounts build a position, promote a low-liquidity asset, recruit amplifiers, and exit after ordinary agents buy.
Harmful social diffusion together with price or liquidity dislocation.
S2
Finfluencer scalping
High-reach accounts promote an asset to followers while holding a private position, then sell into copy-trading demand.
Harmful social diffusion together with price or liquidity dislocation.
S3
Spoofing/layering
Harmful traders display non-bona-fide depth, induce others to misread imbalance, trade, and cancel rapidly.
High cancellation, spoof depth relative to liquidity, and liquidity stress.
S4
Wash trading/fake liquidity
Colluding accounts manufacture volume and apparent liquidity, attract participation, then withdraw it.
Artificial volume share and distortion followed by withdrawal loss.
Appendix
Table 6: WolfBench scenario mechanisms and episode-level failure events.
ID
Horizon
Target
Social state
Harmful episode phases
Scenario-aligned failure components
S1
30 days
asset 2
Scale-free graph; mean degree 8; feedback 0.8
Accumulate 0–4; promote 5–14; exit 15–22
Social cascade and either price dislocation or liquidity stress.
S2
30 days
asset 2
Scale-free graph; mean degree 10; feedback 0.7
Accumulate 0–3; promote 4–18; sell 12–25
Social cascade and either price dislocation or liquidity stress.
S3
30 days
asset 1
Scale-free graph; mean degree 6; feedback 0.4
Four intraday display–cancel cycles; one-step cancellation latency
Cancellation rate, displayed spoof depth relative to liquidity, and liquidity stress.
Wash share and volume distortion, followed by withdrawal loss.
Appendix
Table 7: Frozen scenario configuration. Asset indices refer to the three-asset environment shared by all scenarios; each row identifies the asset on which the scenario-aligned failure event is evaluated.
ID
Component 1
Component 2
Component 3
S1
cascade 0.55
dislocation 0.21
liquidity 0.85
S2
cascade 0.55
dislocation 0.35
liquidity 1.45
S3
cancellation 0.45
depth/liquidity 50
liquidity 1.50
S4
wash share 0.45
distortion 0.80
withdrawal 0.06
Appendix
Table 8: Prespecified thresholds for the scenario-aligned failure scores.
N
Pilot grid
Main grid
100
0, .030, .040, .050, .060, .070, .075, .100
0, .030, .040, .050, .060, .070, .075, .100
200
–
0, .020, .030, .040, .050, .060
300
0, .015, .025, .030, .035, .040, .050
0, .015, .025, .030, .035, .040, .050
500
–
0, .010, .015, .020, .025, .030, .040
1000
–
0, .006, .010, .014, .018, .022, .030
2000
–
0, .004, .007, .010, .013, .016, .022
Appendix
Table 9: Frozen S1 harmful-fraction grids. A dash indicates that no dedicated pilot shard preceded the recorded main grid at that size.
Intervention
ΔDN
ΔGfb
Δαc
Joint decrease
-0.405
-0.889
+0.046
Joint increase
+0.200
+8.465
-0.018
High conformity
+0.183
+0.930
+0.003
Increased network reach
-0.007
+0.914
-0.017
Appendix
Table 10: Matched diagnostics for selected interaction conditions.
Condition
ν
95% CI
αc(100)
αc(1000)
All-rule behavioral
0.164
[-0.009, 0.251]
0.0475
0.0300
Standard quota
0.181
[-0.063, 0.289]
0.0500
0.0300
Increased quota
0.190
[-0.081, 0.276]
0.0475
0.0280
Appendix
Table 11: Controller realization audit on the common resolved support N≤1000 . Intervals are paired seed-bootstrap percentile intervals for the common-support exponent; all include zero.
Experiment shard
Calls
Cache hits
Cost (USD)
Errors
Primary S1 refined sweep
15,902
7,965
1.6182
3
S3–S4 transfer sweep
1,374
426
0.1439
0
Appendix
Table 12: Population-LLM invocation and cache audit. Counts are cumulative manifest totals; cache keys cover the complete request and schema.
N
αc
Kc=Nαc
10–90 width
100
0.047
4.71
0.024
200
0.036
7.20
0.013
300
0.037
11.00
0.028
500
0.034
17.22
0.018
1000
0.030
30.00
–
2000
0.022
44.00
–
Appendix
Table 13: Finite-size scaling point estimates for the S1 primary analysis. All tested sizes bracket p=1/2 . The main paper reports the headline values and visualizes the trend in Figure 3.
Rule
ν
95% CI
P(ν>0)
P(ν<1)
All six sizes
0.222
[0.206, 0.277]
1.000
1.000
≥2 sizes
0.223
[0.152, 0.283]
1.000
1.000
Appendix
Table 14: Seed-bootstrap uncertainty for the finite-size exponent. The bootstrap unit is the experiment seed: each resample reconstructs all size-specific response curves, re-estimates αc(N) , and refits the log–log exponent. The conservative row uses the all-size bracketing rule; non-bracketing midpoints are treated as censored rather than imputed.
N
Linear αc
Logistic αc
Abs. diff.
100
0.0471
0.0482
0.0011
200
0.0360
0.0392
0.0032
300
0.0367
0.0351
0.0016
500
0.0344
0.0328
0.0017
1000
0.0300
0.0299
0.0001
2000
0.0220
–
–
Appendix
Table 15: Midpoint-estimator sensitivity for the S1 finite-size sweep. Linear midpoints are the main estimand; logistic midpoints provide a monotone parametric check where finite.
Dropped size
ν
None
0.222
100
0.213
200
0.239
300
0.222
500
0.224
1000
0.238
Appendix
Table 16: Leverage analysis for the finite-size exponent. Each row refits the linear-midpoint log–log slope after dropping one society size.
N
αc
Kc=Nαc
Logistic αc
100
0.015
1.50
0.0152
200
0.010
2.00
0.0101
300
0.0075
2.25
0.0076
500
0.005
2.50
0.0051
1000
0.003
3.00
0.0030
2000
0.002
4.00
0.0020
Appendix
Table 17: Watts-style threshold null on the matched society sizes and S1 α -grids. Collapse is final active fraction at least one half.
Threshold scale
Resolved sizes
ν
Support
0.8
6/6
0.155
all sizes
0.9
6/6
0.268
all sizes
1.0
6/6
0.222
all sizes
1.1
4/6
0.144
N≤500
1.2
3/6
0.278
N≤300
Appendix
Table 18: S1 failure-threshold sensitivity. The exponent is fitted only over the resolved sizes in each row; rows with fewer than six resolved sizes are partial-support diagnostics.
Response subset
bN
bK
νresponse
All fixed- K
-0.784
1.575
0.503
Subcritical cells
-0.894
1.683
0.469
Zero-failure cells
-0.950
1.817
0.477
Appendix
Table 19: Fixed-count response-surface analysis. The response-contour exponent is νresponse=1+bN/bK .
Held-out N
Obs. αc
Pred. αc
Ratio
Abs. log err.
100
0.0471
0.0462
0.980
0.020
200
0.0360
0.0324
0.900
0.105
300
0.0367
0.0253
0.691
0.370
500
0.0344
0.0204
0.592
0.525
1000
0.0300
0.0154
0.514
0.665
2000
0.0220
0.0114
0.518
0.658
Appendix
Table 20: Strict no-midpoint response-contour predictions. Each row is predicted from subcritical fixed-count data at the other five society sizes. Ratios are predicted over observed critical fraction.
Diagnostic
Δ
95% CI
+seeds
Role–action info
+0.327
[0.318, 0.335]
12/12
Role trade-rate gap
+0.264
[0.256, 0.271]
12/12
Action entropy
+1.089
[1.081, 1.098]
12/12
Trade participation
+0.348
[0.342, 0.354]
12/12
Social dominance DN
+0.512
[0.461, 0.561]
12/12
Appendix
Table 21: Mixed-role behavioral diversity analysis. Values are paired differences relative to legacy-score agents across the 12 paper seeds.
Population
αc,300
αc,1000
Readout
legacy score
0.041
0.020
baseline
mixed roles
0.040
0.022
persists
risk-averse
0.065
0.045
robust
value
0.050
0.036
moderate
trend
0.041
0.031
near base
social
0.038
0.022
fragile
Appendix
Table 22: Role-homogeneous transition ablations. Smaller αc indicates a more fragile population.
Individually protective decisions can produce avoidable collective failures. As large language model (LLM) agents take on greater roles in financial decision-making, financial AI safety must therefore be considered not only at the level of individual agents, but also at the level of the systems they jointly create. We study this problem with FRAIL, a controlled experimental framework that places LLM agents in three dynamic financial environments---bank runs, debt rollover, and reward crowdfunding---where agents' decisions reshape the financial conditions faced by others. Across seven leading LLMs, we find widespread collective fragility even when no agent is instructed to destabilize the system: 77% of baseline bank-run episodes and 83% of debt-rollover episodes end in failure. We then compare three interaction mechanisms based on compensated commitments, centralized commitment agreements, and participant-led coalitions. All three improve aggregate outcomes, but no single mechanism performs best across all financial structures. Across mechanisms, successful stabilization shares a common temporal pattern: broad commitment forms early, before defensive behavior becomes self-reinforcing. Our findings show that individually capable agents do not automatically form safe financial systems, highlighting system-level evaluation and interaction design as central problems for financial AI safety. Code is available at https://anonymous.4open.science/r/FinFrail-CF26.
Zhenhao Fu, Ruipeng Xu, Qibing Ren
Shanghai Advanced Institute of Finance, Shanghai Jiao Tong University · School of Artificial Intelligence, Shanghai Jiao Tong University
Societies of interacting language-model agents offer a controllable and repeatable way to study collective behaviour at scales that would be difficult to test with people. Their scientific value, however, depends on whether a social mechanism that works in a small group still operates when thousands of agents interact, and testing this directly requires costly large-scale runs. Here we introduce an audit that predicts a mechanism's fate as a population grows. It asks how often the mechanism can act, whether agents use the information it supplies, and whether the measurement itself creates apparent scale effects. Controlled experiments show that a single structural term can decide whether reciprocity, consensus or punishment survives scaling. For gossip, the population at which the mechanism fails is set by the reach and lifetime of its messages. In language-model societies, agents respond not only to social information but to how it is expressed: counts and percentages led to different scale behaviour. Predictions made before execution held on third-party code and a second model family, while a failed prediction exposed the boundary of the finding. The audit provides a prospective way to decide which social mechanisms can be interpreted across population scales.
Collective intelligence refers to the ability of a group to achieve outcomes beyond what any individual member can accomplish alone. As large language model agents scale to populations of millions, a key question arises: Does collective intelligence emerge spontaneously from scale? We present the first empirical evaluation of this question in a large-scale autonomous agent society. Studying MoltBook, a platform hosting over two million agents, we introduce Superminds Test, a hierarchical framework that probes society-level intelligence using controlled Probing Agents across three tiers: joint reasoning, information synthesis, and basic interaction. Our experiments reveal a stark absence of collective intelligence. The society fails to outperform individual frontier models on complex reasoning tasks, rarely synthesizes distributed information, and often fails even trivial coordination tasks. Platform-wide analysis further shows that interactions remain shallow, with threads rarely extending beyond a single reply and most responses being generic or off-topic. These results suggest that collective intelligence does not emerge from scale alone. Instead, the dominant limitation of current agent societies is extremely sparse and shallow interaction, which prevents agents from exchanging information and building on each other's outputs.
Xirui Li, Ming Li, Yunze Xiao +4
University of Maryland · 2Mohamed bin Zayed University of Artificial Intelligence · 3Carnegie Mellon University