More Capable, Less Cooperative? When LLMs Fail At Zero-Cost Collaboration

Advait Yadav; Sid Black; Oliver Sourbut

cs.MAApr 9, 2026

More Capable, Less Cooperative? When LLMs Fail At Zero-Cost Collaboration

Authors: Advait Yadav, Sid Black, Oliver Sourbut

Organizations: MATS · University of Illinois Urbana-Champaign · UK AI Security Institute · Future of Life Foundation

Abstract

Large language model (LLM) agents increasingly coordinate in multi-agent systems, yet we lack an understanding of where and why cooperation fails. Many real-world coordination problems are not social dilemmas: helping others -- sharing documentation, unblocking a teammate -- costs the helper almost nothing while producing substantial collective benefit. Whether LLM agents cooperate in this regime, where helping is free and they are explicitly instructed to do so, remains unknown. We build a turn-based multi-agent environment that strips away all strategic complexity, making cooperation costless and trivially optimal. Across eight widely used LLMs, capability does not predict cooperation: OpenAI o3 reaches only 17% of optimal collective performance while the weaker o3-mini reaches 50%, despite identical instructions to maximize group revenue. Using a causal decomposition that automates one side of agent communication, we separate cooperation failures from competence failures, and find that several capable models actively withhold information despite gaining nothing from withholding. Targeted interventions address each mode: explicit protocols roughly double the performance of competence-limited models, while small sharing incentives unlock cooperation-limited ones. Our results suggest that scaling intelligence alone will not solve coordination in multi-agent systems, and will require deliberate cooperative design, even when helping costs nothing.

Figures & tables

Figure 1: The instruction-utility gap. Agent 1 requests information from Agent 3 to complete a task. Agent 3 can cooperate or withhold. While the agents are instructed to maximize overall revenue, sending information has no effect on Agent 3’s individual payoff—only Agent 1 benefits from receiving it. This neutrality for the sender creates the instruction-utility gap and drives cooperative failures.

Figure 2: The two-step pipeline under perfect play. In round T, Agent A requests all missing pieces from holders and fulfills incoming requests from others. Other agents fulfill A’s requests during their turns within the same round. By round T+1, Agent A has received the needed pieces, can submit completed tasks, and receives new tasks to maintain its queue. This two-step flow continually repeats for subsequent rounds.

Figure 3: Final performance is uncorrelated with general capability. We use Chatbot Arena Elo scores as a proxy for capability. The dashed line shows linear fit (R² = 0.025, p = 0.71).

Model	Total Tasks ( $\uparrow$ )	Msgs/Task ( $\downarrow$ )	Gini Coefficient ( $\downarrow$ )	Response Rate ( $\uparrow$ )	Pipeline Efficiency ( $\uparrow$ )
o3-mini	102.8 $\pm$ 17.3 (50.4%)	4.4 $\pm$ 1.0	0.075 $\pm$ 0.039	94.6%	95.4%
GPT-5-mini	78.7 $\pm$ 8.6 (38.6%)	10.6 $\pm$ 8.0	0.133 $\pm$ 0.121	45.4%	95.1%
o3	34.4 $\pm$ 2.6 (16.9%)	29.0 $\pm$ 3.2	0.206 $\pm$ 0.067	60.1%	44.6%
DeepSeek-R1	93.5 $\pm$ 8.7 (45.8%)	10.3 $\pm$ 8.0	0.110 $\pm$ 0.024	52.0%	89.6%
GPT-4.1-mini	11.8 $\pm$ 1.6 (5.8%)	24.0 $\pm$ 7.3	0.443 $\pm$ 0.076	77.0%	11.0%
Claude Sonnet 4	132.0 $\pm$ 9.6 (64.7%)	3.5 $\pm$ 0.3	0.078 $\pm$ 0.016	87.7%	89.7%

Table 1: Baseline performance. Total tasks are also reported as a % of the Perfect-Play row, which provides the performance ceiling.

Figure 4: Failure mode decomposition. Models mapped by their cooperation rate versus competence rate. The diagonal separates cooperation-limited models from competence-limited models.

Model	Baseline	Auto-Request	Auto-Fulfill
		(Cooperation)	(Competence)
o3-mini	50.4%	17.2%	92.1%
GPT-5-mini	38.6%	18.6%	95.3%
o3	16.9%	15.2%	94.9%
DeepSeek-R1	45.8%	70.5%	75.5%
GPT-4.1-mini	5.8%	30.1%	14.4%

Table 2: Causal decomposition of cooperation and competence. Performance shown as % of Perfect-Play ceiling. Full results with all metrics in Appendix A.1 .

Model	Limited	Policy	Incentive
o3-mini	+29.4%	+25.3%	+19.6%
GPT-5-mini	+48.8%	+99.3%	+74.5%
o3	+22.1%	+82.6%	+190.7%
DeepSeek-R1	+26.2%	+78.0%	+46.8%
GPT-4.1-mini	+113.6%	+64.4%	+20.3%
Claude Sonnet 4	$-$ 15.0%	+5.9%	$-$ 4.7%

Table 3: Targeted interventions address distinct failure modes. Performance change relative to baseline (Table 1 ). Full results with all metrics in Appendix A.1 .

Figure 5: Intervention effects. Performance impact of three interventions relative to baseline.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Model	Setting	Total Tasks ( $\uparrow$ )	Msgs/Task ( $\downarrow$ )	Gini Coefficient ( $\downarrow$ )	Response Rate ( $\uparrow$ )	Pipeline Efficiency ( $\uparrow$ )
o3-mini	Auto Fulfill	187.8 $\pm$ 20.5 (92.1%)	1.8 $\pm$ 0.1	0.028 $\pm$ 0.010	104.2%	100.0%
	Auto Request	35.0 $\pm$ 13.5 (17.2%)	23.3 $\pm$ 11.1	0.200 $\pm$ 0.036	76.5%	60.7%
	Baseline	102.8 $\pm$ 17.3 (50.4%)	4.4 $\pm$ 1.0	0.075 $\pm$ 0.039	94.6%	95.4%
GPT-5-mini	Auto Fulfill	194.4 $\pm$ 4.1 (95.3%)	2.1 $\pm$ 0.3	0.019 $\pm$ 0.008	74.0%	99.8%
	Auto Request	38.0 $\pm$ 7.8 (18.6%)	21.5 $\pm$ 7.0	0.280 $\pm$ 0.122	65.0%	70.5%
	Baseline	78.7 $\pm$ 8.6 (38.6%)	10.6 $\pm$ 8.0	0.133 $\pm$ 0.121	45.4%	95.1%

Appendix

Table 4: Full causal decomposition results. Complete metrics for the selective automation experiment. Auto-Request isolates cooperation on the sending dimension; Auto-Fulfill isolates competence on the requesting/submission dimension. Total Tasks shown with % of Perfect-Play ceiling in parentheses.

Model	Intervention	Total Tasks ( $\uparrow$ )	Msgs/Task ( $\downarrow$ )	Gini Coefficient ( $\downarrow$ )	Response Rate ( $\uparrow$ )	Pipeline Efficiency ( $\uparrow$ )
o3-mini	Limited	133.0 $\pm$ 6.5 (+29.4%)	2.9 $\pm$ 0.2	0.047 $\pm$ 0.009	98.1%	98.1%
	Policy	128.8 $\pm$ 7.9 (+25.3%)	3.1 $\pm$ 0.2	0.058 $\pm$ 0.015	101.0%	99.7%
	Incentive	123.0 $\pm$ 9.9 (+19.6%)	3.5 $\pm$ 0.5	0.079 $\pm$ 0.010	103.7%	98.0%
GPT-5-mini	Limited	117.1 $\pm$ 8.3 (+48.8%)	5.1 $\pm$ 3.2	0.087 $\pm$ 0.050	57.9%	96.6%
	Policy	156.8 $\pm$ 5.2 (+99.3%)	3.3 $\pm$ 0.3	0.042 $\pm$ 0.006	62.0%	99.7%
	Incentive	137.3 $\pm$ 10.1 (+74.5%)	3.9 $\pm$ 1.2	0.070 $\pm$ 0.049	59.5%	99.6%

Appendix

Table 5: Full intervention results. Complete metrics for three minimal interventions: Policy (explicit protocols), Incentive (10% sender bonus), and Limited (hidden peer revenues). Total Tasks shows % change from baseline in parentheses.

Category	Example Patterns
Defection Patterns
Explicit	withhold(ing)? , not (share\|send) (until\|unless) , retain(ing)? what I
Leverage	maintain(ing)? leverage , bargaining (position\|power\|chip) , \bleverage\b
Conditional	await(ing)? (replies\|responses) , before (sending\|sharing) , wait (for\|until)
Self-priority	prioritize my (task\|own) , secure what I need first
Cooperation Patterns

Appendix

Table 6: Pattern definitions for reasoning classification.

Model	All Defection	Hard Defection	Leverage Count	Tasks/Run
o3	43.9% $\pm$ 8.2%	39.3% $\pm$ 8.2%	373	34.4
GPT-5-mini	14.8% $\pm$ 3.5%	8.1% $\pm$ 2.3%	159	113.6
GPT-4.1-mini	8.9% $\pm$ 3.8%	0.3% $\pm$ 0.3%	2	11.4
Gemini-2.5-Pro	8.4% $\pm$ 4.3%	0.0%	0	164.8
Claude Sonnet 4	3.3% $\pm$ 1.8%	0.0%	0	115.0
DeepSeek-R1	3.2% $\pm$ 2.0%	2.7% $\pm$ 1.8%	27	156.4

Appendix

Table 7: Defection reasoning rates by model.

Model	Market Terms/1K
o3	27.09 $\pm$ 1.33
GPT-5-mini	14.14 $\pm$ 3.99
DeepSeek-R1	5.20 $\pm$ 5.45
o3-mini	3.20 $\pm$ 2.24
Claude Sonnet 4	1.74 $\pm$ 0.39
GPT-4.1-mini	1.41 $\pm$ 0.62

Appendix

Table 8: Market language density (terms per 1,000 words).

Model	Configuration	Total Tasks ( $\uparrow$ )	Msgs/Task ( $\downarrow$ )	Gini Coefficient ( $\downarrow$ )	Response Rate ( $\uparrow$ )	Pipeline Efficiency ( $\uparrow$ )
o3-mini	10	54.8 $\pm$ 2.0	4.4 $\pm$ 0.3	0.100 $\pm$ 0.037	87.0% $\pm$ 4.9%	54.8% $\pm$ 2.0%
	20	102.8 $\pm$ 17.3	4.4 $\pm$ 1.0	0.075 $\pm$ 0.039	70.5% $\pm$ 9.7%	51.4% $\pm$ 8.6%
	30	154.6 $\pm$ 15.5	4.0 $\pm$ 0.5	0.081 $\pm$ 0.029	62.7% $\pm$ 5.3%	51.5% $\pm$ 5.2%
o3	10	15.2 $\pm$ 10.3	44.1 $\pm$ 42.2	0.286 $\pm$ 0.177	52.1% $\pm$ 15.8%	15.2% $\pm$ 10.3%
	20	34.4 $\pm$ 2.6	29.0 $\pm$ 3.2	0.206 $\pm$ 0.067	48.1% $\pm$ 3.5%	17.2% $\pm$ 1.3%
	30	80.0 $\pm$ 25.1	19.1 $\pm$ 7.3	0.140 $\pm$ 0.053	51.4% $\pm$ 7.8%	26.7% $\pm$ 8.4%

Appendix

Table 9: Effect of episode length on model performance (10, 20, 30 rounds).

Model	Configuration	Total Tasks ( $\uparrow$ )	Gini ( $\downarrow$ )	Response Rate ( $\uparrow$ )	Pipeline Eff ( $\uparrow$ )	Msgs/Task ( $\downarrow$ )
o3-mini	10 agents	102.8 $\pm$ 17.3	0.075 $\pm$ 0.039	94.6% $\pm$ 4.3%	95.4% $\pm$ 2.7%	4.4 $\pm$ 1.0
	20 agents	187.0 $\pm$ 11.7	0.074 $\pm$ 0.015	80.4% $\pm$ 3.7%	46.8% $\pm$ 5.0%	5.9 $\pm$ 0.6
o3	10 agents	34.4 $\pm$ 2.6	0.206 $\pm$ 0.067	60.1% $\pm$ 10.6%	44.6% $\pm$ 7.1%	29.0 $\pm$ 3.2
	20 agents	73.4 $\pm$ 17.9	0.193 $\pm$ 0.085	51.1% $\pm$ 9.0%	18.4% $\pm$ 5.0%	35.2 $\pm$ 8.4
GPT-4.1-mini	10 agents	11.8 $\pm$ 1.6	0.443 $\pm$ 0.076	77.0% $\pm$ 9.4%	11.0% $\pm$ 12.5%	24.0 $\pm$ 7.4
	20 agents	27.0 $\pm$ 4.4	0.372 $\pm$ 0.122	65.5% $\pm$ 8.0%	6.8% $\pm$ 5.0%	20.7 $\pm$ 5.0

Appendix

Table 10: Effect of agent count on model performance (10 vs. 20 agents).

Explore similar work

The Collaboration Tax: How Much LLM Multi-Agent Systems Pay to Coordinate
Aug 23, 2026Weixiang Sun, Zehong Wang, Hong Huang +3Multi-Agent Coordination Multi-LLM Collaboration
University of Notre Dame · Meta Superintelligence Labs · Simon Fraser University
COOP $^2$ : Defining, Observing, and Repairing Cooperation in LLM Multi-Agent Systems
Feb 27, 2026Hanqing Yang, Narjes Nourzad, Shiyu Chen +3Multi-Agent Collaboration Multi-Agent Coordination
CMU · USC · SUTD +1
Competition and Cooperation of LLM Agents in Games
Apr 1, 2026Jiayi Yao, Cong Chen, Baosen ZhangLLM Agents Multi-Agent LLM Systems
Department of Electrical and Computer Engineering, University of Washington, Seattle. · Thayer School of Engineering at Dartmouth College.

Aug 23, 2026J/K move · Enter open · S save

The Collaboration Tax: How Much LLM Multi-Agent Systems Pay to Coordinate

Weixiang Sun, Zehong Wang, Hong Huang, Colby Nelson+2

University of Notre Dame · Meta Superintelligence Labs · Simon Fraser University

Multi-Agent Coordination Multi-LLM Collaboration

Open paper AskPDF arXiv

Multi-agent systems built from large language models are deployed widely, yet how much performance is lost when two LLMs must coordinate rather than act alone remains unclear. We formulate the collaboration tax as the team-decentralisation loss of a two-player cooperative game with private information, with two propositions characterising its sign and its equivalence to a max-superadditivity violation. We operationalise this definition on 32 solo-tractable tasks grouped by source of grounding friction and measure it on 11 models from 7 providers. The tax is structured along two no-exception axes: a category ordering across every model and a monotonic decrease with capability. The proximate mechanism is not a reasoning deficit but a four-stage conversational cascade in which agents make ungrounded claims, fail to query the partner, skip integrating both views, and accept the answer without re-derivation. The tax is mechanically predictable from conversation features and partly tractable: a prompt intervention targeting all four stages closes a substantial fraction of the gap, with the dominant bottleneck differing across categories. In heterogeneous pairs the tax is pulled toward the stronger partner rather than the additive midpoint, empirically realising the max-superadditivity violation predicted by our framework. Together these results recast collaboration in LLM systems as a measurable, predictable, and partly tractable cost.

Abstract

Explore similar work

The Collaboration Tax: How Much LLM Multi-Agent Systems Pay to Coordinate

COOP2^22: Defining, Observing, and Repairing Cooperation in LLM Multi-Agent Systems

Competition and Cooperation of LLM Agents in Games

COOP $^2$ : Defining, Observing, and Repairing Cooperation in LLM Multi-Agent Systems