High Volatility and Action Bias Distinguish LLMs from Humans in Group Coordination

Sahaj Singh Maini; Robert L. Goldstone; Zoran Tiganj

cs.MAApr 2, 2026

High Volatility and Action Bias Distinguish LLMs from Humans in Group Coordination

Authors: Sahaj Singh Maini, Robert L. Goldstone, Zoran Tiganj

Organizations: Department of Computer Science, Indiana University Bloomington · Cognitive Science Program, Indiana University Bloomington · Department of Psychological and Brain Sciences, Indiana University Bloomington

Abstract

Humans exhibit remarkable abilities to coordinate in groups. As large language models (LLMs) become more capable, it remains an open question whether they can demonstrate comparable adaptive coordination and whether they use the same strategies as humans. To better understand this, we compare LLM and human performance on a common-interest game with imperfect monitoring: Group Binary Search. In this $n$ -player game, participants need to coordinate their actions to achieve a common objective. Players independently submit numerical values in an effort to collectively sum to a randomly assigned target number. Without direct communication, they rely on group feedback to iteratively adjust their submissions until they reach the target number. Our findings show that, unlike humans who adapt and stabilize their behavior over time, LLMs often fail to improve across games and exhibit excessive switching, which impairs group convergence. Moreover, richer feedback (e.g., numerical error magnitude) benefits humans substantially but has small effects on LLMs. Finally, we show that GRPO can be effective in reducing the excessive switching. Taken together, by grounding the analysis in human baselines and mechanism-level metrics, including reactivity scaling, switching dynamics, and learning across games, we point to differences in human and LLM groups and provide a behaviorally grounded diagnostic for closing the coordination gap.

Figures & tables

Figure 1: Schematic of a GBS game with three players and numerical feedback. The sum of guesses from each player is compared to the mystery number, and the players are provided feedback about the difference between the sum of their guesses and the mystery number. The players can then adjust their answer (without communicating with each other or knowing the guesses by other players), and the game continues until the sum of guesses matches the mystery number or until 15 rounds have been played.

Zero-Shot
	Small groups		Medium groups		Large groups
	Numerical	Directional	Numerical	Directional	Numerical	Directional
Human	4.34 (0.82)	9.29 (1.99)	8.72 (1.69)	10.84 (2.32)	11.05 (2.99)	12.85 (1.89)
Deepseek-V3	8.46 (3.02)	7.60 (2.02)	10.16 (2.73)	12.48 (0.63)	14.65 (0.70)	14.10 (1.04)
Gemini 2.0 Flash	7.23 (2.16)	8.85 (2.75)	11.36 (3.94)	11.40 (1.12)	11.55 (2.88)	12.40 (2.68)
Llama 3.3	11.33 (3.48)	9.02 (2.75)	15.00 (0.00)	11.08 (2.23)	15.00 (0.00)	13.00 (1.36)

Table 1: Mean rounds to solution (games that remained unsolved at the 15-round limit assigned 15). Standard deviations are computed across session-level means, with each session-level mean averaging the five games with the corresponding feedback type. Success rates, rounds conditional on success, failure rates, and terminal errors are reported separately in Appendix Table A5 . See Table A6 for breakdown of the results for Mixed LLMs .

Figure 2: Example of coordination in 3-player games with numerical feedback and zero-shot prompts. The solid horizontal line indicates the mystery number, the other solid line indicates the sum of the group, and the dashed line represents the decisions of each agent in the group.

Figure 3: Average number of rounds needed to finish the game with zero-shot CoT prompts under numerical feedback.

Figure 4: Group reaction to numerical feedback under zero-shot CoT prompting. Each dot denotes the aggregate adjustment made by a group after the previous round’s feedback. The dotted line indicates the optimal collective correction, and the solid line shows the fitted group reaction. Human groups remain closer to stable underreaction, whereas LLM groups more often respond with steeper collective updates.

Panel of Figure 5 — Figure 5: Average proportion of players switching their guess from the previous round as the group approaches the end of the game (either by termination or by finding a solution), across different experimental conditions: small-group vs medium-group vs large-group and zero-shot vs. zero-shot CoT prompts. Error bars represent standard errors of the mean across contributing games.

Panel of Figure 6 — Figure 6: Coordination signatures by model and group size. Horizontal and vertical bars summarize across-game variability within the condition. Across both feedback types, human conditions occupy a high-stability, high-dispersion region, while most LLM conditions cluster in a relatively lower-stability, lower-dispersion regime.

Panel of Figure 7 — Figure 7: Numerical-feedback GBS performance across the base Qwen2.5-14B-Instruct model and a set of GRPO checkpoints. Panels show stay probability by group size (notice the systematic change), percentage of games solved, and mean rounds played (unsolved games are capped at 15). Error bars show standard errors across 15 games.

Appendix figures & tables42 assets

Supplementary material from the paper’s appendix.

Appendix

Model	API Model ID	Params
Deepseek-V3	deepseek-chat	671B
Deepseek-V3.1-T	deepseek-reasoner	685B
Gemini 2.0 Flash	gemini-2.0-flash	—
Llama 3.3	llama3.3:70b-instruct-fp16	70B

Appendix

Table A1: Model identifiers and parameter counts. “—” indicates the parameter count is not publicly disclosed.

Model	Temp. (Zero-shot)	Temp. (ZS-CoT)	Other
Deepseek-V3	0.6	0.6	Provider defaults
Deepseek-V3.1-T	—	default	Provider defaults
Gemini 2.0 Flash	0.2	1.0	Provider defaults
Llama 3.3	0.4	0.6	num_ctx=128K

Appendix

Table A2: Temperature settings and other hyperparameters. Deepseek-V3.1-T was only used in the Zero-shot CoT condition. “—” indicates the model was not used in that condition.

Panel of Figure 11 — Figure A1: Prompt used by GPT-4.1-mini to evaluate the reasoning supporting a stay decision during GRPO training. The final JSON block shows the response-specific fields appended to the instruction.

	Numerical		Directional
Model	Default	Mixed Temp	Default	Mixed Temp.
Deepseek-V3	8.46 (3.02)	8.68 (2.84)	7.60 (2.02)	7.88 (2.29)
Gemini 2.0 Flash	7.23 (2.16)	9.92 (4.95)	8.85 (2.75)	8.74 (2.24)
Llama 3.3	11.33 (3.48)	12.67 (2.33)	9.02 (2.75)	9.72 (2.17)

Appendix

Table A3: Mean rounds to solution (standard deviation) for LLM models under default setting (single temperature for all agents) versus mixed temperature settings (different temperature for each agent) for small group games with zero-shot prompt. In mixed temperature setting, the temperature values were 0.2, 1.0 for 2-player games and 0.2, 0.6 and 1.0 for 3-player games.

	Numerical		Directional
Model	Default	w/ Strategy, Sum	Default	w/ Strategy, Sum
Deepseek-V3	8.46 (3.02)	10.61 (2.50)	7.60 (2.02)	9.59 (2.56)
Gemini 2.0 Flash	7.23 (2.16)	10.79 (3.20)	8.85 (2.75)	10.48 (2.21)
Llama 3.3	11.33 (3.48)	9.97 (3.70)	9.02 (2.75)	10.09 (3.37)

Appendix

Table A4: Mean rounds to solution (standard deviation) for LLM models comparing default zero-shot prompt (as described in Fig. A7 ) and zero-shot prompt that includes explicit behavioral strategy (as described in Fig. A9 ) for small group games with zero-shot prompt.

Panel of Figure 14 — Figure A2: Performance of LLM models across different temperature values in small-group games with zero-shot prompt. Left panel shows results for directional feedback, right panel shows numerical feedback. Lines represent mean rounds to solution with shaded regions indicating standard deviation.

Panel of Figure 15 — Figure A3: Performance of LLM models across different temperature values in small-group games with zero-shot CoT prompt. Left panel shows results for directional feedback, right panel shows numerical feedback. Lines represent mean rounds to solution with shaded regions indicating standard deviation.

Panel of Figure 16 — Figure A4: Performance of LLM models across different temperature values in small-group games with zero-shot prompt with strategy in the prompt and total sum available in the feedback. Left panel shows results for directional feedback, right panel shows numerical feedback. Lines represent mean rounds to solution with shaded regions indicating standard deviation.

Condition	Group	Feedback	N	Success %	Rounds $\mid$ success	Failure %	Terminal error $\mid$ failure
Human	Small	Numerical	44	97.7	4.09 (1.69)	2.3	1.00
Human	Small	Directional	44	86.4	8.45 (3.26)	13.6	4.83 (3.92)
Human	Medium	Numerical	25	64.0	6.62 (2.70)	36.0	4.44 (4.75)
Human	Medium	Directional	25	48.0	8.25 (4.11)	52.0	6.62 (7.56)
Human	Large	Numerical	20	50.0	7.10 (3.57)	50.0	20.50 (22.97)
Human	Large	Directional	20	35.0	8.86 (2.41)	65.0	8.38 (11.74)

Appendix

Table A5: Performance summary for Group Binary Search. Success rate is the percentage of games solved within the 15-round limit. Rounds are averaged only over successful games, so failures at the 15-round limit are no longer pooled with genuine successes on round 15. Terminal error is the absolute difference between the final group sum and the mystery number, averaged only over failed games. Parentheses denote standard deviations across games.

Mixed LLM Configuration	Numerical Feedback	Directional Feedback
17-player (2 sessions)	15.00 (0.00)	14.5 (0.71)
(6 $\times$ Gemini, 6 $\times$ Llama, 5 $\times$ Deepseek-V3)
16-player (1 session)	15.00 (-)	13.20 (-)
(6 $\times$ Gemini, 5 $\times$ Llama, 5 $\times$ Deepseek-V3)
10-player (1 session)	15.00 (-)	14.00 (-)
(4 $\times$ Gemini, 3 $\times$ Llama, 3 $\times$ Deepseek-V3)

Appendix

Table A6: Breakdown of Mixed LLM Performance (Mean and Standard Deviation). The number of sessions for each group size matched the number of sessions in the human data. LLMs participating in each session are also listed. (Gemini refers to Gemini 2.0 Flash, Llama refers to Llama 3.3 70B)

Figure A5: Representative player-level stay profiles across five numerical games. Columns show one 2-player session (day 10), one 7-player session (day 1), and one 17-player session (day 15). Rows show humans, Deepseek-V3, Deepseek-V3.1-T, Gemini 2.0 Flash, Llama 3.3, and mixed-model groups. Faint points denote per-game stay probabilities for each player, shaded from light to dark for games 1 through 5. Large markers and error bars show mean stay probability and variability across the five numerical games. In the human examples, players are more separated from one another than in the LLM examples, which are typically lower and more compressed. The remaining stay profiles are provided as a part of the Supplemental Material.

Figure A6: Extreme stay behavior in numerical games. For each player and game, stay probability is the fraction of post-initial rounds in which the player repeated the previous guess. The left panel shows the proportion of players with stay probability 0, meaning they changed their guess on every post-initial round of a game. The right panel shows the proportion of players with stay probability 1, meaning they kept the same guess throughout the game. Bars show means across runs and error bars show standard errors across runs. The right panel highlights a qualitative difference between humans and LLMs: fully stable players appear only in human groups and become more common as group size increases.

Panel of Figure 21 — Figure A7: Zero-Shot prompt provided to each agent during the game in the first two rounds of the experiment.

Panel of Figure 22 — Figure A8: Zero-Shot CoT prompt provided to each agent during the game in the first two rounds of the experiment.

Panel of Figure 23 — Figure A9: Zero-shot prompt with an explicit behavioral strategy in the system prompt and the raw group sum from the previous round in the user prompt (both highlighted in red). The raw group sum was provided only in this strategy-plus-sum ablation. In the main experiments, models received their own prior guesses and either directional feedback (too low, too high, or just right) or numerical error-magnitude feedback (too low or too high by $k$ ), but not the raw group sum.

Figure A14: Average number of rounds needed to finish the game across five games under zero-shot prompt and directional feedback (top), zero-shot CoT prompt and directional feedback (middle), and zero-shot prompt and numerical feedback (bottom) conditions.

Model	Feedback	Runs	Mean Slope	95% Bootstrap CI	% Negative
Humans	Directional	18	$-0.91$	$[-1.34,-0.43]$	78%
Humans	Numerical	18	$-0.57$	$[-1.12,-0.04]$	72%
Deepseek-V3	Directional	18	$-0.39$	$[-0.97,0.19]$	56%
Deepseek-V3	Numerical	18	$0.02$	$[-0.61,0.61]$	33%
Gemini 2.0 Flash	Directional	18	$-0.21$	$[-0.94,0.51]$	44%
Gemini 2.0 Flash	Numerical	18	$0.41$	$[-0.17,0.99]$	33%

Appendix

Table A7: Mean slope of rounds-to-solution vs. game number (1–5) for Zero-shot prompts, pooled across group sizes (small, medium, large). A negative slope indicates fewer rounds needed in later games, reflecting cross-game learning. 95% confidence intervals computed via bootstrap resampling (10,000 iterations). “% Negative” reports the fraction of individual runs with a negative slope.

Model	Feedback	Runs	Mean Slope	95% Bootstrap CI	% Negative
Humans	Directional	18	$-0.91$	$[-1.34,-0.43]$	78%
Humans	Numerical	18	$-0.57$	$[-1.12,-0.04]$	72%
Deepseek-V3	Directional	18	$0.31$	$[-0.09,0.70]$	33%
Deepseek-V3	Numerical	18	$0.27$	$[-0.28,0.84]$	39%
Deepseek-V3.1-T	Directional	18	$0.10$	$[-0.31,0.52]$	39%
Deepseek-V3.1-T	Numerical	18	$-0.22$	$[-0.77,0.29]$	50%

Appendix

Table A8: Mean slope of rounds-to-solution vs. game number (1–5) for Zero-shot CoT prompts, pooled across group sizes (small, medium, large). A negative slope indicates fewer rounds needed in later games, reflecting cross-game learning. 95% confidence intervals computed via bootstrap resampling (10,000 iterations). “% Negative” reports the fraction of individual runs with a negative slope.

Figure A15: Group reaction to numerical feedback under zero-shot prompting. Each dot denotes the aggregate adjustment made by a group after the previous round’s numerical feedback. The dotted line indicates the optimal collective correction, and the solid line shows the fitted descriptive group reaction.

Condition	Model	Slope
Small Group	Humans	$-0.589$
Small Group, Zero-shot	Deepseek-V3	$-0.738$
Small Group, Zero-shot	Gemini 2.0 Flash	$-0.969$
Small Group, Zero-shot	Llama 3.3	$-1.794$
Small Group, Zero-shot CoT	Deepseek-V3	$-0.991$
Small Group, Zero-shot CoT	Deepseek-V3.1-T	$-1.018$

Appendix

Table A9: Slopes of best-fit lines for group reaction (change in group sum) vs. numerical response (difference between group sum and target), computed across all observations within each condition. A slope of

-1.0

indicates ideal correction; slopes more negative than

-1.0

indicate overreaction, and slopes less negative than

-1.0

indicate underreaction. Human slopes average

-0.767

across conditions, indicating systematic underreaction. LLM slopes average

-1.386

(computed as the mean of per-condition model averages), indicating systematic overreaction. In 19 of 21 condition–model pairs, LLMs exhibit more negative slopes than the corresponding human baseline.

Figure A16: Representative five-player numerical- and directional-feedback GBS games across the base model and Phase 2 GRPO checkpoints. Colored lines show individual guesses, the solid black line the group sum, and the dashed black line the mystery number.

Figure A17: Distribution of player decisions under the zero-shot condition with directional feedback.

Figure A18: Distribution of player decisions under the zero-shot condition with numerical feedback.

Figure A19: Distribution of player decisions under the zero-shot CoT condition with directional feedback.

Figure A20: Distribution of player decisions under the zero-shot CoT condition with numerical feedback.

Figure A21: Distribution of decision switch magnitudes (change in a player’s guess between consecutive rounds) under the zero-shot condition with directional feedback.

Figure A22: Distribution of decision switch magnitudes (change in a player’s guess between consecutive rounds) under the zero-shot condition with numerical feedback.

Figure A23: Distribution of decision switch magnitudes (change in a player’s guess between consecutive rounds) under the zero-shot CoT condition with directional feedback.

Figure A24: Distribution of decision switch magnitudes (change in a player’s guess between consecutive rounds) under the zero-shot CoT condition with numerical feedback.

Figure A25: Example of coordination in 4-player games with directional feedback and zero-shot prompts. The solid horizontal line indicates the mystery number, the other solid line indicates the sum of the group, and the dashed line represents the decisions of each agent in the group.

Figure A26: Example of coordination in 4-player games with numerical feedback and zero-shot prompts. The solid horizontal line indicates the mystery number, the other solid line indicates the sum of the group, and the dashed line represents the decisions of each agent in the group.

Figure A27: Example of coordination in 6-player games with directional feedback and zero-shot prompts. The solid horizontal line indicates the mystery number, the other solid line indicates the sum of the group, and the dashed line represents the decisions of each agent in the group.

Figure A28: Example of coordination in 10-player games with directional feedback and zero-shot prompts. The solid horizontal line indicates the mystery number, the other solid line indicates the sum of the group, and the dashed line represents the decisions of each agent in the group.

Figure A29: Example of coordination in 16-player games with directional feedback and zero-shot prompts. The solid horizontal line indicates the mystery number, the other solid line indicates the sum of the group, and the dashed line represents the decisions of each agent in the group.

Figure A30: Example of coordination in 17-player games with directional feedback and zero-shot prompts. The solid horizontal line indicates the mystery number, the other solid line indicates the sum of the group, and the dashed line represents the decisions of each agent in the group.

Figure A31: Example of coordination in 10-player games with numerical feedback and zero-shot prompts. The solid horizontal line indicates the mystery number, the other solid line indicates the sum of the group, and the dashed line represents the decisions of each agent in the group.

Figure A32: Example of coordination in 16-player games with numerical feedback and zero-shot prompts. The solid horizontal line indicates the mystery number, the other solid line indicates the sum of the group, and the dashed line represents the decisions of each agent in the group.

Figure A33: Example of coordination in 17-player games with numerical feedback and zero-shot prompts. The solid horizontal line indicates the mystery number, the other solid line indicates the sum of the group, and the dashed line represents the decisions of each agent in the group.

Explore similar work

Nov 6, 2025cs.AI

Large language models replicate and predict human cooperation across experiments in game theory

Large language models (LLMs) are increasingly deployed as decision-making agents in high-stakes domains and as imitators of human behavior in the social and behavioral sciences. Yet how closely LLMs mirror human decision-making remains poorly understood. This gap is critical: misalignment could produce harmful outcomes in practice, while failure to replicate human behavior renders LLMs ineffective as social simulators. Here, we address this gap by replicating large-scale game-theoretic experiments and by introducing a systematic prompting and probing framework for machine-behavioral evaluation. We test three open models typically used to power agents (Llama, Mistral, and Qwen). Across 121 dyadic games spanning four classical game types, Llama reproduces human cooperation patterns with high fidelity, while Qwen aligns closely with Nash equilibrium predictions. Characterizing models through behavioral phenotyping, we find that humans and Llama share an envious decision profile, while Qwen and Mistral exhibit different profiles. An attention-based analysis of payoff salience reveals Llama processes payoff information in a structured, layer-dependent manner absent in Qwen and Mistral, suggesting a mechanistic basis for its closer alignment with human behavior. Population-level behavioral replication is achieved without persona-based prompting, simplifying the simulation process. Extending the experimental parameter space beyond the original human-tested games, we generate and preregister testable hypotheses for novel game configurations. Our findings demonstrate appropriately configured LLMs can replicate aggregate human behavioral patterns, exhibit human-like decision phenotypes, and enable systematic exploration of unexplored experimental spaces, offering a complementary approach to traditional behavioral research that generates new empirical predictions about human social decision-making.

Andrea Cera Palatsi, Samuel Martin-Gutierrez, Ana S. Cardenal +1

Department for Computational Social Sciences and Humanities, Barcelona Supercomputing Center · Grupo de Sistemas Complejos, Universidad Polit´ecnica de Madrid · School of Law and Political Science, Universitat Oberta de Catalunya +1

Large Language Model Decisions Behavioral Foundation Models Game Theory

Ask PDF

Jun 29, 2026physics.soc-ph

Collective cooperation without individual fidelity in LLM agents

Large language models (LLMs) are increasingly used as agents in simulations of social systems, yet it remains unclear when their behavior can be interpreted as a faithful proxy for human decision-making. Here we test LLM agents against a direct empirical benchmark: a large-scale networked Prisoner's Dilemma experiment with human participants. Using the same interaction protocol, payoff structure, and network topologies, we compare nine open-weight LLMs with the human data. The selected model reproduces several macro-level features of cooperation dynamics, including the early decline and later stabilization of cooperation. This aggregate agreement, however, does not extend uniformly to finer levels of behavior. LLM populations underestimate individual-level heterogeneity and generate conditional cooperation patterns that differ from those observed in humans. Adding a fraction of random agents improves some aspects of micro-level agreement, but does not remove the mismatch in decision rules. These findings reveal a macro--micro dissociation in LLM-based social agents: collective outcomes can appear human-like even when the underlying behavioral distributions and mechanisms are not. They suggest that validating LLM agents as human surrogates requires comparisons across aggregate dynamics, individual heterogeneity, and context-dependent decision rules, rather than outcome-level agreement alone.

Henrique Ferraz de Arruda, Carlos Gracia Lázaro, Alberto Aleta +1

ARAID Foundation, Zaragoza, Spain. · Institute for Biocomputation and Physics of Complex Systems, University of Zaragoza, Zaragoza, Spain. · Universidad San Jorge (USJ), Zaragoza, Spain. +1

Large Language Model Agents Cooperation Heterogeneity Fidelity

Ask PDF

Sep 17, 2026cs.AI

Language-model groups overstate consensus when replaying human deliberation on a reasoning task

Full-consensus rates are often treated as indicators of collective cognition, yet depend on how participation and final states are operationalized. We replayed 100 held-out human Wason groups with matched large language model (LLM) agent groups, seeding one belief-anchored agent per participant's pre-discussion answer and scoring agents and people with the same code. Across human scoring definitions, estimates ranged from 24.0% to 57.0%; about one fifth of participants never posted, whereas agents almost always did. Agent groups remained more consensual in two post-unblinding sensitivity analyses: the submit-based comparison (n = 98) yielded gaps of 34.0 and 43.9 percentage points for chat and reasoning modes, and the participation-matched comparison (n = 45) yielded gaps of 34.1 and 44.4 points. These complementary routes reduced different measurement asymmetries yet converged within 0.5 percentage points. The gap persisted without early stopping and under a reparameterization removing the memorizable answer; reasoning-mode groups then agreed nearly unanimously, mostly on incorrect answers. Simulated consensus did not track collective accuracy, and belief-anchored agent groups were biased estimators of the human group-outcome distribution in this setting. These analyses provide a scoring-explicit basis for assessing simulated-group estimates of human deliberative outcomes.

Tengfei Shao

Global Education Center, Waseda University, Tokyo, Japan

Deliberation Large Language Model Decisions Conformity Human Judgment Reasoning Skills Mean Opinion Scores

Ask PDF