cs.MAApr 2, 2026
SaveHigh Volatility and Action Bias Distinguish LLMs from Humans in Group Coordination
Organizations: Department of Computer Science, Indiana University Bloomington · Cognitive Science Program, Indiana University Bloomington · Department of Psychological and Brain Sciences, Indiana University Bloomington
Abstract
Humans exhibit remarkable abilities to coordinate in groups. As large language models (LLMs) become more capable, it remains an open question whether they can demonstrate comparable adaptive coordination and whether they use the same strategies as humans. To better understand this, we compare LLM and human performance on a common-interest game with imperfect monitoring: Group Binary Search. In this -player game, participants need to coordinate their actions to achieve a common objective. Players independently submit numerical values in an effort to collectively sum to a randomly assigned target number. Without direct communication, they rely on group feedback to iteratively adjust their submissions until they reach the target number. Our findings show that, unlike humans who adapt and stabilize their behavior over time, LLMs often fail to improve across games and exhibit excessive switching, which impairs group convergence. Moreover, richer feedback (e.g., numerical error magnitude) benefits humans substantially but has small effects on LLMs. Finally, we show that GRPO can be effective in reducing the excessive switching. Taken together, by grounding the analysis in human baselines and mechanism-level metrics, including reactivity scaling, switching dynamics, and learning across games, we point to differences in human and LLM groups and provide a behaviorally grounded diagnostic for closing the coordination gap.
Figures & tables
Figure 1: Schematic of a GBS game with three players and numerical feedback. The sum of guesses from each player is compared to the mystery number, and the players are provided feedback about the difference between the sum of their guesses and the mystery number. The players can then adjust their answer (without communicating with each other or knowing the guesses by other players), and the game continues until the sum of guesses matches the mystery number or until 15 rounds have been played.
| Small groups | Medium groups | Large groups | ||||
| Numerical | Directional | Numerical | Directional | Numerical | Directional | |
| Human | 4.34 (0.82) | 9.29 (1.99) | 8.72 (1.69) | 10.84 (2.32) | 11.05 (2.99) | 12.85 (1.89) |
| Zero-Shot | ||||||
| Deepseek-V3 | 8.46 (3.02) | 7.60 (2.02) | 10.16 (2.73) | 12.48 (0.63) | 14.65 (0.70) | 14.10 (1.04) |
| Gemini 2.0 Flash | 7.23 (2.16) | 8.85 (2.75) | 11.36 (3.94) | 11.40 (1.12) | 11.55 (2.88) | 12.40 (2.68) |
| Llama 3.3 | 11.33 (3.48) | 9.02 (2.75) | 15.00 (0.00) | 11.08 (2.23) | 15.00 (0.00) | 13.00 (1.36) |
Table 1: Mean rounds to solution (games that remained unsolved at the 15-round limit assigned 15). Standard deviations are computed across session-level means, with each session-level mean averaging the five games with the corresponding feedback type. Success rates, rounds conditional on success, failure rates, and terminal errors are reported separately in Appendix Table A5 . See Table A6 for breakdown of the results for Mixed LLMs .
Figure 2: Example of coordination in 3-player games with numerical feedback and zero-shot prompts. The solid horizontal line indicates the mystery number, the other solid line indicates the sum of the group, and the dashed line represents the decisions of each agent in the group.
Figure 3: Average number of rounds needed to finish the game with zero-shot CoT prompts under numerical feedback.
Figure 4: Group reaction to numerical feedback under zero-shot CoT prompting. Each dot denotes the aggregate adjustment made by a group after the previous round’s feedback. The dotted line indicates the optimal collective correction, and the solid line shows the fitted group reaction. Human groups remain closer to stable underreaction, whereas LLM groups more often respond with steeper collective updates.
Figure 5: Average proportion of players switching their guess from the previous round as the group approaches the end of the game (either by termination or by finding a solution), across different experimental conditions: small-group vs medium-group vs large-group and zero-shot vs. zero-shot CoT prompts. Error bars represent standard errors of the mean across contributing games.
Figure 6: Coordination signatures by model and group size. Horizontal and vertical bars summarize across-game variability within the condition. Across both feedback types, human conditions occupy a high-stability, high-dispersion region, while most LLM conditions cluster in a relatively lower-stability, lower-dispersion regime.
Figure 7: Numerical-feedback GBS performance across the base Qwen2.5-14B-Instruct model and a set of GRPO checkpoints. Panels show stay probability by group size (notice the systematic change), percentage of games solved, and mean rounds played (unsolved games are capped at 15). Error bars show standard errors across 15 games.
Appendix figures & tables42 assets
Supplementary material from the paper’s appendix.
Appendix
| Model | API Model ID | Params |
|---|---|---|
| Deepseek-V3 | deepseek-chat | 671B |
| Deepseek-V3.1-T | deepseek-reasoner | 685B |
| Gemini 2.0 Flash | gemini-2.0-flash | — |
| Llama 3.3 | llama3.3:70b-instruct-fp16 | 70B |
Table A1: Model identifiers and parameter counts. “—” indicates the parameter count is not publicly disclosed.
| Model | Temp. (Zero-shot) | Temp. (ZS-CoT) | Other |
|---|---|---|---|
| Deepseek-V3 | 0.6 | 0.6 | Provider defaults |
| Deepseek-V3.1-T | — | default | Provider defaults |
| Gemini 2.0 Flash | 0.2 | 1.0 | Provider defaults |
| Llama 3.3 | 0.4 | 0.6 | num_ctx=128K |
Table A2: Temperature settings and other hyperparameters. Deepseek-V3.1-T was only used in the Zero-shot CoT condition. “—” indicates the model was not used in that condition.
Figure A1: Prompt used by GPT-4.1-mini to evaluate the reasoning supporting a stay decision during GRPO training. The final JSON block shows the response-specific fields appended to the instruction.
| Numerical | Directional | |||
|---|---|---|---|---|
| Model | Default | Mixed Temp | Default | Mixed Temp. |
| Deepseek-V3 | 8.46 (3.02) | 8.68 (2.84) | 7.60 (2.02) | 7.88 (2.29) |
| Gemini 2.0 Flash | 7.23 (2.16) | 9.92 (4.95) | 8.85 (2.75) | 8.74 (2.24) |
| Llama 3.3 | 11.33 (3.48) | 12.67 (2.33) | 9.02 (2.75) | 9.72 (2.17) |
Table A3: Mean rounds to solution (standard deviation) for LLM models under default setting (single temperature for all agents) versus mixed temperature settings (different temperature for each agent) for small group games with zero-shot prompt. In mixed temperature setting, the temperature values were 0.2, 1.0 for 2-player games and 0.2, 0.6 and 1.0 for 3-player games.
| Numerical | Directional | |||
|---|---|---|---|---|
| Model | Default | w/ Strategy, Sum | Default | w/ Strategy, Sum |
| Deepseek-V3 | 8.46 (3.02) | 10.61 (2.50) | 7.60 (2.02) | 9.59 (2.56) |
| Gemini 2.0 Flash | 7.23 (2.16) | 10.79 (3.20) | 8.85 (2.75) | 10.48 (2.21) |
| Llama 3.3 | 11.33 (3.48) | 9.97 (3.70) | 9.02 (2.75) | 10.09 (3.37) |
Table A4: Mean rounds to solution (standard deviation) for LLM models comparing default zero-shot prompt (as described in Fig. A7 ) and zero-shot prompt that includes explicit behavioral strategy (as described in Fig. A9 ) for small group games with zero-shot prompt.
Figure A2: Performance of LLM models across different temperature values in small-group games with zero-shot prompt. Left panel shows results for directional feedback, right panel shows numerical feedback. Lines represent mean rounds to solution with shaded regions indicating standard deviation.
Figure A3: Performance of LLM models across different temperature values in small-group games with zero-shot CoT prompt. Left panel shows results for directional feedback, right panel shows numerical feedback. Lines represent mean rounds to solution with shaded regions indicating standard deviation.
Figure A4: Performance of LLM models across different temperature values in small-group games with zero-shot prompt with strategy in the prompt and total sum available in the feedback. Left panel shows results for directional feedback, right panel shows numerical feedback. Lines represent mean rounds to solution with shaded regions indicating standard deviation.
| Condition | Group | Feedback | N | Success % | Rounds success | Failure % | Terminal error failure |
|---|---|---|---|---|---|---|---|
| Human | Small | Numerical | 44 | 97.7 | 4.09 (1.69) | 2.3 | 1.00 |
| Human | Small | Directional | 44 | 86.4 | 8.45 (3.26) | 13.6 | 4.83 (3.92) |
| Human | Medium | Numerical | 25 | 64.0 | 6.62 (2.70) | 36.0 | 4.44 (4.75) |
| Human | Medium | Directional | 25 | 48.0 | 8.25 (4.11) | 52.0 | 6.62 (7.56) |
| Human | Large | Numerical | 20 | 50.0 | 7.10 (3.57) | 50.0 | 20.50 (22.97) |
| Human | Large | Directional | 20 | 35.0 | 8.86 (2.41) | 65.0 | 8.38 (11.74) |
Table A5: Performance summary for Group Binary Search. Success rate is the percentage of games solved within the 15-round limit. Rounds are averaged only over successful games, so failures at the 15-round limit are no longer pooled with genuine successes on round 15. Terminal error is the absolute difference between the final group sum and the mystery number, averaged only over failed games. Parentheses denote standard deviations across games.
| Mixed LLM Configuration | Numerical Feedback | Directional Feedback |
|---|---|---|
| 17-player (2 sessions) | 15.00 (0.00) | 14.5 (0.71) |
| (6 Gemini, 6 Llama, 5 Deepseek-V3) | ||
| 16-player (1 session) | 15.00 (-) | 13.20 (-) |
| (6 Gemini, 5 Llama, 5 Deepseek-V3) | ||
| 10-player (1 session) | 15.00 (-) | 14.00 (-) |
| (4 Gemini, 3 Llama, 3 Deepseek-V3) |
Table A6: Breakdown of Mixed LLM Performance (Mean and Standard Deviation). The number of sessions for each group size matched the number of sessions in the human data. LLMs participating in each session are also listed. (Gemini refers to Gemini 2.0 Flash, Llama refers to Llama 3.3 70B)
Figure A5: Representative player-level stay profiles across five numerical games. Columns show one 2-player session (day 10), one 7-player session (day 1), and one 17-player session (day 15). Rows show humans, Deepseek-V3, Deepseek-V3.1-T, Gemini 2.0 Flash, Llama 3.3, and mixed-model groups. Faint points denote per-game stay probabilities for each player, shaded from light to dark for games 1 through 5. Large markers and error bars show mean stay probability and variability across the five numerical games. In the human examples, players are more separated from one another than in the LLM examples, which are typically lower and more compressed. The remaining stay profiles are provided as a part of the Supplemental Material.
Figure A6: Extreme stay behavior in numerical games. For each player and game, stay probability is the fraction of post-initial rounds in which the player repeated the previous guess. The left panel shows the proportion of players with stay probability 0, meaning they changed their guess on every post-initial round of a game. The right panel shows the proportion of players with stay probability 1, meaning they kept the same guess throughout the game. Bars show means across runs and error bars show standard errors across runs. The right panel highlights a qualitative difference between humans and LLMs: fully stable players appear only in human groups and become more common as group size increases.
Figure A7: Zero-Shot prompt provided to each agent during the game in the first two rounds of the experiment.
Figure A8: Zero-Shot CoT prompt provided to each agent during the game in the first two rounds of the experiment.
Figure A9: Zero-shot prompt with an explicit behavioral strategy in the system prompt and the raw group sum from the previous round in the user prompt (both highlighted in red). The raw group sum was provided only in this strategy-plus-sum ablation. In the main experiments, models received their own prior guesses and either directional feedback (too low, too high, or just right) or numerical error-magnitude feedback (too low or too high by ), but not the raw group sum.
Figure A10:
Figure A11:
Figure A12:
Figure A13:
Figure A14: Average number of rounds needed to finish the game across five games under zero-shot prompt and directional feedback (top), zero-shot CoT prompt and directional feedback (middle), and zero-shot prompt and numerical feedback (bottom) conditions.
| Model | Feedback | Runs | Mean Slope | 95% Bootstrap CI | % Negative |
|---|---|---|---|---|---|
| Humans | Directional | 18 | 78% | ||
| Humans | Numerical | 18 | 72% | ||
| Deepseek-V3 | Directional | 18 | 56% | ||
| Deepseek-V3 | Numerical | 18 | 33% | ||
| Gemini 2.0 Flash | Directional | 18 | 44% | ||
| Gemini 2.0 Flash | Numerical | 18 | 33% |
Table A7: Mean slope of rounds-to-solution vs. game number (1–5) for Zero-shot prompts, pooled across group sizes (small, medium, large). A negative slope indicates fewer rounds needed in later games, reflecting cross-game learning. 95% confidence intervals computed via bootstrap resampling (10,000 iterations). “% Negative” reports the fraction of individual runs with a negative slope.
| Model | Feedback | Runs | Mean Slope | 95% Bootstrap CI | % Negative |
|---|---|---|---|---|---|
| Humans | Directional | 18 | 78% | ||
| Humans | Numerical | 18 | 72% | ||
| Deepseek-V3 | Directional | 18 | 33% | ||
| Deepseek-V3 | Numerical | 18 | 39% | ||
| Deepseek-V3.1-T | Directional | 18 | 39% | ||
| Deepseek-V3.1-T | Numerical | 18 | 50% |
Table A8: Mean slope of rounds-to-solution vs. game number (1–5) for Zero-shot CoT prompts, pooled across group sizes (small, medium, large). A negative slope indicates fewer rounds needed in later games, reflecting cross-game learning. 95% confidence intervals computed via bootstrap resampling (10,000 iterations). “% Negative” reports the fraction of individual runs with a negative slope.
Figure A15: Group reaction to numerical feedback under zero-shot prompting. Each dot denotes the aggregate adjustment made by a group after the previous round’s numerical feedback. The dotted line indicates the optimal collective correction, and the solid line shows the fitted descriptive group reaction.
| Condition | Model | Slope |
|---|---|---|
| Small Group | Humans | |
| Small Group, Zero-shot | Deepseek-V3 | |
| Small Group, Zero-shot | Gemini 2.0 Flash | |
| Small Group, Zero-shot | Llama 3.3 | |
| Small Group, Zero-shot CoT | Deepseek-V3 | |
| Small Group, Zero-shot CoT | Deepseek-V3.1-T |
Table A9: Slopes of best-fit lines for group reaction (change in group sum) vs. numerical response (difference between group sum and target), computed across all observations within each condition. A slope of indicates ideal correction; slopes more negative than indicate overreaction, and slopes less negative than indicate underreaction. Human slopes average across conditions, indicating systematic underreaction. LLM slopes average (computed as the mean of per-condition model averages), indicating systematic overreaction. In 19 of 21 condition–model pairs, LLMs exhibit more negative slopes than the corresponding human baseline.
Figure A16: Representative five-player numerical- and directional-feedback GBS games across the base model and Phase 2 GRPO checkpoints. Colored lines show individual guesses, the solid black line the group sum, and the dashed black line the mystery number.
Figure A17: Distribution of player decisions under the zero-shot condition with directional feedback.
Figure A18: Distribution of player decisions under the zero-shot condition with numerical feedback.
Figure A19: Distribution of player decisions under the zero-shot CoT condition with directional feedback.
Figure A20: Distribution of player decisions under the zero-shot CoT condition with numerical feedback.
Figure A21: Distribution of decision switch magnitudes (change in a player’s guess between consecutive rounds) under the zero-shot condition with directional feedback.
Figure A22: Distribution of decision switch magnitudes (change in a player’s guess between consecutive rounds) under the zero-shot condition with numerical feedback.
Figure A23: Distribution of decision switch magnitudes (change in a player’s guess between consecutive rounds) under the zero-shot CoT condition with directional feedback.
Figure A24: Distribution of decision switch magnitudes (change in a player’s guess between consecutive rounds) under the zero-shot CoT condition with numerical feedback.
Figure A25: Example of coordination in 4-player games with directional feedback and zero-shot prompts. The solid horizontal line indicates the mystery number, the other solid line indicates the sum of the group, and the dashed line represents the decisions of each agent in the group.
Figure A26: Example of coordination in 4-player games with numerical feedback and zero-shot prompts. The solid horizontal line indicates the mystery number, the other solid line indicates the sum of the group, and the dashed line represents the decisions of each agent in the group.
Figure A27: Example of coordination in 6-player games with directional feedback and zero-shot prompts. The solid horizontal line indicates the mystery number, the other solid line indicates the sum of the group, and the dashed line represents the decisions of each agent in the group.
Figure A28: Example of coordination in 10-player games with directional feedback and zero-shot prompts. The solid horizontal line indicates the mystery number, the other solid line indicates the sum of the group, and the dashed line represents the decisions of each agent in the group.
Figure A29: Example of coordination in 16-player games with directional feedback and zero-shot prompts. The solid horizontal line indicates the mystery number, the other solid line indicates the sum of the group, and the dashed line represents the decisions of each agent in the group.
Figure A30: Example of coordination in 17-player games with directional feedback and zero-shot prompts. The solid horizontal line indicates the mystery number, the other solid line indicates the sum of the group, and the dashed line represents the decisions of each agent in the group.
Figure A31: Example of coordination in 10-player games with numerical feedback and zero-shot prompts. The solid horizontal line indicates the mystery number, the other solid line indicates the sum of the group, and the dashed line represents the decisions of each agent in the group.
Figure A32: Example of coordination in 16-player games with numerical feedback and zero-shot prompts. The solid horizontal line indicates the mystery number, the other solid line indicates the sum of the group, and the dashed line represents the decisions of each agent in the group.
Figure A33: Example of coordination in 17-player games with numerical feedback and zero-shot prompts. The solid horizontal line indicates the mystery number, the other solid line indicates the sum of the group, and the dashed line represents the decisions of each agent in the group.