Organizations: The Hong Kong University of Science and Technology, Hong Kong SAR, China · Institute of Software, Chinese Academy of Sciences, Beijing, China · Shandong University, Weihai, China · Tsinghua University, Beijing, China
Web systems increasingly predict users' decisions from what users say, yet when interests partly conflict, users may promise to cooperate to win the other side's cooperation and then not do so. Sales or service agents that rely on such promises may grant concessions to users who then do not keep their promises, at a cost to the party they represent. We test how LLMs combine users' statements with decision records to predict users' decisions, using 1,260 rounds in which 42 users played a repeated prisoner's dilemma-style game with five agents, each prompted to express a dominant Big Five trait, and could talk with the agent before every decision. Across eight open-weight LLMs, predictions from decision records separated cooperation from defection well (AUC = .80), and adding dialogue brought no detectable gain. With dialogue, predictions agreed with users' statements more often, and errors concentrated on rounds in which users broke their statements: accuracy on rounds with a statement barely changed (.73 to .74), while it rose on kept statements (.76 to .95) and fell sharply on broken ones (.65 to .17). Predictions weighted statements more heavily than users' behavior warranted: when users stated a decision they had not yet made in that game, the predicted probability of keeping it rose from .13 to .69, whereas users kept .40 of such statements. Users' decisions also varied with the agent they faced, but predictions hardly reflected this: users cooperated more with the Agreeableness agent than with the Extraversion agent (70.6% vs. 54.5%), whereas predicted cooperation was nearly identical (64.5% vs. 63.1%). Users also cooperated more with an agent when they themselves scored higher on that agent's trait, whereas predicted cooperation barely changed with these scores. Dialogue-based predictions should therefore weigh users' statements by their earlier decisions and the agent they face.
Figures & tables
Figure 1. LLM predictions varied with users’ stated decisions but barely varied with the agent users faced, although users’ behavior did. (a) Each user played a six-round game with each of five agents, each prompted to express a dominant Big Five trait, and could talk with the agent before deciding to cooperate or defect in every round. Eight open-weight LLMs predicted each decision from earlier decisions (deeds) and the round’s dialogue (words). (b) Left: adding dialogue barely changed accuracy on rounds with a stated decision, but raised it on kept statements and lowered it on broken ones. Right: users cooperated more with the Agreeableness than with the Extraversion agent, whereas predictions from all decisions and dialogue barely differed. Two panels. (a) Left: a user linked to five agent personas, Openness, Conscientiousness, Extraversion, Agreeableness and Neuroticism. Right: one game with an agent; rounds 1 to 3 show the user's decisions (cooperate, cooperate, defect), labeled deeds, and round 4 shows the user saying I'll cooperate this round, labeled words. Both feed six LLM families (Qwen, Gemma, Seed-OSS, GLM, Ministral, Olmo), whose predicted round-4 decision is compared with the actual decision. (b) Left: accuracy with deeds only and with deeds and words: all rounds .73 and .74, kept .76 and .95, broken .65 and .17. Right: cooperation rate with the Agreeableness and Extraversion agents: users 70.6 and 54.5 percent, LLM predictions 64.5 and 63.1 percent.
Figure 2. Experiment design. ➀ The five materials given to the predictor, defined by game and round. ➁ The 19 combinations of materials shown to eight LLM predictors, and the sentence inserted into rounds without a stated decision. ➂ What each research question measures in users’ behavior and in the predictions. Three panels. Left: small grids of games by rounds showing the five materials given to the predictor. Middle: a four by five matrix of the 19 information conditions and the inserted sentence. Right: the measures used for each research question.
Figure 3. The decision record carried the ranking information, and dialogue brought no detectable gain once the record was shown. AUC under each combination of decisions and dialogue shown to the model (games 2–5), grouped by decisions (a, c) or by dialogue (b, d). Dashed line: AUC of the decision record alone ( R ); dotted line: chance. Bars: 95% user-bootstrap intervals; brackets: Holm-corrected tests. ∗p<.05 , ∗∗p<.01 , ∗∗∗p<.001 . Four panels of AUC with 95 percent intervals for the 19 information conditions. Conditions with a decision record reach AUC near .80; adding dialogue does not raise it further.
Figure 4. Adding what users said moved the predictions toward their statements, also where users then broke them (a) and beyond the observed rate of keeping them (b), while earlier broken statements were only partly used (c); one inserted sentence set the prediction (d). Accuracy, agreement with the recorded decision (d) or predicted probability before (open) and after (filled) adding text to the prompt (games 2–5; group sizes in Appendix G ); black bars: observed rates. Light bars: 95% user-bootstrap intervals; brackets: Holm-corrected tests. ∗p<.05 , ∗∗p<.01 , ∗∗∗p<.001 . Four dot plots of model accuracy or predicted probability before and after adding text to the prompt. Adding the current-round dialogue raises accuracy where the stated decision was kept and lowers it sharply where it was broken.
Figure 5. Users’ decisions shifted with the agent persona and with trait compatibility; the predictions barely varied with either. a, b: users’ rates by persona (all games; dots are games). c–e: AUC and prediction bias by persona (games 2–5). f–i: relative rates and prediction bias against trait compatibility (dots: games; lines: pooled slopes with 95% user-bootstrap bands). Bars: 95% user-bootstrap intervals; brackets: Holm-corrected tests. ∗p<.05 , ∗∗p<.01 . Nine panels. Top two rows by agent persona: users' cooperation and word-keeping rates differ between personas, AUC is similar across personas, and prediction bias is positive for the Extraversion agent and negative for the Agreeableness agent. Bottom row: scatter plots against trait compatibility with pooled slopes.
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
Model
Hugging Face identifier
Qwen3.5-9B
Qwen/Qwen3.5-9B
Qwen3.8-27B
Qwen/Qwen3.8-27B
Gemma-4-12B
google/gemma-4-12B-it
Gemma-4-31B
google/gemma-4-31B-it
Seed-OSS-36B
ByteDance-Seed/Seed-OSS-36B-Instruct
GLM-4-32B
zai-org/GLM-4-32B-0414
Appendix
Table 1. Predictor models.
Subset
Rounds
Users
Findings
All rounds, games 1–5
1,260
42
5, 7
All rounds, games 2–5
1,008
42
1, 6, 8
With a stated decision (Fig. 4 a)
646
41
1, 2
kept / broken
470 / 176
41 / 31
Rounds 2–6 with a stated decision (Fig. 4 b)
532
41
3
chosen earlier / not yet chosen
479 / 53
41 / 20
Appendix
Table 2. Rounds and users in each analysis subset.
Large language models (LLMs) are increasingly used as agents in simulations of social systems, yet it remains unclear when their behavior can be interpreted as a faithful proxy for human decision-making. Here we test LLM agents against a direct empirical benchmark: a large-scale networked Prisoner's Dilemma experiment with human participants. Using the same interaction protocol, payoff structure, and network topologies, we compare nine open-weight LLMs with the human data. The selected model reproduces several macro-level features of cooperation dynamics, including the early decline and later stabilization of cooperation. This aggregate agreement, however, does not extend uniformly to finer levels of behavior. LLM populations underestimate individual-level heterogeneity and generate conditional cooperation patterns that differ from those observed in humans. Adding a fraction of random agents improves some aspects of micro-level agreement, but does not remove the mismatch in decision rules. These findings reveal a macro--micro dissociation in LLM-based social agents: collective outcomes can appear human-like even when the underlying behavioral distributions and mechanisms are not. They suggest that validating LLM agents as human surrogates requires comparisons across aggregate dynamics, individual heterogeneity, and context-dependent decision rules, rather than outcome-level agreement alone.
Henrique Ferraz de Arruda, Carlos Gracia Lázaro, Alberto Aleta +1
ARAID Foundation, Zaragoza, Spain. · Institute for Biocomputation and Physics of Complex Systems, University of Zaragoza, Zaragoza, Spain. · Universidad San Jorge (USJ), Zaragoza, Spain. +1
As LLM-based agents with user-instructed goals are becoming widely deployed, they increasingly encounter each other in strategic interactions, and face challenges of finding mutually beneficial outcomes. Prior literature has argued that cooperation problems such as the Prisoner's Dilemma are resolvable in settings where agents know they follow very similar decision making patterns, as for example in monocultural AI ecosystems. Following that line of work, this paper introduces the first framework for evaluating LLM decision making when agents are provided with graded similarity signals. Among our findings, we establish that different LLM models vary drastically in how they navigate similarity signals, with some modern models showing consistent behavior across cooperation problems, payoff structures, and prompt framing. Perhaps surprisingly, our experiments also show that the dataset based on which the similarity signal is computed has small to no impact on induced cooperation, and that LLM models systematically self-identify as highly similar when asked to evaluate another model's chain-of-thought reasoning by themselves. Finally, we develop an LLM-behavioral-game-theoretic model that captures some of their reasoning rationale, and show that it can support cooperative outcomes in equilibrium under sufficiently high similarity scores.
We examine whether large language models (LLMs) can predict biased decision-making in conversational settings, and whether their predictions capture not only human cognitive biases but also how those effects change under cognitive load. In a pre-registered study (N = 1,648), participants completed six classic decision-making tasks via a chatbot with dialogues of varying complexity. Participants exhibited two well-documented cognitive biases: the Framing Effect and the Status Quo Bias. Increased dialogue complexity resulted in participants reporting higher mental demand. This increase in cognitive load selectively, but significantly, increased the effect of the biases, demonstrating the load-bias interaction. We then evaluated whether LLMs (GPT-4, GPT-5, and open-source models) could predict individual decisions given demographic information and prior dialogue. While results were mixed across choice problems, LLM predictions that incorporated dialogue context were significantly more accurate in several key scenarios. Importantly, their predictions reproduced the same bias patterns and load-bias interactions observed in humans. Across all models tested, the GPT-4 family consistently aligned with human behavior, outperforming GPT-5 and open-source models in both predictive accuracy and fidelity to human-like bias patterns. These findings advance our understanding of LLMs as tools for simulating human decision-making and inform the design of conversational agents that adapt to user biases.