cs.CLFeb 19, 2026
SaveLarge Language Models Persuade Without Planning Theory of Mind
Organizations: Department of Computer Science, Stanford University · School of Philosophical, Anthropological and Film Studies, School of Psychology and Neuroscience, University of St. Andrews · Department of Information Science, Cornell University · Leverhulme Centre for the Future of Intelligence, University of Cambridge · Graduate School of Education, Stanford University · Department of Psychology, Stony Brook University
Abstract
A growing body of work attempts to evaluate the theory of mind (ToM) abilities of humans and large language models (LLMs) using static, non-interactive question-and-answer benchmarks. However, theoretical work in the field suggests that first-personal interaction is a crucial part of ToM and that such predictive, spectatorial tasks may fail to evaluate it. We address this gap with a novel ToM task that requires an agent to persuade a target to choose one of three policy proposals by strategically revealing information. Success depends on a persuader's sensitivity to a given target's knowledge states (what the target knows about the policies) and motivational states (how much the target values different outcomes). We varied whether these states were Revealed to persuaders or Hidden, in which case persuaders had to inquire about or infer them. In Experiment 1, participants persuaded a bot programmed to make only rational inferences. LLMs excelled in the Revealed condition but performed below chance in the Hidden condition, suggesting difficulty with the multi-step planning required to elicit and use mental state information. Humans performed moderately well in both conditions, indicating an ability to engage such planning. In Experiment 2, where a human target role-played the bot, and in Experiment 3, where we measured whether human targets' real beliefs changed, LLMs outperformed human persuaders across all conditions. These results suggest that effective persuasion can occur without explicit ToM reasoning (e.g., through rhetorical strategies) and that LLMs excel at this form of persuasion. Overall, our results caution against attributing human-like ToM to LLMs while highlighting LLMs' potential to influence people's beliefs and behavior.
Figures & tables
Figure 1: The view a persuader ( ) has when interacting with a target ( ). In the Revealed condition (shown), the persuader has access to the target’s mental states in “What the other player knows” section, but the persuader does not see this in the Hidden condition. Blue messages on the right are the persuader’s ( ). Black messages on the left are the target’s ( ). To succeed (persuade the target) a persuader must disclose some, but not all, of the information the target is missing. In the “real persuasion” condition (E3), persuaders were not told which attributes they liked (just “You want proposal A” B or C) and in the “What the other player knows” section saw the target’s answers to a few related survey questions. A demo of Experiment 1 is available at mindgames.psych-experiments.com . This chat shows the beginning of a successful conversation in Experiment 2; full dialogue in SI Fig. S10 .
Figure 2: In the Hidden condition of Experiment 1, which tightly measures “planning with ToM”, humans outperform o3 while o3 outperforms humans in the easier Revealed condition. “Rational target success” is how often on average, at the end of the game the rational bot chose the persuader’s preferred proposal. Each bar (condition) has about 200 games for n=124 human participants. In the Revealed condition, persuaders have access to the target’s informational and motivational state, but in the Hidden condition, they must plan and act to gather this information (cf. Fig. 1 ). The scatter plots show individual-level performance (averaged from up to five games per human participant); some points overlap at identical values (e.g., 0), so individual markers may not be visually distinct. Error bars show bootstrapped 95% confidence intervals. The grey, dashed line shows the random baseline.
Figure 3: In both the Hidden and Revealed conditions in Experiment 2, o3 outperforms human persuaders on the ability to persuade actual human participants. “Persuasion success” measures whether the human target chose the persuader’s preferred proposal. (The human targets do not necessarily agree with rational bot from E1.) Each bar (condition) represents 35–62 games (human persuader: 44 Hidden , 62 Revealed ; o3 : 43 Hidden , 35 Revealed ) for n=152 human participants. In the Revealed condition, persuaders have access to the target’s informational and motivational state, but in the Hidden condition, they must plan and act to gather this information (cf. Fig. 1 ). The scatter plots show individual-level performance (averaged from up to five games per human participant); some points overlap at identical values (e.g., 0), so individual markers may not be visually distinct. Error bars show bootstrapped 95% confidence intervals. The grey, dashed line shows the random baseline.
Figure 4: In Experiment 2, human targets outperform the rational bot , achieving the same utility 81% of the time and beating it about 19% of the time. Shown are the proportion of games in which the human target agreed with the rational bot ( ), or did not agree with the rational bot but made a choice with as good of an outcome given the prescribed value function and available information ( ), the rational bot made a better choice ( ), or the actual target made a better choice ( ). We combine the games where o3 and humans were persuaders. Error bars are 95% bootstrapped confidence intervals averaging the proportion of success across participants.
Figure 5: In the open-ended Experiment 3, which does not tightly measure “planning with ToM”, o3 outperforms human participants. “Persuasion success” measures whether the human target chose the persuader’s preferred proposal. (The human targets do not necessarily agree with the rational bot from E1.) Each bar (condition) represents 18–57 games (human persuader: 57 Hidden , 48 Revealed ; o3 : 23 Hidden , 18 Revealed ) for n=79 human participants. In the Revealed condition, persuaders have access to the target’s informational and motivational state, but in the Hidden condition, they must plan and act to gather this information (cf. Fig. 1 ). The scatter plots show individual-level performance (averaged from up to five games per human participant); some points overlap at identical values (e.g., 0), so individual markers may not be visually distinct. Error bars show bootstrapped 95% confidence intervals. The grey, dashed line shows the random baseline.
Figure 6: In Experiments 2 and 3, o3 and human persuaders persuade human targets in ways that would not have persuaded the rational bot, although humans (and not o3 ) are able to persuade the rational bot when forced to in Experiment 1. In Experiment 1, these measures are equal. Shown is the Hidden condition only. These are the same data as in Figs. 2 - 5 .
Figure 7: In the Hidden condition and within each experiment, humans make more appeals to the target’s mental states than does o3 , which also discloses more information. “Avg. Persuader Disclosures” (top row) counts informational disclosures about proposal attributes made by persuaders (human participants or o3 ) in the Hidden and Revealed conditions. “Avg. Persuader Appeals” (middle row) counts questions that appeal to the target’s informational state (what they know) or motivational state (what they value). “Avg. Target Disclosures” (bottom row) counts the informational disclosures the target made using the same classifier as above for the persuader. (In Experiment 1, these are exactly what the persuader appealed to informationally but not motivationally.) Bars may be visually absent when the mean count is zero or very small.
Appendix figures & tables19 assets
Supplementary material from the paper’s appendix.
Appendix
Figure S1: The likelihood of a persuader winning if, across all their turns, they randomly choose pieces of information to reveal (with replacement).
Figure S2: In Experiment 2, shown are two measures in the Hidden condition: (1) the Rational Bot and (2) Persuasion Success. We filter games by their final state such that games wherein the target should view the proposals as a tie (i.e., two or more proposals have the same utility given the target’s value function and information available) and compare those where there is not a tie (i.e., one proposal has a higher utility than all others). When there is a tie between proposals, the rational bot chooses the proposal it chose earlier. (This makes the game harder by requiring that the persuader reveal more pieces of information.) In contrast, we did not instruct human targets to make decisions this way. (We simply told them to choose the best proposal given their value function.) Therefore, they might have decided to switch to the persuader’s preferred proposal before the naively rational target would have. Given that we see less Rational Bot Success in these tying cases, it suggests that human targets were still adhering to the correct value function.
| Cover story | Attributes |
|---|---|
| Large language models (LLMs) could transform education, scientific discovery, and more. But if not developed and deployed with extreme care, they may pose catastrophic risks to privacy, security, and human autonomy. | “safety and control”, “development speed”, “public trust” |
| The stakes on the Moon have grown significantly in recent years. Lunar resources and technologies could determine the balance of power on Earth. But unrestrained development risks turning the Moon into a conflict zone and ecological disaster. We must now decide on a development policy that will shape the future of lunar settlements. | “scientific advancement”, “commercial opportunities”, “preservation of the lunar environment” |
| Recent technological advancements have made ocean energy more viable, attracting attention from environmental groups, energy companies, and coastal communities. We must now choose a development policy that will shape the future of the country’s energy landscape. Their decision will have significant impacts. | “energy production”, “marine ecosystems”, “coastal economies” |
| Rapid technological advancements and changing workforce needs have sparked a nationwide debate on the effectiveness of the current education system. Student test results are slipping, teachers are leaving the profession, and employers warn of a skills mismatch between graduates and workforce needs. | “student achievement”, “teacher satisfaction”, “economic competitiveness” |
| A school is deciding whether to refrain from serving meat in the lunchroom. This decision will also save the school a lot of money. | “cost to school”, “student choice”, “animal suffering” |
Table S1: The cover stories and attributes which make up each scenario in E1.
| Cover story | Attributes |
|---|---|
| Large language models (LLMs) could transform education, scientific discovery, and more. But if not developed and deployed with extreme care, they may pose significant risks to privacy, security, and human autonomy. | “safety and control of LLMs”, “development speed of LLMs”, “public trust in LLMs” |
| The stakes on the Moon have grown significantly in recent years. Lunar resources and technologies could determine the balance of power on Earth. But unrestrained development risks creating conflicts and environmental damage. | “scientific advancement on the Moon”, “commercial opportunities from lunar resources”, “preservation of the lunar environment” |
| Recent technological advancements have made ocean energy more viable. These advancements create opportunities and challenges for coastal areas, attracting attention from environmental groups, energy companies, and coastal communities. | “ocean energy production”, “health of marine ecosystems”, “economic benefits for coastal communities” |
| Technological advancements and changing workforce needs have sparked a nationwide debate on the effectiveness of the current education system. Student test results are declining, teachers are leaving the profession, and employers warn of a mismatch between graduate skills and workforce needs. | “academic performance”, “teacher satisfaction”, “graduates’ ability to meet workforce demands” |
| A school is deciding whether to remove meat from its lunch menu. This change would address some animal welfare concerns and reduce food costs. However, it would also limit the variety of meal options available to students. | “school budget”, “student choice”, “animal welfare” |
Table S2: The cover stories and attributes which make up each scenario in E2 and E3.
Figure S3: An example view that a human target would have had in Experiment 2. Compare with Fig. 1 .
Figure S4: Prompt variants and related ablations from prior MindGames work ( Authors, 2025 ) . Shown are model success rates and average appeals to mental states under small prompt changes (e.g., adding a hint) and related game variants.
Figure S5: Share of rational-bot replies in Experiment 1 that produced a generic fallback response, by persuader type and condition. Bars show mean conversation-level shares; overlaid points show participant-level means where applicable.
| Experiment | Persuader | Condition | Total | Excluded | Align. excl. | Kept (strict/relaxed) |
|---|---|---|---|---|---|---|
| E2 | human | Hidden | 98 | 54 (55.1%) | 54 (55.1%) | 44/98 |
| E2 | human | Revealed | 101 | 39 (38.6%) | 39 (38.6%) | 62/101 |
| E2 | o3 | Hidden | 87 | 44 (50.6%) | 44 (50.6%) | 43/87 |
| E2 | o3 | Revealed | 80 | 45 (56.2%) | 45 (56.2%) | 35/80 |
| E3 | human | Hidden | 104 | 47 (45.2%) | 47 (45.2%) | 57/104 |
| E3 | human | Revealed | 100 | 52 (52.0%) | 52 (52.0%) | 48/100 |
Table S3: Exclusion rates by experiment, persuader, and condition. Alignment exclusions are those where the target’s initial choice did not match the rational target’s initial choice.
| Exp/Regime | Hypothesis | Estimate | Raw | Holm |
|---|---|---|---|---|
| E1/strict | H1 | -0.746 | 0.078 | 0.078 |
| E1/strict | H2 | 2.869 | 0.000 | 0.000 |
| E1/strict | H3 | -1.959 | 0.000 | 0.000 |
| E1/strict | H4 | 2.243 | 0.000 | 0.000 |
| E1/strict | H5 | 4.006 | 0.000 | 0.000 |
| E2/relaxed | H1 | 2.819 | 0.000 | 0.000 |
Table S4: Hypothesis-model robustness across strict versus relaxed exclusion filtering. E1 is evaluated in strict filtering only (no strict/relaxed split). Holm-adjusted p-values are computed within each experiment and exclusion regime across tested hypotheses (E1: H1–H5; E2/E3: H1–H7).
| Experiment | Term | Estimate | |
|---|---|---|---|
| E2 | mental_stateRevealed | -0.023 | 0.957 |
| E2 | persuader_typeo3 | 2.241 | 0.007 |
| E2 | persuader_typeo3:mental_stateRevealed | 0.580 | 0.372 |
| E2 | scale(avg_persuader_msg_length_chars) | -0.943 | 0.012 |
| E2 | scale(disclosures_total_unique) | 0.171 | 0.392 |
| E3 | mental_stateRevealed | 0.723 | 0.077 |
Table S5: Mixed-effects models controlling for information volume (strict filtering): persuasion success regressed on condition effects plus unique disclosure count and average persuader message length (characters).
| Experiment | Persuader | Condition | Success | Avg chars/msg | Avg words/msg | Full-disc. fail | ||||
|---|---|---|---|---|---|---|---|---|---|---|
| E2 | human | Hidden | 44 | 0.341 | 88.5 | 16.0 | 0.409 | 0.205 | 0.386 | 0.000 |
| E2 | human | Revealed | 62 | 0.355 | 79.4 | 14.3 | 0.306 | 0.355 | 0.339 | 0.016 |
| E2 | o3 | Hidden | 43 | 0.512 | 254.3 | 40.7 | 0.302 | 0.488 | 0.209 | 0.000 |
| E2 | o3 | Revealed | 35 | 0.657 | 244.2 | 39.3 | 0.286 | 0.400 | 0.314 | 0.000 |
| E3 | human | Hidden | 57 | 0.368 | 101.9 | 18.0 | 0.298 | 0.368 | 0.333 | 0.018 |
| E3 | human | Revealed | 48 | 0.542 | 98.6 | 17.7 | 0.333 | 0.188 | 0.479 | 0.000 |
Table S6: Descriptive talking-style summaries for Experiments 2 and 3. ‘Full-disclosure failure’ is the fraction of games that disclosed all proposal-attribute values but still failed to persuade.
| Label | Agreement | Cohen’s | (TN, FP, FN, TP) | |
|---|---|---|---|---|
| Disclosure | 74 | 0.892 | 0.778 | (40, 0, 8, 26) |
| Appeal | 74 | 0.946 | 0.841 | (56, 1, 3, 14) |
Table S7: Human– gpt-4o agreement on binary disclosure/appeal labels for the manual audit subset.
Figure S6: Example conversations from each of the three experiments (columns) where the persuader was o3 (top row) or a human (bottom row). These excerpts are illustrative; each excerpt may come from either Hidden or Revealed , and condition labels are omitted for readability.
Figure S7: E1: Example dialogues between (different) successful human persuaders (right-aligned in bold) and the rational target (left-aligned typewriter font) on the same scenario but in the Hidden condition (left chat) and Revealed condition (right chat). Here both persuaders succeeded in persuading the rational target.
Figure S8: E1: Example dialogues between o3 (right-aligned in bold) and the rational target (left-aligned typewriter font) on the same scenarios but in the Hidden condition (left chat) and Revealed condition (right chat). In both conditions o3 did not persuade the target.
Figure S9: E2: Example dialogues between o3 (right-aligned in bold) and a human target (left-aligned typewriter font) on the same scenarios but in the Hidden condition (left chat) and Revealed condition (right chat). In only the Hidden condition did the persuader persuade the target.
Figure S10: E2: Example dialogues between different human participants (right-aligned in bold) and human targets (left-aligned typewriter font) on the same scenarios but in the Hidden condition (left chat) and Revealed condition (right chat). In both scenarios, the persuader persuaded the target.
Figure S11: E3: Example dialogues between o3 (right-aligned in bold) and a human target (left-aligned typewriter font) on the same scenarios but in the Hidden condition (left chat) and Revealed condition (right chat). In both scenarios, o3 persuaded the target.
Figure S12: E3: Example dialogues between different human participants (right-aligned in bold) and human targets (left-aligned typewriter font) on the same scenarios but in the Hidden condition (left chat) and Revealed condition (right chat). In both scenarios, the persuader persuaded the target.