A Thousand Faces: A Persuasion Benchmark with Experience-Grounded Persuadees Informed by Psychology and Pragmatics

Ziqing Zhuang; Dingyi Zhang; Deyu Zhou

cs.CLOct 18, 2026

A Thousand Faces: A Persuasion Benchmark with Experience-Grounded Persuadees Informed by Psychology and Pragmatics

Authors: Ziqing Zhuang, Dingyi Zhang, Deyu Zhou

Organizations: Southeast University

Abstract

People can express the same beliefs and desires yet need different reasons to change their minds. What makes an argument relevant depends on how those views arose: experiences shape what a person finds credible, what matters to them, and what could prompt reconsideration. These personal grounds may also be expressed indirectly, through contextual clues in conversation. Benchmarks that specify beliefs and desires without these connections can reward arguments that match a description while overlooking whether they address the person's reasons. We introduce A Thousand Faces (ATF-Persu), with 1,000 personas across 200 topics and nine domains. Drawing on personality and attitude theories and conversational implicature, we construct five personas per factual context, linking experiences to judgments, expression, and grounds for revision. Evaluations show lower persuasion success than three comparison benchmarks under their respective protocols. Complete backgrounds yield broader use of supplied personal grounds and higher persona faithfulness than simplified specifications; blinded reviewers also rate ATF-Persu above CToMPersu on identifiable personal reasons, their influence on responses, and contextual expression. The dataset is publicly available at https://huggingface.co/datasets/Slandes/ATF-Persu.

Figures & tables

Figure 1: Illustration of how personal experiences shape persuasion, even when initial beliefs and desires are shared.

Figure 2: Persona construction and persona-grounded interaction in ATF-Persu, informed by psychology and pragmatics.

Model	Method	CToMPersu	DailyPersuasion	INSPIRED	ATF-Persu
DeepSeek	Direct	80.00 $\pm$ 1.01	78.10 $\pm$ 0.76	99.33 $\pm$ 0.58	19.53 $\pm$ 2.48
	Meta	80.95 $\pm$ 1.56	78.35 $\pm$ 1.72	98.65 $\pm$ 0.58	21.53 $\pm$ 1.01
	Strategic	91.81 $\pm$ 0.33	89.08 $\pm$ 1.08	100.00 $\pm$ 0.00	37.43 $\pm$ 0.45
GPT-5 nano	Direct	65.08 $\pm$ 0.86	67.81 $\pm$ 0.83	78.79 $\pm$ 3.64	25.23 $\pm$ 1.55
	Meta	64.51 $\pm$ 2.68	66.10 $\pm$ 1.33	75.42 $\pm$ 3.82	34.53 $\pm$ 1.29
	Strategic	89.40 $\pm$ 1.30	89.46 $\pm$ 2.76	81.82 $\pm$ 1.01	48.93 $\pm$ 0.74

Table 1: Persuasion success (%) across benchmarks. Values are means

\pm

sample standard deviations over three runs. Model groups identify the persuadee and judge; DeepSeek generates and plans all persuasive messages.

Dataset / specification	Effective individualization breadth $\uparrow$	Persona faithfulness (%) $\uparrow$
CToMPersu	2.66 [2.47, 2.87]	61.77 [56.07, 67.52]
DailyPersuasion	0.93 [0.80, 1.06]	34.60 [28.03, 41.68]
INSPIRED	N/A	N/A
ATF-Persu (S)	2.09 [1.91, 2.25]	63.39 [57.92, 68.78]
ATF-Persu (F)	4.10 [3.76, 4.46]	85.01 [81.03, 88.68]

Table 2: Six-turn persona realization with 95% person-bootstrap confidence intervals. Faithfulness is defined for 99 persons in DailyPersuasion and ATF-Persu (S), and 100 in the other scored conditions. INSPIRED lacks a supplied persona reference (N/A).

Dataset	Persona consistency $\uparrow$		Mental-state explicitness
	Score	Gap $\downarrow$	Score	Gap $\downarrow$
P4G (human)	8.018	—	4.038	—
CToMPersu	7.606	0.412	5.594	1.557
ATF-Persu	8.060	0.042	4.953	0.916

Table 3: Automated persona consistency and mental-state explicitness (1–10), using 500 randomly sampled dialogues per group and metric. Gap is the absolute difference from the corresponding P4G mean before rounding; bold marks the smaller gap among simulated groups.

Dataset	Personal reasons $\uparrow$	Reasons shaping responses $\uparrow$	Contextual expression $\uparrow$
CToMPersu	5.91	6.18	5.85
ATF-Persu	6.62	6.63	6.16

Table 4: Human evaluation (1–7; higher is better). Each dimension averages two items scored by three reviewers blind to dataset identity.

Figure 3: First replies to the same opening across three independent runs, without a personal background (a) and with Fiona’s background (b). Ellipses mark omissions; labels and portraits summarize response differences.

Appendix figures & tables17 assets

Supplementary material from the paper’s appendix.

Appendix

Study	Sample	Models	Protocol
Benchmark comparison	CToMPersu: 525; DailyPersuasion: 525; INSPIRED: 99; ATF-Persu: 1,000. Three methods $\times$ three model conditions $\times$ three runs; 58,023 dialogues.	Persuasion and planning: deepseek-flash. Persuadee and independently called success judge: deepseek-flash, gpt-5-nano, or qwen3.6-35b-a3b-fp8.	At most seven rounds; judge after each reply; stop on success. Report mean and sample standard deviation of three run-level success rates.
Persona realization	CToMPersu, DailyPersuasion, ATF-Persu (F), and ATF-Persu (S): 100 each; INSPIRED: 99. Total: 499 dialogues.	deepseek-flash for material processing, both speakers, and evaluation.	Six complete rounds, without early stopping. Evaluate the entire dialogue once; person-bootstrap 95% intervals with 2,000 resamples.
Persona enactment and mental-state explicitness	500 dialogues per group and metric: CToMPersu, ATF-Persu, and P4G.	Simulated persuadees: gpt-4o-mini. Evaluator: GPT-5.4, low reasoning effort, maximum 2,048 completion tokens.	Persona enactment uses personal reference plus dialogue; explicitness uses dialogue alone. Randomly sample 500 P4G dialogues for enactment and 500 from each group for explicitness.
Human evaluation	200 CToMPersu and 200 ATF-Persu dialogues, randomly sampled. Three reviewers rate every dialogue.	Three computer-science graduate students.	Dataset identities hidden; six items rated from 1 to 7. Average reviewers within dialogue, then dialogues; each dimension averages two items.
Fixed-context response stability	1,000 personas $\times$ three fixed probes $\times$ three independent repetitions $\times$ three actor models = 27,000 replies.	Actors: deepseek-flash, gemma-4-26b-a4b-it-fp8-dynamic, and gpt-5-nano. Judge fixed to deepseek-flash.	Each trial generates one reply to the same saved history and probe, followed by one seven-class judgment.

Appendix

Table A1: Study configurations.

Role	Information
Benchmark / persuader and planning	Public scenario and goal; live dialogue. Meta infers preventive/generative beliefs and desires from this information; Strategic estimates stance from it.
Benchmark / CToMPersu and DailyPersuasion persuadee	Domain, background, goal, supplied preventive/generative fields, live dialogue, and final-round flag.
Benchmark / INSPIRED persuadee	The same scenario fields, target-movie card, and live dialogue; all 99 frozen cases have an unknown seeker profile.
Benchmark / ATF-Persu persuadee	Complete private persona and live utterances rendered as “you” and “other person”. The judge’s full target is not added to the actor input.
Benchmark / success judge	External benchmarks: scenario fields and live dialogue, plus movie/profile context for INSPIRED. ATF-Persu: full target and live dialogue.
Persona realization / persuader	Public scenario, public goal, live dialogue, and current turn function.

Appendix

Table A2: Information available to each role.

Method	Procedure
Direct	Generate the next message from the public context and current dialogue in one call.
Meta	Opening: request fewer than five strategies, then generate a message. Later rounds: infer six preventive/generative fields; retrieve the two highest-count domain strategies from the fixed knowledge base (global-count fallback); request five concrete strategies; generate the next message.
Strategic	Estimate a 1–7 receiver support score, opinion, and question before the first message. Generate from the latest estimate and dialogue; update the estimate after every persuadee reply, including the terminal round. The benchmark’s success judge controls stopping.

Appendix

Table A3: Inference-time persuasion methods.

Dimension	Item	Statement
A	Q1	I can identify specific personal reasons for the persuadee’s current stance from the dialogue.
A	Q2	I can understand from the dialogue how these reasons relate to this person’s individual circumstances.
B	Q3	The persuadee provides a substantive evaluation of the specific information or suggestions offered by the other speaker.
B	Q4	The grounds on which the persuadee accepts, questions, or withholds judgment are clearly connected to the personal reasons evident in the dialogue.
C	Q5	The timing of the persuadee’s explanations of their personal views fits the needs of the ongoing conversation.
C	Q6	The level of detail with which the persuadee explains their personal reasons fits the needs of the ongoing conversation.

Appendix

Table B1: Full questionnaire. Items A/Q1–Q2 concern personal reasons, B/Q3–Q4 their influence on responses, and C/Q5–Q6 contextual expression.

Dimension	Low	Middle	High
A. Personal reasons	Only agreement or disagreement is apparent, and the reasons are generic.	Reasons can be identified, but their connection to the person’s own circumstances is limited.	Specific reasons can be identified, and their personal basis can be understood.
B. Reasons shaping responses	The persuadee ignores specific information or repeats the same point regardless of what the other speaker says.	The persuadee responds to some of the content, but the basis for their judgment is unclear.	It is clear which issues the information resolves or leaves unresolved, and how the persuadee judges it on that basis.
C. Contextual expression	The persuadee gives irrelevant self-descriptions, repeatedly unloads the same reasons, or provides no clues when an explanation is needed.	Expression is broadly appropriate, but some explanations come too early, include too much detail, or provide too little.	The timing and level of detail fit the needs of the exchange; either direct or indirect expression can be appropriate.

Appendix

Table B2: Qualitative scoring anchors for the three dimensions. The original seven-point ratings are retained.

Item	CToMPersu	ATF-Persu	$\Delta$
Q1	6.09	6.51	+0.42
Q2	5.73	6.73	+1.00
Q3	6.33	6.82	+0.49
Q4	6.02	6.44	+0.42
Q5	6.25	6.34	+0.09
Q6	5.44	5.98	+0.54

Appendix

Table B3: Item-level human ratings on the 1–7 scale.

\Delta

is ATF-Persu minus CToMPersu, computed from the displayed item means.

Model	Replies	Overall	Message 1	Message 2	Message 3
DeepSeek	9,000	8.83	5.60	8.57	12.33
Gemma	9,000	2.13	1.60	2.43	2.37
GPT-5 nano	9,000	2.92	2.40	2.80	3.57

Appendix

Table C1: Strict acceptance by model and fixed message (%). Each message column contains 3,000 replies per model.

Model pair	Pairs	Binary	Seven-state	$p_{e}$	$\kappa$
DeepSeek / Gemma	9,000	91.06	60.36	89.41	0.155
DeepSeek / GPT-5 nano	9,000	89.42	51.10	88.76	0.059
Gemma / GPT-5 nano	9,000	95.50	58.74	95.07	0.087

Appendix

Table C2: Cross-model agreement at matched checkpoints. Agreement and

p_{e}

are percentages;

\kappa

is dimensionless.

Model	Pairs	Binary	Seven-state	$p_{e}$	$\kappa$
DeepSeek	9,000	90.27	57.39	83.89	0.396
Gemma	9,000	97.47	75.28	95.82	0.393
GPT-5 nano	9,000	95.20	57.60	94.33	0.154

Appendix

Table C3: Agreement between repeated generations of the same model. Each checkpoint contributes three pairs.

Model	Repeat 1	Repeat 2	Repeat 3
DeepSeek	8.97	8.80	8.73
Gemma	2.00	2.17	2.23
GPT-5 nano	2.73	3.33	2.70

Appendix

Table C4: Strict acceptance across the three independent repetitions (%); 3,000 replies per model and repetition.

Figure D1: Success rates by domain, method, and receiver condition. Each cell is the equal-topic average of five persona proportions, each based on three recorded outcomes. The same 0–70% colour scale is used in all three panels. Parentheses give the number of topics; the receiver and success judge both change between panels, while the persuader remains DeepSeek.

Domain	Topics	Personas	Direct	Meta	Strategic
DeepSeek
Climate, environment & energy	22	110	26.4	23.3	40.9
Consumer product safety	23	115	23.8	25.5	44.6
Digital security & AI	22	110	17.0	15.8	32.4
Economy & finance	22	110	21.2	23.3	37.9
Food, nutrition & safety	22	110	24.8	24.8	42.4

Appendix

Table D1: Domain success rates (%). Topic and persona counts apply to every method in the row; each persona contributes three recorded dialogues per method. Values are rounded to one decimal place.

Figure D2: Distribution of the observed within-topic range among five persona success proportions. Each bar represents 200 topics under one fixed method and receiver condition; segments correspond to ranges of 0, 33.3, 66.7, or 100 percentage points. Labels within sufficiently wide segments show their share of topics (%). The underlying persona proportions each use three recorded outcomes.

Method	Success (%)	Range (pp)	Pairwise gap (pp)
(a) Success and mean gaps
DeepSeek
Direct	19.5	46.2	22.6
Meta	21.5	52.3	25.9
Strategic	37.4	68.3	34.3
GPT-5-nano

Appendix

Table D2: Within-topic persona differences and within-person outcome variation. Mean range and mean pairwise absolute gap are in percentage points and average equally over 200 topics. Unequal-persona topics have nonzero range; 0/3–3/3 topics have the full 100-point range. Mixed-outcome personas have one or two successes in their three recorded outcomes; their denominator is 1,000 personas. Panel (a) reports success and mean gaps; panel (b) reports the corresponding counts and percentages.

Method	Unequal topics	Full-range topics	Mixed personas
(b) Topic and persona counts
DeepSeek
Direct	179/200 (89.5%)	21/200 (10.5%)	417/1,000 (41.7%)
Meta	179/200 (89.5%)	24/200 (12.0%)	419/1,000 (41.9%)
Strategic	198/200 (99.0%)	61/200 (30.5%)	555/1,000 (55.5%)
GPT-5-nano

Appendix

Table D2: *

Figure E1: Troy’s replies with and without personal background share the same opening. The lower panel follows the background condition’s own subsequent dialogue. Both conversations run for three rounds with deepseek-flash. Quotation marks identify exact excerpts; arrows show conversation progression.

Figure E2: Personal basis, scope of agreement, and target status in two separate cases. Fiona’s conversation lasts three rounds; Arlene’s Direct/DeepSeek conversation lasts seven and ends with the recorded label strict_reject. R denotes the round. Arrows follow each conversation; quotations are exact excerpts.

Persuasion in Language Models

Persuasion in language models studies how they influence human and AI beliefs or decisions, and how to measure and limit undue influence.

See all 76 papers

A Model of Multi-turn Human Persuadability Using Probabilistic Belief Tracing
Jun 3, 2026Jared Moore, Noah Goodman, Nick Haber +1Opinion Dynamics Human-AI Interaction
Stanford University · University of Washington
Change My View? The Dynamics of Persuasion and Polarization in Online Discourse
May 8, 2026David Freeborn, Malihe Alikani, Anthony SiciliaBelief Revision Opinion Dynamics
$Ψ$ -Bench: Evaluating Persona-Sensitive Influencing in Persuasive Dialogues
Jun 1, 2026Peixuan Han, Hongyi Du, Jiayu Liu +3LLM Personalization LLM Agent Evaluation
University of Illinois Urbana-Champaign

Jun 3, 2026J/K move · Enter open · S save

A Model of Multi-turn Human Persuadability Using Probabilistic Belief Tracing

Jared Moore, Noah Goodman, Nick Haber, Max Kleiman-Weiner

Stanford University · University of Washington

Opinion Dynamics Human-AI Interaction Persuasion in Language Models

Open paper AskPDF arXiv

Large language models can shift human beliefs across high-stakes domains, but most persuasion studies rely on pre/post belief change. These endpoint measures identify whether persuasion occurred, yet miss where and how beliefs moved within a dialogue. We present PERSUASIONTRACE, a framework for studying persuasion in human-LLM interaction. Built on a web-based experimental platform, PERSUASIONTRACE contributes a tool for multi-turn persuasion studies and a process-level evaluation protocol: it records multi-turn belief reports from human or simulated targets of persuasion, annotates persuader turns with rhetorical dimensions (logos/pathos/ethos), and evaluates simulators by fidelity to real human belief dynamics. Using this framework, we find that human targets group into two clusters of multi-turn belief updates and exhibit susceptibility to rhetorical strategies, and that LLMs are persuasive across generic and personalized topics, text and audio modalities, and multi-turn interactions. Prior work has chiefly used vanilla-prompted LLMs to simulate human targets, but we show that these simulators fail to replicate human belief dynamics. We introduce a Bayesian-network simulated target that maintains an explicit latent belief state over time so each persuader message yields cognitively realistic belief updates. In human-likeness evaluation, our Bayesian target scores near a human reference (81 vs 80), while baseline LLM targets score substantially lower (64). PERSUASIONTRACE reframes persuasion evaluation from endpoint movement alone to process fidelity, providing a stronger basis for scientific analysis and safer optimization of persuasive systems.

Abstract

Persuasion in Language Models

Explore similar work

A Model of Multi-turn Human Persuadability Using Probabilistic Belief Tracing

Change My View? The Dynamics of Persuasion and Polarization in Online Discourse

ΨΨΨ-Bench: Evaluating Persona-Sensitive Influencing in Persuasive Dialogues

$Ψ$ -Bench: Evaluating Persona-Sensitive Influencing in Persuasive Dialogues