How Much Does Persuasion Strategy Matter? LLM-Annotated Evidence from Charitable Donation Dialogues

Tatiana Petrova; Stanislav Sokol; Radu State

cs.CLMar 30, 2026

How Much Does Persuasion Strategy Matter? LLM-Annotated Evidence from Charitable Donation Dialogues

Authors: Tatiana Petrova, Stanislav Sokol, Radu State

Organizations: Interdisciplinary Centre for Security, Reliability and Trust (SnT) University of Luxembourg

Abstract

Which persuasion strategies, if any, are associated with donation compliance? Answering this requires fine-grained strategy labels across a full corpus and statistical tests corrected for multiple comparisons. We annotate all 10,600 persuader turns in the 1,017-dialogue PersuasionForGood corpus (Wang et al., 2019), where donation outcomes are directly observable, with a taxonomy of 41 strategies in 11 categories, using three open-source large language models (LLMs; Qwen3:30b, Mistral-Small-3.2, Phi-4). Strategy categories alone explain little variance in donation outcome (pseudo $R^2 \approx 0.015$ , consistent across all three annotators). Guilt Induction is the only strategy significantly associated with lower donation rates ( $Δ\approx -23$ percentage points), an effect that replicates across all three models despite only moderate inter-model agreement. Reciprocity is the most robust positive correlate. Target sentiment and interest predict whether a donation occurs but show at most a weak correlation with donation amount. These findings suggest that strategy identification alone is insufficient to explain persuasion effectiveness, and that guilt-based appeals may be counterproductive in prosocial settings. We release the fully annotated corpus as a public resource.

Figures & tables

Figure 1: Distribution of 11 persuasion strategy categories (Qwen3:30b). Each bar shows the number of persuader turns assigned to the category; percentages indicate the category’s share of all

N{=}10{,}600

persuader turns. Conversation Management turns (44.0%) are omitted.

	Pres.	Abs.	$\boldsymbol{p_{B}}$	$\boldsymbol{p_{FDR}}$
Strategy–donation (22 tests, $\chi^{2}$ )
Guilt	32.7%	56.0%	${<}.001$	${<}.001$
Reciprocity	72.2%	52.2%	.034	.016
Commit./Cons.	63.5%	51.2%	.049	.016
Target response–donation (MWU)
	Don.	No don.	$\boldsymbol{p}$

Table 1: Bivariate associations with donation outcome (Qwen3:30b). Top : donation rates (%) in dialogues where the strategy is present (Pres.) vs. absent (Abs.);

p_{B}

: Bonferroni,

p_{FDR}

: Benjamini-Hochberg correction over 22 tests. Effect sizes are small (

\phi=0.14

for Guilt,

0.10

for Reciprocity). Middle : mean target sentiment (

-1

+1

) and interest (

0

–

2

) in donated (Don.) vs. not-donated (No don.) dialogues, Mann-Whitney

U

test. Bottom : Pearson

r

between target response and donation amount (donors only). Cross-model robustness is reported in Table 4 .

Figure 2: (a) Donation rates in dialogues containing Guilt Induction (

n{=}104

) vs. dialogues without (

n{=}913

); difference of

-23.3

pp (

\chi^{2}{=}19.4

p<0.001

). This effect replicates across all three annotators (Table 4 ). (b) Donation rate by predominant target sentiment across a dialogue’s target turns (

\chi^{2}(2){=}44.3

V{=}0.21

p<0.001

Predictor	OR	95% CI	$\boldsymbol{p}$
Model 3 (parsimonious): pseudo $R^{2}=0.080$ , AUC $=0.67$
Sentiment	3.94	[2.30, 6.74]	${<}.001$
Interest	2.77	[1.64, 4.67]	${<}.001$
Reciprocity	2.41	[1.38, 4.20]	.002
Guilt Ind.	0.60	[0.37, 0.95]	.029

Table 2: Logistic regression predicting donation (binary), parsimonious Model 3. OR: odds ratio (

>1

= higher donation probability). Sentiment: mean target sentiment per dialogue (

-1

+1

). Interest: mean target interest per dialogue (

0

–

2

Strategy	n	Mean sent.	% neg.
Bottom 3 (lowest mean sentiment)
Guilt Induction	114	$+0.02$	27.2%
Fear Appeal	69	$+0.19$	21.7%
Unity	43	$+0.23$	23.3%
Top 3 (highest mean sentiment)
Commit. & Cons.	182	$+0.54$	7.1%

Table 3: Mean target sentiment (

-1

+1

) in the turn immediately following each persuader strategy (

n=5{,}387

persuader–target pairs). Only persuasion strategies with

n\geq 20

pairs shown; Conversation Management labels excluded. % neg.: proportion of negative target responses.

Finding	Qwen	Mistral	Phi-4	Repl.
Guilt $\Delta$ (pp)	$-23.3$ ***	$-23.0$ *	$-23.9$ **	3/3
Recip. $\Delta$ (pp)	$+20.1$ **	$+5.8$	$+10.4$ *	2/3
Categories $R^{2}$	.015	.011	.016	3/3
+ Sent./Int. $R^{2}$	.082	.079	.077	3/3

Table 4: Cross-model robustness.

\Delta

: difference in donation rate (pp) between dialogues with vs. without the strategy.

R^{2}

: McFadden pseudo

R^{2}

. Significance:

{}^{*}p{<}.05

{}^{**}p{<}.01

{}^{***}p{<}.001

(uncorrected

p_{raw}

; multiple-comparison correction is applied only to the primary Qwen model in Table 1 ). Repl.: models where the finding reaches significance (

p_{raw}

) or matches direction (for

R^{2}

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Category	Strategy	n	%
Norms / Morality / Values	Appeal to Values	543	5.1
	Moral Appeal	521	4.9
	Guilt Induction	129	1.2
	Self-feeling Appeal	138	1.3
Rational / Impact Appeal	Rational Appeal	928	8.8
Rational / Impact Appeal	Logical Appeal	45	0.4

Appendix

Table 5: Complete taxonomy: 41 persuasion strategies in 11 categories plus 9 conversation management labels (Qwen3:30b annotations). Columns show absolute turn counts and each label’s percentage of all

N{=}10{,}600

persuader turns (percentages sum to 100%). Five of the 41 taxonomy strategies (Punishing Activity, Overloading, Confusion Induction, Promise, and Activation of Impersonal Commitment) received zero assignments and are not shown.

Persuasion in Language Models

Persuasion in language models studies how they influence human and AI beliefs or decisions, and how to measure and limit undue influence.

See all 79 papers

Make an Offer They Can't Refuse: Grounding Bayesian Persuasion in Real-World Dialogues without Pre-Commitment
Oct 15, 2025Buwei He, Yang Liu, Zhaowei Zhang +6Game Theory LLM Decision-Making
Beijing University of Posts and Telecommunications · Beijing Institute for General Artificial Intelligence · Peking University +1
The Bots of Persuasion: Examining How Conversational Agents' Linguistic Expressions of Personality Affect User Perceptions and Decisions
Feb 19, 2026Uğur Genç, Heng Gu, Chadha Degachi +3Persuasion in Language Models Human-AI Decision Making
Delft University of Technology, Delft, Netherlands
Implicit Beliefs as Scalable Process Rewards for Strategic Persuasion
Oct 24, 2026Minqian Liu, Yuexi Shen, Heajun An +5Persuasion in Language Models Process Reward Models
Virginia Tech · UC Santa Barbara · STIR Lab +1

Oct 15, 2025J/K move · Enter open · S save

Make an Offer They Can't Refuse: Grounding Bayesian Persuasion in Real-World Dialogues without Pre-Commitment

Buwei He, Yang Liu, Zhaowei Zhang, Zixia Jia+5

Beijing University of Posts and Telecommunications · Beijing Institute for General Artificial Intelligence · Peking University+1

Game Theory LLM Decision-Making Persuasion in Language Models Bayesian Persuasion LLM Reasoning

Open paper AskPDF arXiv

Large language models (LLMs) still struggle with strategic persuasion, largely because existing approaches either neglect information asymmetry or rely on unrealistic pre-commitment assumptions. We introduce a type-induced commitment-communication mechanism that grounds Bayesian Persuasion (BP) in natural language dialogue without pre-commitment: the persuader narrates their potential types (e.g., honest vs. dishonest) to dynamically construct an information schema, enabling the persuadee to perform Bayesian belief updates within the conversation itself. We implement two variants: Semi-Formal-Natural-Language (SFNL) and Fully-Natural-Language (FNL), evaluating them against strong baselines across multiple LLMs and human judges. BP strategies consistently outperform baselines: SFNL excels in logical credibility, while FNL shows superior robustness and emotional resonance. We verify that gains stem from genuine Bayesian reasoning rather than superficial formatting, and we further show that supervised fine-tuning enables small models to match the persuasive performance of much larger ones.

Abstract

Persuasion in Language Models

Explore similar work

Make an Offer They Can't Refuse: Grounding Bayesian Persuasion in Real-World Dialogues without Pre-Commitment

The Bots of Persuasion: Examining How Conversational Agents' Linguistic Expressions of Personality Affect User Perceptions and Decisions

Implicit Beliefs as Scalable Process Rewards for Strategic Persuasion