ARGUS: Seeing the Influence of Narrative Features on Persuasion in Argumentative Texts

Sara Nabhani; Federico Pianzola; Khalid Al-Khatib; Malvina Nissim

cs.CLFeb 27, 2026

ARGUS: Seeing the Influence of Narrative Features on Persuasion in Argumentative Texts

Authors: Sara Nabhani, Federico Pianzola, Khalid Al-Khatib, Malvina Nissim

Organizations: University of Groningen, Groningen, Netherlands

Abstract

Can narratives make arguments more persuasive? And to this end, which narrative features matter most? Although stories are often seen as powerful tools for persuasion, their specific role in online, unstructured argumentation remains underexplored. To address this gap, we present ARGUS, a framework for studying the impact of narration on persuasion in argumentative discourse. ARGUS introduces a new ChangeMyView corpus annotated for story presence and six key narrative features, integrating insights from two established theoretical frameworks that capture both textual narrative features and their effects on recipients. Leveraging both encoder-based classifiers and zero-shot large language models (LLMs), ARGUS identifies stories and narrative features and applies them at scale to examine how different narrative dimensions influence persuasion success in online argumentation.

Figures & tables

Figure 1. An example ChangeMyView discussion thread. The original poster User 001 expresses a belief, and other users respond. User 002 received a delta, indicating their comment successfully changed the original poster’s view.

Statistic	Value
Number of posts	20,436
Number of unique original posters	13,704
Number of comments	1,017,724
Number of awarded deltas	11,643
Average number of deltas per post	0.57
Average number of comments per post	49.96

Table 1. Summary statistics of the ChangeMyView dataset after preprocessing.

Figure 2. An example of Story and Story features.

Th.	Feature	Description
T	Agency	Extent to which a narrative centers on consistent, clearly defined characters driving the action.
T	Event Sequencing	Temporal arrangement of events within the narrative.
T	World Making	Construction of a fictional or realistic world through narrative elements.
R	Suspense	Presentation of information that suggests future events, thereby creating a delay in resolution.
R	Curiosity	Presentation of information related to past events, leaving the reader intrigued by missing details.
R	Surprise	Introduction of unexpected information about an event, eliciting a need for revising previous knowledge about the story.

Table 2. Narrative features, their descriptions, and theoretical framework (Th.) – T ext- or R eader oriented

Feature	ADI(mean/median) $\downarrow$	ICC(3,k) $\uparrow$
Suspense	0.612/0.555	0.720
Curiosity	0.626/0.576	0.673
Surprise	0.674/0.620	0.703
Agency	0.428/0.338	0.852
Event Sequencing	0.440/0.356	0.831
World Making	0.262/0.208	0.649

Table 3. Narrative Features Annotation Reliability and Consistency. Average Deviation Index (ADI; mean and median; range: [0–2]; lower is better) and Intraclass Correlation Coefficient (ICC(3,k); range: [0–1]; higher is better)

Predictor	$\boldsymbol{\beta}$	SE	t	$\boldsymbol{p}$	$\boldsymbol{\eta^{2}}$
Agency scalar	0.34	0.06	5.86	$<0.001$	0.05
Event Sequence scalar	0.26	0.06	4.50	$<0.001$	0.03
World Making scalar	0.07	0.03	2.80	$0.005$	0.01
Suspense scalar	0.19	0.03	5.64	$<0.001$	0.05
Curiosity scalar	0.17	0.03	5.39	$<0.001$	0.05
Surprise scalar	0.10	0.03	2.92	$0.004$	0.01

Table 4. Linear regression predicting the Story score from narrative features scores.

\boldsymbol{\beta}

= log-odds coefficients, SE = standard error, t = the t-test statistic,

\boldsymbol{p}

= p-value, and

\boldsymbol{\eta^{2}}

= partial eta squared. The model’s adjusted

\boldsymbol{R^{2}}

= 0.74. Number of samples = 620.

Predictor	$\boldsymbol{\beta}$	SE	z	$\boldsymbol{p}$	OR
Agency scalar	1.16	0.32	3.65	$<0.001$	3.19
Event Sequencing scalar	0.95	0.33	2.86	$0.004$	2.59
World Making scalar	0.18	0.17	1.09	0.27	1.20
Suspense scalar	0.88	0.20	4.41	$<0.001$	2.41
Curiosity scalar	0.40	0.19	2.15	$0.03$	1.49
Surprise scalar	0.33	0.20	1.68	0.09	1.39

Table 5. Logistic regression predicting the presence of Story from narrative features scores.

\beta

= log-odds coefficients, SE = standard error, z = the Wald test statistic,

\boldsymbol{p}

= p-value, and OR = odds ratio (computed as

\exp(\beta)

). Number of samples = 620.

Figure 3. Distribution of the strength of Story occurrence and Story features.

Panel of Figure 4 — Figure 4. Distribution of the features’ strength scores calculated as the average of the annotators’ ratings.

Feature	Model	Brier $\downarrow$	Wasserstein $\downarrow$	RMSE $\downarrow$	Accuracy $\uparrow$	F1 $\uparrow$
Story binary	D	0.120	0.186	0.245	0.815	0.763
Story scalar	R	0.076	0.150	0.194	0.879	0.842
Suspense	B	0.188	0.573	0.554	0.774	0.771
Curiosity	R	0.203	0.583	0.564	0.823	0.871
Surprise	D	0.207	0.635	0.612	0.758	0.795
Agency	M	0.167	0.545	0.843	0.839	0.714

Table 6. Performance on the test set using the best fine-tuned model per feature (DistilBERT(D), BERT(B), RoBERTa(R), and ModernBERT(M)). Metric ranges are as follows: Brier score [0,2], Wasserstein distance and RMSE [0,1] for Story and [0,4] for story features (lower is better); Accuracy and F1 [0,1] (higher is better).

Predictor	$\boldsymbol{\beta}$	SE	z	$\boldsymbol{p}$	OR
$Story_{scalar}Predictor$
Intercept	-5.65	0.03	39.26	$<0.001$	-
Story scalar	0.45	0.01	39.26	$<0.001$	1.57
Text length	0.36	0.01	53.88	$<0.001$	1.52
$Story_{binary}Predictor$
Intercept	-5.69	0.03	-170.84	$<0.001$	-

Table 7. Results of

Model_{1}

and

Model_{2}

\boldsymbol{\beta}

= log-odds coefficients, SE = standard error, z = the Wald test statistic,

\boldsymbol{p}

= p-value, and OR = odds ratio (computed as

\exp(\boldsymbol{\beta})

). Number of samples = 963,253.

$Story_{scalar}$
Predictor	$\boldsymbol{\beta}$	SE	t	$\boldsymbol{p}$	$\boldsymbol{\eta^{2}}$
Intercept	0.00	0.00	0.00	1.00	-
Response score	0.49	0.00	715.34	$<0.001$	0.35
Structural score	0.53	0.00	766.55	$<0.001$	0.38
Text length	-0.06	0.00	-80.05	$<0.001$	0.01
$Story_{binary}$

Table 8. Results of

Model_{3}

and

Model_{4}

\boldsymbol{\beta}

= log-odds coefficients, SE = standard error, z = the Wald test statistic, t = the t-test statistic,

\boldsymbol{p}

= p-value, OR = odds ratio (computed as

\exp(\boldsymbol{\beta})

), and

\boldsymbol{\eta^{2}}

= partial eta squared.

Model_{3}

adjusted

\boldsymbol{R^{2}}

= 0.77. Number of samples = 963,253.

Predictor	$\boldsymbol{\beta}$	SE	z	$\boldsymbol{p}$	OR
Scalar Predictors
Intercept	-5.70	0.03	-175.32	$<0.001$	-
Response score	0.59	0.02	30.24	$<0.001$	1.80
Structural score	0.09	0.01	7.80	$<0.001$	1.09
Text length	0.30	0.01	39.91	$<0.001$	1.35
Binary Predictors

Table 9. Results of

Model_{5}

and

Model_{6}

\boldsymbol{\beta}

= log-odds coefficients, SE = standard error, z = the Wald test statistic,

\boldsymbol{p}

= p-value, and OR = odds ratio (computed as

\exp(\boldsymbol{\beta})

). Number of samples = 963,253.

Predictor	$\boldsymbol{\beta}$	SE	z	$\boldsymbol{p}$	OR
Scalar Predictors
Intercept	-5.72	0.03	-175.572	$<0.001$	-
Agency scalar	0.03	0.01	2.453	0.01	1.03
Event Sequencing scalar	0.06	0.02	3.59	$<0.001$	1.06
Suspense scalar	0.15	0.03	4.90	$<0.001$	1.16
Curiosity scalar	0.60	0.04	15.865	$<0.001$	1.82

Table 10. Results of

Model_{7}

and

Model_{8}

\boldsymbol{\beta}

= log-odds coefficients, SE = standard error, z = the Wald test statistic,

\boldsymbol{p}

= p-value, and OR = odds ratio (computed as

\exp(\boldsymbol{\beta})

). Number of samples = 963,253.

Panel of Figure 5 — Figure 5. The distribution of narrativity scores per delta status in the ChangeMyView corpus.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Feature	Model	Brier AVG/STD $\downarrow$	Wasserstein AVG/STD $\downarrow$	RMSE AVG/STD $\downarrow$	MAE AVG/STD $\downarrow$
Story scalar	D	0.084/0.011	0.161/0.010	0.204/0.0136	0.161/0.010
	B	0.103/0.017	0.179/0.017	0.226/0.020	0.179/0.017
	R	0.100/0.008	0.173/0.013	0.223/0.009	0.173/0.013
	M	0.091/0.010	0.161/0.010	0.213/0.012	0.161/0.010
Suspense	D	0.196/0.020	0.639/0.059	0.607/0.055	0.493/0.052
	B	0.196/0.014	0.629/0.043	0.610/0.056	0.491/0.049

Appendix

Table 11. Performance comparison of soft-label trained DistilBERT(D), BERT(B), RoBERTa(R), and ModernBERT(M) models across 5-fold cross-validation. Results are reported as mean and standard deviation for each evaluation metric. Metric ranges are as follows: Brier score [0,2], Wasserstein distance,RMSE, and MAE [0,1] for Story and [0,4] for story features. The best-performing model for each feature is highlighted in bold (lower is better).

Feature	Model	Accuracy AVG/STD $\uparrow$	F1 AVG/STD $\uparrow$	macro-F1 AVG/STD $\uparrow$	weightd-F1 AVG/STD $\uparrow$
Story binary	D	0.800/0.034	0.765/0.033	0.795/0.033	0.799/0.033
	B	0.796/0.041	0.769/0.045	0.793/0.041	0.797/0.041
	R	0.799/0.060	0.763/0.076	0.794/0.063	0.798/0.061
	M	0.768/0.050	0.725/0.053	0.762/0.049	0.767/0.049

Appendix

Table 12. Performance comparison of hard-label Story trained BERT(B), DistilBERT(D), RoBERTa(R), and ModernBERT(M) models across 5-fold cross-validation. Results are reported as mean and standard deviation for each evaluation metric. The best-performing model is highlighted in bold (higher is better).

Figure 6. Cross-dataset performance of story detection models. Models fine-tuned on our data are evaluated on the StorySeeker and NarrDetect test sets, and the model released with the StorySeeker toolkit is evaluated on our annotated test set. The NarrDetect model is not publicly available and therefore could not be evaluated on our dataset.

Feature	Model	F1 $\uparrow$					RMSE $\downarrow$
Feature	Model	Argus	L.8B	L.8B binarized	L.70B	L.70B binarized	Argus	L.8B	L.70B
Story	R	0.842	0.784	0.614	0.830	0.769	0.194	0.323	0.291
Suspense	B	0.771	0.565	0.584	0.197	0.036	0.554	1.187	1.494
Curiosity	R	0.871	0.733	0.714	0.468	0.409	0.564	1.497	1.492
Surprise	D	0.795	0.709	0.678	0.613	0.484	0.612	1.716	1.340
Agency	M	0.714	0.493	0.522	0.486	0.534	0.843	3.198	3.434

Appendix

Table 13. Performance of the encoder-based models compared to Llama 8B (L.8B) and Llama 70B (L.70B) on the test set, using our best fine-tuned model per feature (DistilBERT(D), BERT(B), RoBERTa(R), and ModernBERT(M)). For Story , we use the scalar-based model given its higher performance. We also report the performance of the binarized LLMs scalar ratings (L.8B binarized & L.70B binarized ). Metric ranges are as follows: F1 [0,1] (higher is better), and RMSE [0,1] for Story and [0,4] for story elements (lower is better).

Computational Argumentation

Computational argumentation studies how to represent, analyze, and apply arguments and their relationships using formal and computational methods.

See all 94 papers

Oct 10, 2026

Have You Been Convinced? A Neuro-symbolic Argumentation Reasoning Approach to Detecting Persuasiveness in ChangeMyView Discussions

Argument structures represent how individual arguments interact within a discourse. However, leveraging these structures to infer meaningful knowledge remains challenging, as it requires a formal representation of how argumentative influence propagates across the structure. We propose Neuro-symbolic Argumentation Reasoning System (NARS) for performing persuasiveness detection directly from online discussions. The neural component employs retrieval-augmented iterative LLM reasoning to extract bipolar argument structures, while the symbolic component applies gradual argumentation semantics to propagate argumentative influence over the structure. Using the Webis-CMV-20 corpus, we construct CMV2BAS, a dataset of long, multi-threaded ChangeMyView discussions annotated with LLM-derived bipolar argument structures and labeled with delta events (view change vs.\ no change). We investigate whether gradual argumentation semantics can effectively detect persuasiveness, whether augmenting them with argument quality and graph-based centrality improves performance, and whether symbolic reasoning remains necessary in the presence of modern LLMs. NARS achieves F1=0.57 and AUC=0.56 under its best configuration, outperforming zero-shot Qwen3.6:27B and GPT-oss:20B. We identify support-dominant discussions and long argumentative chains as key failure modes limiting attack influence propagation. Our work demonstrates the feasibility of operationalizing formal gradual argumentation semantics over noisy real-world discourse for scalable macro-level persuasion analysis, while highlighting key challenges and directions for future research in neuro-symbolic argumentation reasoning. We release both CMV2BAS dataset and the NARS framework for public use.

Siddharth Bhargava, Sara Tonelli, Patricia Martín-Rodilla

Fondazione Bruno Kessler, Trento, Italy · Universidade da Coruña, Coruña, Spain · IEGPS-CSIC, Spanish National Research Council Spain

Computational Argumentation Neuro-Symbolic Reasoning Argument Mining

Ask PDF

Oct 18, 2026

A Thousand Faces: A Persuasion Benchmark with Experience-Grounded Persuadees Informed by Psychology and Pragmatics

People can express the same beliefs and desires yet need different reasons to change their minds. What makes an argument relevant depends on how those views arose: experiences shape what a person finds credible, what matters to them, and what could prompt reconsideration. These personal grounds may also be expressed indirectly, through contextual clues in conversation. Benchmarks that specify beliefs and desires without these connections can reward arguments that match a description while overlooking whether they address the person's reasons. We introduce A Thousand Faces (ATF-Persu), with 1,000 personas across 200 topics and nine domains. Drawing on personality and attitude theories and conversational implicature, we construct five personas per factual context, linking experiences to judgments, expression, and grounds for revision. Evaluations show lower persuasion success than three comparison benchmarks under their respective protocols. Complete backgrounds yield broader use of supplied personal grounds and higher persona faithfulness than simplified specifications; blinded reviewers also rate ATF-Persu above CToMPersu on identifiable personal reasons, their influence on responses, and contextual expression. The dataset is publicly available at https://huggingface.co/datasets/Slandes/ATF-Persu.

Ziqing Zhuang, Dingyi Zhang, Deyu Zhou

Southeast University

Persuasion in Language Models Pragmatic Reasoning in Language Models Persona-Conditioned Generation

Ask PDF

Aug 30, 2026

Evaluating the Capabilities of LLMs for Persuasive Dialogue

Large language models (LLMs) can generate apparently highly persuasive text, but does sounding persuasive mean arguing well? We introduce Persuasio, a multi-agent dialogue platform grounded in a formal argumentation-based theory of persuasion dialogues that adjudicates logical winners during free-text debates. Using this system, we generated 192 debates on a UK political topic between humans and LLMs, and evaluated 22 interlocutors through both automated adjudication and 9,702 crowdsourced pairwise judgements across 1{,}386 annotation instances. We observed a consistent decoupling between subjective and formal persuasiveness: LLMs dominated the subjective ranking yet performed substantially worse under argumentation-theoretic adjudication, where humans remained competitive. Multi-agent and retrieval-augmented variants further widened this divergence. These findings reveal a systematic gap between rhetorical fluency and formal argumentative strength in LLM-based persuasive dialogues.

Jordan Robinson, Angus R. Williams, Katie Atkinson +1

University of Liverpool · 2The Alan Turing Institute · University of Leeds

LLM Evaluation Multi-Agent Debate Computational Argumentation Multi-Turn Dialogue Evaluation Persuasion in Language Models

Ask PDF

Abstract

Computational Argumentation

Explore similar work

Have You Been Convinced? A Neuro-symbolic Argumentation Reasoning Approach to Detecting Persuasiveness in ChangeMyView Discussions

A Thousand Faces: A Persuasion Benchmark with Experience-Grounded Persuadees Informed by Psychology and Pragmatics

Evaluating the Capabilities of LLMs for Persuasive Dialogue