ARGUS: Seeing the Influence of Narrative Features on Persuasion in Argumentative Texts

Sara Nabhani; Federico Pianzola; Khalid Al-Khatib; Malvina Nissim

cs.CLFeb 27, 2026

ARGUS: Seeing the Influence of Narrative Features on Persuasion in Argumentative Texts

Authors: Sara Nabhani, Federico Pianzola, Khalid Al-Khatib, Malvina Nissim

Organizations: University of Groningen, Groningen, Netherlands

Abstract

Can narratives make arguments more persuasive? And to this end, which narrative features matter most? Although stories are often seen as powerful tools for persuasion, their specific role in online, unstructured argumentation remains underexplored. To address this gap, we present ARGUS, a framework for studying the impact of narration on persuasion in argumentative discourse. ARGUS introduces a new ChangeMyView corpus annotated for story presence and six key narrative features, integrating insights from two established theoretical frameworks that capture both textual narrative features and their effects on recipients. Leveraging both encoder-based classifiers and zero-shot large language models (LLMs), ARGUS identifies stories and narrative features and applies them at scale to examine how different narrative dimensions influence persuasion success in online argumentation.

Figures & tables

Figure 1. An example ChangeMyView discussion thread. The original poster User 001 expresses a belief, and other users respond. User 002 received a delta, indicating their comment successfully changed the original poster’s view.

Statistic	Value
Number of posts	20,436
Number of unique original posters	13,704
Number of comments	1,017,724
Number of awarded deltas	11,643
Average number of deltas per post	0.57
Average number of comments per post	49.96

Table 1. Summary statistics of the ChangeMyView dataset after preprocessing.

Figure 2. An example of Story and Story features.

Th.	Feature	Description
T	Agency	Extent to which a narrative centers on consistent, clearly defined characters driving the action.
T	Event Sequencing	Temporal arrangement of events within the narrative.
T	World Making	Construction of a fictional or realistic world through narrative elements.
R	Suspense	Presentation of information that suggests future events, thereby creating a delay in resolution.
R	Curiosity	Presentation of information related to past events, leaving the reader intrigued by missing details.
R	Surprise	Introduction of unexpected information about an event, eliciting a need for revising previous knowledge about the story.

Table 2. Narrative features, their descriptions, and theoretical framework (Th.) – T ext- or R eader oriented

Feature	ADI(mean/median) $\downarrow$	ICC(3,k) $\uparrow$
Suspense	0.612/0.555	0.720
Curiosity	0.626/0.576	0.673
Surprise	0.674/0.620	0.703
Agency	0.428/0.338	0.852
Event Sequencing	0.440/0.356	0.831
World Making	0.262/0.208	0.649

Table 3. Narrative Features Annotation Reliability and Consistency. Average Deviation Index (ADI; mean and median; range: [0–2]; lower is better) and Intraclass Correlation Coefficient (ICC(3,k); range: [0–1]; higher is better)

Predictor	$\boldsymbol{\beta}$	SE	t	$\boldsymbol{p}$	$\boldsymbol{\eta^{2}}$
Agency scalar	0.34	0.06	5.86	$<0.001$	0.05
Event Sequence scalar	0.26	0.06	4.50	$<0.001$	0.03
World Making scalar	0.07	0.03	2.80	$0.005$	0.01
Suspense scalar	0.19	0.03	5.64	$<0.001$	0.05
Curiosity scalar	0.17	0.03	5.39	$<0.001$	0.05
Surprise scalar	0.10	0.03	2.92	$0.004$	0.01

Table 4. Linear regression predicting the Story score from narrative features scores.

\boldsymbol{\beta}

= log-odds coefficients, SE = standard error, t = the t-test statistic,

\boldsymbol{p}

= p-value, and

\boldsymbol{\eta^{2}}

= partial eta squared. The model’s adjusted

\boldsymbol{R^{2}}

= 0.74. Number of samples = 620.

Predictor	$\boldsymbol{\beta}$	SE	z	$\boldsymbol{p}$	OR
Agency scalar	1.16	0.32	3.65	$<0.001$	3.19
Event Sequencing scalar	0.95	0.33	2.86	$0.004$	2.59
World Making scalar	0.18	0.17	1.09	0.27	1.20
Suspense scalar	0.88	0.20	4.41	$<0.001$	2.41
Curiosity scalar	0.40	0.19	2.15	$0.03$	1.49
Surprise scalar	0.33	0.20	1.68	0.09	1.39

Table 5. Logistic regression predicting the presence of Story from narrative features scores.

\beta

= log-odds coefficients, SE = standard error, z = the Wald test statistic,

\boldsymbol{p}

= p-value, and OR = odds ratio (computed as

\exp(\beta)

). Number of samples = 620.

Figure 3. Distribution of the strength of Story occurrence and Story features.

Panel of Figure 4 — Figure 4. Distribution of the features’ strength scores calculated as the average of the annotators’ ratings.

Feature	Model	Brier $\downarrow$	Wasserstein $\downarrow$	RMSE $\downarrow$	Accuracy $\uparrow$	F1 $\uparrow$
Story binary	D	0.120	0.186	0.245	0.815	0.763
Story scalar	R	0.076	0.150	0.194	0.879	0.842
Suspense	B	0.188	0.573	0.554	0.774	0.771
Curiosity	R	0.203	0.583	0.564	0.823	0.871
Surprise	D	0.207	0.635	0.612	0.758	0.795
Agency	M	0.167	0.545	0.843	0.839	0.714

Table 6. Performance on the test set using the best fine-tuned model per feature (DistilBERT(D), BERT(B), RoBERTa(R), and ModernBERT(M)). Metric ranges are as follows: Brier score [0,2], Wasserstein distance and RMSE [0,1] for Story and [0,4] for story features (lower is better); Accuracy and F1 [0,1] (higher is better).

Predictor	$\boldsymbol{\beta}$	SE	z	$\boldsymbol{p}$	OR
$Story_{scalar}Predictor$
Intercept	-5.65	0.03	39.26	$<0.001$	-
Story scalar	0.45	0.01	39.26	$<0.001$	1.57
Text length	0.36	0.01	53.88	$<0.001$	1.52
$Story_{binary}Predictor$
Intercept	-5.69	0.03	-170.84	$<0.001$	-

Table 7. Results of

Model_{1}

and

Model_{2}

\boldsymbol{\beta}

= log-odds coefficients, SE = standard error, z = the Wald test statistic,

\boldsymbol{p}

= p-value, and OR = odds ratio (computed as

\exp(\boldsymbol{\beta})

). Number of samples = 963,253.

$Story_{scalar}$
Predictor	$\boldsymbol{\beta}$	SE	t	$\boldsymbol{p}$	$\boldsymbol{\eta^{2}}$
Intercept	0.00	0.00	0.00	1.00	-
Response score	0.49	0.00	715.34	$<0.001$	0.35
Structural score	0.53	0.00	766.55	$<0.001$	0.38
Text length	-0.06	0.00	-80.05	$<0.001$	0.01
$Story_{binary}$

Table 8. Results of

Model_{3}

and

Model_{4}

\boldsymbol{\beta}

= log-odds coefficients, SE = standard error, z = the Wald test statistic, t = the t-test statistic,

\boldsymbol{p}

= p-value, OR = odds ratio (computed as

\exp(\boldsymbol{\beta})

), and

\boldsymbol{\eta^{2}}

= partial eta squared.

Model_{3}

adjusted

\boldsymbol{R^{2}}

= 0.77. Number of samples = 963,253.

Predictor	$\boldsymbol{\beta}$	SE	z	$\boldsymbol{p}$	OR
Scalar Predictors
Intercept	-5.70	0.03	-175.32	$<0.001$	-
Response score	0.59	0.02	30.24	$<0.001$	1.80
Structural score	0.09	0.01	7.80	$<0.001$	1.09
Text length	0.30	0.01	39.91	$<0.001$	1.35
Binary Predictors

Table 9. Results of

Model_{5}

and

Model_{6}

\boldsymbol{\beta}

= log-odds coefficients, SE = standard error, z = the Wald test statistic,

\boldsymbol{p}

= p-value, and OR = odds ratio (computed as

\exp(\boldsymbol{\beta})

). Number of samples = 963,253.

Predictor	$\boldsymbol{\beta}$	SE	z	$\boldsymbol{p}$	OR
Scalar Predictors
Intercept	-5.72	0.03	-175.572	$<0.001$	-
Agency scalar	0.03	0.01	2.453	0.01	1.03
Event Sequencing scalar	0.06	0.02	3.59	$<0.001$	1.06
Suspense scalar	0.15	0.03	4.90	$<0.001$	1.16
Curiosity scalar	0.60	0.04	15.865	$<0.001$	1.82

Table 10. Results of

Model_{7}

and

Model_{8}

\boldsymbol{\beta}

= log-odds coefficients, SE = standard error, z = the Wald test statistic,

\boldsymbol{p}

= p-value, and OR = odds ratio (computed as

\exp(\boldsymbol{\beta})

). Number of samples = 963,253.

Panel of Figure 5 — Figure 5. The distribution of narrativity scores per delta status in the ChangeMyView corpus.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Feature	Model	Brier AVG/STD $\downarrow$	Wasserstein AVG/STD $\downarrow$	RMSE AVG/STD $\downarrow$	MAE AVG/STD $\downarrow$
Story scalar	D	0.084/0.011	0.161/0.010	0.204/0.0136	0.161/0.010
	B	0.103/0.017	0.179/0.017	0.226/0.020	0.179/0.017
	R	0.100/0.008	0.173/0.013	0.223/0.009	0.173/0.013
	M	0.091/0.010	0.161/0.010	0.213/0.012	0.161/0.010
Suspense	D	0.196/0.020	0.639/0.059	0.607/0.055	0.493/0.052
	B	0.196/0.014	0.629/0.043	0.610/0.056	0.491/0.049

Appendix

Table 11. Performance comparison of soft-label trained DistilBERT(D), BERT(B), RoBERTa(R), and ModernBERT(M) models across 5-fold cross-validation. Results are reported as mean and standard deviation for each evaluation metric. Metric ranges are as follows: Brier score [0,2], Wasserstein distance,RMSE, and MAE [0,1] for Story and [0,4] for story features. The best-performing model for each feature is highlighted in bold (lower is better).

Feature	Model	Accuracy AVG/STD $\uparrow$	F1 AVG/STD $\uparrow$	macro-F1 AVG/STD $\uparrow$	weightd-F1 AVG/STD $\uparrow$
Story binary	D	0.800/0.034	0.765/0.033	0.795/0.033	0.799/0.033
	B	0.796/0.041	0.769/0.045	0.793/0.041	0.797/0.041
	R	0.799/0.060	0.763/0.076	0.794/0.063	0.798/0.061
	M	0.768/0.050	0.725/0.053	0.762/0.049	0.767/0.049

Appendix

Table 12. Performance comparison of hard-label Story trained BERT(B), DistilBERT(D), RoBERTa(R), and ModernBERT(M) models across 5-fold cross-validation. Results are reported as mean and standard deviation for each evaluation metric. The best-performing model is highlighted in bold (higher is better).

Figure 6. Cross-dataset performance of story detection models. Models fine-tuned on our data are evaluated on the StorySeeker and NarrDetect test sets, and the model released with the StorySeeker toolkit is evaluated on our annotated test set. The NarrDetect model is not publicly available and therefore could not be evaluated on our dataset.

Feature	Model	F1 $\uparrow$					RMSE $\downarrow$
Feature	Model	Argus	L.8B	L.8B binarized	L.70B	L.70B binarized	Argus	L.8B	L.70B
Story	R	0.842	0.784	0.614	0.830	0.769	0.194	0.323	0.291
Suspense	B	0.771	0.565	0.584	0.197	0.036	0.554	1.187	1.494
Curiosity	R	0.871	0.733	0.714	0.468	0.409	0.564	1.497	1.492
Surprise	D	0.795	0.709	0.678	0.613	0.484	0.612	1.716	1.340
Agency	M	0.714	0.493	0.522	0.486	0.534	0.843	3.198	3.434

Appendix

Table 13. Performance of the encoder-based models compared to Llama 8B (L.8B) and Llama 70B (L.70B) on the test set, using our best fine-tuned model per feature (DistilBERT(D), BERT(B), RoBERTa(R), and ModernBERT(M)). For Story , we use the scalar-based model given its higher performance. We also report the performance of the binarized LLMs scalar ratings (L.8B binarized & L.70B binarized ). Metric ranges are as follows: F1 [0,1] (higher is better), and RMSE [0,1] for Story and [0,4] for story elements (lower is better).

Computational Argumentation

Computational argumentation studies how to represent, analyze, and apply arguments and their relationships using formal and computational methods.

See all 94 papers

Have You Been Convinced? A Neuro-symbolic Argumentation Reasoning Approach to Detecting Persuasiveness in ChangeMyView Discussions
Oct 10, 2026Siddharth Bhargava, Sara Tonelli, Patricia Martín-RodillaComputational Argumentation Neuro-Symbolic Reasoning
Fondazione Bruno Kessler, Trento, Italy · Universidade da Coruña, Coruña, Spain · IEGPS-CSIC, Spanish National Research Council Spain
A Thousand Faces: A Persuasion Benchmark with Experience-Grounded Persuadees Informed by Psychology and Pragmatics
Oct 18, 2026Ziqing Zhuang, Dingyi Zhang, Deyu ZhouPersuasion in Language Models Pragmatic Reasoning in Language Models
Southeast University
Evaluating the Capabilities of LLMs for Persuasive Dialogue
Aug 30, 2026Jordan Robinson, Angus R. Williams, Katie Atkinson +1LLM Evaluation Multi-Agent Debate
University of Liverpool · 2The Alan Turing Institute · University of Leeds

Oct 10, 2026J/K move · Enter open · S save

Have You Been Convinced? A Neuro-symbolic Argumentation Reasoning Approach to Detecting Persuasiveness in ChangeMyView Discussions

Siddharth Bhargava, Sara Tonelli, Patricia Martín-Rodilla

Fondazione Bruno Kessler, Trento, Italy · Universidade da Coruña, Coruña, Spain · IEGPS-CSIC, Spanish National Research Council Spain

Computational Argumentation Neuro-Symbolic Reasoning Argument Mining

Open paper AskPDF arXiv

Argument structures represent how individual arguments interact within a discourse. However, leveraging these structures to infer meaningful knowledge remains challenging, as it requires a formal representation of how argumentative influence propagates across the structure. We propose Neuro-symbolic Argumentation Reasoning System (NARS) for performing persuasiveness detection directly from online discussions. The neural component employs retrieval-augmented iterative LLM reasoning to extract bipolar argument structures, while the symbolic component applies gradual argumentation semantics to propagate argumentative influence over the structure. Using the Webis-CMV-20 corpus, we construct CMV2BAS, a dataset of long, multi-threaded ChangeMyView discussions annotated with LLM-derived bipolar argument structures and labeled with delta events (view change vs.\ no change). We investigate whether gradual argumentation semantics can effectively detect persuasiveness, whether augmenting them with argument quality and graph-based centrality improves performance, and whether symbolic reasoning remains necessary in the presence of modern LLMs. NARS achieves F1=0.57 and AUC=0.56 under its best configuration, outperforming zero-shot Qwen3.6:27B and GPT-oss:20B. We identify support-dominant discussions and long argumentative chains as key failure modes limiting attack influence propagation. Our work demonstrates the feasibility of operationalizing formal gradual argumentation semantics over noisy real-world discourse for scalable macro-level persuasion analysis, while highlighting key challenges and directions for future research in neuro-symbolic argumentation reasoning. We release both CMV2BAS dataset and the NARS framework for public use.

Abstract

Computational Argumentation

Explore similar work

Have You Been Convinced? A Neuro-symbolic Argumentation Reasoning Approach to Detecting Persuasiveness in ChangeMyView Discussions

A Thousand Faces: A Persuasion Benchmark with Experience-Grounded Persuadees Informed by Psychology and Pragmatics

Evaluating the Capabilities of LLMs for Persuasive Dialogue