Argument Structure Prediction in Online Conversations: A Comparative Study of Modeling Paradigms and Task Architectures
Authors: Siddharth Bhargava, Sara Tonelli, Patricia Martín-Rodilla, Javier Parapar
Organizations: Fondazione Bruno Kessler, Povo 38123, Trento Italy · Universidade da Coruña, 15001 A Coruña Spain · IEGPS-CSIC, Spanish National Research Council, 15704 Santiago de Compostela Spain
Argument structure prediction (ASP) constructs complete argument structures from discourse by identifying argumentative units and their relations. While recent work has explored diverse approaches---including unified neural models, multi-step pipelines, and prompt-based large language models (LLMs)---their relative trade-offs remain under-explored, particularly in dialogical settings. We present a systematic evaluation of ASP under strict schema constraints, comparing supervised fine-tuning and prompt-based LLMs across single- and multi-step task architectures, generating complete argument structures from dialogical input end-to-end. We benchmark them on three diverse dialogical corpora adapted from Inference Anchoring Theory into bipolar argument structures. Under a shared evaluation framework, we assess predictive performance, cross-domain generalization, schema compliance, and computational efficiency. Our results show that ASP remains a challenging task, with identifying argumentative relations emerging as the primary bottleneck, largely due to the implicit and context-dependent nature of dialogical argumentation. To facilitate future research, we release our data processing pipeline and end-to-end modeling framework for computational ASP on dialogical corpora.
Figures & tables
Figure 1: Illustrative example of a conversation excerpt [id:10055] from the US2016reddit corpus (left), the bipolar argument structure representation, annotated by humans (middle)—where green edges denote support relations and red edges denote attack relations—and the JSON output representation (right) indicating how the argument structure has been stored.
Corpus
#Conv. (orig.)
AvgLen
#Units
#Support(%)
#Attack(%)
#LongR(%)
QT30 (train)
669 (1478)
299.2
6451
3580 (85.9)
565 (14.1)
277 (6.7)
QT30 (test)
168 (1478)
304.3
1559
871 (87.1)
141 (12.9)
76 (7.8)
US2016reddit
152 (264)
221.1
1666
895 (71.3)
361 (28.7)
315 (25.6)
RIP1
79 (204)
406.3
765
386 (83.5)
76 (16.5)
113 (25)
Table 1: Corpus statistics after cleaning and filtering . Values in parentheses in the #Conv. column denote the original number of conversations in dataset before filtering. Abbreviations: AvgLen = average conversation length (in tokens); LongR= long-distance relations where source and target are at least 3 sentences apart.
Paradigm
Model
Arch.
Code (MC)
Fine-tuning
BiLSTM-ER
ss
FT
XLM-R-Long + XLM-R-Large
ms
FT
Prompting
Qwen2.5-14B-Instruct
ss / ms
Q0, Q3, Q5
DeepSeek-R1-Distill-Qwen-14B
ss / ms
D0, D3, D5
Gemma3-12B-it
ss / ms
G0, G3, G5
Table 2: Models and configurations evaluated. Paradigms include supervised fine-tuning and prompt-based inference. Task Architectures: single-step (ss) and multi-step (ms). Model codes (MC) denote the model and number of context examples (e.g., Q3 = Qwen with three examples).
Figure 2: Task performance across datasets and configurations (identified by Model Codes [MC]; see Table 2 ) for single- ( ss ) and multi-step ( ms ) settings. Plots (a–c) show argument unit extraction (x: AUE@50) and relation classification (y: RTC@50) at 50% span overlap, with each point representing the weighted F1 score (AUE, RTC).
QT30-test (168 conv.)
RIP1 (79 conv.)
MC
SR(%)
T(s)
Tk
SR(%)
T(s)
Tk
Q0 ss
96.4 +3.6
27.46 +0.6
210k +13k
100.0 +0.0
37.38 +0.0
119k +0k
Q3 ss
100.0 +0.0
34.49 +0.0
623k +0
98.7 +1.3
34.45 +0.0
301k +8k
Q5 ss
100.0 +0.0
37.89 +0.0
830k +0
98.7 +1.3
37.3 +0.3
397k +10k
D3 ss
100.0 +0.0
46.11 +0.0
646k +0
97.4 +0.0
48.78 +2.0
312k +4k
D5 ss
100.0 +0.0
47.76 +0.0
847k +0
97.4 +0.0
52.18 +2.0
405k +4k
Table 3: Computational efficiency of configurations across QT30-test and RIP1 . SR: success rate of schema-compliant predictions in first try; T: Average time taken per conversation in first try; Tk: total tokens (in thousands) outputted for all conversations in first try. Subscripts (+x) denote changes observed when a retry is performed.
QT30-test
US2016reddit
RIP1
MC
AUE P/R
RTC P/R
AUE P/R
RTC P/R
AUE P/R
RTC P/R
FT ms
0.32 / 0.35
0.12 / 0.06
0.42 / 0.36
0.08 / 0.04
0.28 / 0.44
0.12 / 0.08
Q0 ss
0.35 / 0.25
0.06 / 0.02
0.55 / 0.42
0.18 / 0.05
0.21 / 0.25
0.12 / 0.03
Q3 ss
0.52 / 0.61
0.33 / 0.16
0.64 / 0.64
0.35 / 0.16
0.32 / 0.38
0.25 / 0.06
Q5 ms
0.53 / 0.61
0.38 / 0.18
0.67 / 0.60
0.30 / 0.14
0.38 / 0.38
0.24 / 0.06
Q5 ss
0.51 / 0.64
0.44 / 0.22
0.65 / 0.65
0.41 / 0.17
0.34 / 0.43
0.27 / 0.08
Table 4: Performance analysis across model configurations (MC, see Table 2 ). Precision (P) and recall (R) are reported as P / R for argument unit extraction (AUE) and relation classification (RTC). Bold values indicate the best performance within each dataset.
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 6: Visualization of the six types of structural errors observed in our error analysis between the ground truth structures and the predicted structures. The green edges represent support relations; red edges represent attack relations; U1-6 are argument units.
MC
U +
U -
E
R +
R -
C
QT30-test
FT ms
1163
1022
1836
265
934
8
Q0 ss
647
1178
472
71
978
3
Q3 ss
883
630
851
207
825
13
Q5 ms
853
640
799
228
812
14
Q5 ss
982
569
921
204
763
18
Appendix
Table 5: Quantitative Error analysis across model configurations at 50% overlap threshold using seed 42 for each dataset/best configurations. U + and U - denote false positive and false negative argument units. E denotes endpoint-mismatched relations. R + and R - denote false positive and false negative relations. C denotes mis-classified relation types.
Argument Mining(AM) aims to uncover the argumentative structures within a text. Previous methods require several subtasks, such as span identification, component classification, and relation classification. Consequently, these methods need rule-based postprocessing to derive argumentative structures from the output of each subtask. This approach adds to the complexity of the model and expands the search space of the hyperparameters. To address this difficulty, we propose a simple yet strong method based on a text-to-text generation approach using a pretrained encoder-decoder language model. Our method simultaneously generates argumentatively annotated text for spans, components, and relations, eliminating the need for task-specific postprocessing and hyperparameter tuning. Furthermore, because it is a straightforward text-to-text generation method, we can easily adapt our approach to various types of argumentative structures. Experimental results demonstrate the effectiveness of our method, as it achieves state-of-the-art performance on three different types of benchmark datasets: the Argument-annotated Essays Corpus(AAEC), AbstRCT, and the Cornell eRulemaking Corpus(CDCP)
Argument Mining (AM) is a critical NLP task that remains significantly under-resourced in Arabic. This paper presents \testttSTAR−Ar, a BERT-BiLSTM-CRF architecture for argument discourse detection and classification, as our system for Daleel 2026, the inaugural Arabic argument mining shared task. The task requires the identification and classification of argumentative discourse units (ADUs) in debate and editorial texts.We jointly model these two objectives as a token-level sequence labeling task using a BERT-BiLSTM-CRF architecture that combines contextual transformer embeddings with structural transition constraints to support accurate span detection. \testttSTAR−Ar achieves an F1-score of 72.69 on validation and 73.7 on test data. Our domain-specific analysis shows that models trained exclusively on editorials underperform those trained on debates, a disparity we primarily attribute to the smaller size of the editorial dataset. The code for \testttSTAR−Ar is available at [\faGithubTTLabatDaleel2026](https://github.com/ENTAILab/daleel2026Arabic−Argumentative−Discourse−Mining)
Bhuvanesh Verma, Ali Abusaleh, Alexander Mehler
Text Technology Lab (TTLab), Goethe University Frankfurt
Large Language Models (LLMs) are increasingly assessed and utilized in the field of Argument Mining (AM), thanks to their strong general reasoning capabilities. However, standard training-free models often miss sophisticated details, specifically in contexts where two parts of the text have to be analyzed together. Furthermore, self-correction mechanisms tend to reinforce initial hallucinations in reasoning. Overcoming these limitations typically requires expensive, domain-specific supervised fine-tuning. Recent work has shown that a multi-agent paradigm can address such weaknesses for the component classification task through dialectical refinement with a Proponent-Opponent-Judge architecture, setting a promising direction for training-free approaches in the field. In this paper, we extend and evaluate this framework on the Argument Relation Identification and Classification (ARIC) task, reformulating it as a debate over component pairs. Besides that, we introduce a confidence gating mechanism that enables debating only on the uncertain cases and accepting the initial prediction when confidence is high. On the UKP Argument Annotated Essays v2 corpus, we demonstrate that the selective debate achieves the highest Macro F1 among all training-free methods, while debate over all samples degrades performance below that of one of the baselines. All generative approaches also outperform fine-tuned RoBERTa models on Macro F1, suggesting that the under-representation of the Attack class was more damaging to supervised fine-tuning than to inference-only models. Additionally, our framework produces human-readable debate transcripts, offering interpretability absent from both single-agent and supervised classifiers.
Jakub Bąba, Jarosław A. Chudziak
Faculty of Electronics and Information Technology, Warsaw University of Technology, Poland