Large Language Models (LLMs) achieve strong performance in analyzing and generating text, yet they struggle with explicit, transparent, and verifiable reasoning over complex texts such as those containing debates. In particular, they lack structured representations that capture how arguments support or attack each other and how their relative strengths determine overall acceptability. We encompass these limitations by proposing a framework that integrates learning-based argument mining with quantitative reasoning and ontology-based querying. Starting from a raw debate text, the framework extracts a fuzzy argumentative knowledge base, where arguments are explicitly represented as entities, linked by attack and support relations, and annotated with initial fuzzy strengths reflecting plausibility w.r.t. the debate's context. Quantitative argumentation semantics are then applied to compute final argument strengths by propagating the effects of supports and attacks. These results are then embedded into a fuzzy description logic setting, enabling expressive query answering through efficient rewriting techniques. The proposed approach provides a transparent, explainable, and formally grounded method for analyzing debates, overcoming purely statistical LLM-based analyses.
Figures & tables
Figure 1: FAKB workflow: from raw unstructured texts to FAKB generation and querying.
Figure 2: An example of updated FAbox, visualized in the form of a knowledge graph, and extracted from an extended version of debate D of Example 1 . Nodes are color-coded by type: blue for entities, red for argument identifiers, and yellow for concepts. Similarly, edges represent specific relations: green/red denotes support/attacks, while dark blue, light blue, and yellow represent authorship, role instances, and concept instances, respectively. Upper/lower numbers within each argument denote its initial/updated strength. For optimal visualization, the arguments’ text has been omitted, yet they can be found in Section 5 . A full-screen visualization also including additional entities and full texts is provided in the Appendix.
Table 1: Comparison between our approach (left) vs Prompt-based (right) for relation strength estimation on argument pairs (ai,aj) in D1 , with i>j . Red-/Green-colored cells refer to attacks/supports, while grey-shaded cells marked with ‘N’ indicate the absence of a relation. Numeric values denote the relation strength. Solid (resp., dashed) borders refer to pairs of arguments stated by the same author (resp., different authors).
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
Figure 3: An example of updated FAbox, visualized in the form of a knowledge graph, and extracted from an extended version of debate D of Example 1 . Nodes are color-coded by type: violet for text of arguments, blue for entities, red for argument identifiers, and yellow for concepts. Similarly, edges represent specific relations: green/red denotes support/attacks, while dark blue, light blue, and yellow represent authorship, role instances, and concept instances, respectively. Upper/lower numbers within each argument denote its initial/updated strength.
Large Language Models (LLMs) are increasingly assessed and utilized in the field of Argument Mining (AM), thanks to their strong general reasoning capabilities. However, standard training-free models often miss sophisticated details, specifically in contexts where two parts of the text have to be analyzed together. Furthermore, self-correction mechanisms tend to reinforce initial hallucinations in reasoning. Overcoming these limitations typically requires expensive, domain-specific supervised fine-tuning. Recent work has shown that a multi-agent paradigm can address such weaknesses for the component classification task through dialectical refinement with a Proponent-Opponent-Judge architecture, setting a promising direction for training-free approaches in the field. In this paper, we extend and evaluate this framework on the Argument Relation Identification and Classification (ARIC) task, reformulating it as a debate over component pairs. Besides that, we introduce a confidence gating mechanism that enables debating only on the uncertain cases and accepting the initial prediction when confidence is high. On the UKP Argument Annotated Essays v2 corpus, we demonstrate that the selective debate achieves the highest Macro F1 among all training-free methods, while debate over all samples degrades performance below that of one of the baselines. All generative approaches also outperform fine-tuned RoBERTa models on Macro F1, suggesting that the under-representation of the Attack class was more damaging to supervised fine-tuning than to inference-only models. Additionally, our framework produces human-readable debate transcripts, offering interpretability absent from both single-agent and supervised classifiers.
Jakub Bąba, Jarosław A. Chudziak
Faculty of Electronics and Information Technology, Warsaw University of Technology, Poland
Arguments are a fundamental aspect of human reasoning, in which claims are supported, challenged, and weighed against one another. We present an end-to-end large language model (LLM)-based system for reconstructing arguments from natural language text into abstract argument graphs. The system follows a multi-stage pipeline that progressively identifies argumentative components, selects relevant elements, and uncovers their logical relations. These elements are represented as directed acyclic graphs consisting of two component types (premises and conclusions) and three relation types (support, attack, and undercut). We conduct two complementary experiments to evaluate the system. First, we perform a manual evaluation on arguments drawn from an argumentation theory textbook to assess the system's ability to recover argumentative structure. Second, we conduct a quantitative evaluation on benchmark datasets, allowing comparison with prior work by mapping our outputs to established annotation schemes. Results show that the system can adequately recover argumentative structures and, when adapted to different annotation schemes, achieve reasonable performance across benchmark datasets. These findings highlight the potential of LLM-based pipelines for scalable argument mining.
Paulo Pirozelli, Victor Hugo Nascimento Rocha, Fabio G. Cozman +1
Universidade de S˜ao Paulo · Center for Artificial Intelligence (C4AI) · Instituto Mau´a de Tecnologia +1
Argumentation skills are an essential toolkit for large language models (LLMs). These skills are crucial in various use cases, including self-reflection, debating collaboratively for diverse answers, and countering hate speech. In this paper, we create the first benchmark for a standardized evaluation of LLM-based approaches to computational argumentation, encompassing 33 datasets from previous work in unified form. Using the benchmark, we evaluate the generalizability of five LLM families across 46 computational argumentation tasks that cover mining arguments, assessing perspectives, assessing argument quality, reasoning about arguments, and generating arguments. On the benchmark, we conduct an extensive systematic analysis of the contribution of few-shot examples, reasoning steps, model size, and training skills to the performance of LLMs on the computational argumentation tasks in the benchmark.
Yamen Ajjour, Carlotta Quensel, Nedim Lipka +1
Leibniz University Hannover · Adobe Research · Leibniz University Hannover, L3S