cs.CLNov 19, 2026

Incorporating argumentation frameworks for LLM decision-making: Task-specific design and evaluation

Authors: Bochen Han, Anshu Xiong, Mingyang Ma

Organizations: State Key Laboratory of Mathematical Sciences, Academy of Mathematics and Systems Science, Chinese Academy of Sciences, Beijing 100190, CHINA · School of Mathematical Sciences, University of Chinese Academy of Sciences, Beijing 100190, CHINA

Abstract

Argumentation frameworks (AFs) provide explicit, formal mechanisms for representing and resolving conflicts and thus have been incorporated into large language model (LLM) decision-making. The benefits vary and depend on how arguments and their relations are constructed, evaluated, and incorporated into the decision process. {We abstract a workflow consisting of four stages based on existing argumentation-augmented methods} and investigate their design choices over tasks of debate-based decision-making and retrieval-based fact verification. Our evaluation with recent LLMs shows that existing methods do not consistently improve performance across the considered tasks. Motivated by this observation, we develop two task-specific methods. The first is called Qnt-Arg, which organizes arguments around the original speakers in debate materials and aggregates model-estimated argument strengths and candidate-answer priors under gradual semantics. The second is Qlt-Arg that uses credibility estimates to establish preferences between conflicting evidence items and applies grounded semantics to select acceptable evidence for subsequent LLM inference. We examine the roles of generative and decision models within the two methods. In the debate setting, Qnt-Arg consistently outperforms other argumentation-augmented methods across matched configurations; Qlt-Arg improves retrieval-based fact-verification performance compared with the baselines. These findings demonstrate the value of combining formal argumentation and LLM while highlighting the importance of aligning with tasks in argumentation construction, model selection, and the role of formal semantic reasoning.

Figures & tables

Explore similar work

CardsList
  1. LLM-based Argument Mining meets Argumentation and Description Logics: a Unified Framework for Reasoning about Debates

    Mar 3, 2026Gianvincenzo Alfano, Sergio Greco, Lucio La Cava +2Computational ArgumentationArgument Mining

  2. ArgLLM-App: An Interactive System for Argumentative Reasoning with Large Language Models

    Feb 27, 2026Adam Dejl, Deniz Gorur, Francesca ToniLLM Decision-MakingLLM Agents

  3. ArgBench: Benchmarking LLMs on Computational Argumentation Tasks

    Apr 19, 2026Yamen Ajjour, Carlotta Quensel, Nedim Lipka +1LLM EvaluationArgument Mining