Large language models (LLMs) exhibit strong general capabilities, but their deployment in high-stakes domains is hindered by their opacity and unpredictability. Recent work has taken meaningful steps towards addressing these issues by augmenting LLMs with post-hoc reasoning based on computational argumentation, providing faithful explanations and enabling users to contest incorrect decisions. However, this paradigm is limited to pre-defined binary choices and only supports local contestation for specific instances, leaving the underlying decision logic unchanged and prone to repeated mistakes. In this paper, we introduce ArgEval, a framework that shifts from instance-specific reasoning to structured evaluation of general decision options. Rather than mining arguments solely for individual cases, ArgEval systematically maps task-specific decision spaces, builds corresponding option ontologies, and constructs general argumentation frameworks (AFs) for each option. These frameworks can then be instantiated to provide explainable recommendations for specific cases while still supporting global contestability through modification of the shared AFs. We investigate the effectiveness of ArgEval on treatment recommendation for glioblastoma, an aggressive brain tumour, and show that it can produce explainable guidance aligned with clinical practice.
Figures & tables
Figure 1: Illustration of ArgEval inference for one of the treatment options for glioblastoma 3 3 3 Icons from Noun Project ( Patient by Alzam and Data by Anna Riana, CC BY 3.0 ). . Case Parameters extracted from the Case Description are used to instantiate the General QBAF associated with the given option ( Root Argument ), removing the dashed nodes whose conditions are not satisfied. The Prediction After Instantiation is obtained from the instantiated framework, which also serves as a faithful explanation.
Figure 2: Overview of the ArgEval pipeline 5 5 5 Icons from Noun Project ( Policy by Puspito, Decision by Kamin Ginkaew, report analysis by kartini 1, Patient by Alzam and Data by Anna Riana, CC BY 3.0 ). . Top: given natural-language policy documents that specify general criteria for decision-making in a certain domain, ArgEval builds a decision-space ontology and constructs general QBAFs for each candidate decision in the ontology. Bottom: at inference time, the general QBAF is instantiated with the parameters of a specific case, providing faithfully explainable recommendations. Users can contest the decision-space ontology, the general QBAFs, the extracted case parameters and the general parameter schema specifying the properties to be extracted, in response to incorrect recommendations or explanations produced by the model.
Property
Method
LLMs
ArgLLMs ArgRAG
ArgLLMs-O
ArgEval
Open-Ended Decisions
✓
✗
✓
✓
Faithful Explainability
✗
✓
✓
✓
Local Contestability
( ✓ )
✓
✓
✓
Global Contestability
✗
✗
✗
✓
Inference Speed
↯↯
↯
↯
↯↯↯
Table 1: Qualitative comparison between ArgEval and other methods. ( ✓ ) in the third row signifies that local contestability for LLMs relies on their opaque mechanisms rather than being fully transparent. A higher number of ↯ symbols denotes faster inference. More details regarding the listed properties are provided in Section 4.3 .
Major premise
Generally, if a clinical intervention provides a net medical benefit and is comparatively suitable among its mutually exclusive alternatives for a given patient, then it is recommended.
Minor premise
The clinical intervention provides a positive benefit-risk balance for the considered patient.
Minor premise
The clinical intervention is superior or equivalent to its incompatible alternatives for the considered patient.
CQ1
Are the clinical features of the patient free of any contraindications against the considered intervention?
CQ2
Does the available evidence from clinical guidelines establish that the intervention provides a net benefit to the given patient?
CQ3
Is there no alternative intervention that offers a better benefit-risk profile for the given patient and is incompatible with the currently considered intervention?
Table 2: Premises and critical questions of the argument scheme sarg used for the glioblastoma treatment recommendation task. The scheme was developed specifically for this task in collaboration with a domain expert.
LLM Backbone
Method
d
Est. Root
Arg. Scheme
LMR ↑
NDCG ↑
I/O Tokens (M) ↓
gpt-oss-20b (medium)
gpt-oss-20b
Base LLM
–
–
–
0.8775
0.9578
16.95 / 6.59
gpt-oss-20b
ArgLLMs-O
1
✗
✗
0.8614
0.9739
80.51 / 20.12
gpt-oss-20b
ArgLLMs-O
1
✗
✓
0.8676
0.9629
80.41 / 19.89
gpt-oss-20b
ArgLLMs-O
1
✓
✗
0.8429
0.9718
95.91 / 22.69
gpt-oss-20b
ArgLLMs-O
1
✓
✓
0.8444
0.9707
95.98 / 22.72
Table 3: Performance on glioblastoma (GBM) treatment prediction across models and inference configurations. LMR denotes the label match rate, indicating the share of the absolute treatment recommendation scores matching the labels, while NDCG quantifies the level of agreement between the ground-truth treatment ranking and the predicted ranking. The I/O Tokens column reports prompt/completion token usage in millions (M) without the initial decision-space ontology construction, which is shared for all methods.
Figure 4: Illustration of the ArgEval contestability experiment 10 10 10 Icon from Noun Project ( Patient by Alzam, CC BY 3.0 ). . To correct the initially suboptimal recommendations, we make small adjustments to the base scores in the general argumentation framework for the radiotherapy 60 Gy treatment option (with initial scores shown in smaller red font and updated scores in larger green font) and clarify the descriptions of two parameters associated with surgical resection in the parameter schema. These modifications are sufficient to achieve a perfect score on this instance while also substantially improving the overall performance. Treatments recommended by the ground-truth labels are shown in green, with those not recommended in red.
Argumentative LLMs (ArgLLMs) are an existing approach leveraging Large Language Models (LLMs) and computational argumentation for decision-making, with the aim of making the resulting decisions faithfully explainable to and contestable by humans. Here we propose a web-based system implementing ArgLLM-empowered agents for binary tasks. ArgLLM-App supports visualisation of the produced explanations and interaction with human users, allowing them to identify and contest any mistakes in the system's reasoning. It is highly modular and enables drawing information from trusted external sources. ArgLLM-App is publicly available at https://argllm.app, with a video demonstration at https://youtu.be/vzwlGOr0sPM.
Legal reasoning requires not only high accuracy but also the ability to justify decisions through verifiable and contestable arguments. However, existing Large Language Model (LLM) approaches, such as Chain-of-Thought (CoT) and Retrieval-Augmented Generation (RAG), often produce unstructured explanations that lack a formal mechanism for verification or user intervention. To address this limitation, we propose Adaptive Collaboration of Argumentative LLMs (ACAL), a neuro-symbolic framework that integrates adaptive multi-agent collaboration with an Arena-based Quantitative Bipolar Argumentation Framework (A-QBAF). ACAL dynamically deploys expert agent teams to construct arguments, employs a clash resolution mechanism to adjudicate conflicting claims, and utilizes uncertainty-aware escalation for borderline cases. Crucially, our framework supports a Human-in-the-Loop (HITL) contestability workflow, enabling users to directly audit and modify the underlying reasoning graph to influence the final judgment. Empirical evaluations on the LegalBench benchmark demonstrate that ACAL outperforms strong baselines across Gemini-2.5-Flash-Lite and Gemini-2.5-Flash architectures, effectively balancing efficient predictive performance with structured transparency and contestability. Our implementation is available at: https://github.com/loc110504/ACAL.
Computational argumentation offers formal frameworks for transparent, verifiable reasoning but has traditionally been limited by its reliance on domain-specific information and extensive feature engineering. In contrast, LLMs excel at processing unstructured text, yet their opaque nature makes their reasoning difficult to evaluate and trust. We argue that the convergence of these fields will lay the foundation for a new paradigm: Argumentative Human-AI Decision-Making. We analyze how the synergy of argumentation framework mining, argumentation framework synthesis, and argumentative reasoning enables agents that do not just justify decisions, but engage in dialectical processes where decisions are contestable and revisable -- reasoning with humans rather than for them. This convergence of computational argumentation and LLMs is essential for human-aware, trustworthy AI in high-stakes domains.
Stylianos Loukas Vasileiou, Antonio Rago, Francesca Toni +1
New Mexico State University, Las Cruces, NM, USA · King’s College London, London, UK · Imperial College London, London, UK +1