Large language models (LLMs) exhibit strong general capabilities, but their deployment in high-stakes domains is hindered by their opacity and unpredictability. Recent work has taken meaningful steps towards addressing these issues by augmenting LLMs with post-hoc reasoning based on computational argumentation, providing faithful explanations and enabling users to contest incorrect decisions. However, this paradigm is limited to pre-defined binary choices and only supports local contestation for specific instances, leaving the underlying decision logic unchanged and prone to repeated mistakes. In this paper, we introduce ArgEval, a framework that shifts from instance-specific reasoning to structured evaluation of general decision options. Rather than mining arguments solely for individual cases, ArgEval systematically maps task-specific decision spaces, builds corresponding option ontologies, and constructs general argumentation frameworks (AFs) for each option. These frameworks can then be instantiated to provide explainable recommendations for specific cases while still supporting global contestability through modification of the shared AFs. We investigate the effectiveness of ArgEval on treatment recommendation for glioblastoma, an aggressive brain tumour, and show that it can produce explainable guidance aligned with clinical practice.
Figures & tables
Figure 1: Illustration of ArgEval inference for one of the treatment options for glioblastoma 3 3 3 Icons from Noun Project ( Patient by Alzam and Data by Anna Riana, CC BY 3.0 ). . Case Parameters extracted from the Case Description are used to instantiate the General QBAF associated with the given option ( Root Argument ), removing the dashed nodes whose conditions are not satisfied. The Prediction After Instantiation is obtained from the instantiated framework, which also serves as a faithful explanation.
Figure 2: Overview of the ArgEval pipeline 5 5 5 Icons from Noun Project ( Policy by Puspito, Decision by Kamin Ginkaew, report analysis by kartini 1, Patient by Alzam and Data by Anna Riana, CC BY 3.0 ). . Top: given natural-language policy documents that specify general criteria for decision-making in a certain domain, ArgEval builds a decision-space ontology and constructs general QBAFs for each candidate decision in the ontology. Bottom: at inference time, the general QBAF is instantiated with the parameters of a specific case, providing faithfully explainable recommendations. Users can contest the decision-space ontology, the general QBAFs, the extracted case parameters and the general parameter schema specifying the properties to be extracted, in response to incorrect recommendations or explanations produced by the model.
Property
Method
LLMs
ArgLLMs ArgRAG
ArgLLMs-O
ArgEval
Open-Ended Decisions
✓
✗
✓
✓
Faithful Explainability
✗
✓
✓
✓
Local Contestability
( ✓ )
✓
✓
✓
Global Contestability
✗
✗
✗
✓
Inference Speed
↯↯
↯
↯
↯↯↯
Table 1: Qualitative comparison between ArgEval and other methods. ( ✓ ) in the third row signifies that local contestability for LLMs relies on their opaque mechanisms rather than being fully transparent. A higher number of ↯ symbols denotes faster inference. More details regarding the listed properties are provided in Section 4.3 .
Major premise
Generally, if a clinical intervention provides a net medical benefit and is comparatively suitable among its mutually exclusive alternatives for a given patient, then it is recommended.
Minor premise
The clinical intervention provides a positive benefit-risk balance for the considered patient.
Minor premise
The clinical intervention is superior or equivalent to its incompatible alternatives for the considered patient.
CQ1
Are the clinical features of the patient free of any contraindications against the considered intervention?
CQ2
Does the available evidence from clinical guidelines establish that the intervention provides a net benefit to the given patient?
CQ3
Is there no alternative intervention that offers a better benefit-risk profile for the given patient and is incompatible with the currently considered intervention?
Table 2: Premises and critical questions of the argument scheme sarg used for the glioblastoma treatment recommendation task. The scheme was developed specifically for this task in collaboration with a domain expert.
LLM Backbone
Method
d
Est. Root
Arg. Scheme
LMR ↑
NDCG ↑
I/O Tokens (M) ↓
gpt-oss-20b (medium)
gpt-oss-20b
Base LLM
–
–
–
0.8775
0.9578
16.95 / 6.59
gpt-oss-20b
ArgLLMs-O
1
✗
✗
0.8614
0.9739
80.51 / 20.12
gpt-oss-20b
ArgLLMs-O
1
✗
✓
0.8676
0.9629
80.41 / 19.89
gpt-oss-20b
ArgLLMs-O
1
✓
✗
0.8429
0.9718
95.91 / 22.69
gpt-oss-20b
ArgLLMs-O
1
✓
✓
0.8444
0.9707
95.98 / 22.72
Table 3: Performance on glioblastoma (GBM) treatment prediction across models and inference configurations. LMR denotes the label match rate, indicating the share of the absolute treatment recommendation scores matching the labels, while NDCG quantifies the level of agreement between the ground-truth treatment ranking and the predicted ranking. The I/O Tokens column reports prompt/completion token usage in millions (M) without the initial decision-space ontology construction, which is shared for all methods.
Figure 4: Illustration of the ArgEval contestability experiment 10 10 10 Icon from Noun Project ( Patient by Alzam, CC BY 3.0 ). . To correct the initially suboptimal recommendations, we make small adjustments to the base scores in the general argumentation framework for the radiotherapy 60 Gy treatment option (with initial scores shown in smaller red font and updated scores in larger green font) and clarify the descriptions of two parameters associated with surgical resection in the parameter schema. These modifications are sufficient to achieve a perfect score on this instance while also substantially improving the overall performance. Treatments recommended by the ground-truth labels are shown in green, with those not recommended in red.