Adaptive Collaboration of Arena-Based Argumentative LLMs for Explainable and Contestable Legal Reasoning
Authors: Hoang-Loc Cao, Phuc Ho, Truong Thanh Hung Nguyen, Phuc Truong Loc Nguyen, Dinh Thien Loc Nguyen, Hung Cao
Organizations: Ho Chi Minh University of Science, Vietnam · University of New Brunswick, Canada · Friedrich-Alexander-Universität Erlangen-Nürnberg, Germany
Legal reasoning requires not only high accuracy but also the ability to justify decisions through verifiable and contestable arguments. However, existing Large Language Model (LLM) approaches, such as Chain-of-Thought (CoT) and Retrieval-Augmented Generation (RAG), often produce unstructured explanations that lack a formal mechanism for verification or user intervention. To address this limitation, we propose Adaptive Collaboration of Argumentative LLMs (ACAL), a neuro-symbolic framework that integrates adaptive multi-agent collaboration with an Arena-based Quantitative Bipolar Argumentation Framework (A-QBAF). ACAL dynamically deploys expert agent teams to construct arguments, employs a clash resolution mechanism to adjudicate conflicting claims, and utilizes uncertainty-aware escalation for borderline cases. Crucially, our framework supports a Human-in-the-Loop (HITL) contestability workflow, enabling users to directly audit and modify the underlying reasoning graph to influence the final judgment. Empirical evaluations on the LegalBench benchmark demonstrate that ACAL outperforms strong baselines across Gemini-2.5-Flash-Lite and Gemini-2.5-Flash architectures, effectively balancing efficient predictive performance with structured transparency and contestability. Our implementation is available at: https://github.com/loc110504/ACAL.
Figures & tables
Figure 1. Our proposed Adaptive Collaboration of Argumentative LLMs (ACAL) Architecture for Legal Reasoning.
Category
Roles
Adjudication
Judge, Law Clerk / Judicial Clerk
Litigation & Advocacy
Private Practice Lawyer, Prosecutor, Public Defender
Advisory & Transactional
Corporate Counsel, Compliance Officer, IP Attorney
Research & Support
Legal Analyst, Paralegal
Table 1. Legal roles grouped by functional category.
Score Range
Interpretation
0.1−0.2
Incorrect legal analysis or misidentification of key elements
0.3−0.4
Partially correct but missing critical components or overly generic
0.5−0.6
Sound analysis with minor gaps or insufficient case-specific application
0.7−0.8
Strong analysis with specific facts and correct legal reasoning
0.9−1.0
Exceptional precision with authoritative citations and flawless logic
Figure 2. Illustration of an Arena Debating Round through Clash Resolution (CR).
Model
Method
Learned Hands Courts
Hearsay
Acc
Prec
Rec
F1
Acc
Prec
Rec
F1
Gemini-2.5-Flash-Lite
SP
57.8
63.1
57.8
53.1
69.2
73.6
71.5
68.9
CoT
69.3
69.3
69.2
69.2
69.2
74.7
71.8
68.7
RAG
70.3
70.5
70.3
70.3
71.3
70.8
70.9
70.9
MAD
69.8
69.8
69.8
69.8
74.5
75.8
72.4
72.7
ACAL (Ours)
70.8
71.2
70.8
70.7
74.5
77.1
76.3
74.4
Table 3. Comparative results on Learned Hands Courts and Hearsay from LegalBench. All metrics are reported as percentages. The highest scores are in bold .
Table 6
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
Figure 3. Illustrative Case Study of ACAL on the LegalBench Hearsay Task.
Graph-based Retrieval-Augmented Generation (GraphRAG) advances flat document retrieval by structuring knowledge as relational graphs, enabling more coherent and effective reasoning. However, applying it to specific domains like legal reasoning faces critical challenges. (i) Legal corpora are heterogeneous, containing multi-granular knowledge from cases, articles and interpretations. A flat knowledge graph cannot adequately differentiate between factual details, applied rules, and abstract principles, limiting accurate retrieval. (ii) Reliable legal judgment demands transparent, evidence-based reasoning. Traditional RAG passes retrieved context directly to an LLM without verification, resulting in opaque, error-prone reasoning. To this end, we propose LegalGraphRAG, a framework designed for reliable legal reasoning. Our approach introduces two core components: a hierarchical legal graph that hierarchically organizes legal sources to enable retrieval at appropriate abstraction levels, and a multi-agent system for reliable legal reasoning, where a Researcher retrieves candidate evidence, an Auditor rigorously verifies its validity against source documents, and an Adjudicator synthesizes the set of verified evidence to render a final judgment. Extensive experiments show that LegalGraphRAG achieves the state-of-the-art performance, outperforming existing GraphRAG baselines in accurate and trustworthy legal analysis. Our code, datasets and implementation details are available at https://github.com/XMUDeepLIT/LegalGraphRAG.
Zerui Chen, Qinggang Zhang, Zhishang Xiang +5
School of Informatics, Xiamen University · The Hong Kong Polytechnic University · Institute of Artificial Intelligence, Xiamen University
Large language models (LLMs) exhibit strong general capabilities, but their deployment in high-stakes domains is hindered by their opacity and unpredictability. Recent work has taken meaningful steps towards addressing these issues by augmenting LLMs with post-hoc reasoning based on computational argumentation, providing faithful explanations and enabling users to contest incorrect decisions. However, this paradigm is limited to pre-defined binary choices and only supports local contestation for specific instances, leaving the underlying decision logic unchanged and prone to repeated mistakes. In this paper, we introduce ArgEval, a framework that shifts from instance-specific reasoning to structured evaluation of general decision options. Rather than mining arguments solely for individual cases, ArgEval systematically maps task-specific decision spaces, builds corresponding option ontologies, and constructs general argumentation frameworks (AFs) for each option. These frameworks can then be instantiated to provide explainable recommendations for specific cases while still supporting global contestability through modification of the shared AFs. We investigate the effectiveness of ArgEval on treatment recommendation for glioblastoma, an aggressive brain tumour, and show that it can produce explainable guidance aligned with clinical practice.
Adam Dejl, Matthew Williams, Francesca Toni
Department of Computing, Imperial College London · Department of Surgery & Cancer, Imperial College London
Large language models are increasingly applied to high-risk domains such as law, yet complex legal reasoning remains limited by two structural challenges. First, existing RAG and GraphRAG methods emphasize lexical or semantic similarity while overlooking normative relations among legal provisions. Second, vanilla Chain-of-Thought prompting may generate plausible rationales without enforcing the normative structure of legal reasoning. To deal with the bottleneck of pipelines in the legal reasoning domain, we propose LEGO, a dual-module framework that synergizes Legal Expert GraphRAG and expert Chain-of-thought for complex legal reasoning. ExpertGraphRAG uses an expert-annotated civil code graph encoding these normative relations with a greedy normative-coverage retrieval algorithm to dynamically extract instance-specific provision subgraphs, while ExpertCoT organizes the retrieved provisions and case facts into structured Provision-Fact-Conclusion reasoning. With a Qwen3-8B backbone, LEGO achieves 40.53% exact-match accuracy on LawExamQA_Civil, outperforming the evaluated RAG and CoT baselines and performing comparably to the evaluated larger models, while remaining robust on multi-hop questions. It also achieves the best results among the evaluated baselines on the open-ended benchmarks. Ablation studies confirm the individual and complementary contributions of both modules, demonstrating LEGO's effectiveness in improving LLMs' complex legal reasoning ability. Code and dataset can be found in the link: https://github.com/BLK-WHT/LEGO
Qingjing Chen, Junkai Zhang, Shaochun Wang +7
Alma AI, University of Bologna, Italy · Tsinghua University, China · Modelbest Inc. +2