Identifying the issues disputed between litigating parties is a crucial component of real-world litigation. However, legal issues remain comparatively underexplored in legal AI research. In this work, we study the computational modelling of legal issue identification in litigation. We introduce a legally grounded hierarchical schema that represents legal issues through both free-form issue descriptions and structured legal categories, and formulate legal issue identification as two complementary tasks: legal issue generation and legal issue classification. Based on this formulation, we construct LexIssue, a benchmark containing 430 real-world Chinese civil litigation cases and 1,303 expert-annotated disputed legal issues. We further develop an issue-centric legal knowledge base spanning 27 causes of action and 441 candidate legal issue entries to support retrieval-augmented reasoning. Experimental results across a diverse set of models show that retrieval-augmented generation using the constructed legal issue knowledge base consistently improves performance in identifying disputed legal issues and their corresponding legal attributes.
Figures & tables
Figure 1: Overview of the legal issue identification process . Left : An illustrative product liability dispute in which the claimant alleges that a freezer purchased from the defendant is defective and seeks compensation for losses, while the defendant denies the existence of defects, disputes causation and fault, and refuses compensation. The issue analysis content is extracted from the judge’s reasoning section in the original court judgment. Right : An illustration of the legal issue identification task, where models identify disputed legal issues from the litigated case and generate both descriptive issue summaries and structured legal category labels. In the illustrated three-tier legal category hierarchy, the first-tier labels law & fact and law correspond to mixed issues of law and fact and substantive issues of law , respectively. The second-tier labels elements and effect correspond to elements of a claim and legal effect . The third-tier labels act , causation , fault , and liability presence correspond to the schema categories tortious act , causation , fault , and presence or absence of liability , respectively.
Model
LIG
LIC
Tier-1
Tier-2
Tier-3
GPT-5.2
0.542
0.346
0.255
0.185
w/ LP
0.475
0.279
0.191
0.133
w/ RAG
0.634
0.460
0.335
0.243
Gemini-3-Flash
0.701
0.581
0.459
0.357
w/ LP
0.699
0.544
0.394
0.292
Table 1: Evaluation results on the LexIssue benchmark. LIG denotes the legal issue generation task, whilst LIC denotes the legal issue classification task. Entries labelled with the model name alone (e.g., GPT-5.2) correspond to the zero-shot inference setting. The variant “w/ LP” denotes models evaluated under the legal prompting setting, while “w/ RAG” denotes models augmented with RAG using the external legal issue knowledge base.
Rater
Sim.
Align.
Agr.
Qwen-3.5-27B
0.965
0.852
0.668
ROUGE_1
0.870
0.635
0.178
gte-large-zh
0.870
0.665
0.254
Table 2: Comparison of representative instances for three categories of rating approaches: LLM-as-a-Judge (Qwen-3.5-27B), NLG-based metrics (ROUGE_1), and embedding-based methods (gte-large-zh). Sim. denotes the accuracy achieved in the similarity mapping step; Align. denotes the accuracy achieved in the alignment verification step; Agr. denotes the overall agreement with human judgments.
Model
Tier-1
Tier-2
Tier-3
GPT-5.2
0.971
0.707
0.513
Gemini-3-Flash
0.900
0.709
0.550
DeepSeek-R1
0.877
0.653
0.421
DeepSeek-V4-Flash
0.851
0.657
0.467
Qwen-3-8B
0.908
0.583
0.310
Llama-3.1-8B
0.838
0.658
0.326
Table 3: Model performance on legal issue classification under Tier-1, Tier-2, and Tier-3 settings, calculated only on the subset of successfully aligned issues.
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 2: The proposed dual-layer hierarchical schema for representing legal issues.
Figure 3: Prompt specification (Part 1/2): Task overview and execution steps (Sections I & II). Covers the role/task definition, input variables, and the three execution steps (Substantive/Procedural → Legal/Factual → Legal Logic Subdivision) with the Fact-Mapping Procedure for mixed Legal–Factual pairs. The Chain-of-Thought analysis is written exclusively in JSON fields.
Figure 4: Prompt specification (Part 2/2): Scoring framework, semantic adjustments, and evaluation example (Sections III, IV & V). Covers the final assessment step (Step 4), the quantitative scoring framework with the 6-level benchmark table, semantic distance adjustment rules (terminology, redundancy, bonus), and a fully worked example with complete classification path and score breakdown.
Figure 5: Illustration of the alignment workflow used in the issue-level alignment verification stage. The reference and predicted issues are compared by traversing the hierarchical legal issue taxonomy, with the alignment level determined by the deepest hierarchical tier at which their category paths match. The resulting alignment level forms part of the rubric used by the LLM judge to determine whether the prediction and reference represent the same underlying disputed legal issue.
More than half of the global population struggles to meet their civil justice needs due to limited legal resources. While Large Language Models (LLMs) have demonstrated impressive reasoning capabilities, significant challenges remain even at the foundational step of legal issue identification. To investigate LLMs' capabilities in this task, we constructed a dataset from 769 real-world Malaysian Contract Act court cases, using GPT-4o to extract facts and generate candidate legal issues, annotated by senior legal experts, which reveals a critical limitation: while LLMs generate diverse issue candidates, their precision remains inadequate (GPT-4o achieves only 62%). To address this gap, we propose LePREC (Legal Professional-inspired Reasoning Elicitation and Classification), a neuro-symbolic framework combining neural generation with structured statistical reasoning. LePREC consists of: (1) a neuro component leverages LLMs to transform legal descriptions into question-answer pairs representing diverse analytical factors, and (2) a symbolic component applies sparse linear models over these discrete features, learning explicit algebraic weights that identify the most informative reasoning factors. Unlike end-to-end neural approaches, LePREC achieves interpretability through transparent feature weighting while maintaining data efficiency through correlation-based statistical classification. Experiments show a 30-40% improvement over advanced LLM baselines, including GPT-4o and Claude, confirming that correlation-based factor-issue analysis offers a more data-efficient solution for relevance decisions.
Fanyu Wang, Xiaoxi Kang, Paul Burgess +6
Faculty of Information Technology, Monash University, Australia · School of Information Technology, Monash University Malaysia · Faculty of Law, Monash University, Australia +1
Legal AI systems are increasingly used to answer legal questions, yet existing benchmarks assume queries arrive fully specified. In practice, users omit facts that materially determine the legal outcome. We introduce InsufficiencyBench, the first legal benchmark targeting query-side insufficiency: whether a model recognizes when a query lacks legally material information, identifies what is missing, and refrains from premature conclusions. We formalize a taxonomy of eight canonical missing-element categories across three structural failure modes---switch, gating, and fatal prerequisite--- and construct 202 benchmark items (58 base queries, 144 deficient variants) spanning six legal domains and 24 US jurisdictions and annotated by practising attorneys. Evaluating ten frontier models, we find that no model exceeds F2 = 0.46 on missing-element identification and that the median recall is 0.44. Models either hedge indiscriminately or answer silently under fabricated presumptions. No model both identifies and qualifies responses to deficient queries while directly addressing complete ones.
Samuel J. Vincent, Daniel Calloway, Fangyi Yu +2
Thomson Reuters Foundational Research, 5 Canada Square, London · Imperial College London
Trustworthy Legal AI requires systems that can answer legal questions while grounding their responses in authoritative sources. However, existing Vietnamese legal benchmarks provide limited coverage of real-world legal consultations. We introduce ViLegalExpert, a large-scale benchmark constructed from authentic citizen--lawyer consultations, containing over 172K questions across 34 legal domains, together with professional answers and expert-verified legal evidence. ViLegalExpert supports legal information retrieval, extractive QA, and abstractive QA. Experiments with representative retrieval methods and language models reveal substantial challenges in evidence retrieval and grounded answer generation. While pretrained models perform strongly on QA, hybrid retrieval achieves the best retrieval performance. These results demonstrate the difficulty of mapping naturally expressed legal questions to authoritative provisions and establish ViLegalExpert as a challenging benchmark for reliable Vietnamese Legal AI.
Dat Tien Nguyen, Nghia Hieu Nguyen, Anh Thi-Hoang Nguyen +3
University of Information Technology · Vietnam National University, Ho Chi Minh city, Viet Nam