cs.CLMar 12, 2026

LLM-Assisted Causal Structure Disambiguation and Factor Extraction for Legal Judgment Prediction

Authors: Yuzhi Liang, Lixiang Ma, Xinrong Zhu

Organizations: School of Information Technology, Guangdong University of Foreign Studies

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Mainstream methods for Legal Judgment Prediction (LJP) based on Pre-trained Language Models (PLMs) heavily rely on the statistical correlation between case facts and judgment results. This paradigm lacks explicit modeling of legal constituent elements and underlying causal logic, making models prone to learning spurious correlations and suffering from poor robustness. While introducing causal inference can mitigate this issue, existing causal LJP methods face two critical bottlenecks in real-world legal texts: inaccurate legal factor extraction with severe noise, and significant uncertainty in causal structure discovery due to Markov equivalence under sparse features. To address these challenges, we propose an enhanced causal inference framework that integrates Large Language Model (LLM) priors with statistical causal discovery. First, we design a coarse-to-fine hybrid extraction mechanism combining statistical sampling and LLM semantic reasoning to accurately identify and purify standard legal constituent elements. Second, to resolve structural uncertainty, we introduce an LLM-assisted causal structure disambiguation mechanism. By utilizing the LLM as a constrained prior knowledge base, we conduct probabilistic evaluation and pruning on ambiguous causal directions to generate legally compliant candidate causal graphs. Finally, a causal-aware judgment prediction model is constructed by explicitly constraining text attention intensity via the generated causal graphs. Extensive experiments on multiple benchmark datasets, including LEVEN , QA, and CAIL, demonstrate that our proposed method significantly outperforms state-of-the-art baselines in both predictive accuracy and robustness, particularly in distinguishing confusing charges.

Figures & tables

Figure 1: Overall architecture of Ours. For causal diagram nodes, A, B, C, and D represent predicted key nodes, while Y1 and Y2 denote features requiring differentiation.
DatasetTask TypeText Scale (Entries)Average Length (Characters)
LEVENLegal Event Detection8,116502
QA_corpusLegal Consultation Question Classification203,45938
CAIL2018Criminal Charge Prediction30,183671
LEDGARContract Clause Classification80,000113
OverrulingPrecedent Reversal Detection2,40027
Table 1: Dataset Statistics
ModelsLEVENQACAILLEDGAROverruling
BiLSTM2.9319.7569.6376.2491.78
BiLSTM+CRF34.9128.2463.9078.3892.51
BERT72.3778.6488.4587.0195.69
BERT+CRF72.9479.1788.8186.3595.88
Legal-roberta-base35.4676.8762.9387.2996.57
InLegalBERT11.7822.9763.5486.7494.12
Table 2: Comparison of Performance Metrics Across Models
Figure 2: Accuracy Comparison Between LLM Edge Selection and Random Edge Selection Strategies at Different Training Set Ratios
Figure 3: Causal Quality Graph
Figure 4: The Impact of Factor Quality on Outcomes

Explore similar work

CardsList
Aug 30, 2026

JPO: Juris Policy Optimization for Structured Legal Reasoning in Criminal Judgment Prediction

Criminal judgment prediction requires models to infer statutory articles, charges, and sentencing outcomes from case facts. Unlike standard classification tasks, it involves a structured reasoning process in which statutes should be matched with facts, charges should be justified by statutes, and sentencing outcomes should remain consistent with charges. Existing approaches optimize final labels, and while some have attempted to evaluate reasoning quality, their evaluations are indirect, often relying on LLM-generated rubrics that reflect model-internal preferences rather than the inherent logical structure of legal adjudication. We propose Juris Policy Optimization (JPO), a post-training framework for structured legal reasoning in Chinese criminal judgment prediction. JPO first uses teacher-generated rationales to supervise a standardized four-step reasoning process, and then applies reinforcement learning with a composite reward over legal prediction quality, reasoning structure completeness, and cross-step consistency. JPO further introduces token-level advantage reweighting and adaptive clipping for legally salient reasoning segments. Experiments on multiple open-source language models and three Chinese legal benchmarks show that JPO consistently improves both judgment prediction and reasoning quality over supervised fine-tuning and reinforcement learning baselines.

Zhaolu Kang, Yantao Liu, Tailong Luo +14

Hunyuan, Tencent · Peking University · Tsinghua University +5

Legal NLPLegal Judgment PredictionRL for Language Model ReasoningLegal Reasoning
AskPDF
Mar 13, 2026

VERDICT: Verifiable Evolving Reasoning with Directive-Informed Collegial Teams for Legal Judgment Prediction

Legal Judgment Prediction (LJP) predicts applicable law articles, charges, and penalty terms from case facts. Beyond accuracy, LJP calls for intrinsically interpretable and legally grounded reasoning that can reconcile statutory rules with precedent-informed standards. However, existing methods often behave as static, one-shot predictors, providing limited procedural support for verifiable reasoning and little capability to adapt as jurisprudential practice evolves. We propose VERDICT, a self-refining collaborative multi-agent framework that simulates a virtual collegial panel. VERDICT assigns specialized agents to complementary roles (e.g., fact structuring, legal retrieval, opinion drafting, and supervisory verification) and coordinates them in a traceable draft--verify--revise workflow with explicit Pass/Reject feedback, producing verifiable reasoning traces and revision rationales. To capture evolving case experience, we further introduce a Hybrid Jurisprudential Memory (HJM) grounded in the Micro-Directive Paradigm, which stores precedent standards and continually distills validated multi-agent verification trajectories into updated Micro-Directives for continual learning across cases. We evaluate VERDICT on CAIL2018 and a newly constructed CJO2025 dataset with a strict future time-split for temporal generalization. VERDICT achieves state-of-the-art performance on CAIL2018 and demonstrates strong generalization on CJO2025. To facilitate reproducibility and further research, we release our code and the dataset at https://anonymous.4open.science/r/ARR-4437.

Hui Liao, Chuan Qin, Yongwen Ren +4

University of Science and Technology of China · Chinese Academy of Sciences · iFLYTEK AI Research

Legal Judgment PredictionLegal ReasoningMulti-Agent LLM SystemsContinual Learning for LLM AgentsLLM Self-Refinement
AskPDF
Jul 5, 2026

Shortcut Learning in Legal Judgment Prediction: Empirical Evidence from the UK Employment Tribunal

Current Legal Judgment Prediction (LJP) is constrained by its reliance on post-hoc judicial materials, increasing the likelihood that models perform retrospective classification rather than true forecasting. This paper empirically investigates shortcut learning in this context by studying claim-level outcome prediction in UK Employment Tribunal (UKET) decisions. Using a corpus of 33,158 individual claims, we predict outcomes from claim texts and LLM-extracted case summaries, evaluating models ranging from interpretable TF-IDF-based classifiers to black-box LLMs. While headline predictive performance figures appear strong, we demonstrate that such performance in LJP systems trained on post-hoc judicial text can be driven by the retrospective nature of the source material. Stratifying the test data by human judgments of leakage reveals that performance increases where outcome-revealing cues are embedded in the narrative. Moreover, a model trained on just the 4% of features identified as leakage achieves high performance, outperforming human experts. These findings substantiate concerns that LJP performance may be exaggerated by linguistic artefacts. Yet this vulnerability is not fatal to the research agenda. Instead, post-hoc judgments might be treated as potentially contaminated texts, requiring active auditing. Retraining models after masking leakage features results in only a negligible reduction in Macro-F1. Hence, while models will opportunistically exploit shortcuts when available, they remain capable of extracting useful predictive signals when these artefacts are removed.

Joe Watson, Joana Ribeiro de Faria, Marcus Tomalin +6

Faculty of Law, University of Cambridge, Cambridge, United Kingdom · The Psychometrics Centre, Cambridge Judge Business School, University of Cambridge, Cambridge, United Kingdom · Faculty of English, University of Cambridge, Cambridge, United Kingdom +3

Data LeakageLegal NLPLegal Judgment PredictionShortcut Learning
AskPDF