Organizations: School of Artificial Intelligence and Data Science, University of International Business and Economics · Department of Computer Science, University of Chicago · School of Law, University of International Business and Economics · Gaoling School of Artificial Intelligence, Renmin University of China
Legal judgment prediction models may retain a guilty verdict after a legally decisive condition has been altered such that the specified charge is no longer established. We study this guilty bias via selective repair over matched charge-established and charge-not-established cases, where the unsupported verdict should change while its supported counterpart remains stable. We construct CAIL-MC, a legally reviewed dataset of 11,478 matched pairs covering 261 Chinese criminal charges. Held-out activation interventions show that contrasts learned from correctly resolved pairs transfer to false-guilty cases, but target-reaching interventions can also damage matched supported-guilty decisions. Motivated by this observation, we introduce PROSE, which reconstructs case-specific targets from a training-derived residual geometry, aligns them with a local margin-descent signal, attenuates supported-guilty-sensitive components, and realizes bounded targets through a preservation-regularized low-rank parameter update. Across Qwen3-8B and GLM4-9B, PROSE achieves stronger pairwise safe repair and evidence sensitivity, while preserving standard single-pass inference.
Figures & tables
Figure 1: (a) Guilty bias and (b) its correction trade-off.
Figure 2: Qwen3-8B diagnosis. (a) layer-wise NG-margin reduction under forward and control directions, (b) repair feasibility, and (c) paired-G margin damage.
Track
Source
Train
Dev
Test
Pairwise
CAIL2018
9,181
1,147
1,150
Stage distribution: O 35.7%, U 17.8%, C 46.5%
Evidence
LeCaRD
–
–
280
Criminal-S
–
–
800
Table 1: CAIL-MC statistics. Pairwise counts are matched pairs; evidence counts are variants, four per source case (70 LeCaRD; 200 Criminal-S). O/U/C denote Offense/Unlawfulness/Culpability.
CAIL2018 (Pairwise Repair–Preservation)
LeCaRD (Evidence-Sufficiency Generalization)
Criminal-S (Evidence-Sufficiency Generalization)
Case level
Pair level
Case level
Case level
Method
Acc ↑
F1 ↑
FGR ↓
FNR ↓
Rep. ↑
G-Ret. ↑
PSR ↑
Acc ↑
F1 ↑
FGR ↓
FNR ↓
Acc ↑
F1 ↑
FGR ↓
FNR ↓
Qwen3-8B-Base
Base
72.65 ± 1.43
70.58 ± 1.77
53.91 ± 2.82
0.78 ± 0.50
–
–
–
55.71 ± 2.99
46.91 ± 5.22
85.00 ± 6.59
3.57 ± 3.67
66.88 ± 2.58
63.63 ± 3.36
63.00 ± 5.05
3.25 ± 2.00
SFT
98.87 ± 0.44
98.87 ± 0.44
2.00 ± 0.83
0.26 ± 0.30
96.27 ± 1.53
100.00 ± 0.00
96.27 ± 1.53
49.64 ± 2.71
37.60 ± 3.45
94.29 ± 3.79
6.43 ± 5.26
51.00 ± 1.12
36.72 ± 1.83
96.50 ± 1.78
1.50 ± 1.58
Self-Refine
70.35 ± 1.49
68.29 ± 1.79
4.17 ± 1.13
55.13 ± 2.77
95.79 ± 1.63
39.71 ± 3.66
36.79 ± 3.65
54.29 ± 4.56
47.90 ± 6.33
10.71 ± 5.13
80.71 ± 7.98
54.88 ± 1.70
44.86 ± 2.97
2.50 ± 1.53
87.75 ± 3.37
Table 2: Results (%) on both CAIL-MC tracks. Entries are point estimates ± 1.96% bootstrap standard errors (1,000 resamples over pairs for repair–preservation and cases for evidence-sufficiency evaluation). Best and second-best results per backbone, excluding Base, are bold and underlined. Base’s pair-level scores are omitted: Rep. = PSR = 0 and G-Ret. = 100 by construction.
Figure 3: Mean verdict margin as evidence is removed from the 200 Criminal-S cases of Evidence Track, with (a) Qwen3-8B-Base and (b) GLM-4-9B-Base.
Figure 4: Repair and preservation. (a–b) Pair states before and after PROSE. (c) Supported-guilty damage at matched repair; lower is better.
Qwen3-8B-Base
GLM-4-9B-Base
Variant
F1 ↑
Rep. ↑
G-Ret. ↑
PSR ↑
F1 ↑
Rep. ↑
G-Ret. ↑
PSR ↑
PROSE (full)
91.48
85.25
90.76
76.34
87.17
67.37
94.02
62.16
w/o attenuation (Eq. 8 )
76.44
40.84
99.68
40.52
86.82
68.15
93.05
62.36
w/o constraints (Eq. 11 )
57.27
70.83
41.00
21.39
35.40
99.81
2.90
2.70
w/o both
61.16
62.72
50.57
25.12
35.78
99.81
3.67
3.47
Table 3: Preservation ablation on CAIL-MC.
Figure 5: Offline cost on one RTX 4080 SUPER and per-query latency relative to PROSE, both on log scales.
Appendix figures & tables13 assets
Supplementary material from the paper’s appendix.
Appendix
Total
Agree
Disagree
Discarded
Retained
Batch 1
3,141
2,779
362
267
2,874
Batch 2
3,141
2,758
383
274
2,867
Batch 3
3,141
2,791
350
261
2,880
Batch 4
3,141
2,744
397
284
2,857
Total
12,564
11,072
1,492
1,086
11,478
Appendix
Table 4: Human verification results across the four review batches. “Agree” denotes identical decisions independently made by the two junior annotators. Every “Disagree” case was directly referred to the senior criminal-law expert for final adjudication.
Source
Total
Agree
Disagree
Discarded
Retained
LeCaRD
80
65
15
10
70
Criminal-S
200
194
6
0
200
Total
280
259
21
10
270
Appendix
Table 5: Human verification results for the evidence-sufficiency gradient. “Agree” denotes identical retain/discard decisions independently made by the two criminal-law graduate reviewers. Every “Disagree” case is directly referred to the senior criminal-law expert for final adjudication.
Stage
Geometry
Without
With full projection
Offense
Stage-specific
70.6 (108)
17.6 (27)
Shared
70.6 (108)
20.3 (31)
Unlawfulness
Stage-specific
83.2 (154)
15.7 (29)
Shared
76.8 (142)
19.5 (36)
Culpability
Stage-specific
75.9 (217)
10.8 (31)
Shared
76.9 (220)
10.1 (29)
Appendix
Table 6: Repair-target feasibility (%, feasible pairs in parentheses) on Qwen3-8B-Base without and with full projection onto the complement of the supported-guilty gradient subspace ( ρ=0 in Eq. 8 ).
Figure 6: Cross-stage transfer at Layer 22 on Qwen3-8B-Base. Each cell adds the contrast direction of a source stage to cases of a target stage and reports the mean NG-margin reduction; the diagonal (bold) gives each direction’s effect on its own stage.
Figure 9: Repair feasibility and preservation on GLM-4-9B-Base. (a) Repair feasibility under stage-specific and shared contrast geometry for the three legal stages. (b) Paired-G margin damage on matched feasible cases, before and after preservation projection. Lower paired-G damage indicates better preservation of the supported charge-established counterpart.
Symbol
Description
Value / range
ℓ
Edited layer
20, 22
–
Edited modules
attn-out + MLP down
–
Contrast geometry
stage-specific or shared
τ
Target margin endpoint
[−0.02,−0.005]
λ
Contrast–gradient mixture
[0,1]
rq / rp
Contrast / preservation rank
128 / 64
Appendix
Table 7: Hyperparameters of PROSE across both backbones. Ranges give the development-split search space; single values are fixed. Configurations are selected per backbone and legal stage on the development split.
Figure 10: Mean verdict margin on the 1,150 charge-established CAIL-MC test facts rewritten at three evidence levels, with (a) Qwen3-8B-Base and (b) GLM-4-9B-Base. The vertical axis is linear within [−1,1] in (a) and [−0.2,0.2] in (b) (shaded) and logarithmic beyond; error bars are 95% confidence intervals.
Judged established (%)
Method
Zero ↓
Partial
Full ↑
Δs↑
Unedited
93.8
94.8
97.7
0.58
Uniform shift
0.2
34.8
69.4
0.58
SFT
100.0
98.3
99.0
− 0.25
PROSE
10.3
76.0
86.1
0.74
Appendix
Table 8: Share of the rewritten CAIL-MC facts judged as establishing the charge with Qwen3-8B-Base, and the mean margin change from Zero to Full ( Δs ). The uniform shift is the matched-repair control of Sec. 6.3 .
Level
Unedited
SFT
PROSE
Qwen
Zero
0.304 [0.293, 0.315]
4.835 [4.755, 4.921]
− 0.266 [ − 0.281, − 0.251]
Partial
0.568 [0.546, 0.590]
4.130 [4.003, 4.252]
0.314 [0.290, 0.339]
Full
0.888 [0.863, 0.911]
4.589 [4.468, 4.703]
0.478 [0.451, 0.504]
GLM
Zero
0.059 [0.055, 0.063]
4.531 [4.389, 4.674]
− 0.000 [ − 0.004, 0.004]
Partial
0.179 [0.168, 0.191]
6.251 [5.944, 6.564]
0.126 [0.117, 0.135]
Full
0.272 [0.260, 0.284]
6.878 [6.557, 7.205]
0.172 [0.163, 0.181]
Appendix
Table 9: Mean verdict margin [95% bootstrap CI] of the rewritten CAIL-MC facts at each evidence level.
Acc
F1
FGR
FNR
Rep.
G-Ret.
Off.
Unl.
Cul.
All
Qwen3-8B-Base
O
72.30
70.11
54.78
0.61
7.46
100.00
26.02
1.10
3.83
7.46
U
87.96
87.87
20.35
3.74
63.53
96.76
31.71
74.03
63.58
60.29
C
85.91
85.69
26.43
1.74
51.70
99.19
31.71
17.13
77.96
50.89
OU
86.26
86.11
24.00
3.48
59.00
97.08
48.78
67.96
52.40
56.24
OC
82.39
81.94
33.48
1.74
40.52
99.03
44.72
16.57
50.80
39.55
Appendix
Table 10: Stage ablation on CAIL-MC (%). O / U / C denote the Offense, Unlawfulness, and Culpability updates; OUC is PROSE. The right block gives PSR on the strict false-guilty test pairs of each stage and on all of them.
Figure 11: Effect of editing on the ten most frequent charges in the CAIL-MC test split. Beneficial : test predictions that editing flips from incorrect to correct; harmful : predictions it flips from correct to incorrect.
Figure 12: Change in verdict margin of the charge-not-established member ( ΔsNG ) and of its charge-established counterpart ( ΔsG ) for each strict false-guilty CAIL-MC test pair under PROSE, on (a) Qwen3-8B-Base and (b) GLM-4-9B-Base. The diamond marks the medians; a uniform output shift would place every pair at a single point on the dashed diagonal.
Figure 13: Verdict margins of the charge-established member ( sG , horizontal) and the charge-not-established member ( sNG , vertical) for every CAIL-MC test pair under the base model and PROSE, on (a) Qwen3-8B-Base and (b) GLM-4-9B-Base. The dashed lines are the decision boundaries, so the four quadrants are the four joint pair states; the marginal panels show the distribution of each margin.
Legal Judgment Prediction (LJP) has become a core benchmark for evaluating AI in the criminal legal domain, but it only sees criminal cases that have already passed prosecutorial review and been formally indicted. As a result, LJP leaves a substantial blind spot in assessing criminal liability, overlooking cases involving insufficient evidence, no criminal liability, or guilt exempted from punishment. To fill this gap, we propose Prosecution Decision Prediction (PDP), the first Legal AI task built around prosecutorial review, which classifies each case into prosecution or one of three non-prosecution decisions and reflects legal AI's capabilities in evidence evaluation, legal subsumption, and value-based discretion. We further construct PDP-Bench, a benchmark of 4{,}630 real Chinese prosecutorial decisions spanning 190 charges. Extensive experiments show that state-of-the-art LLMs perform substantially worse on PDP than on LJP and that mainstream enhancement routes fail to close the gap. Moreover, controlled RLVR interventions show that simple outcome rewards fail to produce generalizable PDP discrimination.
Junyu Lu, Qi Wei, Peishuo Zheng +6
1Beijing Institute of Technology, Zhuhai · 2Osaka University · 3Xi’an University of Technology +2
Criminal judgment prediction requires models to infer statutory articles, charges, and sentencing outcomes from case facts. Unlike standard classification tasks, it involves a structured reasoning process in which statutes should be matched with facts, charges should be justified by statutes, and sentencing outcomes should remain consistent with charges. Existing approaches optimize final labels, and while some have attempted to evaluate reasoning quality, their evaluations are indirect, often relying on LLM-generated rubrics that reflect model-internal preferences rather than the inherent logical structure of legal adjudication. We propose Juris Policy Optimization (JPO), a post-training framework for structured legal reasoning in Chinese criminal judgment prediction. JPO first uses teacher-generated rationales to supervise a standardized four-step reasoning process, and then applies reinforcement learning with a composite reward over legal prediction quality, reasoning structure completeness, and cross-step consistency. JPO further introduces token-level advantage reweighting and adaptive clipping for legally salient reasoning segments. Experiments on multiple open-source language models and three Chinese legal benchmarks show that JPO consistently improves both judgment prediction and reasoning quality over supervised fine-tuning and reinforcement learning baselines.
Zhaolu Kang, Yantao Liu, Tailong Luo +14
Hunyuan, Tencent · Peking University · Tsinghua University +5
Mainstream methods for Legal Judgment Prediction (LJP) based on Pre-trained Language Models (PLMs) heavily rely on the statistical correlation between case facts and judgment results. This paradigm lacks explicit modeling of legal constituent elements and underlying causal logic, making models prone to learning spurious correlations and suffering from poor robustness. While introducing causal inference can mitigate this issue, existing causal LJP methods face two critical bottlenecks in real-world legal texts: inaccurate legal factor extraction with severe noise, and significant uncertainty in causal structure discovery due to Markov equivalence under sparse features. To address these challenges, we propose an enhanced causal inference framework that integrates Large Language Model (LLM) priors with statistical causal discovery. First, we design a coarse-to-fine hybrid extraction mechanism combining statistical sampling and LLM semantic reasoning to accurately identify and purify standard legal constituent elements. Second, to resolve structural uncertainty, we introduce an LLM-assisted causal structure disambiguation mechanism. By utilizing the LLM as a constrained prior knowledge base, we conduct probabilistic evaluation and pruning on ambiguous causal directions to generate legally compliant candidate causal graphs. Finally, a causal-aware judgment prediction model is constructed by explicitly constraining text attention intensity via the generated causal graphs. Extensive experiments on multiple benchmark datasets, including LEVEN , QA, and CAIL, demonstrate that our proposed method significantly outperforms state-of-the-art baselines in both predictive accuracy and robustness, particularly in distinguishing confusing charges.
Yuzhi Liang, Lixiang Ma, Xinrong Zhu
School of Information Technology, Guangdong University of Foreign Studies