As LLM-based agents develop, they have been increasingly applied in the legal domain. Legal Judgment Prediction~(LJP) requires models to conduct multi-step reasoning and external legal evidence retrieval. Existing legal search agents' performance is often constrained by the quality of retrieval queries. They often formulate queries directly from the initial user request, overlooking latent legal issues and legally salient details. To address this issue, we propose LegalEvolver, a query optimization framework specifically designed to generate legally informative queries for LJP-oriented retrieval. Sampling high-quality optimized queries and their corresponding search trajectories is challenging, as relying solely on unguided high-temperature rollouts is both inefficient and constrained by the model's initial capabilities. We therefore heuristically optimize queries as seeds, improving the efficiency of trajectory generation. Furthermore, we summarize rollout experience into rubrics and use them to guide multi-round iterative self-evolution. Iteratively performing rubric optimization and rubric-guided RL enables the model to surpass its initial performance ceiling. End-to-end performance on extensive benchmarks demonstrates the effectiveness of query optimization on LJP. Comparisons with direct SFT followed by RL demonstrate the effectiveness of rubric guidance. Evolving rubrics further outperform fixed rubrics, confirming the benefits of self-evolution.
Figures & tables
Figure 1: Task description and main pipeline of LegalEvolver.
Methods
CAIL2018
CMDL
Overall
Law Article
Charge
Prison Term
Law Article
Charge
Prison Term
Acc.
Ma-F1
Acc.
Ma-F1
Acc.
Ma-F1
Acc.
Ma-F1
Acc.
Ma-F1
Acc.
Ma-F1
Classification Methods
TopJudge
75.7
64.6
71.8
63.6
20.1
11.6
80.4
82.6
79.5
80.2
20.2
11.8
55.2
NeurJudge
74.8
62.0
71.2
63.0
18.9
10.6
77.1
79.0
74.0
74.7
20.3
12.1
53.1
BERT
80.6
68.3
76.4
68.7
19.2
9.8
81.6
83.3
79.3
80.1
20.8
13.1
56.8
Table 1: Performance comparison on the CAIL2018 and CMDL datasets. The best results are indicated in bold, and the second-best results are underlined.
Figure 2: Stage-wise RL training curves of LegalEvolver on CAIL2018 and CMDL.
Figure 3: Query failures and quality scores across methods: (a) distribution of five query failure types and (b) mean scores on four query-quality dimensions.
Figure 4: Evolution of the statute_search rubric from the initial policy through two optimization stages, shown for legal qualification and sentencing.
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 5: Test-set statistics and the distribution of the 201 normalized charge categories covered by the two datasets.
Table 7
Method
CAIL2018
CMDL
Overall
Acc.
Ma-F1
Acc.
Ma-F1
LegalEvolver (Ours)
61.20
54.53
65.23
65.17
61.53
w/o Rubric Evolution
60.03 (-1.17)
52.50 (-2.03)
64.33 (-0.90)
64.63 (-0.54)
60.38 (-1.15)
w/o Rubric Rewards
60.03 (-1.17)
53.03 (-1.50)
63.43 (-1.80)
63.23 (-1.94)
59.93 (-1.60)
w/o Targeted Rubrics
60.01 (-1.19)
52.77 (-1.76)
63.88 (-1.35)
63.93 (-1.24)
60.15 (-1.38)
w/o GRPO
58.83 (-2.37)
53.00 (-1.53)
61.27 (-3.96)
60.87 (-4.30)
58.49 (-3.04)
Appendix
Table 4: Ablation results on CAIL2018 and CMDL. Acc. and Ma-F1 are averaged over law-article, charge, and prison-term prediction within each dataset. Parentheses show percentage-point changes from LegalEvolver; higher is better for all metrics. The best results are shown in bold.
Automating the drafting of judgment documents is pivotal to judicial efficiency, yet it remains challenging due to the dual requirements of comprehensive retrieval of legal information and rigorous logical reasoning. Existing approaches, typically relying on standard Retrieval-Augmented Generation and Supervised Fine-Tuning, often suffer from insufficient evidence recall, hallucinated statutory references, and logically flawed legal reasoning. To bridge this gap, we propose Judge-R1, a unified framework designed to enhance LLM-based judgment document generation by jointly improving legal information collection and judgment document generation. First, we introduce Agentic Legal Information Collection, which employs a dynamic planning agent to retrieve precise statutes and precedents from multiple sources. Second, we implement Rubric-Guided Optimization, a reinforcement learning phase utilizing Group Relative Policy Optimization (GRPO) with a comprehensive legal reward function to enforce adherence to judicial standards and reasoning logic. Extensive experiments on the JuDGE benchmark demonstrate that Judge-R1 significantly outperforms state-of-the-art baselines in both legal accuracy and generation quality.
Weihang Su, Xuanyi Chen, Yueyue Wu +2
Tsinghua University · Beijing, China · Quan Cheng Laboratory +1
Legal question answering requires answers that are not only plausible, but also grounded in legally valid and citable evidence. Standard retrieval-augmented generation often relies on one-shot semantic matching, which is insufficient when legal evidence must be selected according to authority, applicability, and diversity across heterogeneous sources. We introduce Legal-R1, an agentic retrieval framework for evidence-based legal reasoning. Legal-R1 decomposes legal reasoning into a multi-agent workflow, where a Lawyer agent learns when to retrieve, which legal source to query, what evidence to seek, and when the collected evidence is sufficient. A Legal Assistant agent retrieves from normative legal sources and precedent cases with authority-aware and diversity-aware selection, while a Judge agent generates the final answer from the complete evidence trajectory. We post-train only the Lawyer agent with masked supervised fine-tuning and trajectory-aware reinforcement learning. Experiments on multiple Chinese legal benchmarks show that Legal-R1 improves over general-purpose, legal-specific, and retrieval-augmented baselines.
Huaiwen Pang, Jiale Han, Junying Chen +2
The Chinese University of Hong Kong, Shenzhen · Shenzhen Loop Area Institute · Shenzhen Shenten Digital Technology Co., Ltd.
This work explores the role of prompt design and judge selection in LLM-as-a-Judge evaluations of free text legal question answering. We examine whether automatic task prompt optimization improves over human-centered design, whether optimization effectiveness varies by judge feedback style, and whether optimized prompts transfer across judges. We systematically address these questions on the LEXam benchmark by optimizing task prompts using the ProTeGi method with feedback from two judges (Qwen3-32B, DeepSeek-V3) across four task models, and then testing cross-judge transfer. Automatic optimization consistently outperforms the baseline, with lenient judge feedback yielding higher and more consistent gains than strict judge feedback. Prompts optimized with lenient feedback transfer better to strict judges than the reverse direction. Analysis reveals that lenient judges provide permissive feedback, yielding prompts with broader applicability, whereas strict judges produce restrictive feedback, leading to judge-specific overfitting. Our findings demonstrate algorithmically optimizing prompts on training data can outperform human-centered prompt design and that judges' dispositions during optimization shape prompt generalizability.
Mohamed Hesham Elganayni, Runsheng Chen, Sebastian Nagl +1