Towards Better Legal Reasoning LLMs: Signal Balancing and Reward Scheduling
Authors: Yinghao Hu, Qian Xiao, Yuting Huang, Kaisong Song, Yiquan Wu, Ang Li, Junjie Cao, Jun Lin, Kun Kuang, Fei Wu
Organizations: Zhejiang University, Hangzhou, China · Alibaba Group, Hangzhou, China · Northeastern University, Shenyang, China · University of Macau, Macao, China
In the legal domain, LLMs can improve reliability and interpretability by generating explicit reasoning traces. However, training LLMs to produce high-quality reasoning traces remains challenging. Existing supervised fine-tuning methods struggle when reasoning traces are significantly longer than final answers, as the learning signal for the answer becomes diluted. Meanwhile, reinforcement learning methods such as Group Relative Policy Optimization also have several limitations, including costly reward design, performance plateaus, and reward hacking. We therefore propose a two-stage training framework. In Stage I, JurisCoT-SFT employs a length-normalized dual objective to balance learning signals between reasoning traces and final answers. In Stage II, Lifecycle-Aware Backtrackable Policy Optimization dynamically evaluates and retires auxiliary reward signals based on their impact on final evaluation objectives, enabling efficient reward utilization. Our framework substantially improves legal reasoning performance on relevant tasks selected from multiple legal benchmarks, including ChJGBench, LawBench, and LegalBench. Experimental results demonstrate consistent improvements in both reasoning trace quality and final answer accuracy, while our models remain competitive with larger models.
Figures & tables
Figure 1: An anonymized and abridged example of a legal reasoning trace.
Figure 2: The framework has two stages. In Stage I, λ balances learning between reasoning traces and final answers. In Stage II, auxiliary reward signals are activated, evaluated, and retired, with checkpoint rollback when needed.
Signal
Description
S1
Tag Format Validation
S2
JSON Schema Validation
S3
Statute Citation Format Verification
S4
Language Style Validation
S5
Statute Authenticity Verification
S6
Statute-Conclusion Consistency Verification
Table 1: Auxiliary reward signals used during training. Detailed definitions are provided in Appendix D .
Models
Avg.
Law
ISS
JUS
Criminal Case
Civil Case
Admin.
Fine
MIT
Charge
Pen_M
REC
SUP
PAY
DIS
Specialized Legal LLMs
WisdomInterrogatory
0.3728
0.158
0.582
0.384
0.135
0.408
0.436
0.608
0.108
0.649
0.244
0.326
DISC-LawLLM-7B
0.4482
0.181
0.729
0.546
0.080
0.727
0.414
0.466
0.370
0.670
0.056
0.420
Open-source LLMs
Llama-3.1-405B
0.6006
0.342
0.712
0.575
0.476
0.885
0.957
0.602
0.500
0.682
0.616
0.712
Table 2: Evaluation results of different models on ChJGBench. The best results among all models are shown in bold , and the second-best results are underlined .
Models
AP ↑
CP ↑
SP ↑
Specialized Legal LLMs
WisdomInterrogatory
0.328
0.351
0.804
DISC-LawLLM-7B
0.054
0.127
0.670
Open-source LLMs
Llama-3.1-405B
0.306
0.352
0.807
DeepSeek-R1
0.831
0.606
0.891
Table 3: Evaluation results of different models on LawBench. AP , CP , and SP denote Article Prediction, Charge Prediction, and Sentence Prediction, respectively. The best results among all models are shown in bold , and the second-best results are underlined .
Models
LRC ↑
NYSJE ↑
Sara-N ↓
Specialized Legal LLMs
WisdomInterrogatory
0.491
0.387
–
DISC-LawLLM-7B
0.746
0.630
19.330
Open-source LLMs
Llama-3.1-405B
0.909
0.709
7.720
DeepSeek-R1
0.927
0.771
0.250
Table 4: Evaluation results of different models on LegalBench. The best results among all models are shown in bold , and the second-best results are underlined .
Figure 3: Training dynamics of auxiliary-signal scheduling in LABPO. Detailed trajectories for S1 – S4 are omitted for clarity.
Figure 4: Pairwise reasoning-trace win rates against DeepSeek-R1 on ChJGBench. Ours: our two-stage model; Baseline: Qwen3-8B. Purple, blue, and green denote a DeepSeek-R1 loss, tie, and win, respectively.
No.
SFT
RL
Avg. ↑
Used
Variant
Used
Retire
Rollback
1
✓
✓
✓
✓
✓
0.7786
2
✓
✓
✓
✓
×
0.7718
3
✓
✓
∙
×
×
0.7671
4
✓
✓
∘
×
×
0.7684
5
✓
✓
×
×
×
0.7667
Table 5: Ablation study of different training configurations. ✓indicates the component is enabled and × indicates it is disabled. Row 3 reduces to GRPO with aggregated primary and auxiliary rewards, while Row 4 reduces to GRPO using only the primary reward.
Appendix figures & tables11 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 5: An overview of the Attempt-Bounded Self-Verification framework, in which a query and its reasoning trace are fed into the target model, and the pair is retained only if the model can infer the correct answer within a bounded number of attempts.
Signal Order
Avg. ↑
S1→S2→S3→S4→S5→S6
0.6735
S6→S5→S4→S3→S2→S1
0.6682
S4→S3→S6→S1→S5→S2
0.6671
S5→S1→S2→S4→S3→S6
0.6723
S3→S4→S1→S6→S5→S2
0.6704
Appendix
Table 6: Ablation study on the effect of auxiliary signal ordering using Qwen3-8B. Reported values are average performance scores computed using Eq. ( 22 ).
Model
Law
S5 Reward Score
Avg. Statute Citations per Response
Avg. Mentions per Distinct Correct and Relevant Statute
JurisCoT-SFT
0.667
0.542
3.18
1.21
JurisCoT-SFT + GRPO
0.669
0.934
12.46
4.87
JurisCoT-SFT + LABPO
0.681
0.761
4.27
1.48
Appendix
Table 7: Results of the controlled diagnostic experiment on reward hacking using a fixed subset of ChJGBench.
Objective
Setting
Avg. ↑
Standard SFT
–
0.7399
Answer-Token Upsampling
k=4
0.7428
JurisCoT-SFT
λ=0.25
0.7629
JurisCoT-SFT
λ=0.50
0.7667
JurisCoT-SFT
λ=0.75
0.7612
Appendix
Table 8: Ablation study of SFT objectives without LABPO. All variants are trained with Qwen3-8B under the same experimental configuration. Avg. denotes the average score on ChJGBench.
α
Majority
LLM Acc.
LLM κ
0.615
96.0%
0.84
0.756
Appendix
Table 9: Human inter-annotator agreement and human–LLM alignment results on 500 queries.
No.
SFT
RL
Avg. ↑
Used
Variant
Used
Retire
Rollback
1
✓
✓
✓
✓
✓
0.7893
2
✓
✓
×
×
×
0.7593
3
✓
×
✓
✓
✓
0.7651
4
✓
×
×
×
×
0.7484
5
×
×
×
×
×
0.7220
Appendix
Table 10: Ablation study on LawBench. A ✓indicates that the component is enabled, while × indicates that it is disabled. Average denotes the mean of all metrics in the range [0, 1].
No.
SFT
RL
Avg. ↑
Sara-N ↓
Used
Var
Used
Ret
Roll
1
✓
✓
✓
✓
✓
0.826
1.753
2
✓
✓
×
×
×
0.777
3.949
3
✓
×
✓
✓
✓
0.787
5.906
4
✓
×
×
×
×
0.759
7.231
5
×
×
×
×
×
0.735
8.093
Appendix
Table 11: Ablation study on LegalBench. A ✓indicates that the component is enabled, and × indicates that it is disabled. Abbreviations: Var = Variant, Ret = Retire, Roll = Rollback. Average denotes the mean of all accuracy metrics. Sara-N corresponds to the Log MSE metric and is therefore reported separately.
Item
Count
Training instances (x,t,a)
2,734,100
Distinct statutes (clause level)
35,010
Causes of action
>400
Criminal cases
581,579
Civil cases
1,625,584
Administrative cases
526,937
Appendix
Table 12: Statistics of the statutory-law training dataset, including the total number of training instances, the number of distinct statutes at the clause level, the distribution of legal cases, and the average token counts per reasoning trace and per answer.
Item
Count
Training instances (x,t,a)
35,965
Average token count per reasoning trace
2,369.2
Average token count per answer
3.5
Appendix
Table 13: Statistics of the case-law training dataset, including the total number of training instances and the average token counts in reasoning traces and answers.
Task
Task Full Name
Litigation Type
Benchmark
Metric
Statutory Law Tasks
Law
Statutory Provision Application
General
ChJGBench
Case-level set F1
ISS
Disputed Issues Identification
General
ChJGBench
LLM-as-a-judge
JUS
Judicial Justification
General
ChJGBench
LLM-as-a-judge
Charge
Criminal Charge Prediction
Criminal
ChJGBench
Defendant-level set F1
Fine
Monetary Fine
Criminal
ChJGBench
NVS
Appendix
Table 14: Summary of evaluation tasks, including statutory law tasks and case law tasks, with their corresponding litigation types, benchmark sources, and evaluation metrics. In ChJGBench Hu et al. (2026) , NVS stands for Normalized Value Similarity.
Figure 6: Case study: legal reasoning accuracy and completeness in a judgment concerning illegal business operations.
Small language models (SLMs) are promising for real-world deployment due to their efficiency and low operational cost. However, their limited capacity struggles with high-stakes legal reasoning tasks that require coherent statute interpretation and logically consistent deduction. Furthermore, training SLMs for such tasks demands high-quality, concise reasoning trajectories, which are prohibitively expensive to manually collect and difficult to curate via standard rejection sampling, lacking granularity beyond final verdicts. To address these challenges, we propose {LegalDrill}, a diagnosis-driven synthesis framework that extracts and iteratively refines reasoning trajectories from a capable teacher via fine-grained prompting, then a self-reflective verification is employed to adaptively select the most effective data for the SLM student. The resulting data empower SLM training through supervised fine-tuning and direct preference optimization. Extensive experiments on several legal benchmarks demonstrate that {LegalDrill} significantly bolsters the legal reasoning capabilities of representative SLMs while bypassing the need for scarce expert annotations, paving a scalable path toward practical legal reasoning systems.
Tianchun Li, Haochen Liu, Vishwa Pardeshi +5
Purdue University, West Lafayette, IN, USA · Fidelity Investments, Boston, MA, USA
Large language models (LLMs) have demonstrated strong reasoning abilities across specialized domains, motivating research into their application to legal reasoning. However, existing legal benchmarks often conflate factual recall with genuine inference, fragment the reasoning process, and overlook the quality of reasoning. To address these limitations, we introduce MSLR, the first Chinese multi-step legal reasoning dataset grounded in real-world judicial decision making. MSLR adopts the IRAC framework (Issue, Rule, Application, Conclusion) to model structured expert reasoning from official legal documents. In addition, we design a scalable Human-LLM collaborative annotation pipeline that efficiently produces fine-grained step-level reasoning annotations and provides a reusable methodological framework for multi-step reasoning datasets. Evaluation of multiple LLMs on MSLR shows only moderate performance, highlighting the challenges of adapting to complex legal reasoning. Further experiments demonstrate that Self-Initiated Chain-of-Thought prompts generated by models autonomously improve reasoning coherence and quality, outperforming human-designed prompts. MSLR contributes to advancing LLM reasoning and Chain-of-Thought strategies and offers open resources for future research. The dataset and code are available at https://github.com/yuwenhan07/MSLR-Bench and https://law.sjtu.edu.cn/flszyjzx/index.html.
Wenhan Yu, Xinbo Lin, Lanxin Ni +2
School of Artificial Intelligence, Beihang University, Beijing, China · KoGuan School of Law, Shanghai Jiao Tong University, Shanghai, China · School of Criminal Law, China University of Political Science and Law, Beijing, China
Large language models are increasingly applied to high-risk domains such as law, yet complex legal reasoning remains limited by two structural challenges. First, existing RAG and GraphRAG methods emphasize lexical or semantic similarity while overlooking normative relations among legal provisions. Second, vanilla Chain-of-Thought prompting may generate plausible rationales without enforcing the normative structure of legal reasoning. To deal with the bottleneck of pipelines in the legal reasoning domain, we propose LEGO, a dual-module framework that synergizes Legal Expert GraphRAG and expert Chain-of-thought for complex legal reasoning. ExpertGraphRAG uses an expert-annotated civil code graph encoding these normative relations with a greedy normative-coverage retrieval algorithm to dynamically extract instance-specific provision subgraphs, while ExpertCoT organizes the retrieved provisions and case facts into structured Provision-Fact-Conclusion reasoning. With a Qwen3-8B backbone, LEGO achieves 40.53% exact-match accuracy on LawExamQA_Civil, outperforming the evaluated RAG and CoT baselines and performing comparably to the evaluated larger models, while remaining robust on multi-hop questions. It also achieves the best results among the evaluated baselines on the open-ended benchmarks. Ablation studies confirm the individual and complementary contributions of both modules, demonstrating LEGO's effectiveness in improving LLMs' complex legal reasoning ability. Code and dataset can be found in the link: https://github.com/BLK-WHT/LEGO
Qingjing Chen, Junkai Zhang, Shaochun Wang +7
Alma AI, University of Bologna, Italy · Tsinghua University, China · Modelbest Inc. +2