cs.CLNov 11, 2026

ChJGBench: A Comprehensive Benchmark for Chinese Legal Judgment Generation with Diagnostic Evaluation

Authors: Yinghao Hu, Linggang Kong, Qian Xiao, Yuting Huang, Yunlong Zhang, Yujie Zhang, Yiquan Wu, Fei Duan, 

Organizations: Zhejiang University, Hangzhou, China · National University of Defense Technology, Changsha, China · Alibaba Group, Hangzhou, China · Northeastern University, Shenyang, China

Abstract

With the rapid development of LLMs, increasing attention has been paid to whether they can reliably generate complete and legally valid judicial documents. This calls for a benchmark capable of evaluating whether models can accurately apply statutes and resolve outcome-critical issues, including claims, charges, penalties, and monetary obligations. However, existing benchmarks typically focus on isolated legal tasks or coarse-grained evaluation. We introduce ChJGBench, a benchmark for generating the reasoning and dispositive sections of Chinese judgments across civil, criminal, and administrative cases. Built from 3,462 real-world judgments, ChJGBench provides a case-type-aware diagnostic evaluation framework comprising 13 metric types and adopts a hybrid paradigm. Experiments reveal no uniform advantage for legal-domain models over general-purpose models, with model strengths varying across diagnostic dimensions. Performance is generally stronger on criminal cases, while civil and administrative cases continue to present substantial challenges.

Figures & tables

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. TW-LegalBench: Measuring Taiwanese Legal Understanding

    Jun 17, 2026Fei-Yueh Chen, Chun Huang Lin, Chan Wei Hsu +4LLM EvaluationLegal Judgment Prediction

  2. LexRubric: A Rubric-Guided Diagnostic Benchmark for Open-Ended Legal Tasks

    Jun 8, 2026Yifan Chen, Haitao Li, Yiran Hu +6LLM EvaluationRubric-Based Evaluation

  3. Magis-Bench: Evaluating LLMs on Magistrate-Level Legal Tasks

    May 8, 2026Ramon Pires, Thales Sales Almeida, Celio Larcher Junior +6Legal NLPLLM Evaluation