cs.AISep 30, 2026

JusticeAxis: Benchmarking Legal Judgment between Rigid Rule Application and Ungrounded Discretion

Authors: Zhengkai Tu, Mingda Zhang, Zijia Wang, Xiaoying Tang, Jimmy Huang

Organizations: The Chinese University of Hong Kong · The Chinese University of Hong Kong, Shenzhen · University of Oxford · York University

Abstract

A sound judgment applies the law to established facts and weighs the circumstances in which they arose. However, existing methods swing between rigid statute matching and ungrounded discretion, benchmarks score a label or a rubric, and the experience that would supply the balance stays unverified. We formalize legal judgment as a reference-anchored task, whose object is a single decision that stays tied to the statute and to the circumstances at once. We introduce JusticeAxis, 256 real-world criminal cases from 18 countries with audio, image, and text evidence, and three lawyer-written judgments for every case: the recorded one and one for each failure. We further propose JusticeAgent, a harness whose element agents establish the facts and whose judge agent applies the law under skills carrying experience of the circumstances. Skills are distilled from execution trajectories and admitted only under Bayesian credible bounds. Experiments show that failure turns direction with scale: open-weight backbones drift to unsupported grounds, frontier models to the statutory default. We further verify that JusticeAgent, as a simple yet effective plugin, carries a frozen open-weight backbone to commercial level. Project resources are available at https://github.com/beita6969/JusticeAxis.

Figures & tables

Explore similar work

CardsList
  1. JudgmentBench: Comparing Rubric and Preference Evaluation for Quality Assessment

    May 24, 2026Russell Yang, Ruishi Chen, Pierce Kelaita +6Human Preference EvaluationLLM Evaluation

  2. Towards Explainable Adjudicative Variance: Quantifying Judicial Discretion via Gated Multi-Task Learning

    Jun 25, 2026Stanisław Sójka, Felix Steffek, Matthias GrabmairLegal NLPAI-Assisted Decision Making

  3. LegalRewardBench: Benchmarking Reward Models for Legal Preference Evaluation

    Oct 13, 2026Hui Huang, Jie Zhou, Hongli Zhou +7Reward Model EvaluationPairwise Preference Evaluation