In case-based legal reasoning, legal professionals identify legally significant fact patterns in a new dispute and compare them with those of decided cases. In AI & Law, HYPO dimensions and CATO factors represent these patterns symbolically, organizing expert knowledge distilled from case narratives as explicit, structured, interpretable, and auditable legal concepts and relations that support case comparison, legal argument, and precedent-based reasoning. Large language models (LLMs) are increasingly used for legal tasks, yet their internal organization of legal knowledge remains difficult to trace. We investigate whether LLMs form internal representations corresponding to all 26 CATO factors. Two research questions examine linear decodability and sparse localization with sparse autoencoders. Experiments cover three open models, Qwen3-1.7B, Qwen3-8B, and Gemma-2 2B, supported by cross-model comparison and human audit of controlled natural-language passages. Results show that all 26 factors are linearly decodable in all three models. Single-feature sparse readouts cover the complete factor catalog, with best-layer test AUCs ranging from 0.86 to 1.00. Five selected features approach full-vector readout performance. We report 78 model-specific layer-feature coordinates, one for each factor in each model. Together, these results show that information corresponding to expert-defined legal factors can be read from the internal activations of LLMs processing legal narratives. By locating these readouts, our study connects legal language and the symbolic abstractions of HYPO and CATO to learned internal representations.
Figures & tables
Figure 1: Overview of legal-factor analysis. (a) HYPO and CATO employ legally significant fact patterns abstracted from natural-language narratives. Subsequent work prompts LLMs to identify these factors in text. (b) This work examines internal representations in pretrained LLMs: RQ1 reads factor presence from full activations, and RQ2 identifies one or five SAE features supporting the readout.
ID
Factor (side)
Meaning (short)
F1
Disclosure-in-negotiations (D)
Plaintiff disclosed its information to defendant in negotiations.
F2
Bribe-employee (P)
Defendant paid plaintiff’s employee to defect with its information.
F3
Employee-sole-developer (D)
Defendant’s employee solely developed plaintiff’s product.
F4
Agreed-not-to-disclose (P)
Defendant signed a nondisclosure agreement with plaintiff.
F5
Agreement-not-specific (D)
The agreement did not specify what was confidential.
F6
Security-measures (P)
Plaintiff adopted concrete security measures.
Table 1: The CATO factor catalog (F1–F27; F9 is vacant). (P)/(D) marks the party the factor favors.
Example
Corpus excerpt
Example A Supplier network
BlueFern Genetics developed a supplier network for its biotech reagent line. The company kept the specifications on a shared drive with no access controls.
Example B Information type
Tessara Analytics developed a formula for industrial chemicals used in polymer curing. The company kept the specifications on a shared drive with no access controls.
Example C Industry
Almara Labs maintained a supplier network for its aerospace components on a shared drive with no access controls. The company did not restrict who inside the organization could open the files.
Example D Narrative style
Complaint excerpt: Vantor Gaming is said to mirror Veltrix Systems’ textiles pricing steps. The plaintiff took no steps to limit which employees could view the process.
Table 2: Corpus examples illustrating variation in industry, information type, and narrative style. All four passages express F19; labels highlight changes relative to Example A.
Figure 2: Mean AUC across 26 factors at each model’s middle sampled layer (14/18/13). Random-5 averages 20 draws; features are selected on readout-fitting data. The dashed line marks chance-level ranking.
Figure 3: Selected layer–feature locations for all 26 factors. In each model panel, the filled square identifies the layer with the highest test single-feature AUC; the adjacent number is the feature ID within that layer’s SAE. Dots mark AUC on horizontal tracks running from 0.85 (left) to 1.00 (right), with exact values printed alongside.
Large language model (LLM) interpretability studies how internal representations and computations produce outputs, and how they can be explained or controlled.
Large language models show promise for legal applications, but deploying frontier models raises concerns about cost, latency, and data privacy. We evaluate whether sub-10B parameter models can serve as practical alternatives by testing nine models across three legal benchmarks (ContractNLI, CaseHOLD, and ECtHR) using five prompting strategies (direct, chain-of-thought, few-shot, BM25 RAG, and dense RAG). Across 405 experiments with three random seeds per configuration, we find that a Mixture-of-Experts model activating only 3B parameters matches GPT-4o-mini in mean accuracy while surpassing it on legal holding identification, and that architecture and training quality matter more than raw parameter count. Our largest model (9B parameters) performs worst overall. Chain-of-thought prompting proves sharply task-dependent, improving contract entailment but degrading multiple-choice legal reasoning, while few-shot prompting emerges as the most consistently effective strategy. Comparing BM25 and dense retrieval for RAG, we find near-identical results, suggesting the bottleneck lies in the language model's utilization of retrieved context rather than retrieval quality. All experiments were conducted via cloud inference APIs at a total cost of $62, demonstrating that rigorous LLM evaluation is accessible without dedicated GPU infrastructure.
Large Language Models (LLMs) achieve strong performance on reasoning tasks, but whether this reflects faithful logical inference or heuristic approximation remains unclear. We study this question in legal entailment by comparing three paradigms, including pure LLM classification, LLM-based Formal Reasoning, and solver-based Formal Reasoning using the Z3 SMT solver, on a re-annotated subset of ContractNLI across five LLMs. Our re-annotation reveals a systematic and measurable gap between pragmatic legal interpretation and strict formal entailment, where a substantial proportion of legally sound inferences are not formally grounded without additional unstated assumptions. While introducing formal structure improves accuracy, with LLM-based Formal Reasoning achieving the highest benchmark performance, we show that this gain does not imply faithful reasoning. We identify three recurring failure modes: scope laundering, where LLMs report solver-inconsistent classifications without executing the underlying formal reasoning, producing conclusions that appear logically grounded but are not; implicit constraint blindness, where LLMs overlook logical constraints present in formal representations; and program synthesis failures, where LLMs generate incorrect Z3 code despite structured prompting. Critically, scope laundering persists across all models, raising serious concerns about the faithfulness of LLM-based formal reasoning as a proxy for symbolic execution. These results reveal a fundamental gap between benchmark accuracy and logical faithfulness.
Large language models are increasingly applied to high-risk domains such as law, yet complex legal reasoning remains limited by two structural challenges. First, existing RAG and GraphRAG methods emphasize lexical or semantic similarity while overlooking normative relations among legal provisions. Second, vanilla Chain-of-Thought prompting may generate plausible rationales without enforcing the normative structure of legal reasoning. To deal with the bottleneck of pipelines in the legal reasoning domain, we propose LEGO, a dual-module framework that synergizes Legal Expert GraphRAG and expert Chain-of-thought for complex legal reasoning. ExpertGraphRAG uses an expert-annotated civil code graph encoding these normative relations with a greedy normative-coverage retrieval algorithm to dynamically extract instance-specific provision subgraphs, while ExpertCoT organizes the retrieved provisions and case facts into structured Provision-Fact-Conclusion reasoning. With a Qwen3-8B backbone, LEGO achieves 40.53% exact-match accuracy on LawExamQA_Civil, outperforming the evaluated RAG and CoT baselines and performing comparably to the evaluated larger models, while remaining robust on multi-hop questions. It also achieves the best results among the evaluated baselines on the open-ended benchmarks. Ablation studies confirm the individual and complementary contributions of both modules, demonstrating LEGO's effectiveness in improving LLMs' complex legal reasoning ability. Code and dataset can be found in the link: https://github.com/BLK-WHT/LEGO
Qingjing Chen, Junkai Zhang, Shaochun Wang +7
Alma AI, University of Bologna, Italy · Tsinghua University, China · Modelbest Inc. +2