Legal Natural Language Processing Benchmarks

Ask about this topic

Recent momentum

+33%

8 papers in the last 28 days · 0.1% of indexed attention

Twelve weeks of publication activity for this topic as it is defined today.

Also in Natural Language Processing

Neighbouring directions

Natural Language ProcessingNatural Language Processing TasksNatural Language Datasets

Weekly history

Recent digests

What was published in this topic, kept on the site without email delivery.

Period ending 2026-09-21

2 new papers

A weekly snapshot of new work published in Legal Natural Language Processing Benchmarks.

Period ending 2026-09-07

5 new papers

A weekly snapshot of new work published in Legal Natural Language Processing Benchmarks.

73 papers

Latest in Legal Natural Language Processing Benchmarks

Browse all topics →
CardsList

  1. A Semiotics-Aware Framework for Evaluating Fidelity and Coverage in Natural Language Generation

    Sep 22, 2026Lorenzo Zangari, Davide PiccaGeo-Contextual AlignmentLegal Natural Language Processing Benchmarks

  2. Financial Language Models as Applied Artificial Intelligence Systems for News-Based Trading under Market Frictions

    Sep 20, 2026Kemal KirtacFinancial Sentiment AnalysisLegal Natural Language Processing Benchmarks

  3. How Much is a Human Right Worth? ECtHR-NPD: A Benchmark for Predicting Non-Pecuniary Damage Awards

    Sep 16, 2026Yanyi Pu, Damian A. Gonzalez-Salzberg, Zheng Yuan +1Legal DomainLegal Natural Language Processing Benchmarks

  4. Right Tool, Right Job: Native-Language Evaluation, Tokenizer Sensitivity, and Methodological Findings from a French-Only BabyLM

    Sep 15, 2026Adam Zachary Wasserman, David BeaucheminLegal Natural Language Processing BenchmarksTranslation Quality

  5. E-CONAN (Entailment, CONtradition And Neutral) Benchmarks: Arabic Textual Entailment and Natural Inference Datasets

    Sep 12, 2026Khloud AL Jallad, Nada Ghneim, Ghaida RebdawiArabic Natural Language ProcessingLegal Natural Language Processing Benchmarks

  6. IGT @ FinMMEval 2026 Task 2: Question-Type Prompting with Targeted Extraction for Multilingual Financial QA

    Sep 8, 2026Yuwen ChiuMultilingual Financial Question AnsweringFinance Domain

  7. KC-Bench: A Dynamic Interactive Benchmark for Evaluating Knowledge Conflicts in LLM Agents

    Sep 3, 2026Yaxing Lyu, Shengjie Zhou, Binbin Toh +2Knowledge ConflictsLarge Language Model Agents

  8. FrameBench:A Language Understanding Benchmark Based on Frame Semantics

    Sep 3, 2026Chihiro Yano, Ryohei SasanoLegal Natural Language Processing BenchmarksHuman-Annotated Benchmark

  9. Do General NLP Embeddings Capture Ontological Reasoning?

    Aug 31, 2026Hamed Babaei Giglou, Jennifer D'Souza, Sören AuerOntologyLegal Natural Language Processing Benchmarks

  10. Where You Measure Decides What You Measure: Position Selection in Ablation-Based SAE Evaluation

    Aug 13, 2026Valentin NoëlAblationLegal Natural Language Processing Benchmarks

  11. When the Knowledge Base Becomes the Gold Standard: Measuring Resource-Shared Evaluation Loops in Entity-Level Machine Translation

    Aug 12, 2026Jinhyung Bae, Dain Kil, Seongmin Oh +1Legal Natural Language Processing BenchmarksHuman Annotations

  12. Geo-Spatial Concept Probing of Large Language Models: Abstraction, Compositionality, and Grounding

    Aug 7, 2026Karim Radouane, Jose G Moreno, Lynda TamineCompositionalityLegal Natural Language Processing Benchmarks

  13. From Financial Sentiment Classification to Return Predictability: A QLoRA Benchmark of Large Language Models

    Aug 4, 2026Fusheng LuoFinancial Sentiment AnalysisLegal Natural Language Processing Benchmarks

  14. Can Language Models Identify Shadow Trading Targets? An NLP Evaluation of SEC Enforcement Theory

    Aug 2, 2026Sarah Wilson, Michael MacKay, Anthony Marello +1PeersLegal Natural Language Processing Benchmarks

  15. SCHEDBench: A Benchmark for Evaluating LLM Constraint Faithfulness in Natural-Language Combinatorial Scheduling

    Aug 2, 2026Shrenil Shaun Sharma, Avi SharmaOptimal SchedulingScheduling

  16. Earnings25: A Comprehensive 500-Hour Speech Benchmark for Finance

    Jul 26, 2026Denglin Jiang, Haoran Zhou, Anshul Wadhawan +7Finance BenchmarksWord Error Rate

  17. Choosing a Text Embedding Model: A Practical Benchmarking and Decision Framework

    Jul 26, 2026Madhav S BaidyaLegal Natural Language Processing Benchmarks

  18. Language-Routed RAG and Direct Option Scoring for Multilingual Financial QA: DS@GT at FinMMEval

    Jul 24, 2026Justice Ayela, Kabir SahniMultilingual Financial Question AnsweringFinance Domain

  19. A Comparative Evaluation of Embeddings and LLMs in a Greek Book Publisher Setting - The CUP Dataset

    Jul 23, 2026Katerina Papantoniou, Panagiotis Papadakos, Theodore Patkos +3RetrieversLegal Natural Language Processing Benchmarks

  20. RW-Voice-EQ Bench: A Real World Benchmark for Evaluating Voice AI Systems

    Jul 16, 2026David Ayllon, Alice Baird, Jeffrey Brooks +11Voice AgentsLegal Natural Language Processing Benchmarks

  21. DS@GT ARC at LongEval: Citation Integrity and Factual Grounding in Scientific QA

    Jul 15, 2026Brandon Michaels, Brendon JohnsonCitationsHuman Grounded Evaluation

  22. The Sound of Absence: Audio-Language Embedding Models Struggle with Negation

    Jul 14, 2026Chun-Yi Kuan, Hung-yi LeeNegationLegal Natural Language Processing Benchmarks

  23. SPEARBench: A Benchmark for Naturalness Evaluation in Streaming Speech-to-Speech Language Models

    Jul 6, 2026Thomas Thebaud, Yuzhe Wang, Hao Zhang +5Conversational BenchmarksLegal Natural Language Processing Benchmarks

  24. RusFinChain: A Russian Benchmark for Verifiable Chain-of-Thought Reasoning in Finance with Fuzzy-Aligned Evaluation

    Jul 1, 2026M. K. ArabovReasoning ChainLegal Natural Language Processing Benchmarks

  25. ALEE: Any-Language Evaluation of Embeddings via English-Centric Minimal Pairs

    Jun 30, 2026Andrianos Michail, Stylianos Psychias, Michelle Wastl +3Legal Natural Language Processing BenchmarksDomain-Invariant Text Embeddings

  26. Fast Numbers, Slow Language: Bridging Quantitative and Qualitative Earnings Signals

    Jun 29, 2026Ding Yu, Zhuo Liu, Hao Zhang +1Financial Sentiment AnalysisQuantifying

  27. Anisotropy Decides Cosine vs. Rank Metrics for Text Embeddings

    Jun 28, 2026V. S. Raghu ParupudiCosine SimilarityReference-Free Metric

  28. Preference-ASR: A Preference-Aware Test Set for Benchmarking ASR in the Era of Speech LLMs

    Jun 28, 2026Nithin Rao Koluguri, Sasha Meister, Nikolay Karpov +4Speech Large Language ModelsLegal Natural Language Processing Benchmarks

  29. Customized Generative AI Agent for Transportation Engineering Practice: A Development and Continued Pre-training Guideline

    Jun 27, 2026Dianwei Chen, Yuan-Zheng Lei, Zifan Zhang +2Generative Artificial IntelligenceLow-Rank Adaptation

  30. Causal Connections: Leveraging Multilingual Fine-Tuning for Financial QA@FinCausal 2026

    Jun 25, 2026Akash Kumar Gautam, Serhii Hamotskyi, Christian HänigMultilingual Financial Question AnsweringFinance Domain

  31. QuechuaTok: Morphological Boundary Accuracy as a Necessary Metric for Tokenizer Evaluation in Agglutinative Low-Resource Languages

    Jun 22, 2026Maria ContrerasLegal Natural Language Processing BenchmarksLow-Resource Languages

  32. KaLM-Reranker-V1: Fast but Not Late Interaction for Compressed Document Reranking

    Jun 22, 2026Xinping Zhao, Jiaxin Xu, Ziqi Dai +9RerankingRelevance

  33. FinBalance: A Multi-Document Accounting Reconciliation Benchmark

    Jun 14, 2026Sasank Tumpati, Devansh Agarwal, Ayush Kedia +6Finance DomainLegal Natural Language Processing Benchmarks

  34. AmchiBias: Measuring Stereotypical Bias in Goan Identity Groups with a Minimal Pair Dataset in English and Konkani

    Jun 13, 2026Michelle Barbosa, Sebastian Padó, Franziska WeeberStereotype MitigationLegal Natural Language Processing Benchmarks

  35. Causally Evaluating the Learnability of Formal Language Tasks

    Jun 8, 2026Vésteinn Snæbjarnarson, Anej Svete, Josef Valvoda +3Legal Natural Language Processing BenchmarksLearn

  36. From Statute to Control Flow: Span-Grounded Deontic Trees for Defeasible Scope Parsing

    Jun 8, 2026Jian Chen, Siyuan Li, Chucheng Wan +1Predictive LawLegal Natural Language Processing Benchmarks

  37. Meaning in Order, Order in Meaning: Semantic R-precision for Keyphrase Evaluation

    Jun 5, 2026Shamira Venturini, Steffen KinkelInformation Retrieval MetricsEvaluation Metrics

  38. SANE Schema-aware Natural-language Evaluation of Biological Data

    Jun 3, 2026Rolf Gattung, Martin Krueger, Markus ReischlText-To-SqlLegal Natural Language Processing Benchmarks

  39. SEA-NLI: Natural Language Inference as a Lens into Southeast Asian Cultural Understanding

    Jun 2, 2026Peerawat Chomphooyod, Jian Gang Ngui, Yosephine Susanto +5PhilippinesLegal Natural Language Processing Benchmarks

  40. Sample-Size Scaling of the African Languages NLI Evaluation

    Jun 2, 2026Anuj Tiwari, Oluwapelumi Ogunremu, Terry Oko-odion +2Legal Natural Language Processing BenchmarksMultilingual Language Models

  41. Multi-Legal-Bench: Evaluating LLMs on Legal Reasoning Across Jurisdictions, Languages, and Legal Traditions

    May 28, 2026Volodymyr OvcharovLegal DomainLegal Natural Language Processing Benchmarks

  42. UA-Legal-Bench: A Benchmark for Evaluating Large Language Models on Ukrainian Legal Reasoning

    May 27, 2026Volodymyr OvcharovLegal Natural Language Processing BenchmarksUkrainian

  43. Same Question, Different Source, Different Answer: Auditing Source-Dependence in Medical Multi-Source RAG

    May 27, 2026Yubo Li, Rema Padman, Ramayya KrishnanMulti-Source Retrieval-Augmented GenerationMulti-Hop Question Answering Benchmarks

  44. Bilinear Coordinate Alignment for Training-Free Task-Vector Transfer

    May 27, 2026Jungyong Son, Jinwook Jung, Minhee Park +1Transfer LearningModel Pretraining

  45. MATCHA: Matching Text via Contrastive Semantic Alignment

    May 26, 2026Siran Li, Ece Sena Etoglu, Carsten Eickhoff +1Geo-Contextual AlignmentLegal Natural Language Processing Benchmarks

  46. From Norms to Indicators (N2I-RAG): An Agentic Retrieval-Augmented Generation Framework for Legal Indicator Computation

    May 26, 2026Youssef Al Mouatamid, Marie Bonnin, Jihad ZahirLegal DomainLegal Natural Language Processing Benchmarks

  47. Granuscore: A Reference-Free Measure of Granularity for Text Analysis and Question Answering

    May 26, 2026Lukas Ellinger, Alexander Fichtl, Miriam Anschütz +1Legal Natural Language Processing Benchmarks

  48. StakeBench: Evaluating Language Understanding Grounded in Market Commitment

    May 25, 2026Yunhua Pei, Jingyu Hu, Yiwei Shi +3MarketLegal Natural Language Processing Benchmarks

  49. The Tokenizer Tax Across 25 European Languages: Domain Invariance, Cross-Lingual Few-Shot Effects, and the Ukrainian Penalty

    May 23, 2026Volodymyr OvcharovSubword TokenizationTokenization

  50. Temporal Concept Drift in Legal Judgment Prediction: Neural Baselines Across Three Epochs of Ukrainian Court Decisions

    May 23, 2026Volodymyr OvcharovPredictive LawLegal Natural Language Processing Benchmarks

  51. IdioLink: Retrieving Meaning Beyond Words Across Idiomatic and Literal Expressions

    May 21, 2026Kai Golan Hashiloni, Daniel Fadlon, Lior Livyatan +3Multilingual IdiomsFigurative Language

  52. Fine-grained Claim-level RAG Benchmark for Law

    May 20, 2026Souvick Das, Sallam Abualhaija, Domenico BianculliLegal DomainHievi-Rag

  53. The Annotation Scarcity Paradox in Low-Resource NLP Evaluation: A Decade of Acceleration and Emerging Constraints

    May 18, 2026Vukosi MarivateLegal Natural Language Processing BenchmarksData Scarcity

  54. Beyond Catalogue Counts: the Dataset Visibility Asymmetry in Low-Resource Multilingual NLP

    May 17, 2026Zhiyin Tan, Changxu DuanLow-Resource LanguagesMultilingual Data

  55. Tokenizer Fertility and Zero-Shot Performance of Foundation Models on Ukrainian Legal Text: A Comparative Study

    May 14, 2026Volodymyr OvcharovZero-Shot BenchmarkLegal Natural Language Processing Benchmarks

  56. Knowledge Beyond Language: Bridging the Gap in Multilingual Machine Unlearning Evaluation

    May 14, 2026Kyomin Hwang, Hyeonjin Kim, Sangyeon Cho +1Large Language Model UnlearningLegal Natural Language Processing Benchmarks

  57. CodeEvolve: LLM-Driven Evolutionary Optimization with Runtime-Enriched Target Selection for Multi-Language Code Enhancement

    May 6, 2026Ajay Krishna Borra, Wenzhuo Yang, Samarth Arora +9Code OptimizationCodebases

  58. From prompting to evidence-based translation: A RAG+prompt system for Japanese-Chinese translation and its pedagogical potential

    May 5, 2026Wenshi GuLegal Natural Language Processing BenchmarksPedagogy

  59. Multi-Level Narrative Evaluation Outperforms Lexical Features for Mental Health

    Apr 30, 2026Yuxi Ma, Jieming Cui, Muyang Li +6Mental HealthNarratives

  60. Measurement Risk in Supervised Financial NLP: Rubric and Metric Sensitivity on JF-ICR

    Apr 30, 2026Sidi Chang, Peiying Zhu, Yuxiao Chen +1Legal Natural Language Processing BenchmarksRubrics