cs.CLApr 15, 2026

BenGER Platform: A Collaborative Web Platform for End-to-End Benchmarking of German Legal Tasks

Authors: Sebastian Nagl, Matthias Grabmair

Organizations: Technical University of Munich, Munich, Germany

Abstract

Evaluating large language models (LLMs) for legal reasoning requires workflows that span task design, expert annotation, model execution, and metric-based evaluation. In practice, these steps are split across platforms and scripts, limiting transparency, reproducibility, and participation by non-technical legal experts. We present the BenGER (Benchmark for German Law) framework, an open-source web platform that integrates task creation, collaborative annotation, configurable LLM runs, and evaluation with lexical, semantic, factual, and judge-based metrics. BenGER supports multi-organization projects with tenant isolation and role-based access control, and can optionally provide formative, reference-grounded feedback to annotators. We will demonstrate a live deployment showing end-to-end benchmark creation and analysis.

Explore similar work

CardsList
  1. BenGER: Benchmarking LLM Systems on Subsumption-Based Legal Reasoning in German Law

    May 27, 2026Sebastian Nagl, Ann-Kristin Mayrhofer, Martin Heidebach +6Legal NLPLLM Evaluation

  2. Magis-Bench: Evaluating LLMs on Magistrate-Level Legal Tasks

    May 8, 2026Ramon Pires, Thales Sales Almeida, Celio Larcher Junior +6Legal NLPLLM Evaluation

  3. LexRubric: A Rubric-Guided Diagnostic Benchmark for Open-Ended Legal Tasks

    Jun 8, 2026Yifan Chen, Haitao Li, Yiran Hu +6LLM EvaluationRubric-Based Evaluation