cs.CLApr 6, 2026

HUKUKBERT: Domain-Specific Language Model for Turkish Law

Authors: Mehmet Utku Öztürk, Tansu Türkoğlu, Buse Buz-Yalug

Organizations: Kalitte Inc., Türkiye · Aibrite Inc., Türkiye · University of Eastern Finland, Finland

Abstract

Natural language processing (NLP) advances have powered a generation of LegalTech systems, but Turkish law remains under-served by domain-specific data and models. English has legal encoders such as LEGAL-BERT; no comparable high-volume Turkish counterpart exists. We introduce HukukBERT, a Turkish legal language model trained on a 19 GB cleaned corpus using a hybrid domain-adaptive pre-training (DAPT) recipe that mixes Whole-Word Masking, Token Span Masking, Word Span Masking, and targeted Keyword Masking. We compared our 48K WordPiece tokenizer and DAPT pipeline against general-purpose and existing domain-specific Turkish models. On the Legal Cloze Test - a masked legal term prediction benchmark over Turkish court decisions - HukukBERT reaches 84.40% Top-1 accuracy and beats every baseline we tested. The Legal Cloze Test is synthetically constructed, so its passages are absent from the pre-training corpus by construction, eliminating train-test contamination. On the downstream task of structural segmentation of official Turkish court decisions, it reaches a 92.8% document pass rate. We release HukukBERT to support Turkish legal NLP work in named entity recognition, judgment prediction, and document classification.

Explore similar work

CardsList
  1. Legal Domain Adaptation of Modern BERT Models

    Jun 26, 2026Dominik Stammbach, Peter HendersonLegal NLPBidirectional Encoder Representations from Transformers

  2. Domain adaptation of Russian ModernBERT for long legal documents

    Oct 5, 2026I. Litvak, D. Gvozdetsky, F. Lashkin +7Language Model PretrainingLegal NLP