cs.CL · 2604.25374 Copy arXiv ID · Apr 28, 2026 Save Language corpora for the Dutch medical domain Authors: B. van Es
Organizations: Central Diagnostic Laboratory, University Medical Center Utrecht, Utrecht, The Netherlands. · R&D, B-lab, Castricum, The Netherlands.
Abstract \textbf{Background:} Dutch medical corpora are scarce, limiting NLP development. \ \textbf{Methods:} We translated English datasets, identified medical text in generic corpora, and extracted open Dutch medical resources. \ \textbf{Results:} The resulting corpus comprises ± \pm ±
Explore similar work Jun 20, 2026 · Bofeng Huang, Jacques Sun, Diane Bouchacourt +2 Large Language Model Pretraining Text Corpora
Aug 30, 2026 · Nhu Vo, Phuong Nguyen, Nu Uyen Phuong Le +4 Medical Ontology Named-Entity Recognition
Sep 22, 2026 · Julien Knafou, Luc Mottin, Anaïs Mottaz +2
Jun 20, 2026 · cs.CL J/K move · Enter open · S save
Bofeng Huang, Jacques Sun, Diane Bouchacourt, Nicolas Barascud +1
Doctolib · Work done while at Doctolib.
Web data curation has been widely studied for decoder Large Language Model (LLM) pretraining. Encoders for dense-terminology domains such as medicine, by contrast, are pretrained on small, manually-curated corpora that limit scalability and writing style diversity, a bottleneck even more severe in non-English clinical settings. Whether web-scale data curation also benefits encoder Masked Language Modeling (MLM) in a dense-terminology domain remains an open question. To address this, we introduce two complementary levers. Medical-term density filtering selects documents rich in medical terms. Signal-amplifying rephrasing uses an LLM to rewrite documents into denser variants with broader entity contexts. We instantiate the recipe on French medical NLP. The medical-term density filter outperforms the widely-used educational quality filter on downstream medical tasks, and the two complement each other. Signal-amplifying rephrasing alone improves on raw web data, and mixing it with filtered web data produces the largest gain. The recipe yields FineMed, a French medical pretraining corpus, and DoctoBERT, a state-of-the-art French medical encoder family evaluated on both the public benchmark DrBenchmark and a proprietary clinical Named Entity Recognition (NER) task.