cs.LGFeb 27, 2026

A medical coding language model trained on clinical narratives from a population-wide cohort of 1.8 million patients

Authors: Joakim Edin, Sedrah Butt Balaganeshan, Annike Kjølby Kristensen, Lars Maaløe, Ioannis Louloudis, Søren Brunak

Organizations: Section for Health Data Science and AI, Department of Public Health, University of Copenhagen, Denmark · Corti, Copenhagen, Denmark · Novo Nordisk Foundation Center for Protein Research, University of Copenhagen · Danish Research Institute for Suicide Prevention, Mental Health Centre Copenhagen, Denmark

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Medical coding translates clinical documentation into standardized codes for billing, research, and public health, but manual coding is time-consuming and error-prone. Existing automation efforts rely on small datasets that poorly represent real-world patient heterogeneity. We trained a language model on 5.8 million electronic health records from 1.8 million patients across nearly all specialties in Eastern Denmark (2006--2016) to predict ICD-10 codes from clinical notes, medications, and laboratory results. Evaluated on 270,000 held-out patients, the model achieved a micro F1 of 71.8% and a top-10 recall of 95.5%. Performance varied by specialty (F1: 53--91%), with higher scores in specialties with well-defined diagnostic criteria. Codes appearing predominantly as secondary diagnoses had markedly lower F1 scores. For three such codes (suicide-related behaviors, weight disorders, and hypertension), the model identified thousands of uncoded cases, of which 76-86% were confirmed valid upon manual review, suggesting systematic under-coding rather than model error. These findings suggest under-coding of secondary diagnoses in Eastern Denmark during this period, with potential implications for epidemiological research, public health surveillance, and understanding of multimorbidity. Similar time constraints and reimbursement structures in other healthcare systems suggest this may not be isolated to this dataset. The model can automate coding for approximately 50% of cases and provide accurate suggestions for most others, and may offer a practical solution to help capture missed secondary conditions.

Figures & tables

 ICD-10 Level 3Level 5
Training data50K100K200K500K1M1.8M1M
F1 Micro59.9%61.7%65.8%69.0%70.1%71.8%54.6%
F1 Macro25.6%26.3%30.5%41.0%43.6%47.3%10.1%
Exact Match Ratio37.0%42.5%47.3%51.7%52.0%54.6%33.2%
Recall@582.2%85.3%87.8%89.8%90.5%91.4%78.4%
Recall@1088.7%90.9%93.0%94.4%94.9%95.5%86.0%
Table 1: Model performance when trained on progressively larger data sets. All models are evaluated on the same test set using the human-annotated codes as the ground-truth labels.
Figure 1: The model’s F1 score for each specialty. Each dot is the model’s median F1 score for a specific department. The figure reveals a pattern in which the top specialties often involve planned admissions, while the bottom specialties represent patients with an average of many comorbidities. Departments with more than one hundred examples in the test set are included only.
Figure 2: a) The F1 score (y-axis) versus the number of occurrences in the training data (x-axis) for each ICD-10 code (level-3). b) The frequency of codes for secondary diagnoses in the training data (x-axis) and their F1 score (y-axis).
Figure 3: a) Recall@5 for primary diagnoses and secondary diagnoses, and b) Recall@10. Recall@5 and Recall@10 measure the frequency of human-annotated codes among the model’s top 5 and top 10 predictions.
Figure 4: The data preprocessing pipeline for the study.
Figure 5: The interface we used for our manual explainability analysis. The clinical note is shown to the left, and the model’s predictions are to the right. When hovering the mouse over a code prediction, the AttInGrad explanations are highlighted in the text of the clinical note. By visualizing the factors that caused the model prediction, we could quickly identify the elements that led to the human-model disagreements. The text in this example is AI-generated and does not relate to the data used in the study.
More in this topic

Automated Medical Coding

Automated medical coding uses clinical records and coding rules to assign standardized diagnosis and procedure codes for billing, care, and research.

See all 23 papers

Explore similar work

CardsList

  1. Bridging the Version Gap: Multi-version Training Improves ICD Code Prediction, Especially for Rare Codes

    May 18, 2026Jinghui Liu, Anthony NguyenAutomated Medical CodingHealthcare

    Australian e-Health Research Centre, CSIRO

  2. Training a Large Language Model for Medical Coding Using Privacy-Preserving Synthetic Clinical Data

    Mar 6, 2026John Cook, Michael Wyatt, Peng Wei +11Automated Medical CodingMedical LLMs

    Veradigm · Snowflake

  3. CoLa-ICD: A Knowledge-Enhanced Framework for Long-Tail Automated Medical Coding

    Aug 31, 2026Yihang Cheng, Veronica Liesaputra, Andrew TrotmanLong-Tailed ClassificationMulti-Label Classification

    University of Otago · Dunedin, New Zealand