Phase 1 Implementation of LLM-generated Discharge Summaries showing high Adoption in a Dutch Academic Hospital
Authors: Nettuno Nadalini, Tarannom Mehri, Anne H Hoekman, Katerina Kagialari, Job N Doornberg, Tom P van der Laan, Jacobien H F Oosterhoff, Rosanne C Schoonbeek, Charlotte M H H T Bootsma-Robroeks
Organizations: Department of Health Information Office, Data & Digitalization, University Medical Center, Groningen, the Netherlands · Department of Trauma Surgery / Orthopedics, University Medical Center, Groningen, the Netherlands · Department of Orthopaedic Surgery, Massachusetts General Hospital, Boston, MA, USA · Harvard Medical School Orthopaedic Trauma Initiative, Boston, MA, USA · Department of Orthopaedic Surgery and Sports Medicine, Amsterdam UMC location University of Amsterdam, Meibergdreef 9, Amsterdam, The Netherlands · Department of Orthopaedic Trauma, Flinders University Medical Centre, Adelaide, Australia · Department of Otolaryngology – Head and Neck Surgery, University Medical Center, Groningen, the Netherlands · Department of Pediatrics, Pediatrics Nephrology, Beatrix Children’s Hospital, University Medical Center, Groningen, the Netherlands
Writing discharge summaries to transfer medical information is an important but time-consuming process that can be assisted by Large Language Models (LLMs). This prospective mixed methods pilot study evaluated an Electronic Health Record (EHR)-integrated LLM to generate discharge summaries drafts. In total, 379 discharge summaries were generated in clinical practice by 21 residents and 4 physician assistants during 9 weeks in our academic hospital. LLM-generated text was copied in 58.5% of admissions, and identifiable LLM content could be traced to 29.1% of final discharge letters. Notably, 86.9% of users self-reported a reduction in documentation time, and 60.9% a reduction in administrative workload. Intent to use after the pilot phase was high (91.3%), supporting further implementation of this use-case. Accurately measuring the documentation time of users on discharge summaries remains challenging, but will be necessary for future extrinsic evaluation of LLM-assisted documentation.