This study presents a fully automated methodology for early prediction studies in clinical settings, leveraging information extracted from unstructured discharge reports. The proposed pipeline uses discharge reports to support the three main steps of early prediction: cohort selection, dataset generation, and outcome labeling. By processing discharge reports with natural language processing techniques, we can efficiently identify relevant patient cohorts, enrich structured datasets with additional clinical variables, and generate high-quality labels without manual intervention. This approach addresses the frequent issue of missing or incomplete data in codified electronic health records (EHR), capturing clinically relevant information that is often underrepresented. We evaluate the methodology in the context of predicting atrial fibrillation (AF) progression, showing that predictive models trained on datasets enriched with discharge report information achieve higher accuracy and correlation with true outcomes compared to models trained solely on structured EHR data, while also surpassing traditional clinical scores. These results demonstrate that automating the integration of unstructured clinical text can streamline early prediction studies, improve data quality, and enhance the reliability of predictive models for clinical decision-making.
Figures & tables
Figure 1: Clinical scenario of AF progression.
Figure 2: Example of a semi-structured discharge report.
Figure 3: Example of the codified structured data.
Figure 4: End to end overview of the proposed methodology. The pipeline starts with automatic cohort selection, followed by dataset generation by combining structured EHR data with information extracted from clinical reports. An NLP module performs automatic labeling, and the resulting silver and gold datasets are used to train and evaluate a TabPFN model for AF progression prediction.
Figure 5: Overview of the vector generation process. For each patient in the AF onset cohort, all discharge reports (free text) and codified data (structured data stored in the Business Intelligence system) are collected and processed using the Report2Vector (R2V) and Structured2Vector (S2V) tools, respectively. Each tool generates a corresponding set of vectors, which are then merged by the VectorMerger (VM) tool to produce a patient-specific vector that integrates both sources of clinical information.
Size
% Positives
Train-Silver
1023
65.40%
Train-Gold
541
66.17%
Test
278
64.03%
Table 1: Characteristics of the dataset. The second column indicates the number of patients and the last column the percentage of AF progression.
Figure 6: Amount of missing values. Difference in percentage between the original and enriched datasets. The bar plots illustrate the recovery of features that were absent in the original codified dataset but retrieved from the information contained in the discharge reports.
Dataset
Accuracy
MCC
Original-Silver
0.65
0.11
Enriched-Silver
0.66
0.20
Enriched-Gold
0.66
0.21
Table 2: Results obtained in the AF progression experiments.
ACC
MCC
CHADS2-VASc
0.6043
-0.0052
HATCH
0.6043
0.0832
APPLE
0.4820
0.0510
Table 3: Results obtained in the AF progression experiments by the clinical scores.
Coronary artery disease (CAD) remains one of the leading causes of death globally, highlighting the need for reliable predictive systems to support early diagnosis and risk assessment. While traditional machine learning models perform well on structured clinical data, large language models (LLMs) present new possibilities to interpret medical information expressed in natural language. In this work, we develop a hybrid framework that bridges structured clinical data and natural-language representations for CAD prediction. Using a publicly available dataset of 1,190 patient records with 11 clinical attributes, structured variables are converted into interpretable feature representations and synthetic clinical narratives using LLMs. A validation pipeline performs reverse extraction of clinical variables and computes a consistency score with the original records, achieving an average fidelity of 94.61%. We then evaluate four conventional machine learning models and compare their performance with LLM-based classification under zero-shot and few-shot prompting settings. We use two LLMs here, GPT and Gemini. Experimental results show that Random Forest achieves the highest accuracy. Despite this advantage, LLM-based classification remains beneficial in real-world clinical settings. This is because LLMs operate directly on natural language patient descriptions, meaning that sensitive numerical patient data such as exact lab values, blood pressure readings, and diagnostic codes are kept private. Findings suggest that combining structured clinical data with LLM-generated narratives can enable new directions for hybrid clinical prediction systems.
Jeba Maliha, Md Rafiul Kabir
Central Michigan University, Mount Pleasant, MI, USA
Autoregressive foundation models for electronic health records can make predictions for diverse clinical tasks without finetuning. These methods work by generating possible "synthetic futures" for patients, and then inferring downstream predictions over these simulated trajectories. Despite their strong capabilities, these methods have several flaws. (1) Inference is extremely computationally expensive, as each prediction requires simulating many trajectories, each with many observations. (2) Their performance is noisy due to the variance inherent to simulation, which in particular reduces efficacy on rare events, which are often of high importance clinically. (3) They are not promptable, as their only input is the patient's medical history, requiring indirect aggregation at inference time. We introduce EveryQuery, a promptable foundation model for structured EHR data. Building on the insight that clinical prediction tasks can be written in a structured query language, EveryQuery takes a query as a prompt alongside the patient's history and estimates the outcome directly. This allows EveryQuery to make predictions across diverse clinical tasks, such as classification and survival modeling, without simulation or finetuning. Across three EHR datasets, EveryQuery outperforms a competitive autoregressive baseline on clinically relevant classification and time-to-event tasks, with mean AUROC gains of 14.3% and 12.7% on MIMIC-IV and a large academic medical center and comparable performance on the smaller NWICU dataset. Its advantage is largest for rare outcomes, and it requires 580x less inference compute.
To overcome the limitations of manual administrative coding in geriatric Cardiovascular Risk Management, this study introduces an automated classification framework leveraging unstructured Electronic Health Records (EHRs). Using a dataset of 3,482 patients, we benchmarked three distinct modeling paradigms on longitudinal Dutch clinical narratives: classical machine learning baselines, specialized deep learning architectures optimized for large-context sequences, and general-purpose generative Large Language Models (LLMs) in a zero-shot setting. Additionally, we evaluated a late fusion strategy to integrate unstructured text with structured medication embeddings and anthropometric data. Our analysis reveals that the custom Transformer architecture outperforms both traditional methods and generative \acs{llm}s, achieving the highest F1-scores and Matthews Correlation Coefficients. These findings underscore the critical role of specialized hierarchical attention mechanisms in capturing long-range dependencies within medical texts, presenting a robust, automated alternative to manual workflows for clinical risk stratification.
Jacopo Vitale, David Della Morte, Luca Bacco +5
Research Unit of Intelligent Health Technologies, Universit`a Campus Bio-Medico di Roma, Rome, Italy. · Central Diagnostic Laboratory, University Medical Center Utrecht, Utrecht, The Netherlands.