This study presents a fully automated methodology for early prediction studies in clinical settings, leveraging information extracted from unstructured discharge reports. The proposed pipeline uses discharge reports to support the three main steps of early prediction: cohort selection, dataset generation, and outcome labeling. By processing discharge reports with natural language processing techniques, we can efficiently identify relevant patient cohorts, enrich structured datasets with additional clinical variables, and generate high-quality labels without manual intervention. This approach addresses the frequent issue of missing or incomplete data in codified electronic health records (EHR), capturing clinically relevant information that is often underrepresented. We evaluate the methodology in the context of predicting atrial fibrillation (AF) progression, showing that predictive models trained on datasets enriched with discharge report information achieve higher accuracy and correlation with true outcomes compared to models trained solely on structured EHR data, while also surpassing traditional clinical scores. These results demonstrate that automating the integration of unstructured clinical text can streamline early prediction studies, improve data quality, and enhance the reliability of predictive models for clinical decision-making.
Figures & tables
Figure 1: Clinical scenario of AF progression.
Figure 2: Example of a semi-structured discharge report.
Figure 3: Example of the codified structured data.
Figure 4: End to end overview of the proposed methodology. The pipeline starts with automatic cohort selection, followed by dataset generation by combining structured EHR data with information extracted from clinical reports. An NLP module performs automatic labeling, and the resulting silver and gold datasets are used to train and evaluate a TabPFN model for AF progression prediction.
Figure 5: Overview of the vector generation process. For each patient in the AF onset cohort, all discharge reports (free text) and codified data (structured data stored in the Business Intelligence system) are collected and processed using the Report2Vector (R2V) and Structured2Vector (S2V) tools, respectively. Each tool generates a corresponding set of vectors, which are then merged by the VectorMerger (VM) tool to produce a patient-specific vector that integrates both sources of clinical information.
Size
% Positives
Train-Silver
1023
65.40%
Train-Gold
541
66.17%
Test
278
64.03%
Table 1: Characteristics of the dataset. The second column indicates the number of patients and the last column the percentage of AF progression.
Figure 6: Amount of missing values. Difference in percentage between the original and enriched datasets. The bar plots illustrate the recovery of features that were absent in the original codified dataset but retrieved from the information contained in the discharge reports.
Dataset
Accuracy
MCC
Original-Silver
0.65
0.11
Enriched-Silver
0.66
0.20
Enriched-Gold
0.66
0.21
Table 2: Results obtained in the AF progression experiments.
ACC
MCC
CHADS2-VASc
0.6043
-0.0052
HATCH
0.6043
0.0832
APPLE
0.4820
0.0510
Table 3: Results obtained in the AF progression experiments by the clinical scores.
Research Unit of Intelligent Health Technologies, Universit`a Campus Bio-Medico di Roma, Rome, Italy. · Central Diagnostic Laboratory, University Medical Center Utrecht, Utrecht, The Netherlands.