Foresight-England: Development of a National-Scale Generative AI Model of Electronic Health Records for Medical Event Prediction across the COVID-19 Pandemic

Simon Ellershaw; Christopher Tomlinson; Zeljko Kraljevic; Spiros Denaxas; Harry Hemingway; Cathie Sudlow; Angela M. Wood; Anoop D. Shah

cs.LGAug 17, 2026

Foresight-England: Development of a National-Scale Generative AI Model of Electronic Health Records for Medical Event Prediction across the COVID-19 Pandemic

Authors: Simon Ellershaw, Christopher Tomlinson, Zeljko Kraljevic, Spiros Denaxas, Harry Hemingway, Cathie Sudlow, Angela M. Wood, Anoop D. Shah,

Organizations: Institute of Health Informatics, University College London, London, UK · Department of Biostatistics and Health Informatics, Institute of Psychiatry, Psychology and Neuroscience, King’s College London, London, UK · King’s Institute for Artificial Intelligence, King’s College London, London, UK · University College London Hospitals National Institute for Health Research Biomedical Research Centre, London, UK · Interdisciplinary Transformation University, Linz, Austria · British Heart Foundation Data Science Centre, Health Data Research UK, London, UK · Health Data Research UK, London, UK · Usher Institute, School of Population Health Sciences, The University of Edinburgh · British Heart Foundation Cardiovascular Epidemiology Unit, Department of Public Health and Primary Care, University of Cambridge, Cambridge, UK · Victor Phillip Dahdaleh Heart and Lung Research Institute, University of Cambridge, Cambridge, UK · British Heart Foundation Centre of Research Excellence, University of Cambridge, Cambridge, UK · National Institute for Health and Care Research Blood and Transplant Research Unit in Donor Health and Behaviour, University of Cambridge, Cambridge, UK · Health Data Research UK Cambridge, Wellcome Genome Campus and University of Cambridge, Cambridge, UK · Cambridge Centre of Artificial Intelligence in Medicine, University of Cambridge, Cambridge, UK · Department of Clinical Pharmacology, University College London Hospitals NHS Foundation Trust, London, UK · National Institute for Health Research Biomedical Research Centre at South London and Maudsley NHS Foundation Trust and King’s College London, London, UK

Abstract

Foresight-England (Foresight-E) is the first national-scale generative foundation model of electronic health records (EHRs), developed as a research pilot strictly for COVID-19 research. We evaluated its ability to model the direct and indirect effects of the pandemic. Trained from scratch entirely within the NHS England Secure Data Environment, Foresight-E is a 243-million-parameter transformer decoder. It was trained and evaluated on de-identified, longitudinal EHRs of approximately 61 million individuals, integrating primary/secondary care, death registrations, and COVID-19 data. Training and validation used a 90% subset (54.9 million) spanning November 2018 to December 2022; the remaining 10% (6.1 million) was held out for evaluation. Foresight-E models patient timelines autoregressively, predicting the next medical event given their prior history. At inference, it operates zero-shot, predicting any concept in its ~40,000-code vocabulary without task-specific training. Our tokenisation scheme retains the clinical granularity of ICD-10, OPCS-4, and SNOMED CT codes, jointly representing absolute and relative timing. We designed an evaluation framework for 30-day COVID-19 hospitalisation and mortality, including subgroup analyses by demographic factors and vaccination status. To assess generalisation to unseen future data and the pandemic's indirect effects, we tested the model on medical events from 2023 (beyond its training period), benchmarking against logistic regression and XGBoost. As detailed in the Project Status section, NHS England has paused access to data for the Foresight-E project, meaning quantitative results are currently unavailable. Instead, we share our strategy for tokenisation, architecture, training, inference, and evaluation as a methodological template and case study in the challenges of building population-scale EHR foundation models.

Figures & tables

	Train	Validation	Test: 30-day COVID-19	Test: 1 Year
Sex
Female	$\_$ ( $\_$ %)	$\_$ ( $\_$ %)	$\_$ ( $\_$ %)	$\_$ ( $\_$ %)
Male	$\_$ ( $\_$ %)	$\_$ ( $\_$ %)	$\_$ ( $\_$ %)	$\_$ ( $\_$ %)
Ethnicity
Asian	$\_$ ( $\_$ %)	$\_$ ( $\_$ %)	$\_$ ( $\_$ %)	$\_$ ( $\_$ %)
Black	$\_$ ( $\_$ %)	$\_$ ( $\_$ %)	$\_$ ( $\_$ %)	$\_$ ( $\_$ %)

Table 1: PLACEHOLDER Selected characteristics of the training, validation, and test sets (used in Sections 5.1 and 5.2 ). Ages are computed at the time of the last input event in each patient’s timeline. Counts are rounded to the nearest five to comply with the NHSE SDE’s disclosure control requirements. Total sample sizes for each split are derived from previously reported counts [ 14 ] . The rationale for placeholders is provided in Section 5 .

Panel of Figure 1 — Figure 1: Datasets and cohort splits used in the training and evaluation of Foresight-E. Total sample sizes for each split are derived from previously reported counts [ 14 ] .

Figure 2: Schematic of next token prediction training of Foresight-E on a synthetic patient timeline. Tokens at position IDs 2–7 are omitted for clarity. Given the sequence of tokens up to position $i$ , the model predicts the probability distribution over the vocabulary for the token at position $i+1$ . These probabilities are compared with the true next token to compute the cross-entropy loss for self-supervised training. Note that the initial static demographic tokens are masked ([Ignore]) during the loss calculation to prevent the model from learning to predict sequence progression solely from demographic features. The final and padding tokens are also ignored because they have no label to compare against.

Figure 3: Schematic of idealised zero-shot inference on a synthetic patient timeline. Given a history ending with a positive SARS-CoV-2 test, Foresight-E predicts a hospital admission within the next 30 days. Tokens 2–7 are omitted for clarity.

Figure 4: Visualisation of an idealised Foresight-E zero-shot inference for the task of 30-day hospitalisation following a positive SARS-CoV-2 test. A tokenised input timeline, shortened for visualisation, is used to prompt three independent predicted timelines in this example. ‘Time Diff‘ tokens are in units of days. In two of the predicted timelines, hospitalisation is predicted within the next 30 days (at 20 and 25 days, respectively). The remaining timeline predicts the next event at 100 days, falling outside the target window. Therefore, the predicted probability is 2/3.

Panel of Figure 5 — Figure 5: PLACEHOLDER. ROC, PR, and calibration curves showing Foresight-E’s prediction performance for 30-day mortality and hospitalisation following a positive SARS-CoV-2 test. Shaded regions represent 95% confidence intervals estimated via 1,000 bootstrap iterations. Counts are rounded to the nearest five to comply with the NHSE SDE’s disclosure control requirements.

Figure 6: PLACEHOLDER. Temporal evaluation of 30-day mortality prediction across the COVID-19 pandemic. Monthly performance of Foresight-E in predicting death within 30 days of a positive SARS-CoV-2 test, measured by AUROC (top panel) and AUPRC (second panel). Error bars represent $95\%$ confidence intervals estimated via 1,000 bootstrap iterations. Also shown are monthly mortality occurrence rates (third panel) and the number of positive SARS-CoV-2 tests (bottom panel), illustrating changes in disease dynamics over time. Shaded background regions denote the predominant SARS-CoV-2 variants [ 60 ] with labels shown along the upper x-axis. Counts are rounded to the nearest five to comply with the NHSE SDE’s disclosure control requirements.

Figure 7: PLACEHOLDER. Variation in AUROC and AUPRC metrics for the Foresight-E model predicting 30-day mortality after a positive SARS-CoV-2 test, stratified by age, ethnicity, sex, and COVID-19 vaccination status. Error bars show 95% confidence intervals calculated by 1,000 bootstrap iterations. Also shown is the mortality occurrence rate, as well as the count of patients for each group. Performance is consistent across ethnicity and sex, but varies notably by age group and vaccination status. Counts are rounded to the nearest five to comply with the NHSE SDE’s disclosure control requirements.

Figure 8: PLACEHOLDER. Variation in AUROC and AUPRC metrics for the Foresight-E model predicting 30-day mortality after a positive SARS-CoV-2 test with the number of past medical events recorded in a patient’s timeline. Also shown are the binned frequency of past events and corresponding mortality occurrence rates. A bin size of 16 events was used, and the final bin includes all patients with more than 498 events. Counts are rounded to the nearest five to comply with the NHSE SDE’s disclosure control requirements.

Panel of Figure 9 — Figure 9: PLACEHOLDER. Prediction performance of Foresight-E in 2023 using ROC, PR, and calibration curves. Shaded regions represent 95% confidence intervals estimated via 1,000 bootstrap iterations. Counts are rounded to the nearest five to comply with the NHSE SDE’s disclosure control requirements.

Figure 10: PLACEHOLDER. Relationship between AUROC and event occurrence rate for Foresight-E’s 1-year prediction in 2023. $\_$ AUROCs were observed for more $\_$ Phecodes, such as $\_$ .

EHR Foundation Models

See all 19 papers

Scaling Recurrence-aware Foundation Models for Clinical Records via Next-Visit Prediction
Mar 25, 2026Haresh Rengaraj Rajamohan, Xiang Gao, Weicheng Zhu +10EHR Foundation Models Longitudinal EHR Modeling
Center for Data Science, New York University, New York, NY, USA · Department of Radiology, NYU Grossman School of Medicine, New York, NY, USA · Center for Biomedical Imaging, NYU Grossman School of Medicine, New York, NY, USA +1
DT-Transformer: A Foundation Model for Disease Trajectory Prediction on a Real-world Health System
May 14, 2026Yunying Zhu, Andrew R Weckstein, Kueiyu Joshua Lin +1Clinical Prediction Time Series Transformers
Division of Pharmacoepidemiology and Pharmacoeconomics, Department of Medicine, Brigham and Women’s Hospital, Harvard Medical School, Boston, MA, USA · Harvard T.H. Chan School of Public Health, Harvard University, Boston, MA, USA · Kempner Institute for the Study of Natural and Artificial Intelligence, Harvard University, MA, USA +2
Sequence Composition in EHR Foundation Model Pretraining
Oct 12, 2026Joshua Placidi, Inigo Stibbe, Alfred J. Balston +2EHR Foundation Models Electronic Health Records
Department of Computing, Imperial College London

Mar 25, 2026J/K move · Enter open · S save

Scaling Recurrence-aware Foundation Models for Clinical Records via Next-Visit Prediction

Haresh Rengaraj Rajamohan, Xiang Gao, Weicheng Zhu, Shih-Lun Huang+9

Center for Data Science, New York University, New York, NY, USA · Department of Radiology, NYU Grossman School of Medicine, New York, NY, USA · Center for Biomedical Imaging, NYU Grossman School of Medicine, New York, NY, USA+1

EHR Foundation Models Longitudinal EHR Modeling Electronic Health Records Scaling Laws Clinical Outcome Prediction

Open paper AskPDF arXiv

While large-scale pretraining has revolutionized language modeling, its potential remains underexplored in healthcare with structured electronic health records (EHRs). We present RAVEN, a novel generative pretraining strategy for sequential EHR data based on Recurrence-Aware next-Visit EveNt prediction. Leveraging a dataset of over one million unique individuals, our model learns to autoregressively generate tokenized clinical events for the next visit conditioned on patient history. We introduce regularization on predicting repeated events and highlight a key pitfall in EHR-based foundation model evaluations: repeated event tokens can inflate performance metrics when new onsets are not distinguished from subsequent occurrences. Furthermore, we empirically investigate the scaling behaviors in a data-constrained, compute-saturated regime, showing that simply increasing model size is suboptimal without commensurate increases in data volume. We evaluate our model via zero-shot prediction for forecasting the incidence of a diverse set of diseases, where it rivals fully fine-tuned representation-based Transformer models and outperforms both standard simulation-based next-token approaches and a prompted medical large language model baseline. Finally, without additional parameter updates, we show that RAVEN can generalize to an external patient cohort under lossy clinical code mappings and feature coverage gaps.

Abstract

EHR Foundation Models

Explore similar work

Scaling Recurrence-aware Foundation Models for Clinical Records via Next-Visit Prediction

DT-Transformer: A Foundation Model for Disease Trajectory Prediction on a Real-world Health System

Sequence Composition in EHR Foundation Model Pretraining