FlatASCEND: Autoregressive Clinical Sequence Generation with Continuous Time Prediction and Association-Based Pharmacological Testing
Authors: Chris Sainsbury, Feng Dong, Andreas Karwath
Organizations: School of Cardiovascular & Metabolic Health, University of Glasgow, UK · School of Medicine, University of Dundee, UK · NHS Greater Glasgow and Clyde, UK · Department of Computer and Information Sciences, University of Strathclyde, UK · Cancer and Genomic Sciences, School of Medical Sciences, College of Medicine and Health, University of Birmingham, UK
Autoregressive models can predict clinical events, but generating patient-conditioned multi-step trajectories that respond to intervention tokens and testing whether those responses preserve known pharmacological associations has received limited attention. We present FlatASCEND, a 14.5M-parameter autoregressive clinical sequence model using flat composite tokens and a zero-inflated log-normal time head. Standard distributional metrics (Jaccard 0.889-0.954) do not distinguish FlatASCEND from trivial baselines; the model's value lies in conditional generation from patient-specific prefixes. A prompt-shuffle ablation shows patient-specific conditioning amplifies mechanistic pharmacological effects (2.0-2.2x for steroid to glucose, diuretic to potassium) while leaving confounding-driven associations unchanged (0.9x for insulin to glucose). An incident-user framework assesses directional consistency against prior pharmacological knowledge on MIMIC-IV (N=500 per comparison): 4/10 recover correct mechanistic directions, 2 reproduce treatment-context associations, 4 are incorrect (9/10 significant, Wilcoxon p<0.05). This pattern - partial recovery under residual confounding - is consistent with learned observational associations without causal distinction. Direct preference optimisation with surrogate reward destroys all correct associations (3/3 to 0/3), illustrating reward exploitation when reward and evaluation share an outcome domain. Generative evidence is strongest for short-horizon ICU data; outpatient temporal fidelity is weaker (median 10 vs 154 days on INSPECT), and zero-shot cross-site transfer degrades without adaptation.
Figures & tables
Dataset
Setting
Hospitals
N
Vocabulary
Mortality
Access
Dev (diabetes)
Outpatient
1 system
61K patients
76 tokens
—
Proprietary
INSPECT
Outpatient
1
19K patients
220 tokens
2.8%
PhysioNet
MIMIC-IV
ICU
1
425K admissions
220 tokens
2.6%
PhysioNet
eICU-CRD
ICU
208
200K stays
220 tokens ‡
5.3%
PhysioNet
Table 1: Dataset characteristics. Dev = proprietary development dataset used for architecture design. Open-access results are the primary evidence; development-set results are reported for architectural context only. ‡ eICU-CRD uses the 220-token MIMIC-IV class-level vocabulary; 0.7% of eICU tokens absent from this vocabulary were mapped to the padding token.
Dataset
Type Jaccard
Collapse
TF perplexity
Dev (diabetes)
0.857
2%
1.24
MIMIC-IV
0.954
0%
6.30
INSPECT
0.889
0%
5.16
eICU-CRD (zero-shot) †
0.682
0%
776
eICU-CRD (adapted, 5K steps) ††
0.820
0%
27.4
Table 2: Generation quality across datasets. All models use the 14.5M-parameter architecture with factored embeddings and EMD loss. Collapse = percentage of generated sequences with ≥ 20 consecutive identical tokens. TF = teacher-forcing. Dev results are for architectural context; MIMIC-IV and INSPECT are the primary open-data evaluations. † Zero-shot: MIMIC-IV-trained model with no adaptation; high perplexity reflects distributional shift across 208 hospitals. †† Adapted: MIMIC-IV model fine-tuned on eICU-CRD for 5,000 steps (see Methods).
Metric
MIMIC-IV
INSPECT
Timing calibration
Zero-fraction (gen / GT)
0.872 / 0.857
0.795 / 0.782
Median inter-event time, days (gen / GT)
0.22 / 0.17
0.14 / 0.11
Mean inter-event time, days (gen / GT)
0.28 / 0.26
7.19 / 15.56
KS statistic (non-zero times)
0.152
0.158
Bhattacharyya coefficient (log-space)
0.982
0.993
Table 3: Trajectory validation metrics. Timing calibration compares observed versus generated inter-event time distributions. Conditional dependencies compare category-level transition probabilities. Longitudinal fidelity measures per-patient similarity between generated and held-out continuations. BC = Bhattacharyya coefficient (1 = perfect overlap). JSD = Jensen-Shannon divergence. DX → LAB is not evaluated for MIMIC-IV because diagnosis tokens comprise only 1.5% of MIMIC-IV token occurrences and are too sparse for reliable transition estimation.
Dataset
Metric
Unigram
Bigram
FlatASCEND
MIMIC-IV
Pooled Jaccard
0.842
0.989
0.989
MIMIC-IV
Cat. cosine (mean)
0.991
0.989
0.940
MIMIC-IV
Timing KS
0.040
0.027
0.160
MIMIC-IV
Trans. JSD
0.052
0.0004
0.002
INSPECT
Pooled Jaccard
0.915
0.973
0.857
INSPECT
Cat. cosine (mean)
0.965
0.958
0.903
Table 4: Baseline comparison ( N = 200 patients). Pooled Jaccard = token-set overlap across all patients. Cat. cosine = per-patient category distribution cosine similarity. Timing KS = Kolmogorov-Smirnov statistic on non-zero inter-event times. Trans. JSD = Jensen-Shannon divergence of category-level transition matrices. Lower KS and JSD indicate better match to held-out data.
Comparison
Condition
DiD
Wilcoxon p
Ratio
Steroid → glucose
Ordered
+0.155
<0.001
2.0 ×
(mechanistic)
Shuffled
+0.077
<0.001
Diuretic → potassium
Ordered
− 0.048
<0.001
2.2 ×
(mechanistic)
Shuffled
− 0.022
0.23
Insulin → glucose
Ordered
+0.047
<0.001
0.9 ×
(confounding)
Shuffled
+0.054
0.006
Table 5: Prompt-shuffle ablation on MIMIC-IV ( N = 200 patients, 100 samples per arm). Ordered = normal clinical sequence; Shuffled = prompt tokens randomly permuted (first two positions preserved; see Methods). DiD = difference-in-differences in mean outcome quintile. Ratio = ∣ ordered DiD ∣ / ∣ shuffled DiD ∣ . Effect sizes differ from the main incident-user table due to smaller N and different random patient selection (see Methods).
Comparison
Type
Effect
Wilcoxon p
Correct?
Warfarin → INR
Surrogate
DiD +0.052
<0.001
Yes
Steroid → glucose
Surrogate
DiD +0.119
<0.001
Yes
Vasopressor → mortality
Mortality
Δ +0.069
<0.001
Yes a
Statin sepsis → mortality
Mortality
Δ− 0.004
0.0001
Yes b
Furosemide vs spiro → K
Surrogate
DiD − 0.061
<0.001
Yes
Diuretic → K
Surrogate
DiD − 0.040
<0.001
Yes
Table 6: MIMIC-IV incident-user pharmacological association results ( N = 500 patients per comparison). DiD = difference-in-differences in mean outcome quintile; Δ = difference in mortality rate. All tests are patient-level Wilcoxon signed-rank. Direction assessed against published trial evidence.
Dataset
Correct (non-null)
Significant
Expected-null
Total
Method
MIMIC-IV (primary)
6/10
9/10
—
10
Incident-user v2
INSPECT
5/7
0/7
1/1
8
Standard
Open-data total
11/17
9/17
1/1
18
Table 7: Unique pharmacological comparisons by dataset. Each comparison counted once. Correct = direction matches published trial evidence. The incident-user design on MIMIC-IV is the primary analysis.
Dataset
Method
Correct
Significant
Expected-null
Total
Dev (Level 1)
Standard
2/3
2/3
2/2
5
Dev (Level 2)
Standard
3/3
0/3
2/2
5
MIMIC-IV class
Standard
6/7
2/7
—
7
MIMIC-IV drug
Head-to-head
5/8
0/8
—
8
Table 8: Method-specific MIMIC-IV analyses (partial overlap with Table 7 ). Some drug–outcome pairs are evaluated by multiple methods. Development-set results shown for architectural context only.
Autoregressive electronic health record (EHR) foundation models predict the next event token in patient trajectories: sequences of structured medical events. A typical pretraining strategy is to concatenate all trajectories into a global stream and sample token windows from it. This gives patients with longer sequences more training exposure and allows tokens to attend across patient boundaries. We argue that these choices create a misalignment between pretraining and downstream clinical tasks. To explore this, we investigate how the composition of pretraining sequences affects clinical prediction performance. Using the global-stream approach as a baseline, we propose alternative sequence composition methods that control how patient trajectories contribute to pretraining sequences. Across four MIMIC-IV v2.2 tasks, our best method improves macro AUROC, AUPRC, F1, and balanced accuracy, with relative gains of 9.3% in F1 and 2.7% in AUPRC under a fixed training recipe. We find that these gains generalize across four additional evaluation settings, including the INSPIRE and eICU-CRD datasets, with improvements in macro AUROC, AUPRC, and F1 score. Our results show that simply changing how EHR data are presented to the model during pretraining can improve downstream performance, without changing the model or collecting additional data.
Joshua Placidi, Inigo Stibbe, Alfred J. Balston +2
We introduce Prior-Fitted Functional Flows, a generative foundation model for pharmacokinetics that enables zero-shot population synthesis and individual forecasting without manual parameter tuning. We learn functional vector fields, explicitly conditioned on the sparse, irregular data of an entire study population. This enables the generation of coherent virtual cohorts as well as forecasting of partially observed patient trajectories with calibrated uncertainty. We construct a new open-access literature corpus to inform our priors, and demonstrate state-of-the-art predictive accuracy on extensive real-world datasets.
César Ojeda, Niklas Hartung, Wilhelm Huisinga +6
University of Potsdam, Germany · Technical University Berlin, Germany · Freie Universität Berlin, Germany +2
Many time-series forecasts depend not only on prior observations but also on actions specified during the forecast period. In intensive care units (ICUs), future vital signs and laboratory values are influenced by treatments such as vasopressors. However, models that predict the full future sequence all at once make little use of these treatments, whereas autoregressive models can accumulate errors. We introduce DRIFT, a hybrid framework in which a direct model produces the primary forecast and a recursive, action-conditioned model contributes constrained corrections. We evaluate DRIFT on 6,046 admissions from MIMIC-IV and 8,345 admissions from eICU-CRD. Averaged across the 8-, 24-, and 48-hour forecast endpoints, DRIFT reduces mean absolute error for mean arterial pressure (MAP) by 0.673% relative to an action-conditioned Temporal Fusion Transformer (TFT-action) on MIMIC-IV and achieves the lowest corresponding error among the compared models on eICU-CRD. Although the overall accuracy improvement is modest, a MIMIC-IV audit restricted to windows in which the supplied treatment sequence was altered showed that DRIFT achieved lower observed-target MAP error than TFT-action at 8 and 24 hours. Treatment-sequence alteration increased DRIFT's MAP error by 0.21-0.26 mmHg more than it increased TFT-action's error, with prediction changes occurring primarily after the supplied paths diverged. In a separate robustness experiment, the MAP advantage persisted under three shared checkpoint-selection rules emphasizing overall endpoint error, MAP error, or both equally.
Weixin Liu, Juming Xiong, Congning Ni +4
Vanderbilt University, Nashville, Tennessee, USA · Vanderbilt University Medical Center, Nashville, Tennessee, USA