FlatASCEND: Autoregressive Clinical Sequence Generation with Continuous Time Prediction and Association-Based Pharmacological Testing

Chris Sainsbury; Feng Dong; Andreas Karwath

cs.LGApr 13, 2026

FlatASCEND: Autoregressive Clinical Sequence Generation with Continuous Time Prediction and Association-Based Pharmacological Testing

Authors: Chris Sainsbury, Feng Dong, Andreas Karwath

Organizations: School of Cardiovascular & Metabolic Health, University of Glasgow, UK · School of Medicine, University of Dundee, UK · NHS Greater Glasgow and Clyde, UK · Department of Computer and Information Sciences, University of Strathclyde, UK ·

Abstract

Autoregressive models can predict clinical events, but generating patient-conditioned multi-step trajectories that respond to intervention tokens and testing whether those responses preserve known pharmacological associations has received limited attention. We present FlatASCEND, a 14.5M-parameter autoregressive clinical sequence model using flat composite tokens and a zero-inflated log-normal time head. Standard distributional metrics (Jaccard 0.889-0.954) do not distinguish FlatASCEND from trivial baselines; the model's value lies in conditional generation from patient-specific prefixes. A prompt-shuffle ablation shows patient-specific conditioning amplifies mechanistic pharmacological effects (2.0-2.2x for steroid to glucose, diuretic to potassium) while leaving confounding-driven associations unchanged (0.9x for insulin to glucose). An incident-user framework assesses directional consistency against prior pharmacological knowledge on MIMIC-IV (N=500 per comparison): 4/10 recover correct mechanistic directions, 2 reproduce treatment-context associations, 4 are incorrect (9/10 significant, Wilcoxon p<0.05). This pattern - partial recovery under residual confounding - is consistent with learned observational associations without causal distinction. Direct preference optimisation with surrogate reward destroys all correct associations (3/3 to 0/3), illustrating reward exploitation when reward and evaluation share an outcome domain. Generative evidence is strongest for short-horizon ICU data; outpatient temporal fidelity is weaker (median 10 vs 154 days on INSPECT), and zero-shot cross-site transfer degrades without adaptation.

Figures & tables

Dataset	Setting	Hospitals	N	Vocabulary	Mortality	Access
Dev (diabetes)	Outpatient	1 system	61K patients	76 tokens	—	Proprietary
INSPECT	Outpatient	1	19K patients	220 tokens	2.8%	PhysioNet
MIMIC-IV	ICU	1	425K admissions	220 tokens	2.6%	PhysioNet
eICU-CRD	ICU	208	200K stays	220 tokens $\ddagger$	5.3%	PhysioNet

Table 1: Dataset characteristics. Dev = proprietary development dataset used for architecture design. Open-access results are the primary evidence; development-set results are reported for architectural context only.

\ddagger

eICU-CRD uses the 220-token MIMIC-IV class-level vocabulary; 0.7% of eICU tokens absent from this vocabulary were mapped to the padding token.

Dataset	Type Jaccard	Collapse	TF perplexity
Dev (diabetes)	0.857	2%	1.24
MIMIC-IV	0.954	0%	6.30
INSPECT	0.889	0%	5.16
eICU-CRD (zero-shot) $\dagger$	0.682	0%	776
eICU-CRD (adapted, 5K steps) $\dagger\dagger$	0.820	0%	27.4

Table 2: Generation quality across datasets. All models use the 14.5M-parameter architecture with factored embeddings and EMD loss. Collapse = percentage of generated sequences with

\geq

20 consecutive identical tokens. TF = teacher-forcing. Dev results are for architectural context; MIMIC-IV and INSPECT are the primary open-data evaluations.

\dagger

Zero-shot: MIMIC-IV-trained model with no adaptation; high perplexity reflects distributional shift across 208 hospitals.

\dagger\dagger

Adapted: MIMIC-IV model fine-tuned on eICU-CRD for 5,000 steps (see Methods).

Timing calibration
Metric	MIMIC-IV	INSPECT
Zero-fraction (gen / GT)	0.872 / 0.857	0.795 / 0.782
Median inter-event time, days (gen / GT)	0.22 / 0.17	0.14 / 0.11
Mean inter-event time, days (gen / GT)	0.28 / 0.26	7.19 / 15.56
KS statistic (non-zero times)	0.152	0.158
Bhattacharyya coefficient (log-space)	0.982	0.993

Table 3: Trajectory validation metrics. Timing calibration compares observed versus generated inter-event time distributions. Conditional dependencies compare category-level transition probabilities. Longitudinal fidelity measures per-patient similarity between generated and held-out continuations. BC = Bhattacharyya coefficient (1 = perfect overlap). JSD = Jensen-Shannon divergence. DX

\to

LAB is not evaluated for MIMIC-IV because diagnosis tokens comprise only 1.5% of MIMIC-IV token occurrences and are too sparse for reliable transition estimation.

Dataset	Metric	Unigram	Bigram	FlatASCEND
MIMIC-IV	Pooled Jaccard	0.842	0.989	0.989
MIMIC-IV	Cat. cosine (mean)	0.991	0.989	0.940
MIMIC-IV	Timing KS	0.040	0.027	0.160
MIMIC-IV	Trans. JSD	0.052	0.0004	0.002
INSPECT	Pooled Jaccard	0.915	0.973	0.857
INSPECT	Cat. cosine (mean)	0.965	0.958	0.903

Table 4: Baseline comparison (

N

= 200 patients). Pooled Jaccard = token-set overlap across all patients. Cat. cosine = per-patient category distribution cosine similarity. Timing KS = Kolmogorov-Smirnov statistic on non-zero inter-event times. Trans. JSD = Jensen-Shannon divergence of category-level transition matrices. Lower KS and JSD indicate better match to held-out data.

Comparison	Condition	DiD	Wilcoxon $p$	Ratio
Steroid $\to$ glucose	Ordered	+0.155	$<0.001$	2.0 $\times$
(mechanistic)	Shuffled	+0.077	$<0.001$	2.0 $\times$
Diuretic $\to$ potassium	Ordered	$-$ 0.048	$<0.001$	2.2 $\times$
(mechanistic)	Shuffled	$-$ 0.022	0.23	2.2 $\times$
Insulin $\to$ glucose	Ordered	+0.047	$<0.001$	0.9 $\times$
(confounding)	Shuffled	+0.054	0.006	0.9 $\times$

Table 5: Prompt-shuffle ablation on MIMIC-IV (

N

= 200 patients, 100 samples per arm). Ordered = normal clinical sequence; Shuffled = prompt tokens randomly permuted (first two positions preserved; see Methods). DiD = difference-in-differences in mean outcome quintile. Ratio =

|

ordered DiD

|

|

shuffled DiD

|

. Effect sizes differ from the main incident-user table due to smaller

N

and different random patient selection (see Methods).

Comparison	Type	Effect	Wilcoxon $p$	Correct?
Warfarin $\to$ INR	Surrogate	DiD +0.052	$<0.001$	Yes
Steroid $\to$ glucose	Surrogate	DiD +0.119	$<0.001$	Yes
Vasopressor $\to$ mortality	Mortality	$\Delta$ +0.069	$<0.001$	Yes a
Statin sepsis $\to$ mortality	Mortality	$\Delta$ $-$ 0.004	0.0001	Yes b
Furosemide vs spiro $\to$ K	Surrogate	DiD $-$ 0.061	$<0.001$	Yes
Diuretic $\to$ K	Surrogate	DiD $-$ 0.040	$<0.001$	Yes

Table 6: MIMIC-IV incident-user pharmacological association results (

N

= 500 patients per comparison). DiD = difference-in-differences in mean outcome quintile;

\Delta

= difference in mortality rate. All tests are patient-level Wilcoxon signed-rank. Direction assessed against published trial evidence.

Dataset	Correct (non-null)	Significant	Expected-null	Total	Method
MIMIC-IV (primary)	6/10	9/10	—	10	Incident-user v2
INSPECT	5/7	0/7	1/1	8	Standard
Open-data total	11/17	9/17	1/1	18

Table 7: Unique pharmacological comparisons by dataset. Each comparison counted once. Correct = direction matches published trial evidence. The incident-user design on MIMIC-IV is the primary analysis.

Dataset	Method	Correct	Significant	Expected-null	Total
Dev (Level 1)	Standard	2/3	2/3	2/2	5
Dev (Level 2)	Standard	3/3	0/3	2/2	5
MIMIC-IV class	Standard	6/7	2/7	—	7
MIMIC-IV drug	Head-to-head	5/8	0/8	—	8

Table 8: Method-specific MIMIC-IV analyses (partial overlap with Table 7 ). Some drug–outcome pairs are evaluated by multiple methods. Development-set results shown for architectural context only.

Comparison	Type	DiD	95% CI	Wilcoxon $p$	Dir.	Evidence
Warfarin $\to$ INR	Surr.	+0.052	[+0.038, +0.066]	$<0.001$	Correct	Vit. K inhibition
Steroid $\to$ glucose	Surr.	+0.119	[+0.104, +0.133]	$<0.001$	Correct	Glucocorticoid effect
Furosemide vs spiro $\to$ K	Surr.	$-$ 0.061	[ $-$ 0.071, $-$ 0.050]	$<0.001$	Correct	Loop vs K-sparing
Diuretic $\to$ K	Surr.	$-$ 0.040	[ $-$ 0.050, $-$ 0.030]	$<0.001$	Correct	Loop hypokalemia
Vasopressor $\to$ mortality	Mort.	+0.069	[+0.062, +0.076]	$<0.001$	Correct a	Confounding by indication
Statin sepsis $\to$ mortality	Mort.	$-$ 0.004	[ $-$ 0.005, $-$ 0.002]	0.0001	Correct b	SAILS near-null

Table 9

Comparison	Trial emulated	OR	$p$	Direction
Vasopressin vs norepinephrine	VASST	0.986	0.70	Correct (null)
Atorvastatin vs pravastatin	PROVE-IT	0.965	—	Correct
Carvedilol vs metoprolol	COMET	0.916	0.094	Correct
Apixaban vs warfarin	ARISTOTLE	—	—	—

Table 10

Comparison	Type	OR	$p$	Direction
Anticoagulant $\to$ INR	Surrogate	1.080	—	Correct
ACE inhibitor $\to$ potassium	Surrogate	1.020	—	Correct
Diuretic $\to$ potassium	Surrogate	—	—	Correct
Insulin $\to$ glucose	Surrogate	—	—	Incorrect
Statin $\to$ mortality	Mortality	—	—	Correct
Metformin vs SU $\to$ glucose	Null	—	—	Correct (null)

Table 11

Comparison	Stage	OR	$p$	Direction
GLP-1RA vs SU	Level 2 (pre-DPO)	0.98	—	Correct
GLP-1RA vs SU	Level 3 (post-DPO)	2.03	$<0.001$	Incorrect
Insulin vs SU	Level 2 (pre-DPO)	0.92	0.012	Correct
Insulin vs SU	Level 3 (post-DPO)	—	—	Incorrect
SGLT2i vs SU	Level 2 (pre-DPO)	0.96	—	Correct
SGLT2i vs SU	Level 3 (post-DPO)	—	—	Incorrect

Table 12

Supplementary Figure S1a: MIMIC-IV timing calibration.

Supplementary Figure S1b: INSPECT timing calibration.

Autoregressive Generation

Autoregressive generation creates structured outputs step by step, conditioning each prediction on what has already been generated.

See all 109 papers

Sequence Composition in EHR Foundation Model Pretraining
Oct 12, 2026Joshua Placidi, Inigo Stibbe, Alfred J. Balston +2EHR Foundation Models Electronic Health Records
Department of Computing, Imperial College London
Prior-Fitted Functional Flow: In-Context Generative Models for Pharmacokinetics
Apr 19, 2026César Ojeda, Niklas Hartung, Wilhelm Huisinga +6Probabilistic Time Series Forecasting Healthcare
University of Potsdam, Germany · Technical University Berlin, Germany · Freie Universität Berlin, Germany +2
DRIFT: Direct-Recursive Intervention-Conditioned Forecasting of ICU Physiological Trajectories
Jul 28, 2026Weixin Liu, Juming Xiong, Congning Ni +4Multivariate Time Series Forecasting Time Series Forecasting
Vanderbilt University, Nashville, Tennessee, USA · Vanderbilt University Medical Center, Nashville, Tennessee, USA

Oct 12, 2026J/K move · Enter open · S save

Sequence Composition in EHR Foundation Model Pretraining

Joshua Placidi, Inigo Stibbe, Alfred J. Balston, Marek Rei+1

Department of Computing, Imperial College London

EHR Foundation Models Electronic Health Records Longitudinal EHR Modeling Self-Supervised Pre-Training Clinical Outcome Prediction

Open paper AskPDF arXiv

Autoregressive electronic health record (EHR) foundation models predict the next event token in patient trajectories: sequences of structured medical events. A typical pretraining strategy is to concatenate all trajectories into a global stream and sample token windows from it. This gives patients with longer sequences more training exposure and allows tokens to attend across patient boundaries. We argue that these choices create a misalignment between pretraining and downstream clinical tasks. To explore this, we investigate how the composition of pretraining sequences affects clinical prediction performance. Using the global-stream approach as a baseline, we propose alternative sequence composition methods that control how patient trajectories contribute to pretraining sequences. Across four MIMIC-IV v2.2 tasks, our best method improves macro AUROC, AUPRC, F1, and balanced accuracy, with relative gains of 9.3% in F1 and 2.7% in AUPRC under a fixed training recipe. We find that these gains generalize across four additional evaluation settings, including the INSPIRE and eICU-CRD datasets, with improvements in macro AUROC, AUPRC, and F1 score. Our results show that simply changing how EHR data are presented to the model during pretraining can improve downstream performance, without changing the model or collecting additional data.

Abstract

Autoregressive Generation

Explore similar work

Sequence Composition in EHR Foundation Model Pretraining

Prior-Fitted Functional Flow: In-Context Generative Models for Pharmacokinetics

DRIFT: Direct-Recursive Intervention-Conditioned Forecasting of ICU Physiological Trajectories