Block Diffusion for Longitudinal EHR: Accuracy, Efficiency, and Representation Trade-offs

Ashvin Gupta; Joshua Placidi; Brendan Delaney; Alessandra Russo

cs.LGOct 21, 2026

Block Diffusion for Longitudinal EHR: Accuracy, Efficiency, and Representation Trade-offs

Authors: Ashvin Gupta, Joshua Placidi, Brendan Delaney, Alessandra Russo

Organizations: Imperial College London

Abstract

Autoregressive (AR) models have proved effective for zero-shot prediction from longitudinal electronic health records, but impose strictly sequential generation on trajectories containing temporally clustered and interdependent clinical events. We investigate block-based masked diffusion (BD) as an alternative on MIMIC-IV under matched tokenisation, context length, model capacity, and evaluation conditions. Across block sizes, BD exhibits a clear quality--efficiency trade-off, with small-block configurations retaining much of the predictive performance of AR while enabling substantially faster inference. A representative $B{=}8$ configuration achieves approximately $1.43\times$ faster evaluation. Although generative rollout remains below AR in most settings, frozen linear probes on BD and AR representations achieve near-identical performance across clinical prediction tasks, indicating that BD learns comparably informative clinical representations. Increasing the denoising budget yields diminishing returns, while alternative decoding strategies provide limited gains. Overall, our results show that block diffusion is a viable and computationally attractive approach to longitudinal EHR modelling, with further improvements in iterative decoding offering a promising route toward closing the remaining generative performance gap.

Figures & tables

Panel of Figure 1 — Figure 1: Quality, efficiency, and representation trade-offs of block diffusion. Top-left: BD–AR AUROC difference under rollout and linear probing. Top-right: AUROC across denoising budgets. Bottom: ICU mortality AUROC and evaluation time across block sizes at $S/B=0.5$ . Dashed lines denote AR.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 2: Illustration of AR and block-diffusion decoding for longitudinal EHR trajectories. Clinical events frequently occur in temporally clustered groups. AR generates each event sequentially, whereas BD preserves causal generation across blocks while jointly denoising multiple positions within the active block.

Figure 3: Clinical event multiplicity in MIMIC-IV. Survival distribution of the number of non-time-gap event tokens sharing a recorded timestamp in the ETHOS representation.

Statistic	Value
Number of timestamps	23 274 367
Mean events per timestamp	16.16
Median	6
25th / 75th percentile	3 / 20
$P(N>1)$	96.8%
$P(N\geq 4)$	67.6%

Appendix

Table 1: Multiplicity of clinical event tokens sharing a timestamp in the MIMIC-IV / ETHOS training set.

N

is the number of non-time-gap event tokens at a timestamp.

	AR	BD ( $B{=}8$ , $S{=}4$ )
AUROC	$0.844$ $[0.831,0.858]$	$0.883$ $[0.873,0.892]$
AUPRC	$0.306$ $[0.276,0.335]$	$0.204$ $[0.183,0.227]$
Precision@100	$0.74$	$0.47$
Mean score (deaths / survivors)	$0.242$ / $0.015$	$0.353$ / $0.090$
Score $=0$ (deaths / survivors)	$23.7\%$ / $86.7\%$	$2.5\%$ / $39.1\%$

Appendix

Table 2: Hospital-mortality rollout scores on the test set. AUROC/AUPRC brackets are

95\%

instance-bootstrap CIs. Score statistics are Monte Carlo

P(\texttt{MEDS\_DEATH})

over

20

samples.

Figure 4: Predictive performance versus evaluation speed on ICU mortality. Each point denotes a block-diffusion configuration with a particular block size and denoising budget. The x-axis shows evaluation speedup relative to the autoregressive (AR) baseline, while the y-axis shows AUROC. The highlighted $B=8$ , $S=4$ configuration provides a representative operating point, achieving approximately $1.43\times$ faster evaluation while retaining much of the predictive performance of AR.

Panel of Figure 5 — Figure 5: Left: Difference in AUPRC between block diffusion and AR under generative rollout and frozen linear probing across four clinical tasks. Middle: ICU mortality AUPRC as block size varies under the fixed normalised denoising budget $S/B=0.5$ . Dashed lines denote the AR baseline. Shaded regions and error bars indicate 95% bootstrap confidence intervals. Right: ICU mortality AUPRC as the normalized denoising budget $S/B$ increases for fixed block sizes.

Strategy	AUROC	AUPRC
Random	0.8476 [0.8331, 0.8614]	0.3440 [0.3123, 0.3792]
ReMDM $\eta=0.05$	0.8421 [0.8268, 0.8562]	0.3338 [0.3028, 0.3679]
ReMDM $\eta=0.10$	0.8446 [0.8307, 0.8590]	0.3515 [0.3176, 0.3871]
ReMDM $\eta=0.20$	0.8433 [0.8287, 0.8580]	0.3474 [0.3158, 0.3800]
Confidence	0.8089 [0.7921, 0.8257]	0.3487 [0.3154, 0.3844]

Appendix

Table 3: ICU mortality decoding strategies for block size

B{=}8

and

S{=}4

diffusion steps. Metrics are medians over completed full-validation runs where available; Confidence uses the available full split run. Brackets are

95\%

CIs over the three random seeds.

Longitudinal EHR Modeling

Longitudinal electronic health record modeling learns from patient histories over time to forecast outcomes, characterize disease trajectories, and guide care.

See all 158 papers

BlockGen: Flexible Blockwise Sequence Modeling with Hybrid Samplers
Jun 1, 2026Justin Deschenaux, Caglar GulcehreMasked Diffusion Models Diffusion Sampling
EPFL Lausanne, Switzerland · EPFL, Lausanne, Switzerland · Microsoft AI
Rethinking the Generation Order of Block Diffusion Language Models
Jul 27, 2026Kai Syun Hou, James KwokDiffusion Model Sampling Blockwise Diffusion Language Models
Department of Computer Science and Engineering The Hong Kong University of Science and Technology
Are Joint-Embedding Predictive Objectives Effective for Longitudinal EHRs?
Oct 21, 2026Ashvin Gupta, Brendan Delaney, Alessandra RussoRepresentation Learning Joint-Embedding Predictive Architecture
Imperial College London

Jun 1, 2026J/K move · Enter open · S save

BlockGen: Flexible Blockwise Sequence Modeling with Hybrid Samplers

Justin Deschenaux, Caglar Gulcehre

EPFL Lausanne, Switzerland · EPFL, Lausanne, Switzerland · Microsoft AI

Masked Diffusion Models Diffusion Sampling Blockwise Diffusion Language Models Few-Step Diffusion Sampling Diffusion Language Model Decoding Diffusion Language Models+2

Open paper AskPDF arXiv

Is the uniform-state diffusion framework a more powerful paradigm for discrete diffusion? Recent studies indicate that this may be the case. In combination with predictor-corrector samplers, uniform-state diffusion models (USDMs) produce samples of higher-quality than masked diffusion models (MDMs), and USDMs equal or outperform MDMs in downstream tasks, even though they exhibit greater perplexity. Two issues remain unresolved. First, existing work compares uniform and masked diffusion with un-informed correctors that re-inject noise at random positions, rather than targeting tokens most likely to be wrong. Second, prior work compares full-sequence diffusion models, so we do not know whether the same conclusion holds when tokens are generated block by block. To address these issues, we introduce BlockGen, a blockwise sequence model that we instantiate with both masked and uniform diffusion. BlockGen trains on a mixture of block sizes and its likelihood interpolates between AR and pure diffusion more finely than models with a fixed block size. BlockGen enables AR-informed predictor-corrector sampling (ARPC), which combines AR and diffusion predictions to re-generate unlikely tokens without an auxiliary verifier. Under ancestral sampling, uniform outperforms masked in the block-by-block setting, especially in the few-step regime. Under ARPC, the gap closes and reverses at high NFE. With block size

16

on GSM8K, MDMs reach slightly higher accuracy than USDMs, and we observe a similar trend in Generative Perplexity on OpenWebText. Find our code at https://github.com/jdeschena/blockgen.

Abstract

Longitudinal EHR Modeling

Explore similar work

BlockGen: Flexible Blockwise Sequence Modeling with Hybrid Samplers

Rethinking the Generation Order of Block Diffusion Language Models

Are Joint-Embedding Predictive Objectives Effective for Longitudinal EHRs?