Preprocessing is an under-measured reliability risk in clinical prediction: a model can keep a similar AUROC and still change who receives an alert. Imputation quality is commonly evaluated using reconstruction error or aggregate discrimination, neither of which directly measures agreement on which patients are flagged. At a matched alert rate, pipelines with similar aggregate performance can produce different patient lists. This question is especially relevant under the frequent, structured missingness of real ICU records. We therefore introduce alert-list concordance, which asks whether two pipelines identify the same high-risk patients, measured by the Jaccard similarity of their top-k% alert sets with patient-level bootstrap confidence intervals. On PhysioNet/CinC 2019 (40,336 patients, 71.9% missingness), we compare nine imputation strategies and a native-missingness baseline using a fixed XGBoost configuration, fitted and Platt-calibrated separately for each strategy. After Bonferroni correction, 22 of the 28 reconstruction-method pairs showed no statistically significant difference in patient-level AUROC, yet their alert lists overlap at Jaccard 0.518 to 0.750. Similar aggregate discrimination can therefore accompany different patient selections. ForwardFill is a separate outlier, pairing higher discrimination with the lowest overlap in the matrix, 0.40 to 0.45. The broad pattern holds within each hospital system and under six patient-level aggregation rules. At the row level, eight of ten methods fall within an AUROC range of 0.011. In this row-level pipeline, deep imputers did not outperform mean imputation on discrimination, and the sequential implementations cost 5 to 12 times mean imputation in total pipeline wall-clock time on our hardware.
Figures & tables
Method
AUROC
AUPRC
Brier
ECE
Time (min)
ForwardFill
0.787
0.068
0.01718
0.00172
0.69
XGBoostNative ‡
0.744
0.059
0.01728
0.00071
90.13
Mean
0.733
0.054
0.01733
0.00064
0.62
SAITS
0.732
0.052
0.01734
0.00048
7.54
BRITS
0.732
0.052
0.01733
0.00032
3.18
DAE
0.730
0.051
0.01735
0.00047
1.63
TABLE I: Row-level aggregate metrics on the held-out test set, sorted by AUROC, with total pipeline time. ECE is post-Platt.
Fig. 1: Pairwise Jaccard of top-10% alert lists, lower triangle (the matrix is symmetric). Rules separate the three tiers: ForwardFill, the native-handling baseline, and the eight reconstruction imputers.
Pooled, k (%)
k=10% by site
Method pair
5
8
10
12
15
20
A
B
DAE-MIDA
0.679
0.741
0.750
0.785
0.787
0.808
0.707
0.744
Mean-XGBNative
0.584
0.651
0.670
0.703
0.703
0.750
0.645
0.683
BRITS-Mean
0.554
0.607
0.606
0.646
0.656
0.700
0.602
0.593
BRITS-SAITS
0.540
0.607
0.631
0.633
0.649
0.679
0.569
0.610
GRU-D-MICE
0.538
0.571
0.605
0.628
0.658
0.678
0.545
0.589
TABLE II: Jaccard for ten representative pairs across alert rates (pooled), and at k=10% within each hospital system with thresholds set per site (A: n=6,118 ; B: n=5,983 ).
Fig. 2: Hourly calibrated risk in four only-ForwardFill encounters: septic patients ForwardFill flags at k=10% while SAITS and MICE do not. Dotted line: first Sepsis-3 labeled hour. Shaded band: the flagged region. Panels share a vertical scale.
k=5%
k=10%
k=15%
k=20%
Method
Prec
Rec
Prec
Rec
Prec
Rec
Prec
Rec
ForwardFill
0.371
0.256
0.297
0.409
0.253
0.522
0.225
0.619
XGBoostNative ‡
0.347
0.239
0.298
0.410
0.249
0.515
0.224
0.616
Mean
0.340
0.234
0.291
0.400
0.247
0.510
0.220
0.605
SAITS
0.328
0.226
0.285
0.392
0.248
0.512
0.220
0.605
BRITS
0.314
0.216
0.273
0.375
0.235
0.484
0.210
0.578
TABLE III: Patient-level precision@ k and recall@ k on the held-out test set ( n=12,101 patients, 880 ever-septic).
Method
AUROC
95% CI
AUPRC
Site A
Site B
ForwardFill
0.816
0.803–0.830
0.292
0.801
0.830
XGBoostNative ‡
0.812
0.798–0.826
0.274
0.801
0.818
Mean
0.804
0.790–0.819
0.252
0.788
0.814
SAITS
0.802
0.788–0.817
0.252
0.791
0.809
BRITS
0.797
0.784–0.812
0.241
0.780
0.810
MICE
0.797
0.782–0.812
0.239
0.785
0.809
TABLE IV: Patient-level discrimination on the maximum calibrated risk: pooled AUROC with 1,000-resample stratified bootstrap 95% CI, AUPRC, and AUROC within each hospital system.
Rule
FF vs others
Other pairs (med.)
LOS FF/MIDA (h)
AUROC recon.
Max over stay
0.40–0.45
0.47–0.75 (0.55)
55 / 43
0.786–0.804
Mean of top 3 h
0.42–0.46
0.53–0.86 (0.63)
47 / 46
0.788–0.800
90th percentile
0.39–0.43
0.53–0.84 (0.65)
65 / 23
0.790–0.803
Mean of last 6 h
0.36–0.40
0.59–0.88 (0.71)
61 / 22
0.816–0.828
Max, first 24 h
0.40–0.46
0.45–0.75 (0.53)
33 / 39
0.740–0.763
Mean over stay
0.38–0.41
0.59–0.90 (0.73)
72 / 20
0.782–0.795
TABLE V: Sensitivity to the per-patient aggregation rule at k=10% . Jaccard ranges cover the nine FF pairs and the 36 pairs among the nine non-ForwardFill methods, including XGBoostNative; LOS is the median for septic patients flagged by only one method.
Group
n
ICU LOS (h)
Risk A
Risk B
Panel A: ForwardFill (A) vs. MIDA (B)
Both flagged
249
75.0 [30.0, 130.0] (91.1 ± 72.4)
0.152
0.129
Only ForwardFill
111
55.0 [27.0, 102.0] (76.1 ± 64.8)
0.132
0.078
Only MIDA
74
43.0 [18.0, 79.5] (58.3 ± 54.7)
0.067
0.116
Neither
446
22.0 [11.0, 46.0] (37.7 ± 42.4)
0.040
0.054
Panel B: ForwardFill (A) vs. SAITS (B)
TABLE VI: Agreement groups among the 880 sepsis-positive test encounters at k=10% . ICU LOS in hours as median [IQR] (mean ± SD); risks are calibrated probabilities.
Clinical care often relies on key laboratory indicators, yet real-world patient visits are sparse and tests are ordered irregularly, leading to pervasive missingness. While many imputation methods improve average accuracy, they provide limited guidance on which imputed values are reliable enough for high-stakes downstream use. In this work, we study reliable clinical imputation, aiming to produce accurate imputations while selectively releasing the reliable results, with statistical control over clinically unacceptable errors. To achieve this goal, we propose SafeImpute, a reliable imputation framework for irregular and sparse clinical longitudinal records. SafeImpute constructs an event graph that captures both intra-patient temporal trajectories and inter-patient clinical similarity, and learns imputations with a two-relation GNN and adaptive fusion, regularized by an auxiliary masked reconstruction objective. For reliability guarantees, SafeImpute converts a proxy risk score into conformal p-values and applies the Benjamini--Hochberg procedure to control the false discovery rate (FDR) of unacceptable errors among released imputations at a user-specified tolerance. Experiments on our Mayo Clinic data, the public MIMIC-III and MIMIC-IV datasets show that SafeImpute achieves strong imputation accuracy while providing reliable error control, outperforming diverse baselines in both standard imputation evaluation and FDR-controlled selective-release evaluation.
Xinrui He, Mengting Ai, Junting Wang +2
University of Illinois Urbana-Champaign Champaign, IL, USA · Mayo Clinic Scottsdale, AZ, USA
Early sepsis prediction from electronic health records is challenged by irregular sampling, high missingness, and class imbalance. We systematically compare four modeling paradigms -- self-supervised Joint Embedding Predictive Architecture (JEPA) via masked latent prediction, self-supervised VICReg (variance-invariance-covariance regularization) with two-view augmentation, semi-supervised fine-tuning of a VICReg-pretrained encoder, and supervised Temporal Convolutional Network (TCN) -- alongside raw-feature baselines. All models share a common preprocessing pipeline of hourly binning with forward-fill imputation applied to 7 biomarkers selected via sparsity analysis from the MIMIC-III dataset. Our best model (JEPA + XGBoost + mean pooling) achieves AUPRC 0.636 at the time of onset (H0), approaching the SupMix benchmark (0.667) while using 83% fewer biomarkers. The Tier 1 pipeline -- VICReg pretraining followed by semi-supervised fine-tuning and XGBoost -- achieves AUPRC 0.510 at H0, a 3.1× improvement over the raw-feature baseline (0.165) and a 7.6% improvement over the end-to-end supervised TCN (0.474). Crucially, the fine-tuned VICReg encoder exhibits the most temporally persistent representations, degrading only 16.8% from H0 to H10 compared to 47.5% for supervised TCN and 65.3% for JEPA, demonstrating that self-supervised pretraining with task-aware fine-tuning yields features that are both sharp near onset and robust across prediction horizons.
Umair bin Mansoor, Munaf Rashid, Roomi Naqvi
Electrical Engg. Department, DHA Suffa University, Karachi, Pakistan · Salim Habib University, Karachi, Sindh, Pakistan · Intel Foundry Services, Intel, Tempe, Arizona, United States
Imputing physiological time series (arterial blood pressure, blood glucose, etc.) is essential for addressing the missingness that pervades clinical data. Yet modern imputation methods perform poorly in this domain: a recent benchmark found that simple linear interpolation outperformed every learned imputer on real-world clinical signals with realistic gaps. We show that this reflects two properties of physiological missingness that generic imputers ignore: gaps may occur when the signal is clinically extreme rather than typical, and gap lengths can easily span orders of magnitude. To this end, we introduce Curriculum-Aware Interpolate-then-Refine (CAIR), a two-stage framework for physiological time-series imputation. Our key motivation is to learn a coarse base curve and then repeatedly correct it toward physiological realism, rather than predict a gap in a single pass. Consequently, CAIR couples a bidirectional-GRU interpolator with a Transformer refiner that corrects its own estimate over three successive passes, trained jointly under a broad, signal-agnostic random-gap curriculum. We evaluate imputers stratified by gap length and missingness mechanism (MCAR, MAR, NMAR) rather than by a single average, and CAIR is the most accurate under every mechanism on continuous glucose monitoring (AI-READI) and arterial pressure in intensive care (MIMIC-III). Its margin over the strongest baseline grows with difficulty, from 9% under MCAR to 19% under value-dependent dropout, where generic learned imputers are weakest. We further show low reconstruction error alone does not recover the burden metrics clinicians act on: interpolants matching CAIR's error fail to preserve those metrics, imputers that recover them are far less accurate, and CAIR alone ranks among the best on both axes.
Yu-Chao Huang, Haochen Zhang, Nicholas Konz +1
UNITES Lab, University of North Carolina at Chapel Hill