When2What: A Timing-Conditioned Framework for Long-Horizon Forecasting of Continuous Cardiovascular Waveforms
Organizations: Department of Computer Science, Luddy School of Informatics, Computing, and Engineering, Indiana University Bloomington, Bloomington, IN, USA · Department of Anesthesia, Indiana University School of Medicine, Indianapolis, IN, USA · Department of Intelligent Systems Engineering, Luddy School of Informatics, Computing, and Engineering, Indiana University Bloomington, Bloomington, IN, USA · Department of Critical Care Medicine, Xiangya Hospital, Central South University, Changsha, Hunan, China
Abstract
Long-horizon forecasting of quasi-periodic cardiovascular waveforms, such as the electrocardiogram (ECG), arterial blood pressure (ABP), and photoplethysmogram (PPG), requires predicting both when future cardiac events will occur (beat timing) and what the local structural pattern looks like around each event (beat morphology). However, standard sequence forecasters trained with pointwise mean squared error (MSE) average across uncertain future heartbeat phases, flattening sharp R peaks and pulse upstrokes into featureless baselines. Our analysis of intraoperative ECG reveals a clear temporal asymmetry: beat-time predictability from history decays within seconds due to stochastic heart-rate variability, whereas local beat morphology aligned at true beat times remains stable over several minutes. Guided by this insight, we introduce When2What, a timing-conditioned framework comprising three stages: Predict, Commit, and Generate. Given a 120-s history, When2What predicts a dense 60-s timing map, selects and sharpens discrete beat events into unit-height Gaussian impulses, and conditions a neural waveform generator on both these committed events and the observed history. Across intraoperative ECG, When2What substantially outperforms existing time-series and physiological forecasters, achieving a near-term waveform event skill of 0.644 and reproducing 95.62% of its predicted heartbeats in the generated waveform where direct long-horizon forecasters collapse into flat baselines. Furthermore, When2What transfers zero-shot to external hospital and atrial fibrillation cohorts, extends effectively to ABP and PPG pulse waveforms, and is supported by ablations and synthetic analyses that establish explicit timing conditioning as an effective paradigm for quasi-periodic long-horizon waveform forecasting.
Figures & tables
| Signal | Dataset | Reference | Set | Before screen | Evaluable |
| ECG | VitalDB (intraoperative) | Single-lead R peaks | Training | 3,383 patients 10,722 archived h | |
| Fused R peaks | Validation | 2,731 / 911 | 2,425 / 888 | ||
| Test | 2,577 / 859 | 2,286 / 845 | |||
| Lockbox | 2,575 / 860 | 2,267 / 832 | |||
| ABP | VitalDB | Pulse anchors | Test | – | 1,118 / 438 |
| PPG | VitalDB | Pulse anchors | Test | – | 1,860 / 792 |
| Method | Evaluation metric | Baseline score | When2What gain ( ) |
| Long-horizon waveform forecasting: 120 s history 60 s forecast (waveform channel) | |||
| Informer [ 13 ] | Event skill (0–10 s) | 0.069 | 0.713 |
| HTF [ 15 ] | Event skill (0–10 s) | 0.072 | 0.716 |
| BeatLex [ 10 ] | Event skill (0–10 s) | 0.152 | 0.491 |
| DLinear [ 14 ] (100 Hz) | Event skill (0–10 s) | 0.061 | 0.704 |
| PatchTST [ 5 ] (100 Hz) | Event skill (0–10 s) | 0.096 | 0.739 |
| Model variant | Timing loss on | Generator timing input (training) | Generator timing input (inference) |
| Direct | – | zero map | zero map |
| Direct-MT | zero map | zero map | |
| When2What (Ours) | ground-truth map | committed map |
| Ablation variant | Event skill | Waveform skill | Skill vs. Ours | Waveform MSE |
| Conditioning path ablations (original waveform loss) | ||||
| Direct (w/o , , and ) | – | 0.537 | 0.106 | 0.959 |
| Direct-MT (with , w/o ) | 0.670 | 0.533 | 0.110 | 0.957 |
| When2What (full: ) | 0.672 | 0.644 | Ref. | 1.687 |
| Component ablations on Commit ( ), Predictor ( ), and Generator ( ) | ||||
| w/o Commit (raw map ) | 0.672 | 0.441 | 0.203 | – |
| Experimental setting | Cohort | Direct-MT | When2What (Ours) | Gain ( ) |
| Waveform loss | ||||
| Original loss | Test | 0.533 | 0.644 | 0.110 |
| Split-MSE loss | Test | 0.560 | 0.627 | 0.067 |
| Split-MSE loss (tuned) | Test | 0.557 | 0.631 | 0.074 |
| Backbone architecture | ||||
| TCN backbone, original loss | Test | 0.534 | 0.631 | 0.097 |
| Signal | Metric (0–10 s) | Baseline | When2What (Ours) | Gain ( ) |
| PPG | Waveform event (vs. Direct) | 0.671 | 0.686 | 0.015 |
| PPG | Event skill (vs. simple RR rule) | 0.621 | 0.666 | 0.045 |
| ABP | Waveform event (vs. Direct) | 0.681 | 0.719 | 0.038 |
| ABP | Event skill (vs. simple RR rule) | 0.660 | 0.679 | 0.019 |
| Cohort and physiological stratum | Predictor | Fitted (s) | 95% CI (s) |
| VitalDB ECG test cohort (all windows) | Simple RR extrapolation | 7.26 | [6.85, 7.70] |
| VitalDB ECG test cohort (all windows) | When2What timing head ( ) | 7.36 | [7.02, 7.70] |
| VitalDB ECG Lockbox cohort (all windows) | Simple RR extrapolation | 6.89 | [6.49, 7.32] |
| VitalDB ECG test, lowest RMSSD quintile (most regular) | When2What timing head ( ) | 8.84 | [8.16, 9.60] |
| VitalDB ECG test, highest RMSSD quintile (most variable) | When2What timing head ( ) | 6.46 | [5.83, 7.13] |
| External NSRDB (ambulatory awake sinus rhythm) | When2What timing head ( ) | 1.86 | [1.56, 2.17] |
| Component | Specification |
| Input and output | 120-s history ( ), 60-s horizon ( ) at 500 Hz, z-scored per window |
| Autoencoder ( ) | Convolutional autoencoder, strides (2-s tokens), channels 128 and 256 |
| Spectral embedding | Historical Fourier magnitudes below 10 Hz with a learned frequency gate |
| Sequence backbone | 4 layers, width 512, bidirectional Mamba blocks [ 30 ] with self-attention every second layer |
| Timing target | Unit-height Gaussian bumps, ms, truncated at and summed across events |
| Timing loss | Peak-weighted MSE ( ) plus soft overlap loss ( ) |
| Pipeline stage | Operation description | Median latency (ms) |
| Preprocessing | Window buffering and z-score normalization | 2.61 |
| History RR estimation | Causal R-peak detection on historical window | 1.93 |
| Predict ( ) | First network pass (encoder, Mamba–Attention backbone, timing head) | 2.23 |
| Commit ( ) | Local-threshold peak selection, refractory filtering, and Gaussian encoding | 8.90 |
| Subtotal: Predict ( ) + Commit ( ) event-prediction chain | 15.43 | |
| Generate ( ) | Second network pass (conditioned backbone, decoder, and 500-Hz fusion) | 4.43 |
| Hardware slice | Model | Batch 1 latency (ms) | Batch 1 memory (MB) | Batch 32 latency (ms) | Batch 32 memory (MB) |
| H200 MIG 1g.18gb | Direct | 6.0 | 404 | 76.7 | 678 |
| H200 MIG 1g.18gb | When2What (Ours) | 6.0 | 429 | 76.8 | 699 |
| H200 MIG 4g.71gb | Direct | 3.4 | 404 | 18.7 | 678 |
| H200 MIG 4g.71gb | When2What (Ours) | 3.3 | 429 | 18.7 | 699 |
| Cohort or annotated subset | Metric | Before screen | Included | Excluded |
| VitalDB ECG validation cohort | Windows / patients | 2,731 / 911 | 2,425 / 888 | 11.2% |
| VitalDB ECG test cohort | Windows / patients | 2,577 / 859 | 2,286 / 845 | 11.3% |
| VitalDB ECG Lockbox cohort | Windows / patients | 2,575 / 860 | 2,267 / 832 | 12.0% |
| External MOVER ECG cohort | Windows / patients | 1,694 / 577 | 1,641 / 574 | 3.1% |
| VitalDB test tachycardia rate | Share with HR bpm | – | 3.7% | 14.8% |
| VitalDB Arrhythmia DB [ 36 ] | AF or atrial flutter windows | – | 1,611 / 6,963 | 142 / 310 |
| Verification stratum | Inspected scope or prevalence | Observed error rate [95% CI] |
| Final-round newly sampled windows | 2,790 peaks (100% inter-annotator agreement) | 0.00% [0.00%, 0.132%] |
| All verification rounds combined | 7,126 peaks inspected (4 residual errors) | 0.056% |
| Beats without synchronous PPG adjudication | 9.5% of reference beats | 0.14% [0.00%, 0.80%] |
| Beats flagged as borderline by rule | 2.8% of reference beats | 0.91% [0.11%, 3.26%] |
| Beats at refinement window boundary | 0.045% of reference beats | – |
| Verification metric | PPG (PLETH) | ABP (ART) |
| Evaluable windows / patients | 1,860 / 792 | 1,118 / 438 |
| Unmatched ECG beats (missed anchor rate) [95% CI] | 1.2% [0.8%, 1.6%] | 1.0% [0.6%, 1.4%] |
| Unmatched pulse anchors (extra anchor rate) [95% CI] | 0.9% [0.6%, 1.3%] | 0.3% [0.1%, 0.6%] |
| Median R-to-anchor physiological delay (ms) | 714 | 212 |
| Interquartile range of R-to-anchor delay (ms) | 694–734 | 196–234 |
| Within-patient delay residual SD (ms) | 20 | 14 |
| Method and configuration | Metric | Baseline [95% CI] | When2What baseline [95% CI] |
| Raw waveform, 500 Hz, 120 s history 60 s forecast (waveform channel) | |||
| Informer [ 13 ] , primary (robust recipe) | Skill , 0–10 s | 0.069 [ 0.074, 0.065] | 0.713 [ 0.697, 0.729] |
| Informer [ 13 ] , sensitivity (original recipe) | Skill , 0–10 s | 0.060 [ 0.064, 0.057] | 0.704 [ 0.688, 0.719] |
| BeatLex [ 10 ] , primary | Skill , 0–10 s | 0.152 [0.139, 0.166] | 0.491 [ 0.474, 0.508] |
| HTF [ 15 ] , primary (original setting, 3 seeds) | Skill , 0–10 s | 0.072 [ 0.077, 0.067] | 0.716 [ 0.700, 0.731] |
| Raw waveform, 500 Hz, 10 s history 2 s or 10 s forecast (waveform channel) | |||
| Metric | Direct | Direct-MT | When2What (Ours) |
| Original waveform loss | |||
| Event-channel skill (0–10 s) | – | 0.670 [0.654, 0.685] | 0.672 [0.657, 0.688] |
| Waveform-channel skill (0–10 s) | 0.537 [0.521, 0.554] | 0.533 [0.517, 0.550] | 0.644 [0.628, 0.659] |
| Waveform MSE (0–60 s) | 0.959 [0.950, 0.969] | 0.957 [0.948, 0.966] | 1.687 [1.674, 1.701] |
| Split-MSE waveform loss | |||
| Event-channel skill (0–10 s) | – | 0.669 [0.654, 0.685] | 0.655 [0.640, 0.670] |
| Metric | Direct | Direct-MT | When2What (Ours) |
| Original waveform loss | |||
| Event-channel skill (0–10 s) | – | 0.677 [0.662, 0.691] | 0.682 [0.667, 0.695] |
| Waveform-channel skill (0–10 s) | 0.544 [0.530, 0.559] | 0.541 [0.526, 0.555] | 0.651 [0.637, 0.665] |
| Waveform MSE (0–60 s) | 0.964 [0.953, 0.976] | 0.962 [0.951, 0.974] | 1.696 [1.681, 1.711] |
| Split-MSE waveform loss | |||
| Event-channel skill (0–10 s) | – | 0.676 [0.662, 0.690] | 0.666 [0.652, 0.679] |
| Ablation or diagnostic comparison | Metric | Estimate [95% CI] |
| Conditioning path and Commit ( ) ablations (vs. full When2What) | ||
| Direct (w/o , , and ) When2What | Waveform skill (0–10 s) | 0.106 [ 0.114, 0.099] |
| Direct-MT (with , w/o ) When2What | Waveform skill (0–10 s) | 0.110 [ 0.117, 0.102] |
| w/o Commit (raw map ) When2What | Waveform skill (0–10 s) | 0.203 [ 0.224, 0.184] |
| w/o Commit (raw map ) When2What | Per-beat RMSE (mV) | 0.114 [ 0.107, 0.120] |
| Direct waveform peaks When2What | Waveform skill (0–10 s) | 0.148 [ 0.159, 0.136] |
| Model variant | s | s | s | s | s | s | s | Overall |
| Waveform-channel event skill ( ms; Overall = 0–10 s primary mean) | ||||||||
| Direct | 0.795 | 0.629 | 0.187 | 0.068 | 0.088 | 0.088 | 0.088 | 0.537 |
| Direct-MT | 0.791 | 0.627 | 0.183 | 0.072 | 0.078 | 0.066 | 0.059 | 0.533 |
| When2What timing Template | 0.806 | 0.704 | 0.407 | 0.142 | 0.036 | 0.011 | 0.010 | 0.639 |
| When2What (Ours) | 0.810 | 0.708 | 0.412 | 0.152 | 0.045 | 0.016 | 0.011 | 0.644 |
| Samplewise waveform MSE in history-normalized units ( ; Overall = 0–60 s pooled) | ||||||||
| Model variant | s | s | s | s | s | s | s | Overall |
| Waveform-channel event skill ( ms; Overall = 0–10 s primary mean) | ||||||||
| Direct | 0.815 | 0.647 | 0.170 | 0.071 | 0.089 | 0.091 | 0.089 | 0.544 |
| Direct-MT | 0.814 | 0.642 | 0.167 | 0.078 | 0.078 | 0.069 | 0.063 | 0.541 |
| When2What timing Template | 0.827 | 0.718 | 0.396 | 0.130 | 0.030 | 0.007 | 0.009 | 0.647 |
| When2What (Ours) | 0.831 | 0.721 | 0.402 | 0.141 | 0.041 | 0.011 | 0.010 | 0.651 |
| Samplewise waveform MSE in history-normalized units ( ; Overall = 0–60 s pooled) | ||||||||
| Metric and tolerance | Channel | Direct-MT [95% CI] | When2What [95% CI] | When2What Direct-MT [95% CI] |
| Event skill ( ms) | Waveform | 0.428 [0.413, 0.443] | 0.476 [0.461, 0.491] | 0.049 [ 0.043, 0.054] |
| Event skill ( ms, primary) | Waveform | 0.533 [0.517, 0.550] | 0.644 [0.628, 0.659] | 0.110 [ 0.102, 0.117] |
| Event skill ( ms) | Waveform | 0.552 [0.534, 0.571] | 0.767 [0.752, 0.782] | 0.214 [ 0.203, 0.225] |
| Cumulative precision ( ms) | Waveform | 0.764 [0.750, 0.777] | 0.638 [0.623, 0.653] | – |
| Cumulative recall ( ms) | Waveform | 0.473 [0.458, 0.488] | 0.622 [0.608, 0.637] | – |
| Cumulative ( ms) | Waveform | 0.557 [0.543, 0.572] | 0.629 [0.614, 0.644] | – |
| Cohort | Model or paired contrast | Expression rate [95% CI] | Strict expression rate [95% CI] |
| Test | Direct-MT | 72.04% [70.83%, 73.38%] | 72.00% [70.79%, 73.34%] |
| Test | When2What (Ours) | 95.62% [95.38%, 95.88%] | 95.62% [95.38%, 95.88%] |
| Test | When2What Direct-MT ( ) | 0.2358 [ 0.2232, 0.2475] | 0.2362 [ 0.2235, 0.2480] |
| Lockbox | Direct-MT | 72.15% [70.93%, 73.40%] | 72.11% [70.89%, 73.36%] |
| Lockbox | When2What (Ours) | 95.49% [95.22%, 95.75%] | 95.49% [95.22%, 95.75%] |
| Lockbox | When2What Direct-MT ( ) | 0.2334 [ 0.2212, 0.2452] | 0.2338 [ 0.2216, 0.2458] |
| Cohort | Model | Unmatched head | Unmatched wave | ||||
| Test | Direct-MT | 12730.2 | 4941.0 | 356.0 | 9280.8 | 8902.4 | 3224.6 |
| Test | When2What (Ours) | 16992.2 | 778.4 | 111.0 | 9426.4 | 8889.0 | 9390.0 |
| Lockbox | Direct-MT | 12798.4 | 4939.6 | 335.4 | 9145.6 | 8942.0 | 3125.6 |
| Lockbox | When2What (Ours) | 17064.2 | 806.2 | 110.0 | 9238.6 | 8892.8 | 9358.0 |
| Cohort | Model | Head out | Head in | Wave out | Wave in | Zero-head windows | Zero-head patients |
| Test | Direct-MT | 11.6 | 11.4 | 2.0 | 3.2 | 54.4 | 2.0 |
| Test | When2What (Ours) | 10.6 | 13.0 | 11.0 | 13.0 | 52.0 | 2.0 |
| Lockbox | Direct-MT | 11.6 | 15.2 | 3.2 | 5.2 | 34.4 | 0.0 |
| Lockbox | When2What (Ours) | 11.8 | 15.4 | 12.0 | 15.6 | 32.8 | 0.0 |
| Experimental setting | Cohort (clustering unit) | Direct-MT | When2What (Ours) | Gain [95% CI] |
| Original loss | Test (845 patients) | 0.533 | 0.644 | 0.110 [ 0.102, 0.117] |
| Split-MSE loss | Test (845 patients) | 0.560 | 0.627 | 0.067 [ 0.061, 0.074] |
| Split-MSE loss (tuned) | Test (845 patients) | 0.557 | 0.631 | 0.074 [ 0.068, 0.081] |
| TCN backbone, original loss | Test (845 patients) | 0.534 | 0.631 | 0.097 [ 0.089, 0.104] |
| Original loss | Lockbox (832 patients) | 0.541 | 0.651 | 0.111 [ 0.103, 0.118] |
| Split-MSE loss | Lockbox (832 patients) | 0.567 | 0.637 | 0.069 [ 0.063, 0.076] |
| Cohort and stratum | Windows / clusters | Model comparison | Channel | Difference [95% CI] |
| VitalDB ECG Test | 2,286 / 845 | When2What Direct-MT | Waveform | 0.110 [ 0.102, 0.117] |
| VitalDB ECG Test | 2,286 / 845 | When2What Guarded cadence | Event | 0.032 [ 0.023, 0.041] |
| VitalDB ECG Lockbox | 2,267 / 832 | When2What Direct-MT | Waveform | 0.111 [ 0.103, 0.118] |
| VitalDB ECG Lockbox | 2,267 / 832 | When2What Guarded cadence | Event | 0.030 [ 0.021, 0.039] |
| External MOVER (all) | 1,641 / 574 | When2What Direct-MT | Waveform | 0.097 [ 0.089, 0.106] |
| External MOVER (all) | 1,641 / 574 | When2What Guarded cadence | Event | 0.0005 [ 0.0094, 0.0106] |
| Signal | Evaluation metric | Baseline | When2What (Ours) | Difference [95% CI] |
| PPG | Waveform-channel event (0–10 s, vs. Direct) | 0.671 | 0.686 | 0.015 [ 0.010, 0.020] |
| PPG | Event-channel skill (0–10 s, vs. simple RR rule) | 0.621 | 0.666 | 0.045 [ 0.035, 0.056] |
| PPG | Predictability time scale (s, vs. simple RR rule) | – | – | 0.20 [ 0.53, 0.21] |
| ABP | Waveform-channel event (0–10 s, vs. Direct) | 0.681 | 0.719 | 0.038 [ 0.033, 0.044] |
| ABP | Event-channel skill (0–10 s, vs. simple RR rule) | 0.660 | 0.679 | 0.019 [ 0.008, 0.032] |
| ABP | Predictability time scale (s, vs. simple RR rule) | – | – | 0.11 [ 0.58, 0.46] |
| Metric | NeuroKit2 (standard) | Shared threshold detector | Per-model threshold detector |
| on ground-truth ECG (validation) | 0.968 | 0.951 | – |
| on ground-truth ECG (test) | 0.970 [0.967, 0.973] | 0.947 [0.941, 0.953] | – |
| Conditioning gain, original loss (test) | 0.110 [ 0.102, 0.117] | 0.135 [ 0.123, 0.146] | 0.165 [ 0.154, 0.176] |
| Split-MSE / original gain ratio (test) | 0.61 [0.57, 0.65] | 0.48 [0.44, 0.52] | 0.57 [0.53, 0.60] |
| Timing uncertainty level | T1 | T2 | T3 | T4 |
| Per-beat RR innovation SD (ms) | 2 | 6 | 18 | 54 |
| Conditional beat-time SD over first 8 beats (ms) | 4 | 11 | 34 | 104 |
| Morphology variability level | M1 | M2 | M3 | M4 |
| Amplitude/width log-normal jitter | 0.02 | 0.06 | 0.15 | 0.35 |
| Per-beat template-switching probability | 0.00 | 0.01 | 0.04 | 0.12 |
| Grid cell | Event difference [95% CI] | Squared error difference [95% CI] |
| T1M1 | 0.009 [ 0.006, 0.011] | 0.0024 [ 0.0020, 0.0028] |
| T1M2 | 0.034 [ 0.028, 0.040] | 0.0006 [ 0.0001, 0.0011] |
| T1M3 | 0.014 [ 0.012, 0.017] | 0.0020 [ 0.0016, 0.0025] |
| T1M4 | 0.020 [ 0.018, 0.023] | 0.0015 [ 0.0010, 0.0019] |
| T2M1 | 0.139 [ 0.128, 0.150] | 0.0098 [ 0.0091, 0.0105] |
| T2M2 | 0.132 [ 0.122, 0.142] | 0.0089 [ 0.0081, 0.0098] |
| Grid cell | Event difference [95% CI] | Squared error difference [95% CI] |
| T1M1 | 0.002 [ 0.000, 0.005] | 0.0008 [ 0.0005, 0.0011] |
| T1M2 | 0.027 [ 0.022, 0.034] | 0.0015 [ 0.0020, 0.0010] |
| T1M3 | 0.012 [ 0.010, 0.015] | 0.0001 [ 0.0005, 0.0003] |
| T1M4 | 0.020 [ 0.018, 0.023] | 0.0006 [ 0.0010, 0.0002] |
| T2M1 | 0.045 [ 0.035, 0.055] | 0.0023 [ 0.0020, 0.0026] |
| T2M2 | 0.072 [ 0.062, 0.082] | 0.0018 [ 0.0013, 0.0022] |
| Simulation task and setting | Cohort | Relative reduction | 95% CI |
| Immediate next heartbeat ( ms tolerance, all beats) | VitalDB Test | 19.8% | [11.0%, 28.5%] |
| Immediate next heartbeat ( ms tolerance, non-ectopic) | VitalDB Test | 25.5% | [14.9%, 35.1%] |
| Immediate next heartbeat ( ms tolerance, ectopic beats) | VitalDB Test | 1.8% | [ 16.8%, 10.9%] |
| Immediate next heartbeat ( ms tolerance, all beats) | VitalDB Lockbox | 15.8% | [2.2%, 28.2%] |
| Streaming device trigger (50-ms delay, ms tolerance) | VitalDB Test | 19.6% | [12.9%, 26.2%] |
| Streaming device trigger (50-ms delay, ms tolerance) | VitalDB Lockbox | 21.8% | [16.3%, 27.7%] |
Time Series Forecasting
Time series forecasting predicts future values from ordered observations using statistical, neural, and probabilistic models to support decisions under uncertainty.
See all 930 papers