Objective: To distinguish the transferable predictive value of physiological trajectories from that of observation patterns in ICU mortality prediction, and to assess whether measurement timing affects performance independently of observation quantity. Materials and Methods: In-hospital mortality was predicted at 48 hours using MIMIC-III, MIMIC-IV, and eICU. Nested representations progressively integrated latest-value snapshots (R1), longitudinal physiological summaries (R2), and measurement frequency and timing features (R3). Models fitted on 7,762 MetaVision patients were evaluated on 21,682 MIMIC-IV patients and 43,878 eICU patients across 204 hospitals. The effect of measurement timing was assessed while holding the number of retained observations constant. Results: Physiological trajectories improved external prediction in eICU (R2-R1 log-loss difference -0.00922, 95% CI -0.01148 to -0.00704). Observation-pattern features improved locally developed eICU models (R3-R2 -0.00257, 95% CI -0.00370 to -0.00146), but not clearly after transfer from MetaVision (-0.00012, 95% CI -0.00206 to +0.00208). R3 was favoured in 10 of 30 source bootstrap refits. At matched observation counts, retaining measurements spread throughout the window reduced log loss relative to random retention by 0.03280 for R3 versus 0.00703 for R2. Removing the final 12 hours increased log loss relative to count-matched random removal by 0.01947 for R3 versus 0.00822 for R2. Under reduced availability, the local R3 increment changed from beneficial to harmful. Discussion: Physiological trajectories transferred more consistently, while observation-pattern features were particularly sensitive to the timing and availability of measurements. Conclusion: Clinical prediction models should be evaluated not only in new patient populations but also under the measurement conditions that supply their inputs.
Figures & tables
Cohort
Patients
Deaths (%)
Age
Female
Hospitals
CareVue
12,518
1,733 (13.8)
67 (53–78)
5,488
1
MetaVision
7,762
1,005 (12.9)
66 (54–79)
3,485
1
MIMIC-IV
21,682
2,978 (13.7)
67 (56–77)
9,387
1
eICU
43,878
5,393 (12.3)
66 (55–77)
19,909
204
Table 1: Eligible MIMIC cohorts and the eICU evaluation population. Age is reported as median (interquartile range) in years. CareVue, MetaVision, and MIMIC-IV relate to the same institution. Seven eICU test records have sex outside the female/male categories shown. The separate 10,909-patient adaptation population is not included.
Tier
Interpretation
MIMIC
Shared
Information added
R0
Baseline
10
10
Static inputs and latest values of six physiological variables.
R1
Snapshot
43
42
Additional scalar values, urine output, and ventilation indicators.
R2
Trajectory
335
334
Physiological severity, trend, and variability summaries.
R3
Observation patterns
524
514
Measurement frequency, timing, and observation-pattern summaries.
R4
Rich summary
940
918
Additional physiological and observation summaries.
Table 2: Nested information representations. Feature counts are reported before categorical variables are expanded during encoding. Shared counts indicate the subset retained after harmonisation between MetaVision and eICU.
Cohort
Model
Log loss
AUROC
Brier
Intercept
Slope
MIMIC-IV
R1
0.29589
0.85140
0.08970
0.298
1.196
R2
0.28291
0.87258
0.08659
0.546
1.272
R3
0.27519
0.88026
0.08431
0.504
1.265
eICU
R1
0.30116
0.80666
0.08895
-0.089
1.055
R2
0.29194
0.82397
0.08616
0.043
1.141
R3
0.29182
0.82292
0.08670
0.047
1.078
Table 3: External performance of MetaVision-trained models in MIMIC-IV (n = 21,682) and eICU (n = 43,878). Results use uncalibrated predictions, with negative paired log-loss differences favouring the richer representation. Unadjusted 95% CIs use patient bootstrapping for MIMIC-IV and hospital bootstrapping for eICU.
Figure 1: Incremental value of physiological trajectories (R2–R1; A) and observation patterns (R3–R2; B) across evaluation settings. Points show patient-weighted log-loss differences with 95% paired-bootstrap CIs, using patient resampling for MIMIC and hospital resampling for eICU. Negative differences favour the richer representation.
Figure 2: External eICU effects of observation timing at matched observation counts. A: most recent observations versus random retention. B: observations spread through the window versus random retention. C: final 12-hour outage versus matched random removal. Negative differences favour recent or spread retention, while positive outage differences indicate worse performance. Points show patient-weighted log-loss differences averaged across restriction iterations, with 95% hospital-bootstrap CIs.
Figure 3: Local eICU effects of changing observation availability. A: R2–R1 and R3–R2 differences when models trained on complete records are evaluated on complete or restricted records. B: R3–R2 differences before and after restriction-matched recalibration using a separate calibration cohort. Points show patient-weighted log-loss differences averaged across allocations, with 95% hospital-bootstrap CIs.