NeoTriFuse: Reliability-Aware Multimodal Fusion under Missingness Heterogeneity for Neonatal Mortality Risk Prediction
Authors: Jiyuan Tian, Qincheng Shen, Ye Lin, Yu Gao, Haohui Lu
Organizations: Discipline of Business Analytics, The University of Sydney, Sydney, Australia · Molly Wardaguga Institute for First Nations Birth Rights, Charles Darwin University, Darwin, Australia
Neonatal mortality risk prediction from bedside monitoring data remains challenging due to extreme class imbalance, heterogeneous clinical risk factors, multi-scale temporal dynamics, and substantial missingness. We propose NeoTriFuse, a reliability-aware multimodal fusion framework for missingness-heterogeneous neonatal monitoring data. Unlike conventional multimodal approaches that treat missingness primarily as a preprocessing issue, NeoTriFuse models missingness as an explicit reliability signal that dynamically modulates modality contributions during fusion. The framework integrates static perinatal variables, local-global temporal encoders, and patient-level statistical summaries through reliability-guided gating mechanisms, while jointly optimizing mortality prediction and an auxiliary length-of-stay objective. NeoTriFuse achieves competitive performance, with an F1 score of 0.6736 +/- 0.0216 and an AUROC of 0.9454 +/- 0.0056. Ablation studies indicate that the local-global temporal architecture and patient-level summary branch contribute most substantially to predictive performance, while reliability-aware gating provides additional improvements on threshold-dependent metrics under heterogeneous observation completeness. Sensitivity analyses further suggest stable performance across nearby hyperparameter settings. Overall, the findings support reliability-aware multimodal fusion as a practical approach for neonatal mortality prediction under realistic clinical missingness conditions.
Figures & tables
Figure 1: Overview of NeoTriFuse for reliability-aware multimodal fusion under heterogeneous missingness. The model integrates temporal HR and SpO 2 sequences, static clinical variables, patient-level summary statistics, and a reliability vector derived from missingness and observation coverage. Local and global temporal encoders generate htemporal , while reliability-guided gates modulate temporal, static, and summary representations before fusion. The fused representation is used for mortality prediction with auxiliary length-of-stay prediction.
Model
F1
Precision
Recall
AUROC
TIM-MSFL
0.6570 ± 0.0126
0.7780 ± 0.0370
0.5888 ± 0.0170
0.9292 ± 0.0014
GRU
0.6446 ± 0.0117
0.7246 ± 0.0421
0.6147 ± 0.0245
0.9450 ± 0.0019
CNN
0.6348 ± 0.0120
0.7445 ± 0.0310
0.5916 ± 0.0183
0.9317 ± 0.0042
Transformer
0.6201 ± 0.0164
0.6397 ± 0.0314
0.6430 ± 0.0211
0.9316 ± 0.0037
TNformer-MP
0.6155 ± 0.0202
0.6314 ± 0.0357
0.6273 ± 0.0205
0.9303 ± 0.0039
GRU-D
0.5690 ± 0.0223
0.6442 ± 0.0438
0.5739 ± 0.0234
0.9250 ± 0.0030
Table 1: Comparison with traditional, neural, and missingness-aware baselines. Values are mean ± std over 10 seeds.
Figure 2: Heatmap of classification performance across baselines and the proposed model.
Variant
F1
Precision
Recall
AUROC
No Local CNN
0.6632 ± 0.0095
0.6961 ± 0.0319
0.6824 ± 0.0210
0.9450 ± 0.0038
No Global Transformer
0.6412 ± 0.0171
0.6633 ± 0.0381
0.6750 ± 0.0267
0.9292 ± 0.0078
No Tabular Summary
0.6470 ± 0.0115
0.6949 ± 0.0332
0.6633 ± 0.0232
0.9420 ± 0.0080
Concat Fusion
0.6679 ± 0.0090
0.7427 ± 0.0309
0.6539 ± 0.0213
0.9436 ± 0.0048
No Reliability Vector
0.6685 ± 0.0127
0.7157 ± 0.0513
0.6663 ± 0.0335
0.9473 ± 0.0052
No LOS Auxiliary
0.6641 ± 0.0156
0.7076 ± 0.0588
0.6735 ± 0.0309
0.9372 ± 0.0045
Table 2: Ablation study of the proposed model.
Config
F1
Precision
Recall
AUROC
hd128_los0.3
0.6690 ± 0.0162
0.7144 ± 0.0370
0.6778 ± 0.0233
0.9419 ± 0.0037
hd96_los0.2
0.6662 ± 0.0164
0.6999 ± 0.0403
0.6824 ± 0.0252
0.9439 ± 0.0060
hd128_los0.2
0.6658 ± 0.0144
0.6945 ± 0.0414
0.6785 ± 0.0329
0.9457 ± 0.0044
hd64_los0.3
0.6653 ± 0.0142
0.7218 ± 0.0573
0.6653 ± 0.0424
0.9409 ± 0.0058
hd96_los0.1
0.6629 ± 0.0204
0.7060 ± 0.0407
0.6711 ± 0.0307
0.9419 ± 0.0079
hd128_los0.1
0.6617 ± 0.0128
0.7112 ± 0.0336
0.6667 ± 0.0237
0.9443 ± 0.0049
Table 3: Ten-seed sensitivity analysis of hidden dimension and LOS loss weight.
Figure 3: Sensitivity of F1 and AUROC across hidden dimension and LOS loss weight settings.
Recent multimodal survival prediction models have demonstrated strong predictive performance by leveraging complementary information across modalities. However, such models generally assume data completeness and exhibit limited robustness toward missing modalities, which are frequently encountered in real-world clinical settings. We propose the Evidential Missing Modality Survival Fusion (EMMS) model for multimodal survival prediction under missing modalities. EMMS offers a straightforward, computationally effective approach to survival analysis without requiring a generative phase for missing data. By employing Dempster-Shafer theory and Gaussian Random Fuzzy Numbers for multimodal decision fusion, it considers both aleatoric and epistemic uncertainty alongside modality reliability for fusion. Moreover, the model treats missing modalities as vacuous evidence, preventing interference with available inputs and naturally reflecting increased uncertainty and calibrated predictions. Extensive experiments on four cancer datasets demonstrate state-of-the-art performance while providing calibrated and interpretable uncertainty estimates under incomplete multimodal observations, without introducing additional computational overhead.
Yucheng Xing, Hailan Mo, Zi Wang +2
National University of Singapore, Singapore · Imperial College London, London, UK
Multimodal inputs may be missing or degraded, raising distinct questions of evidence availability and trust. We introduce a reliability-aware Gaussian Product-of-Experts (PoE) in which an input-dependent reliability weight tempers each modality's latent precision, while missing experts are omitted. We characterize how reliability and availability affect posterior information and expert influence. Across two clinical multimodal tasks, the full framework remains competitive with complete inputs and achieves its clearest mean-AUROC advantage under modality absence, particularly on HAIM-MIMIC-MM. Ablations show a substantial gain from missing-aware training and an additional HAIM improvement from reliability weighting under matched PoE, missing-aware, and consistency training. Unseen test-time corruptions are used diagnostically and reveal modality-dependent reliability-weight and latent-uncertainty responses, including failure under severe degradation. Thus, reliability weighting provides an explicit evidence-control mechanism, while learning dependable quality signals remains an open challenge.
Accurate inpatient risk prediction requires modeling heterogeneous clinical evidence that accumulates irregularly over time. This challenge spans in-hospital mortality, prolonged length of stay, and readmission, each with task-specific observation windows. Existing multimodal approaches combine structured health record data, chest x-rays, and clinical text, but primarily focus on modality fusion or missing-modality compensation, with limited attention to temporal trajectories over an admission. We propose TRACE (TRAjectory modeling of multimodal Clinical Events), a daily-scale framework organizing clinical events into day-level trajectories, separating day-invariant global context from day-specific local events drawn from EHR data, chest x-rays, radiology reports, and discharge summaries. The trajectory is encoded using temporal attention, allowing unavailable modalities to emit no tokens. A phenotype-guided contrastive objective further organizes admissions under heterogeneous disease distributions. Across cohorts from MIMIC-IV, eICU, and MIMIC-III comprising nearly 180,000 total stays, we evaluate TRACE on mortality, length of stay, and readmission prediction. TRACE achieves the highest AUROC across all nine dataset-task settings and the highest ACC in eight, outperforming baselines. Ablation studies validate our attention masks and contrastive learning, while trajectory-level attribution localizes risk-associated hospital days and modalities.