Data-Driven Priors for Uncertainty-Aware Risk Prediction of Clinical Deterioration using Multimodal Data

L. Julián Lechuga López; Tim G. J. Rudner; Farah E. Shamout

cs.LGMar 9, 2026

Data-Driven Priors for Uncertainty-Aware Risk Prediction of Clinical Deterioration using Multimodal Data

Authors: L. Julián Lechuga López, Tim G. J. Rudner, Farah E. Shamout

Organizations: Tandon School of Engineering, New York University · New York University Abu Dhabi · University of Toronto

Abstract

Safe predictions are a crucial requirement for integrating predictive models into clinical decision support systems. One approach to improving trustworthiness is to enable models to express uncertainty about individual predictions. However, current machine learning models frequently lack reliable uncertainty estimation, hindering real-world deployment. This limitation is particularly evident in multimodal settings, where models must effectively integrate heterogeneous information. In this work, we propose MedCertAIn, a predictive uncertainty framework that leverages multimodal clinical data to improve model performance and reliability for in-hospital mortality risk prediction as an indicator of patient deterioration. We design data-driven priors over neural network parameters using a hybrid strategy that considers cross-modal similarity in self-supervised latent representations and modality-specific data corruptions. We train and evaluate the models with such priors using clinical time-series and chest X-ray images from the publicly available datasets MIMIC-IV and MIMIC-CXR. Our results show that MedCertAIn achieves competitive predictive performance and substantial gains in selective prediction compared with the evaluated deterministic and stochastic baselines. These findings highlight the promise of data-driven priors in advancing robust, uncertainty-aware AI tools for high-stakes clinical applications. Our implementation is publicly available at: https://github.com/jlaitue/medcertain.

Figures & tables

Figure 1: Uncertainty-Aware Clinical Decision-Making. Multimodal patient data are processed by an uncertainty-aware model to produce a prediction and corresponding uncertainty estimate. Clinicians can accept confident predictions while deferring uncertain cases for further review, improving workflow efficiency and supporting reliable clinical decision-making.

Figure 2: MedCertAIn : A Predictive Uncertainty Framework. a) We construct paired multimodal samples for training and evaluation and generate high-uncertainty context data. From the training set, we create corrupted pairs using noise and transformations and pretrain a self-supervised model to identify low-similarity pairs in latent space, representing high cross-modal uncertainty. We combine these context sets to define a data-driven prior, fine-tune the model using Bayesian learning, and evaluate standard and selective performance. b) Selective Prediction Metrics. We rank predictions by Shannon entropy and evaluate performance on retained subsets across coverage thresholds (0–100%). Averaging across thresholds yields selective AUROC and AUPRC, while the coverage curve supports selection of an operating point for deferring the least reliable predictions.

Panel of Table 1 — Table 1: Dataset Summary : Summary of multimodal data samples used during training for the in-hospital mortality task. Only our proposed method MedCertAIn makes use of the high-uncertainty set (5,347 samples) which is constructed by combining corrupted training examples with 862 additional samples selected via cross-modal similarity in the self-supervised latent space.

Figure 3: MedCertAIn Achieves Strong Predictive Performance and Reliable Selective Prediction. a) Standard AUROC and AUPRC curves for MedCertAIn and representative deterministic and stochastic baselines show relatively small differences in predictive performance. b) Larger differences emerge under selective prediction, where MedCertAIn achieves stronger and more stable performance across rejection thresholds and random seeds. c) Calibration and relative-improvement analyses further characterize the differences among the evaluated methods. Shaded regions represent variation across five random seeds.

Panel of Table 2 — Table 2: MedCertAIn Achieves Strong Predictive Performance and Reliable Selective Prediction. Performance on 48-hour in-hospital mortality prediction compared with deterministic, post-hoc calibration, ensemble, and stochastic uncertainty baselines. MedCertAIn achieves the strongest performance across all metrics, with particularly large improvements in selective AUROC and selective AUPRC relative to both MedFuse and the uninformative-prior BNN.

Ablation	AUROC	AUPRC	Selective AUROC	Selective AUPRC
Uninformative Prior	0.820 $\pm$ 0.004	0.485 $\pm$ 0.006	0.711 $\pm$ 0.034	0.357 $\pm$ 0.058
Random Corruptions	0.832 $\pm$ 0.001	0.494 $\pm$ 0.002	0.841 $\pm$ 0.006	0.579 $\pm$ 0.011
Inter-Modal Similarity	0.821 $\pm$ 0.001	0.470 $\pm$ 0.002	0.749 $\pm$ 0.013	0.423 $\pm$ 0.032
Inter- and Intra-Modal Similarity	0.821 $\pm$ 0.001	0.470 $\pm$ 0.002	0.755 $\pm$ 0.007	0.434 $\pm$ 0.020
MedCertAIn I	0.835 $\pm$ 0.001	0.498 $\pm$ 0.002	0.857 $\pm$ 0.005	0.599 $\pm$ 0.002
MedCertAIn II	0.834 $\pm$ 0.001	0.497 $\pm$ 0.001	0.848 $\pm$ 0.008	0.590 $\pm$ 0.006

Table 3: Comparison Across Multimodal Data-Driven Priors. Performance on 48-hour in-hospital mortality prediction across alternative strategies for constructing the high-uncertainty context set. Combining latent-space similarity with random corruptions in MedCertAIn I and MedCertAIn II improves both predictive and selective performance, with MedCertAIn I achieving the strongest results across all metrics.

Modality	Model	AUROC	AUPRC	Selective AUROC	Selective AUPRC
Unimodal CXR	MedFuse	0.615 $\pm$ 0.007	0.225 $\pm$ 0.007	0.569 $\pm$ 0.012	0.167 $\pm$ 0.015
Unimodal CXR	MedCertAIn	0.632 $\pm$ 0.014	0.242 $\pm$ 0.014	0.567 $\pm$ 0.015	0.146 $\pm$ 0.004
Unimodal EHR	MedFuse	0.760 $\pm$ 0.003	0.393 $\pm$ 0.008	0.670 $\pm$ 0.017	0.235 $\pm$ 0.007
Unimodal EHR	MedCertAIn	0.825 $\pm$ 0.004	0.480 $\pm$ 0.004	0.807 $\pm$ 0.014	0.499 $\pm$ 0.023
Multimodal	MedFuse	0.819 $\pm$ 0.000	0.466 $\pm$ 0.002	0.659 $\pm$ 0.005	0.203 $\pm$ 0.006
Multimodal	MedCertAIn	0.835 $\pm$ 0.001	0.498 $\pm$ 0.002	0.857 $\pm$ 0.005	0.599 $\pm$ 0.002

Table 4: Impact of Multimodal Fusion on Predictive and Selective Performance. Performance on 48-hour in-hospital mortality prediction using CXR-only, EHR-only, and multimodal inputs. Multimodal fusion yields the strongest overall performance, while MedCertAIn improves predictive and selective performance over MedFuse in the EHR-only and multimodal settings.

Patient Subgroup	MedFuse				MedCertAIn
Patient Subgroup	AUROC	AUPRC	Selective AUROC	Selective AUPRC	AUROC	AUPRC	Selective AUROC	Selective AUPRC
	Age
18-45 years	0.803 $\pm$ 0.001	0.603 $\pm$ 0.023	0.559 $\pm$ 0.002	0.262 $\pm$ 0.001	0.824 $\pm$ 0.001	0.600 $\pm$ 0.002	0.723 $\pm$ 0.005	0.387 $\pm$ 0.017
45-60 years	0.788 $\pm$ 0.001	0.545 $\pm$ 0.027	0.554 $\pm$ 0.003	0.231 $\pm$ 0.001	0.811 $\pm$ 0.002	0.485 $\pm$ 0.002	0.730 $\pm$ 0.008	0.299 $\pm$ 0.014
$>$ 60 years	0.814 $\pm$ 0.000	0.528 $\pm$ 0.032	0.631 $\pm$ 0.004	0.207 $\pm$ 0.002	0.832 $\pm$ 0.000	0.480 $\pm$ 0.002	0.863 $\pm$ 0.004	0.559 $\pm$ 0.002
	Sex

Table 5: Analysis of MedCertAIn Across Patient Subpopulations. Comparison of MedFuse and MedCertAIn across age and sex subgroups. MedCertAIn improves AUROC and both selective metrics across all subgroups, while standard AUPRC varies across age groups.

Figure 4: Relative Improvements Across Patient Subpopulations. Percentage improvement of MedCertAIn over MedFuse across age and sex subgroups, highlighting the largest gains in selective prediction.

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Table 10 — Table A.1: 48-Hour Mortality Calibration. Calibration across MedFuse-based baselines and MedCertAIn . MedCertAIn achieves the lowest Brier score, whereas MedFuse obtains the lowest ECE.

Panel of Table 11 — Table A.2: 3-Month Mortality Prediction Performance. Comparison across MedFuse-based baselines and MedCertAIn , which achieves the highest mean performance across all standard and selective metrics, with the largest improvement in selective AUPRC.

Panel of Table 12 — Table A.3: 3-Month Mortality Calibration. Calibration and probabilistic accuracy across MedFuse-based baselines and MedCertAIn . Deep Ensemble achieves the lowest ECE, while MedFuse, Deep Ensemble, and MC Dropout share the lowest reported Brier score.

Panel of Table 13 — Table A.4: 6-Month Mortality Prediction Performance. Comparison across MedFuse-based baselines and MedCertAIn . MedCertAIn achieves the highest mean AUROC and substantially improves both selective metrics, although its AUPRC is lower than those of the evaluated baselines.

Panel of Table 14 — Table A.5: 6-Month Mortality Calibration. Calibration and probabilistic accuracy across MedFuse-based baselines and MedCertAIn . Deep Ensemble achieves the lowest ECE, while Deep Ensemble, MC Dropout, and the uninformative-prior BNN share the lowest reported Brier score.

Panel of Table 15 — Table A.6: 1-Year Mortality Prediction Performance. Comparison across MedFuse-based baselines and MedCertAIn . MedCertAIn achieves the highest mean AUROC and selective prediction performance, with the largest improvement in selective AUPRC.

Panel of Table 16 — Table A.7: 1-Year Mortality Calibration. Calibration and probabilistic accuracy across MedFuse-based baselines and MedCertAIn . MC Dropout achieves the lowest ECE, while all MedFuse-based baselines except the uninformative-prior BNN share the lowest reported Brier score.

Model	AUROC	AUPRC	Selective AUROC	Selective AUPRC	ECE $\downarrow$	Brier $\downarrow$
48h Mortality
DrFuse	0.817 $\pm$ 0.005	0.472 $\pm$ 0.013	0.785 $\pm$ 0.007	0.437 $\pm$ 0.035	8.943 $\pm$ 0.597	0.111 $\pm$ 0.006
MeTra	0.783 $\pm$ 0.015	0.411 $\pm$ 0.037	0.724 $\pm$ 0.034	0.296 $\pm$ 0.083	2.062 $\pm$ 0.918	0.110 $\pm$ 0.005
MedFuse	0.819 $\pm$ 0.000	0.466 $\pm$ 0.002	0.659 $\pm$ 0.005	0.203 $\pm$ 0.006	3.215 $\pm$ 0.242	0.102 $\pm$ 0.000
3-month Mortality
DrFuse	0.647 $\pm$ 0.039	0.150 $\pm$ 0.035	0.574 $\pm$ 0.034	0.091 $\pm$ 0.024	4.159 $\pm$ 2.585	0.079 $\pm$ 0.018

Appendix

Table A.8: Deterministic Multimodal Models Across Mortality Horizons. Performance of DrFuse, MeTra, and MedFuse on 48-hour, 3-month, 6-month, and 1-year mortality prediction. We report standard, selective, and calibration metrics as mean

\pm

standard error across five random seeds. The best and second-best results for each horizon and metric are shown in bold and underlined, respectively.

Model	AUROC	AUPRC	Selective AUROC	Selective AUPRC
MedFuse HEM ( Shrivastava et al., 2016 )	0.233 $\pm$ 0.003	0.094 $\pm$ 0.001	0.393 $\pm$ 0.010	0.070 $\pm$ 0.010
MedCertAIn (HEM 20%)	0.821 $\pm$ 0.002	0.477 $\pm$ 0.002	0.790 $\pm$ 0.011	0.502 $\pm$ 0.023
Absolute Difference (pp)	+58.75	+38.30	+39.65	+43.26

Appendix

Table A.9: Comparison with Hard Example Mining. Performance when high-loss samples are used for direct HEM fine-tuning of MedFuse or as context for the uncertainty-aware prior in MedCertAIn .

Figure A.1: Comparison Across Multimodal Data-Driven Priors. Standard AUROC and AUPRC curves show modest differences across prior constructions, whereas selective AUROC and AUPRC reveal clearer separation across deferral rates. Combining similarity-based selection with random corruptions in MedCertAIn I and MedCertAIn II yields the strongest selective performance.

Dataset	Training	Validation	Testing
Unimodal CXR (images)	124,671	8,813	20,747
Unimodal EHR (time-series records)	42,628	4,802	11,914
Multimodal (CXR + EHR)	4,485	488	1,242

Appendix

Table C.10: Task-Specific Dataset Sizes. Numbers of unimodal and paired multimodal samples derived from MIMIC-IV (EHR) and MIMIC-CXR (CXR) ( Johnson et al., 2019 ; Johnson et al., 2024 ) for in-hospital mortality prediction. The multimodal cohort is smaller because it includes only samples for which both modalities are available.

Hyperparameter	Values
Prior variance	${0.1,1,10,1000}$
Prior likelihood scale	${0.1,1,10}$
Prior likelihood f-scale	${0,1,10}$
Prior likelihood covariance scale	${0.1,0.01,0.001}$
Prior likelihood covariance diagonal	${0.5,1,5}$

Appendix

Table C.11: Stochastic Hyperparameter Search Space. Hyperparameters and candidate values evaluated for the uncertainty-aware prior in MedCertAIn .

Explore similar work

CURA: Clinical Uncertainty Risk Alignment for Language Model-Based Risk Prediction
Apr 16, 2026Sizhe Wang, Ziqi Xu, Claire Najjuuko +2Clinical Prediction Large Language Model Uncertainty
Washington University in St. Louis
OC-Distill: Ontology-aware Contrastive Learning with Cross-Modal Distillation for ICU Risk Prediction
Apr 18, 2026Zhongyuan Liang, Junhyung Jo, Hyang-Jung Lee +2Cross-Modal Distillation International Classification Of Disease Coding
Computational Precision Health UC Berkeley, UCSF · Samsung Advanced Institute of Technology (SAIT) · Computational Precision Health UC Berkeley, UCSF[cs.LG]
MedBayes-Lite: A Clinical Uncertainty Governance Layer for Risk-Aware Medical Decision Support
Nov 20, 2025Elias Hossain, Md Mehedi Hasan Nipu, Maleeha Sheikh +5Clinical Decision Support Large Language Model Uncertainty
College of Engineering and Computer Science, University of Central Florida, Orlando, FL 32816, USA · Department of Computer Science and Engineering, North South University, Dhaka 1229, Bangladesh · Department of Electrical and Computer Engineering, Purdue University Fort Wayne, Fort Wayne, IN 46805, USA +4

Apr 16, 2026, cs.CLJ/K move · Enter open · S save

CURA: Clinical Uncertainty Risk Alignment for Language Model-Based Risk Prediction

Sizhe Wang, Ziqi Xu, Claire Najjuuko, Charles Alba+1

Washington University in St. Louis

Clinical Prediction Large Language Model Uncertainty Risk Regularization

Open paper AskPDF arXiv

Clinical language models (LMs) are increasingly applied to support clinical risk prediction from free-text notes, yet their uncertainty estimates often remain poorly calibrated and clinically unreliable. In this work, we propose Clinical Uncertainty Risk Alignment (CURA), a framework that aligns clinical LM-based risk estimates and uncertainty with both individual error likelihoods and cohort-level ambiguities. CURA first fine-tunes domain-specific clinical LMs to obtain task-adapted patient embeddings, and then performs uncertainty fine-tuning of a multi-head classifier using a bi-level uncertainty objective. Specifically, an individual-level calibration term aligns predictive uncertainty with each patient's likelihood of error, while a cohort-aware regularizer pulls risk estimates toward event rates in their local neighborhoods in the embedding space and places extra weight on ambiguous cohorts near the decision boundary. We further show that this cohort-aware term can be interpreted as a cross-entropy loss with neighborhood-informed soft labels, providing a label-smoothing view of our method. Extensive experiments on MIMIC-IV clinical risk prediction tasks across various clinical LMs show that CURA consistently improves calibration metrics without substantially compromising discrimination. Further analysis illustrates that CURA reduces overconfident false reassurance and yields more trustworthy uncertainty estimates for downstream clinical decision support.

Abstract

Explore similar work

CURA: Clinical Uncertainty Risk Alignment for Language Model-Based Risk Prediction

OC-Distill: Ontology-aware Contrastive Learning with Cross-Modal Distillation for ICU Risk Prediction

MedBayes-Lite: A Clinical Uncertainty Governance Layer for Risk-Aware Medical Decision Support