Diagnostics for Individual-Level Prediction Instability in Machine Learning for Healthcare

Elizabeth W. Miller; Jeffrey D. Blume

cs.LGFeb 27, 2026

Diagnostics for Individual-Level Prediction Instability in Machine Learning for Healthcare

Authors: Elizabeth W. Miller, Jeffrey D. Blume

Organizations: School of Data Science University of Virginia Charlottesville

Abstract

In healthcare, predictive models increasingly inform patient-level decisions, yet little attention is paid to the variability in individual risk estimates and its impact on treatment decisions. For overparameterized models, now standard in machine learning, a substantial source of variability often goes undetected. Even when the data and model architecture are held fixed, randomness introduced by optimization and initialization can lead to materially different risk estimates for the same patient. This problem is largely obscured by standard evaluation practices, which rely on aggregate performance metrics (e.g., log-loss, accuracy) that are agnostic to individual-level stability. As a result, models with indistinguishable aggregate performance can nonetheless exhibit substantial procedural arbitrariness, which can undermine clinical trust. We propose an evaluation framework that quantifies individual-level prediction instability by using two complementary diagnostics: empirical prediction interval width (ePIW), which captures variability in continuous risk estimates, and empirical decision flip rate (eDFR), which measures instability in threshold-based clinical decisions. We apply these diagnostics to simulated data and GUSTO-I clinical dataset. Across observed settings, we find that for flexible machine-learning models, randomness arising solely from optimization and initialization can induce individual-level variability comparable to that produced by resampling the entire training dataset. Neural networks exhibit substantially greater instability in individual risk predictions compared to logistic regression models. Risk estimate instability near clinically relevant decision thresholds can alter treatment recommendations. These findings that stability diagnostics should be incorporated into routine model validation for assessing clinical reliability.

Figures & tables

Abbreviation	Model Family	Specification & Fitting Routine	$p$
Log-LBFGS	Logistic Regression	Quasi-Newton (L-BFGS), $L_{2}$ reg.	6
Log-G	Logistic Regression	Quasi-Newton (L-BFGS), $L_{2}$ reg.	9
Log-SGD	Logistic Regression	SGD, $L_{2}$ reg.	6
Log-Poly	Logistic Regression	L-BFGS, $L_{2}$ reg., polynomial features	21
NN-1L	Neural Network	1 hidden layer (width 40), ReLU, SGD	281
NN-2L	Neural Network	2 hidden layers (width 180), ReLU, SGD	33,841

Table 1: Model specifications for candidate architectures used in simulation and GUSTO-I experiments. The parameter count

p

reflects the total number of trainable weights and biases.

	Resampled training data				Fixed training data
	$n_{\text{train}}=500$		$n_{\text{train}}=5000$		$n_{\text{train}}=500$		$n_{\text{train}}=5000$
Model	BCE	Acc.	BCE	Acc.	BCE	Acc.	BCE	Acc.
Log-LBFGS	$0.523\pm.004$	$73.4\%\pm.4$	$0.518\pm.001$	$74.1\%\pm.1$	$0.517\pm.000$	$74.1\%\pm.0$	$0.519\pm.000$	$74.0\%\pm.0$
Log-SGD	$0.520\pm.003$	$73.8\%\pm.3$	$0.517\pm.001$	$74.1\%\pm.1$	$0.517\pm.000$	$74.0\%\pm.1$	$0.517\pm.000$	$74.0\%\pm.1$
Log-poly	$0.534\pm.006$	$72.4\%\pm.6$	$0.520\pm.001$	$74.1\%\pm.1$	$0.527\pm.001$	$73.2\%\pm.0$	$0.520\pm.001$	$74.0\%\pm.0$
NN-1L	$0.529\pm.004$	$73.6\%\pm.3$	$0.519\pm.001$	$74.1\%\pm.1$	$0.527\pm.001$	$73.7\%\pm.1$	$0.519\pm.001$	$74.3\%\pm.1$

Table 2: Aggregate out-of-sample performance on simulated data (mean

\pm

1 SD across

B=100

retrainings) evaluated on Binary Cross-entropy (BCE) and accuracy

Figure 1: Individual-level prediction instability in simulation (top) and the GUSTO-I clinical dataset (bottom). Panels (A–D) report empirical prediction interval width (ePIW); panels (E–H) report empirical decision flip rate (eDFR). Columns correspond to resampled training data versus fixed training data (random-seed variation only), each evaluated at $n_{\text{train}}\in\{500,5000\}$ . In the simulated setting, instability is plotted as a function of true risk; in GUSTO-I, instability is plotted as a function of developed model risk. Across both settings, prediction dispersion is consistently larger for neural networks than for logistic regression and attenuates with increasing training sample size under resampled training data. Decision instability concentrates near clinical decision thresholds and diminishes with increased data and reduced sources of randomness.

Figure 2: Individual-level prediction distributions across model families. Each panel displays the empirical distribution of predicted risks (

\widehat{P}_{i}

) for 12 representative individuals (

x_{1}

through

x_{12}

) across

B

repeated instantiations of the learning pipeline from resampling the training data. From left to right, the panels represent: (1) Log-LBFGS, (2) Log-poly, (3) Log-SGD, (4) NN-1L, and (5) NN-2L. While all models satisfy the competitiveness criterion, the ridge plots reveal varying degrees of individual-level dispersion, particularly as model complexity evolves to neural network architectures.

Figure 3: Prediction instability for a fixed out-of-sample individual (true risk

=0.381

) under

B=100

learning pipeline (LP) instances with

n_{\text{train}}=500

from simulation data. Each point denotes the predicted risk from one fitted model instance. Left: variability induced by resampling the training data. Right: variability induced by random seed initialization with fixed training data.

	Resampled training data				Fixed training data
	$n=500$		$n=5000$		$n=500$		$n=5000$
Model	BCE	Acc.	BCE	Acc.	BCE	Acc.	BCE	Acc.
Log-G	0.22 $\pm$ .01	0.93 $\pm$ .00	0.21	0.93 $\pm$ .00	0.22	0.93 $\pm$ .00	0.21	0.93 $\pm$ .00
NN-G	0.22 $\pm$ .01	0.93 $\pm$ .01	0.22	0.93 $\pm$ .00	0.23	0.92 $\pm$ .01	0.21	0.93 $\pm$ .01

Table 3: Aggregate out-of-sample performance on GUSTO-I (mean

\pm

1 SD across

B=100

retrainings).

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Resampled training data, $n_{\text{train}}=500$
True Risk (bin)	Log-LBFGS	Log-SGD	Log-poly	NN-1L	NN-2L
$[0.0,0.1)$	0.104	0.089	0.230	0.184	0.205
$[0.1,0.2)$	0.177	0.161	0.314	0.279	0.322
$[0.2,0.3)$	0.223	0.205	0.369	0.342	0.411
$[0.3,0.4)$	0.250	0.232	0.398	0.375	0.469
$[0.4,0.5)$	0.260	0.240	0.407	0.389	0.491

Appendix

Table 4: Empirical prediction interval width (ePIW) by true-risk bin across training conditions.

Resampled training data, $n_{\text{train}}=500$
True Risk (bin)	Log-LBFGS	Log-SGD	Log-poly	NN-1L	NN-2L
$[0.0,0.1)$	0.000	0.000	0.001	0.000	0.000
$[0.1,0.2)$	0.000	0.000	0.007	0.001	0.002
$[0.2,0.3)$	0.000	0.000	0.027	0.010	0.022
$[0.3,0.4)$	0.026	0.015	0.115	0.082	0.127
$[0.4,0.5)$	0.304	0.275	0.390	0.369	0.398

Appendix

Table 5: Empirical decision flip rate (eDFR) by true-risk bin across training conditions.

Figure 4: Predicted risk versus true risk across model classes, training sample sizes, and sources of stochasticity. Each row corresponds to a model specification (Log-LBFGS, Log-SGD, Log-poly, NN-1L, NN-2L). Columns report results under resampled training data and fixed training data (random-seed variation only), each evaluated at $n_{\text{train}}\in\{500,5000\}$ . Points represent individual-level predicted risks evaluated on a common test set across $B=100$ model retrainings. The dashed $45^{\circ}$ line indicates perfect calibration ( $\widehat{p}=p_{\text{true}}$ ).

Figure 5: Individual-level prediction instability, bias, and mean squared error as functions of true risk in the simulated setting. Columns correspond to resampled training data and fixed training data, each evaluated at

n_{\text{train}}\in\{500,5000\}

. Rows report empirical prediction interval width (ePIW; top), prediction bias (middle), and mean squared error (MSE; bottom), with LOESS smoothing applied to highlight systematic trends across the true-risk spectrum. Curves correspond to different model classes.

Clinical Risk Prediction

Clinical risk prediction estimates patients’ future health outcomes from medical records, physiological signals, imaging, and other health data.

See all 217 papers

From Static Risk to Dynamic Trajectories: Toward World-Model-Inspired Clinical Prediction
May 16, 2026Pujun Feng, Xiaoyu Guo, Seyed Ehsan Saffari +10Individualized Treatment Effect Estimation Off-Policy Evaluation
Faculty of Applied Sciences, Macao Polytechnic University, Macao SAR 999078, China. · Medin.ai, Beijing 100871, China. · School of Software & Microelectronics, Peking University, Beijing 100871, China. +8
An empirical evaluation of the risks of AI model updates using clinical data: stability, arbitrariness, and fairness
Apr 27, 2026Ioannis Bilionis, Ricardo C. Berrios, Luis Fernandez-Luque +1Algorithmic Fairness Clinical Outcome Prediction
Adhera Health and Universitat Pompeu Fabra Barcelona, Spain · Adhera Health, USA · Adhera Health, Spain +1
Reliability Auditing for Downstream LLM tasks in Psychiatry: LLM-Generated Hospitalization Risk Scores
Apr 23, 2026Shevya Panda, Shinjini Bose, Ananya JoshiPrompt Sensitivity LLM Auditing
Johns Hopkins University Baltimore, USA

May 16, 2026J/K move · Enter open · S save

From Static Risk to Dynamic Trajectories: Toward World-Model-Inspired Clinical Prediction

Pujun Feng, Xiaoyu Guo, Seyed Ehsan Saffari, Min Hun Lee+9

Faculty of Applied Sciences, Macao Polytechnic University, Macao SAR 999078, China. · Medin.ai, Beijing 100871, China. · School of Software & Microelectronics, Peking University, Beijing 100871, China.+8

Individualized Treatment Effect Estimation Off-Policy Evaluation Causal Effect Estimation Healthcare Clinical Outcome Prediction Counterfactual Prediction+3

Open paper AskPDF arXiv

Clinical decision-making is a feedback system where risk estimates influence treatment, which in turn changes disease trajectories, and both shape clinicians' measurement practices. Static prediction often fails clinically: models trained on observational care logs conflate disease biology with clinician behavior, particularly under treatment confounder feedback and irregular or informative observation. This Review focuses on intervention-aware disease trajectory modeling in clinical AI--methods estimating patient-specific longitudinal disease evolution and assessing trajectory changes under alternative treatments. We organize the field around six linked components: three decision tasks (factual forecasting, counterfactual estimation, policy evaluation) and three data-generating mechanisms (disease evolution, treatment assignment, observation process) that determine identifiability. We present the first unified framework bridging forecasting, counterfactual trajectories, and policy evaluation across discrete/continuous time, explicitly addressing treatment assignment, time-varying confounding, and observation bias. We synthesize key method families (multistate/joint models, temporal point-process, deep sequence architectures, longitudinal causal inference), map them to relevant components, and align evaluation with claim strength via overlap diagnostics, uncertainty quantification, off-policy robustness, and target-trial validation. This synthesis advances benchmark prediction to decision-grade clinical evidence, enabling treatment-sensitive individualized futures, pre-deployment policy stress-testing, and safer closed-loop learning health systems that adapt/abstain when evidence is insufficient.

Abstract

Clinical Risk Prediction

Explore similar work

From Static Risk to Dynamic Trajectories: Toward World-Model-Inspired Clinical Prediction

An empirical evaluation of the risks of AI model updates using clinical data: stability, arbitrariness, and fairness

Reliability Auditing for Downstream LLM tasks in Psychiatry: LLM-Generated Hospitalization Risk Scores