Diagnostics for Individual-Level Prediction Instability in Machine Learning for Healthcare

Elizabeth W. Miller; Jeffrey D. Blume

cs.LGFeb 27, 2026

Diagnostics for Individual-Level Prediction Instability in Machine Learning for Healthcare

Authors: Elizabeth W. Miller, Jeffrey D. Blume

Organizations: School of Data Science University of Virginia Charlottesville

Abstract

In healthcare, predictive models increasingly inform patient-level decisions, yet little attention is paid to the variability in individual risk estimates and its impact on treatment decisions. For overparameterized models, now standard in machine learning, a substantial source of variability often goes undetected. Even when the data and model architecture are held fixed, randomness introduced by optimization and initialization can lead to materially different risk estimates for the same patient. This problem is largely obscured by standard evaluation practices, which rely on aggregate performance metrics (e.g., log-loss, accuracy) that are agnostic to individual-level stability. As a result, models with indistinguishable aggregate performance can nonetheless exhibit substantial procedural arbitrariness, which can undermine clinical trust. We propose an evaluation framework that quantifies individual-level prediction instability by using two complementary diagnostics: empirical prediction interval width (ePIW), which captures variability in continuous risk estimates, and empirical decision flip rate (eDFR), which measures instability in threshold-based clinical decisions. We apply these diagnostics to simulated data and GUSTO-I clinical dataset. Across observed settings, we find that for flexible machine-learning models, randomness arising solely from optimization and initialization can induce individual-level variability comparable to that produced by resampling the entire training dataset. Neural networks exhibit substantially greater instability in individual risk predictions compared to logistic regression models. Risk estimate instability near clinically relevant decision thresholds can alter treatment recommendations. These findings that stability diagnostics should be incorporated into routine model validation for assessing clinical reliability.

Figures & tables

Abbreviation	Model Family	Specification & Fitting Routine	$p$
Log-LBFGS	Logistic Regression	Quasi-Newton (L-BFGS), $L_{2}$ reg.	6
Log-G	Logistic Regression	Quasi-Newton (L-BFGS), $L_{2}$ reg.	9
Log-SGD	Logistic Regression	SGD, $L_{2}$ reg.	6
Log-Poly	Logistic Regression	L-BFGS, $L_{2}$ reg., polynomial features	21
NN-1L	Neural Network	1 hidden layer (width 40), ReLU, SGD	281
NN-2L	Neural Network	2 hidden layers (width 180), ReLU, SGD	33,841

Table 1: Model specifications for candidate architectures used in simulation and GUSTO-I experiments. The parameter count

p

reflects the total number of trainable weights and biases.

	Resampled training data				Fixed training data
	$n_{\text{train}}=500$		$n_{\text{train}}=5000$		$n_{\text{train}}=500$		$n_{\text{train}}=5000$
Model	BCE	Acc.	BCE	Acc.	BCE	Acc.	BCE	Acc.
Log-LBFGS	$0.523\pm.004$	$73.4\%\pm.4$	$0.518\pm.001$	$74.1\%\pm.1$	$0.517\pm.000$	$74.1\%\pm.0$	$0.519\pm.000$	$74.0\%\pm.0$
Log-SGD	$0.520\pm.003$	$73.8\%\pm.3$	$0.517\pm.001$	$74.1\%\pm.1$	$0.517\pm.000$	$74.0\%\pm.1$	$0.517\pm.000$	$74.0\%\pm.1$
Log-poly	$0.534\pm.006$	$72.4\%\pm.6$	$0.520\pm.001$	$74.1\%\pm.1$	$0.527\pm.001$	$73.2\%\pm.0$	$0.520\pm.001$	$74.0\%\pm.0$
NN-1L	$0.529\pm.004$	$73.6\%\pm.3$	$0.519\pm.001$	$74.1\%\pm.1$	$0.527\pm.001$	$73.7\%\pm.1$	$0.519\pm.001$	$74.3\%\pm.1$

Table 2: Aggregate out-of-sample performance on simulated data (mean

\pm

1 SD across

B=100

retrainings) evaluated on Binary Cross-entropy (BCE) and accuracy

Figure 1: Individual-level prediction instability in simulation (top) and the GUSTO-I clinical dataset (bottom). Panels (A–D) report empirical prediction interval width (ePIW); panels (E–H) report empirical decision flip rate (eDFR). Columns correspond to resampled training data versus fixed training data (random-seed variation only), each evaluated at $n_{\text{train}}\in\{500,5000\}$ . In the simulated setting, instability is plotted as a function of true risk; in GUSTO-I, instability is plotted as a function of developed model risk. Across both settings, prediction dispersion is consistently larger for neural networks than for logistic regression and attenuates with increasing training sample size under resampled training data. Decision instability concentrates near clinical decision thresholds and diminishes with increased data and reduced sources of randomness.

Figure 2: Individual-level prediction distributions across model families. Each panel displays the empirical distribution of predicted risks (

\widehat{P}_{i}

) for 12 representative individuals (

x_{1}

through

x_{12}

) across

B

repeated instantiations of the learning pipeline from resampling the training data. From left to right, the panels represent: (1) Log-LBFGS, (2) Log-poly, (3) Log-SGD, (4) NN-1L, and (5) NN-2L. While all models satisfy the competitiveness criterion, the ridge plots reveal varying degrees of individual-level dispersion, particularly as model complexity evolves to neural network architectures.

Figure 3: Prediction instability for a fixed out-of-sample individual (true risk

=0.381

) under

B=100

learning pipeline (LP) instances with

n_{\text{train}}=500

from simulation data. Each point denotes the predicted risk from one fitted model instance. Left: variability induced by resampling the training data. Right: variability induced by random seed initialization with fixed training data.

	Resampled training data				Fixed training data
	$n=500$		$n=5000$		$n=500$		$n=5000$
Model	BCE	Acc.	BCE	Acc.	BCE	Acc.	BCE	Acc.
Log-G	0.22 $\pm$ .01	0.93 $\pm$ .00	0.21	0.93 $\pm$ .00	0.22	0.93 $\pm$ .00	0.21	0.93 $\pm$ .00
NN-G	0.22 $\pm$ .01	0.93 $\pm$ .01	0.22	0.93 $\pm$ .00	0.23	0.92 $\pm$ .01	0.21	0.93 $\pm$ .01

Table 3: Aggregate out-of-sample performance on GUSTO-I (mean

\pm

1 SD across

B=100

retrainings).

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Resampled training data, $n_{\text{train}}=500$
True Risk (bin)	Log-LBFGS	Log-SGD	Log-poly	NN-1L	NN-2L
$[0.0,0.1)$	0.104	0.089	0.230	0.184	0.205
$[0.1,0.2)$	0.177	0.161	0.314	0.279	0.322
$[0.2,0.3)$	0.223	0.205	0.369	0.342	0.411
$[0.3,0.4)$	0.250	0.232	0.398	0.375	0.469
$[0.4,0.5)$	0.260	0.240	0.407	0.389	0.491

Appendix

Table 4: Empirical prediction interval width (ePIW) by true-risk bin across training conditions.

Resampled training data, $n_{\text{train}}=500$
True Risk (bin)	Log-LBFGS	Log-SGD	Log-poly	NN-1L	NN-2L
$[0.0,0.1)$	0.000	0.000	0.001	0.000	0.000
$[0.1,0.2)$	0.000	0.000	0.007	0.001	0.002
$[0.2,0.3)$	0.000	0.000	0.027	0.010	0.022
$[0.3,0.4)$	0.026	0.015	0.115	0.082	0.127
$[0.4,0.5)$	0.304	0.275	0.390	0.369	0.398

Appendix

Table 5: Empirical decision flip rate (eDFR) by true-risk bin across training conditions.

Figure 4: Predicted risk versus true risk across model classes, training sample sizes, and sources of stochasticity. Each row corresponds to a model specification (Log-LBFGS, Log-SGD, Log-poly, NN-1L, NN-2L). Columns report results under resampled training data and fixed training data (random-seed variation only), each evaluated at $n_{\text{train}}\in\{500,5000\}$ . Points represent individual-level predicted risks evaluated on a common test set across $B=100$ model retrainings. The dashed $45^{\circ}$ line indicates perfect calibration ( $\widehat{p}=p_{\text{true}}$ ).

Figure 5: Individual-level prediction instability, bias, and mean squared error as functions of true risk in the simulated setting. Columns correspond to resampled training data and fixed training data, each evaluated at

n_{\text{train}}\in\{500,5000\}

. Rows report empirical prediction interval width (ePIW; top), prediction bias (middle), and mean squared error (MSE; bottom), with LOESS smoothing applied to highlight systematic trends across the true-risk spectrum. Curves correspond to different model classes.

Clinical Risk Prediction

Clinical risk prediction estimates patients’ future health outcomes from medical records, physiological signals, imaging, and other health data.

See all 217 papers

May 16, 2026

From Static Risk to Dynamic Trajectories: Toward World-Model-Inspired Clinical Prediction

Clinical decision-making is a feedback system where risk estimates influence treatment, which in turn changes disease trajectories, and both shape clinicians' measurement practices. Static prediction often fails clinically: models trained on observational care logs conflate disease biology with clinician behavior, particularly under treatment confounder feedback and irregular or informative observation. This Review focuses on intervention-aware disease trajectory modeling in clinical AI--methods estimating patient-specific longitudinal disease evolution and assessing trajectory changes under alternative treatments. We organize the field around six linked components: three decision tasks (factual forecasting, counterfactual estimation, policy evaluation) and three data-generating mechanisms (disease evolution, treatment assignment, observation process) that determine identifiability. We present the first unified framework bridging forecasting, counterfactual trajectories, and policy evaluation across discrete/continuous time, explicitly addressing treatment assignment, time-varying confounding, and observation bias. We synthesize key method families (multistate/joint models, temporal point-process, deep sequence architectures, longitudinal causal inference), map them to relevant components, and align evaluation with claim strength via overlap diagnostics, uncertainty quantification, off-policy robustness, and target-trial validation. This synthesis advances benchmark prediction to decision-grade clinical evidence, enabling treatment-sensitive individualized futures, pre-deployment policy stress-testing, and safer closed-loop learning health systems that adapt/abstain when evidence is insufficient.

Pujun Feng, Xiaoyu Guo, Seyed Ehsan Saffari +10

Faculty of Applied Sciences, Macao Polytechnic University, Macao SAR 999078, China. · Medin.ai, Beijing 100871, China. · School of Software & Microelectronics, Peking University, Beijing 100871, China. +8

Individualized Treatment Effect Estimation Off-Policy Evaluation Causal Effect Estimation Healthcare Clinical Outcome Prediction Counterfactual Prediction+3

Ask PDF

Apr 27, 2026

An empirical evaluation of the risks of AI model updates using clinical data: stability, arbitrariness, and fairness

Artificial Intelligence and Machine Learning (AI/ML) models used in clinical settings are increasingly deployed to support clinical decision-making. However, when training data become stale due to changes in demographics, environment, or patient behaviors, model performance can degrade substantially. While updating models with new training data is necessary, such updates may also introduce new risks. We evaluated the proposed monitoring framework on four publicly available U.S.-based Type 1 Diabetes datasets containing high-resolution continuous glucose monitoring (CGM) data, comprising approximately 11,300 weekly observations from 496 participants under 20 years of age. All datasets included structured sociodemographic information. Using the prediction of severe hyperglycemia events in children with type 1 diabetes as a case study, we examine how different model update strategies can adversely affect model stability (e.g., by causing predictions to "flip" for a large number of cases after an update), increase arbitrariness in predictions, or worsen accuracy equity and the balance of error rates across subpopulations. We propose multiple dimensions for continuous monitoring to detect these issues and argue that such monitoring is essential for the development of trustworthy clinical decision support systems.

Ioannis Bilionis, Ricardo C. Berrios, Luis Fernandez-Luque +1

Adhera Health and Universitat Pompeu Fabra Barcelona, Spain · Adhera Health, USA · Adhera Health, Spain +1

Algorithmic Fairness Clinical Outcome Prediction

Ask PDF

Apr 23, 2026

Reliability Auditing for Downstream LLM tasks in Psychiatry: LLM-Generated Hospitalization Risk Scores

Large language models (LLMs) are increasingly utilized in clinical reasoning and risk assessment. However, their interpretive reliability in critical and indeterminate domains such as psychiatry remains unclear. Prior work has identified algorithmic biases and prompt sensitivity in these systems, raising concerns about how contextual information may influence model outputs, but there remains no systematic way to assess these, especially in the psychiatric domain. We propose an approach for reliability auditing downstream LLM tasks by structuring evaluation around the impact of prompt design and the inclusion of medically insignificant inputs on predicted hospitalization risk scores, which is often the first downstream AI clinical-decision-making task. In our audit, a cohort of synthetic patient profiles (n = 50) is generated, each consisting of 15 clinically relevant features and up to 50 clinically insignificant features, across four prompt reframings (neutral, logical, human impact, clinical judgment). We audit four LLMs (Gemini 2.5 Flash, LLaMa 3.3 70b, Claude Sonnet 4.6, GPT-4o mini), and our results show that including medically insignificant variables resulted in a statistically significant increase in the absolute mean predicted hospitalization risk and output variability across all models and prompts, indicating reduced predictive stability as contextual noise increased. Clinically insignificant features had an effect on instability across many model-prompt conditions, and prompt variations independently affected the trajectory of instability in a model-dependent manner. These findings quantify how LLM-based psychiatric risk assessments are sensitive to non-clinical information, highlighting the need for systematic evaluations of attributional stability and uncertainty behavior like this before clinical deployments.

Shevya Panda, Shinjini Bose, Ananya Joshi

Johns Hopkins University Baltimore, USA

Prompt Sensitivity LLM Auditing Mental Health Clinical Language Model Evaluation Clinical Risk Prediction

Ask PDF

Abstract

Clinical Risk Prediction

Explore similar work

From Static Risk to Dynamic Trajectories: Toward World-Model-Inspired Clinical Prediction

An empirical evaluation of the risks of AI model updates using clinical data: stability, arbitrariness, and fairness

Reliability Auditing for Downstream LLM tasks in Psychiatry: LLM-Generated Hospitalization Risk Scores