When uncertainty quantification fails: structural vulnerabilities and corrections for selective prediction under clinical class imbalance
Authors: Walter Endrizzi, Flavio Ragni, Raffaele Marchesi, Stefano Bovo, Monica Moroni, Giuseppe Jurman, Venet Osmani
Organizations: Data Science for Health, Fondazione Bruno Kessler, Trento, Italy. · Department of Cellular, Computational and Integrative Biology, University of Trento, Trento, Italy. · Department of Mathematics, University of Pavia, Pavia, Italy. · Department of Biomedical Sciences, Humanitas University, Milan, Italy. · Digital Environment Research Institute, Queen Mary University of London, London, E1 1HH, United Kingdom.
Background. Selective prediction systems improve the safety of clinical artificial intelligence by deferring ambiguous cases to human experts while automating high-confidence decisions. However, this safety mechanism relies entirely on the premise that uncertainty scores accurately reflect clinical ambiguity. In prognostic tasks characterized by severe class imbalance, where the clinical outcome of interest is a minority, this assumption breaks down. Standard uncertainty measures cease to capture clinically aligned ambiguity; instead, they become a proxy for class membership, systematically penalizing minority patients regardless of their individual predictive clarity. Results. Across 56 synthetic conditions and three neurodegenerative cohorts spanning severe class imbalance to near-balanced conditions (multiple sclerosis, Parkinson's Disease, Alzheimer's Disease), our framework consistently maintained bounded, balanced risk across trajectories. In contrast, global metrics either collapsed minority Sensitivity or protected the minority at the expense of majority patients. Applying CCR with standard entropy fails to protect the minority because it breaks within-class monotonicity when tau=0.5. Combining CCR with threshold-aware metrics restores strict monotonicity, correctly identifies ambiguous patients, and safely mitigates minority exclusion. Conclusions. Under clinical class imbalance, standard uncertainty quantification measures dataset sparsity rather than clinical ambiguity, creating a systemic illusion of safety. This bias is a correctable structural flaw. Clinical pipelines deploying selective prediction under imbalance should adopt threshold-aligned uncertainty measures and class-stratified evaluation. This reduces class-specific risk imbalance, supporting safer selective prediction under the evaluated conditions without silently excluding the most vulnerable patients.
Figures & tables
Figure 1: Structural vulnerabilities of standard uncertainty quantification under clinical class imbalance. The three panels conceptualize the mathematical failure modes of global uncertainty metrics in a binary classification setting when the clinical decision threshold ( τ ) is shifted from 0.50. (A) Threshold-entropy mismatch. Standard Shannon entropy (red curve) is mathematically anchored to a perfectly balanced decision space. Evaluated at a naturally shifted clinical threshold (e.g., τ=0.15 ), a critical mismatch gap emerges. Maximum mathematical entropy fails to align with maximum clinical ambiguity ( p=τ ), causing standard entropy to operate as a distance metric from 0.5 rather than a true measure of clinical uncertainty. (B) The spatial penalty. Anchoring uncertainty to the absolute decision margin M(p)=∣p−τ∣ exposes the asymmetric geometry of the probability space. The majority class is structurally compressed into a narrow margin ( M∈[0,τ] ), whereas the minority class lies in a vastly larger space ( M∈[0,1−τ] ). Applying a single global rejection window systematically distorts the retained cohort by disproportionately penalizing or entirely excluding the compressed majority class. (C) Structural uncertainty divergence. An idealized illustration demonstrating the structural vulnerabilities introduced by extreme class imbalance. A combination of aleatoric noise and epistemic sparsity compresses minority predictions (red) into a bounded space, imposing a theoretical probability glass ceiling ( g<1 , dotted line). Forced into this compressed regime, the minority class theoretically exhibits a structural uncertainty floor (dashed line), meaning their ensemble disagreement remains artificially elevated above majority patients (blue) even at identical mathematical distances from the decision boundary ( τ , solid line). We empirically validate the real-world manifestation of this structural asymmetry in Section 3 .
Figure 2: Asynchronous ensemble pipeline. For each of 30 bootstrap iterations, the dataset was split into stratified 80% training and 20% test partitions while preserving the natural class distribution. Within each iteration, five heterogeneous machine learning classifiers were independently optimized using randomized grid-search cross-validation and subsequently calibrated by Platt scaling on a held-out calibration fold to obtain reliable predicted probabilities. For each patient appearing in a test set, calibrated predictions across classifiers and iterations were aggregated into a patient-specific prediction pool. The mean prediction μi represents the ensemble consensus, while the variance σi2 quantifies prediction variability across the ensemble and is used as a proxy for epistemic uncertainty. No external resampling strategy was applied, ensuring that the resulting uncertainty estimates reflect model behavior under the natural class imbalance.
Method
Uncertainty signal
Rejection
HTotal (Global)
Shannon entropy H(pi)
Global
Margin (Global)
Decision margin ∣pi−τ∣
Global
Hτ (Global)
Ambiguity Score Hτ(pi)
Global
HTotal + CCR
Shannon entropy H(pi)
Class-conditioned
Hτ + CCR
Ambiguity Score Hτ(pi)
Class-conditioned
Random + CCR
Random
Class-conditioned
Table 1: Uncertainty methods compared across all experiments. Summary of the evaluated quantification and rejection strategies. The proposed architecture ( Hτ + CCR) is isolated against standard global baselines (Margin, HTotal ), an ablation of the rejection mechanism ( Hτ ), a standard entropy CCR baseline ( HTotal + CCR) to demonstrate the necessity of threshold-aware scoring, and a randomized control (Random + CCR) to isolate the mechanical benefit of class preservation.
Figure 3: Failure landscape of selective prediction under class imbalance. Heatmaps illustrate minority class Sensitivity at a fixed 30% global rejection rate across 56 synthetic conditions, parameterized by class prior ( π∈[0.05,0.50] , x-axis) and class separability ( d∈[0.5,3.0] , y-axis). (A) Standard global HTotal exhibits a severe structural failure boundary. Under conditions of severe imbalance and overlapping distributions ( d≤1.0,π≤0.20 ) the baseline disproportionately defers minority patients (Sensitivity ≈0 ). (B) The proposed Hτ + CCR dual-architecture successfully rescues minority coverage within this diagnostic failure zone. Crucially, the architectural correction scales proportionally with the degree of imbalance, smoothly converging to identical standard baseline behavior when the natural prior is perfectly balanced ( π=0.50 ).
Figure 4: Scaling law of minority Sensitivity under dynamic rejection across class priors. Panels show the mean (solid lines) and standard deviation (shaded regions) aggregated across varying levels of intrinsic class separability ( d∈[0.5,3.0] ) at a sample size of n=1000 . (A) Sensitivity stability, defined as the linear slope of minority Sensitivity across the 0%–50% rejection range. Standard global HTotal (red) exhibits a severe non-linear scaling law, plunging into severe minority exclusion as class imbalance increases. The proposed Hτ + CCR dual-architecture (green) fully neutralizes this artifact, maintaining a safe trajectory near zero across all priors. (B) Sensitivity evaluated at a fixed 30% global rejection rate. Standard UQ systematically collapses under severe imbalance ( π≤0.20 ), whereas the proposed framework guarantees robust minority retention. (C) Specificity (majority class retention) evaluated at the exact same 30% rejection rate. Under a fixed global rejection constraint, standard UQ maximizes Specificity by disproportionately concentrating deferrals within the minority class. By recovering minority Sensitivity (Panel B), the proposed framework mathematically redistributes these deferrals, resulting in a balanced and class-proportional rejection profile.
Figure 5: Empirical confirmation of asymmetric uncertainty divergence. Mean ensemble disagreement (used as a proxy for epistemic uncertainty) is evaluated as a function of the prediction’s relative distance from the class-conditioned decision boundary, binned by class-specific distance deciles. Statistical significance is evaluated via a paired Wilcoxon signed-rank test across the distance deciles (*** p<0.001 , ** p<0.01 , * p<0.05 , ns: not significant). Because data is partitioned into deciles, the sample size per bin remains rigorously constant within each cohort. (A) Under severe class imbalance (MS cohort), a profound uncertainty asymmetry gap emerges (). As minority progressor predictions (red) move away from the decision threshold, their variance proportionally increases, whereas majority stable predictions (blue) converge near zero ( n≈23 minority / 48 majority patients per decile bin). (B) In the moderately imbalanced PD cohort, minority variance remains structurally elevated across boundary distances (; n≈5 minority / 9 majority per bin). (C) In the near-balanced AD cohort, this asymmetry naturally dissolves (ns; n≈8 minority / 6 majority per bin). This cross-cohort progression empirically confirms that uncertainty divergence is a mechanical artifact of class imbalance rather than intrinsic clinical noise.
Figure 6: Rejection trajectories across real-world clinical cohorts. Performance is evaluated across the full continuum of rejection rates ( r∈[0,0.80] ) for severe imbalance (MS, top row), moderate imbalance (PD, middle row), and near-balance (AD, bottom row). Under severe imbalance, standard global HTotal (red dashed line) induces a complete Sensitivity collapse, systematically excluding minority progressor patients from the retained cohort. Conversely, the global Margin baseline (orange) empirically instantiates the spatial penalty: while it artificially protects minority Sensitivity, it systematically depletes majority class Specificity. The proposed Hτ + CCR dual-architecture (green) breaks this Sensitivity-Specificity trade-off, strictly bounding risk for both clinical trajectories and safely improving overall clinical utility (AUPRC). In the near-balanced AD cohort, these structural vulnerabilities inherently dissolve, and all frameworks safely converge.
Dataset
Uncertainty Framework
Progressor ccAUGRC (95% CI)
Stable ccAUGRC (95% CI)
Class Disparity
Global AUGRC (95% CI)
MS ( τ=π=0.13 )
HTotal (Global)
0.2756 [0.2080, 0.3357] ↑∗∗∗
0.0413 [0.0317, 0.0518] ↓∗∗∗
0.2343
0.0679 [0.0556, 0.0813]
Margin (Global)
0.0931 [0.0562, 0.1378] ↓∗∗
0.1529 [0.1313, 0.1750] ↑∗∗∗
0.0598
0.1431 [0.1235, 0.1625]
Hτ (Global)
0.2178 [0.1535, 0.2762] ↑∗∗∗
0.0561 [0.0449, 0.0680] ↓∗∗∗
0.1617
0.0734 [0.0605, 0.0867]
Random + CCR
0.1605 [0.1084, 0.2123] ↑∗∗∗
0.1374 [0.1201, 0.1538] ↑∗∗
0.0231
0.1404 [0.1231, 0.1577]
HTotal + CCR
0.1467 [0.0941, 0.2013] ↑∗∗
0.1395 [0.1230, 0.1562] ↑∗
0.0072
0.1404 [0.1235, 0.1577]
Hτ + CCR
0.1115 [0.0682, 0.1564]
0.1301 [0.1135, 0.1466]
0.0186
0.1275 [0.1119, 0.1442]
Table 2: Generalized risk-coverage evaluation across clinical cohorts. Performance is measured via the class-conditioned Area Under the Generalized Risk-Coverage Curve (ccAUGRC) and Global AUGRC. The class disparity column ( ∣Progressor ccAUGRC−Stable ccAUGRC∣ ) explicitly quantifies between-class equity. Values are point estimates with 95% Confidence Intervals derived from 1000 post-prediction bootstrap iterations. Significance testing compares each baseline to the proposed reference framework ( Hτ + CCR), with p -values adjusted for multiple comparisons using the Holm-Bonferroni method. Applying CCR with a threshold-aware metric (instantiated here by Hτ + CCR) is the only evaluated method to safely bound risk for both clinical trajectories under severe imbalance while minimizing class disparity. Lower values indicate better safety/lower risk.
Appendix figures & tables10 assets
Supplementary material from the paper’s appendix.
Appendix
Feature
Stable MCI
MCI-to-AD
Total Cohort
p-value
(N=144)
(N=170)
(N=314)
Age (years)
74.4 (70.4, 80.7)
75.3 (69.8, 79.7)
74.6 (70.1, 80.3)
0.996
Education (years)
16.0 (13.2, 18.0)
16.0 (14.0, 18.0)
16.0 (14.0, 18.0)
0.552
MMSE
28.0 (26.0, 29.0)
27.0 (25.0, 28.0)
27.0 (26.0, 29.0)
<0.001
CDR-SB
1.5 (1.0, 1.5)
2.0 (1.0, 2.5)
1.5 (1.0, 2.0)
<0.001
FAQ
1.0 (0.0, 3.0)
4.0 (1.0, 8.0)
2.0 (0.0, 6.0)
<0.001
Appendix
Table 3: Baseline Characteristics of the ADNI Analytic Cohort. Categorical variables are presented as count (%). Continuous variables are presented as median (Q1, Q3). P-values are calculated using the Mann-Whitney U test for continuous variables and the Pearson’s chi-square test for categorical variables. Abbreviations: CDR-SB = Clinical Dementia Rating Sum of Boxes, FAQ = Functional Activities Questionnaire. Gender encoding in the dataset: 0.0 = Male, 1.0 = Female.
Classifier & Hyperparameter
Search Space / Distribution
Random Forest
n_estimators
[100,1000] (step: 5)
max_depth
[1,110]
min_samples_split
[2,30]
max_features
{log2,sqrt}
class_weight
{balanced,balanced_subsample}
Appendix
Table 4: Hyperparameter search space for the heterogeneous ensemble base classifiers. Categorical options are listed in braces; uniform and log-uniform continuous distributions are denoted by U and logU , respectively; discrete integer ranges are denoted by [min,max] .
Figure 7: Failure landscape of selective prediction under class imbalance. Heatmaps illustrate minority class Sensitivity at a fixed 30% global rejection rate across 56 synthetic conditions, parameterized by class prior ( π∈[0.05,0.50] , x-axis) and class separability ( d∈[0.5,3.0] , y-axis). (A) Standard global HTotal exhibits a severe structural failure boundary. Under conditions of severe imbalance and overlapping distributions ( d≤1.0,π≤0.20 ) the baseline actively defers minority patients (Sensitivity ≈0 ). (B) The proposed Hτ + CCR dual-architecture successfully rescues minority coverage within this diagnostic failure zone. Crucially, the architectural correction scales proportionally with the degree of imbalance, smoothly converging to identical standard baseline behavior when the natural prior is perfectly balanced ( π=0.50 ).
Figure 8: Scaling law of minority Sensitivity under dynamic rejection across class priors (n=300). (A) Sensitivity stability, defined as the linear slope of minority Sensitivity across the 0%–50% rejection range. Standard global HTotal (red) exhibits a severe non-linear scaling law, plunging into active minority exclusion as class imbalance increases. The proposed Hτ + CCR dual-architecture (green) fully neutralizes this artifact, maintaining a safe trajectory near zero across all priors. (B) Sensitivity evaluated at a fixed 30% global rejection rate. Standard UQ systematically collapses under severe imbalance ( π≤0.20 ), whereas the proposed framework guarantees robust minority retention. (C) Specificity (majority class retention) evaluated at the exact same 30% rejection rate. Under a fixed global rejection constraint, standard UQ maximizes Specificity by disproportionately concentrating deferrals within the minority class. By recovering minority Sensitivity (Panel B), the proposed framework mathematically redistributes these deferrals, resulting in a balanced and class-proportional rejection profile.
Figure 9: Rejection curves on synthetic data ( n=1000 ). Minority class Sensitivity is evaluated across a grid of class priors ( π ) and separability ( d ), comparing the proposed framework against global baselines.
Figure 10: Rejection curves on synthetic data ( n=300 ). Minority class Sensitivity is evaluated across a grid of class priors ( π ) and separability ( d ), comparing the proposed framework against global baselines.
Figure 11: Structural uncertainty divergence at a constrained sample size ( N=300 ). Evaluation of ensemble variance across cumulative class proportions in synthetic datasets. At a high class imbalance ( π=0.15 ), a severe and statistically significant uncertainty gap emerges between the classes ( p<0.01 ). This demonstrates that even when minority instances are correctly identified with high probability, they suffer from structurally higher ensemble variance simply due to the epistemic sparsity of the local region. This divergence narrows at moderate imbalance ( π=0.30 , p<0.01 ) and fully dissolves under balanced conditions ( π=0.50 , p=n.s. ), mirroring the real-world clinical vulnerabilities.
Figure 12: Structural uncertainty divergence at a large sample size ( N=1000 ). Replication of the asymmetry test with an expanded overall cohort ( N=1000 ). While the absolute magnitude of the ensemble variance decreases globally compared to N=300 , the relative structural divergence remains firmly intact. The uncertainty asymmetry gap persists significantly at π=0.15 ( p<0.01 ) and π=0.30 ( p<0.01 ), only dissipating when parity ( π=0.50 , p=n.s. ) is achieved.
Figure 13: Continuous error-filtering dynamics within the isolated triage queues. The plots trace the percentage of boundary errors (solid lines) versus confident correct predictions (dotted lines) deferred from the D0 queue (top row) and D1 queue (bottom row). By isolating the queues, the proposed Hτ + CCR architecture (green) forces monotonic uncertainty metrics to aggressively target boundary errors. Consequently, the framework symmetrically rejects true errors (FNs and FPs) at substantially higher rates than correct predictions. The ablation metric ( HTotal + CCR, purple) mimics the proposed framework in the D0 queue due to its monotonic behavior below the threshold, but fails in the D1 queue where standard entropy peaks at 0.5 (deferring true positives faster than false positives). This confirms that while queue isolation (CCR) is sufficient for filtering false negatives, threshold-aware geometry ( Hτ ) is strictly required to filter false positives.
Dataset
Uncertainty Framework
Progressor ccAUGRC (95% CI)
Stable ccAUGRC (95% CI)
Class Disparity
Global AUGRC (95% CI)
MS ( τ=π=0.13 )
HTotal (Global)
0.2044 [0.1300, 0.2863] ↑∗∗∗
0.0413 [0.0317, 0.0518] ↓∗∗∗
0.1631
0.0679 [0.0556, 0.0813]
Margin (Global)
0.0931 [0.0562, 0.1378] ↓ns
0.1463 [0.1247, 0.1672] ↑∗∗∗
0.0532
0.1430 [0.1233, 0.1623]
Hτ (Global)
0.1693 [0.1100, 0.2331] ↑∗∗∗
0.0561 [0.0449, 0.0680] ↓∗∗∗
0.1132
0.0734 [0.0605, 0.0867]
Random + CCR
0.1418 [0.0914, 0.1953] ↑∗∗∗
0.1362 [0.1192, 0.1525] ↑∗∗
0.0056
0.1404 [0.1231, 0.1577]
HTotal + CCR
0.1241 [0.0773, 0.1741] ↑∗∗
0.1392 [0.1223, 0.1558] ↑∗
0.0151
0.1404 [0.1235, 0.1577]
Hτ + CCR
0.1025 [0.0614, 0.1486]
0.1288 [0.1123, 0.1454]
0.0263
0.1275 [0.1119, 0.1442]
Appendix
Table 5: Generalized risk-coverage evaluation computed over a strict common support. This table recalculates the class-conditioned Area Under the Generalized Risk-Coverage Curve (ccAUGRC) and Global AUGRC using a standardized integration domain across all methods. The class disparity column has been added to mirror Table 2. Values are point estimates with 95% Confidence Intervals derived from 1000 post-prediction bootstrap iterations. Significance testing compares each baseline to the proposed reference framework ( Hτ + CCR), with p -values adjusted for multiple comparisons using the Holm-Bonferroni method. Lower values indicate better safety (lower risk).
Selective prediction lets models answer only when sufficiently reliable, trading coverage for lower error through abstention, referral, or extra evidence gathering.
As artificial intelligence systems move toward clinical deployment, ensuring reliable prediction behavior is fundamental for safety-critical decision-making tasks. One proposed safeguard is selective prediction, where models can defer uncertain predictions to human experts for review. In this work, we empirically evaluate the reliability of uncertainty-based selective prediction in multilabel clinical condition classification using multimodal ICU data. Across a range of state-of-the-art unimodal and multimodal models, we find that selective prediction can substantially degrade performance despite strong standard evaluation metrics. This failure is driven by severe class-dependent miscalibration, whereby models assign high uncertainty to correct predictions and low uncertainty to incorrect ones, particularly for underrepresented clinical conditions. Our results show that commonly used aggregate metrics can obscure these effects, limiting their ability to assess selective prediction behavior in this setting. Taken together, our findings characterize a task-specific failure mode of selective prediction in multimodal clinical condition classification and highlight the need for calibration-aware evaluation to provide strong guarantees of safety and robustness in clinical AI.
L. Julián Lechuga López, Farah E. Shamout, Tim G. J. Rudner
New York University · New York University Abu Dhabi · University of Toronto
Selective prediction systems can mitigate harms resulting from language model hallucinations by abstaining from answering in high-risk cases. Uncertainty quantification techniques are often employed to identify such cases, but are rarely evaluated in the context of the wider selective prediction policy and its ability to operate at low target error rates. We identify a model-dependent failure mode of entropy-based uncertainty methods that leads to unreliable abstention behaviour, and address it by combining entropy scores with a correctness probe signal. We find that across three QA benchmarks (TriviaQA, BioASQ, MedicalQA) and four model families, the combined score generally improves both the risk--coverage trade-off and calibration performance relative to entropy-only baselines. Our results highlight the importance of deployment-facing evaluation of uncertainty methods, using metrics that directly reflect whether a system can be trusted to operate at a stated risk level.
Edward Phillips, Fredrik K. Gustafsson, Sean Wu +2
Department of Engineering Science, University of Oxford · Oxford Suzhou Centre for Advanced Research, University of Oxford, Suzhou
Clinical language models (LMs) are increasingly applied to support clinical risk prediction from free-text notes, yet their uncertainty estimates often remain poorly calibrated and clinically unreliable. In this work, we propose Clinical Uncertainty Risk Alignment (CURA), a framework that aligns clinical LM-based risk estimates and uncertainty with both individual error likelihoods and cohort-level ambiguities. CURA first fine-tunes domain-specific clinical LMs to obtain task-adapted patient embeddings, and then performs uncertainty fine-tuning of a multi-head classifier using a bi-level uncertainty objective. Specifically, an individual-level calibration term aligns predictive uncertainty with each patient's likelihood of error, while a cohort-aware regularizer pulls risk estimates toward event rates in their local neighborhoods in the embedding space and places extra weight on ambiguous cohorts near the decision boundary. We further show that this cohort-aware term can be interpreted as a cross-entropy loss with neighborhood-informed soft labels, providing a label-smoothing view of our method. Extensive experiments on MIMIC-IV clinical risk prediction tasks across various clinical LMs show that CURA consistently improves calibration metrics without substantially compromising discrimination. Further analysis illustrates that CURA reduces overconfident false reassurance and yields more trustworthy uncertainty estimates for downstream clinical decision support.