Robustness Quantification and Uncertainty Quantification: Comparing Two Methods for Assessing the Reliability of Classifier Predictions

Adrián Detavernier; Jasper De Bock

cs.LGMar 24, 2026

Robustness Quantification and Uncertainty Quantification: Comparing Two Methods for Assessing the Reliability of Classifier Predictions

Authors: Adrián Detavernier, Jasper De Bock

Organizations: Foundations Lab for imprecise probabilities, Ghent University, Belgium

Abstract

We consider two approaches for assessing the reliability of the individual predictions of a classifier: Robustness Quantification (RQ) and Uncertainty Quantification (UQ). We explain the conceptual differences between the two approaches, compare both approaches on a number of benchmark datasets and show that RQ is capable of outperforming UQ, both in a standard setting and in the presence of distribution shift. Beside showing that RQ can be competitive with UQ, we also demonstrate the complementarity of RQ and UQ by showing that a combination of both approaches can lead to even better reliability assessments.

Figures & tables

Panel of Figure 1 — Figure 1: Situating robustness quantification and uncertainty quantification with respect to reliability quantification.

Table 1: All datasets (UCI) used in the experiments.

Table 2: The AU-ARCs for the NBC for all datasets in the standard setting.

Table 3: The AU-ARCs for the GeFs for all datasets in the standard setting.

Figure 2: Each graph represents the mean of the ARCs for the NBC on the Student Performance Port (D14) dataset for a combination of training set size

N

and feature noise

\beta

Table 4: The AU-ARCs for the NBC for all datasets with training set size

N=50

and feature noise

\beta=0.2

Figure 3: This point cloud (logarithmic scale) depicts for each instance if its predicted class was correct (green) or wrong (red). This plot was made for the dataset D4, performing classification with the NBC.

Table 5: The AU-ARCs for the NBC of

u_{a}

r_{\mathrm{glob}}

r_{\mathrm{loc}}

and their combinations. The columns next to each hybrid show the found

\gamma^{*}

and the optimal

\gamma_{\mathrm{opt}}

(on the test set).

Table 6: The AU-ARCs for the GeFs of

u_{t}

r_{\mathrm{glob}}

r_{\mathrm{loc}}

and their combinations. The columns next to each hybrid show the found

\gamma^{*}

and the optimal

\gamma_{\mathrm{opt}}

(on the test set).

Uncertainty Quantification

Uncertainty quantification estimates how reliable predictions and scientific inferences are, and how uncertainty propagates into decisions.

See all 636 papers

Robustness Quantification for Discriminative Models: a New Robustness Metric and its Application to Dynamic Classifier Selection
Mar 24, 2026Rodrigo F. L. Lassance, Jasper De BockAdversarial Robustness Classification
Foundations Lab for imprecise probabilities (FLip), Ghent University, Ghent, Belgium · Statistics Dept., Federal University of São Carlos, São Carlos, São Paulo, Brazil · Institute of Mathematic and Computer Sciences, University of São Paulo, São Carlos, São Paulo, Brazil
ReliableNet: A Chance-Constrained Approach to Trustworthy Classification in Deep Learning
Aug 10, 2026Ange-Clément Akazan, Ineza Remy Mugenga, Abebe Geletu +2Chance-Constrained Optimization Classification
School of Agriculture and Science, University of KwaZulu-Natal, Pietermaritzburg, KwaZulu-Natal, South Africa · 2AIMS Research and Innovation Centre, African Institute for Mathematical Sciences, Kigali, Rwanda · 3African Institute for Mathematical Sciences, Kigali, Rwanda
From Misclassifications to Outliers: Joint Reliability Assessment in Classification
Mar 4, 2026Yang Li, Youyang Sha, Yinzhi Wang +4Confidence Estimation Classification
Intellindust AI Lab, Shenzhen, 518000, China. · Samsung AI Center, Cambridge, CB1 2GA, United Kingdom.

Mar 24, 2026J/K move · Enter open · S save

Robustness Quantification for Discriminative Models: a New Robustness Metric and its Application to Dynamic Classifier Selection

Rodrigo F. L. Lassance, Jasper De Bock

Foundations Lab for imprecise probabilities (FLip), Ghent University, Ghent, Belgium · Statistics Dept., Federal University of São Carlos, São Carlos, São Paulo, Brazil · Institute of Mathematic and Computer Sciences, University of São Paulo, São Carlos, São Paulo, Brazil

Adversarial Robustness Classification Robust Classification

Open paper AskPDF arXiv

Among the different possible strategies for evaluating the reliability of individual predictions of classifiers, robustness quantification stands out as a method that evaluates how much uncertainty a classifier could cope with before changing its prediction. However, its applicability is more limited than some of its alternatives, since it requires the use of generative models and restricts the analyses either to specific model architectures or discrete features. In this work, we propose a new robustness metric applicable to any probabilistic discriminative classifier and any type of features. We demonstrate that this new metric is capable of distinguishing between reliable and unreliable predictions, and use this observation to develop new strategies for dynamic classifier selection.

Abstract

Uncertainty Quantification

Explore similar work

Robustness Quantification for Discriminative Models: a New Robustness Metric and its Application to Dynamic Classifier Selection

ReliableNet: A Chance-Constrained Approach to Trustworthy Classification in Deep Learning

From Misclassifications to Outliers: Joint Reliability Assessment in Classification