Conformalized Credal Regions for Classification with Ambiguous Ground Truth
Authors: Michele Caprio, David Stutz, Shuo Li, Arnaud Doucet
Organizations: Department of Computer Science University of Manchester · DeepMind · Department of Computer and Information Science University of Pennsylvania · Department of Statistics, University of Oxford
An open question in Imprecise Probabilistic Machine Learning is how to empirically derive a credal region (i.e., a closed and convex family of probabilities on the output space) from the available data, without any prior knowledge or assumption. In classification problems, credal regions are a tool that is able to provide provable guarantees under realistic assumptions by characterizing the uncertainty about the distribution of the labels. Building on previous work, we show that credal regions can be directly constructed using conformal methods. This allows us to provide a novel extension of classical conformal prediction to problems with ambiguous ground truth, that is, when the exact labels for given inputs are not exactly known. The resulting construction enjoys desirable practical and theoretical properties: (i) conformal coverage guarantees, (ii) smaller prediction sets (compared to classical conformal prediction regions) and (iii) disentanglement of uncertainty sources (epistemic, aleatoric). We empirically verify our findings on both synthetic and real datasets.
Figures & tables
Figure 1: Suppose we are in a 3 -class classification setting, so Y={1,2,3} . Then, any probability measure P on Y can be seen as a probability vector. For example, suppose P({1})=0.6 , P({2})=0.3 , and P({3})=0.1 . We have that P≡(0.6,0.3,0.1)⊤ . Since its elements are positive and sum up to 1 , probability vector P belongs to the unit simplex Δ2 in R3 , the purple (2D) triangle in the figure. Then, plausibility region c(Xn+1) is a closed and convex body in Δ2 , such as the depicted orange (2D) heptagon (we depict it with a solid black border to highlight the fact that it is indeed closed).
Figure 2: Empirical distribution coverage levels. The left plot is on toy dataset; the middle plot is on Dermatology DDx dataset; the right plot is on CIFAR-10H.
Figure 3: True label coverage levels. The left plot is on the toy dataset; the middle plot is on Dermatology DDx; the third is on CIFAR-10H.
Figure 4: Average inefficiencies. The left plot is on the toy dataset; the middle plot is on Dermatology DDx; the third is on CIFAR-10H.
Figure 5: Inefficiency using different combinations of α and δ .
Table 1: Qualitative Examples from the DDx Dataset: The left column presents the original medical images, the middle column shows the label distributions annotated by experts, and the right column displays the credal regions constructed using our method.
Conformal prediction (CP) is a widely used frequentist framework to quantify uncertainty by constructing prediction sets with user-specified marginal coverage guarantees. In practice, CP is typically applied on top of probabilistic classifiers, which are able to express aleatoric but not epistemic uncertainty. In this paper, we consider the question of how to optimally employ CP on top of a more expressive formalism, namely credal sets, which can express both aleatoric and epistemic uncertainty. More specifically, we propose probabilistic Bernoulli prediction sets (BPS) and derive a variant that achieves conditional coverage for valid credal sets while remaining minimal in expected size. We then address the more realistic scenario in which the validity of the credal sets is not guaranteed. Assuming access to calibration data with ground-truth distributions over labels, we apply conformal risk control to BPS and derive a PAC-style guarantee: with high probability over the data, the achieved conditional coverage is at least the desired level. We validate our theoretical findings empirically over various datasets.
Alireza Javanmardi, Soroush H. Zargarbashi, Santo M. A. R. Thies +3
Prediction-set size offers a coarse summary of conformal uncertainty. We develop a richer perspective through the geometry of credal sets: families of probability distributions compatible with conformal prediction (CP) regions. We establish this representation for split CP, extending a correspondence recently identified for full CP. Building on this representation, we derive analytical characterisations and exact algorithms for several widely used credal uncertainty measures, reducing potentially exponential computations to linear time after sorting the conformal p-values. We further unify several of these measures with classical conformal summaries by integrating set-size penalties based on Tsallis entropies across error levels. We also prove an impossibility theorem: for common split-conformal regression constructions, uncertainty summaries of the induced credal sets cannot distinguish between inputs. Classification experiments on selective prediction and distribution-shift detection show that these summaries often recover useful ranking information lost by prediction-set size at a fixed error level, enabling finer-grained uncertainty assessment. These results establish credal geometry as a principled framework for quantifying conformal uncertainty and understanding its limitations.
Siu Lun Chau, Soroush H. Zargarbashi, Yusuf Sale +1
Epistemic Intelligence & Computation Lab, College of Computing & Data Science, Nanyang Technological University, Singapore · CISPA Helmholtz Center for Information Security, Germany · Munich Center for Machine Learning; Institute of Informatics, LMU Munich, Germany +3
Conformal prediction provides distribution-free coverage guarantees, but in many-class classification it may still under-cover specific classes or subpopulations, preventing safe deployment in high-stakes applications. We propose Cluster Frequency Conformal Prediction (CFCP), a plug-in framework that adapts conformal prediction to local structure in a learned representation space. CFCP clusters learned embeddings, estimates cluster-level label-frequency distributions from calibration data, and for each test point constructs a sample-specific probability vector by softly mixing nearby cluster distributions regularized with global-prior and reliability-aware shrinkage. This vector is then conformalized using standard set constructors. In the disjoint-split regime, CFCP inherits standard finite-sample marginal validity. Under additional assumptions, CFCP further admits a local-validity interpretation. Since representation clusters aggregate locally similar samples, their empirical class frequencies provide a stable estimate of local label ambiguity. Across image and text benchmarks, CFCP achieves the best class coverage in 15/16 dataset/score-family comparisons and a competitive prediction set size efficiency, with several settings substantially more efficient. Overall, our results show that cluster-frequency information provides an effective localized signal for improving classwise reliability in many-class conformal prediction.
Tomer Lavi, Bracha Shapira, Nadav Rappoport
Institute of Applied AI Research (AAIR) Faculty of Computer and Information Science Ben-Gurion University of the Negev, Beer-Sheva 84105, ISRAEL · Institute for Interdisciplinary Computational Science (ICS) Faculty of Computer and Information Science Ben-Gurion University of the Negev, Beer-Sheva 84105, ISRAEL