Beyond Prediction-Set Size: Quantifying Conformal Uncertainty through Credal Geometry

Siu Lun Chau; Soroush H. Zargarbashi; Yusuf Sale; Michele Caprio

cs.LGFeb 2, 2026

Beyond Prediction-Set Size: Quantifying Conformal Uncertainty through Credal Geometry

Authors: Siu Lun Chau, Soroush H. Zargarbashi, Yusuf Sale, Michele Caprio

Organizations: Epistemic Intelligence & Computation Lab, College of Computing & Data Science, Nanyang Technological University, Singapore · CISPA Helmholtz Center for Information Security, Germany · Munich Center for Machine Learning; Institute of Informatics, LMU Munich, Germany · Munich Center for Machine Learning, Germany · Manchester Centre for AI Fundamentals, Manchester, United Kingdom · Department of Computer Science, The University of Manchester, United Kingdom

Abstract

Prediction-set size offers a coarse summary of conformal uncertainty. We develop a richer perspective through the geometry of credal sets: families of probability distributions compatible with conformal prediction (CP) regions. We establish this representation for split CP, extending a correspondence recently identified for full CP. Building on this representation, we derive analytical characterisations and exact algorithms for several widely used credal uncertainty measures, reducing potentially exponential computations to linear time after sorting the conformal p-values. We further unify several of these measures with classical conformal summaries by integrating set-size penalties based on Tsallis entropies across error levels. We also prove an impossibility theorem: for common split-conformal regression constructions, uncertainty summaries of the induced credal sets cannot distinguish between inputs. Classification experiments on selective prediction and distribution-shift detection show that these summaries often recover useful ranking information lost by prediction-set size at a fixed error level, enabling finer-grained uncertainty assessment. These results establish credal geometry as a principled framework for quantifying conformal uncertainty and understanding its limitations.

Figures & tables

Figure 1: Prediction-set size at

\alpha=0.05

are identical, yet the left-hand input has smaller p-values for every competing label. This motivates finer-grained conformal uncertainty quantification.

Table 1: LAC: mean

\pm

standard error over ten paired splits. Higher is better; set sizes use normalised contours. ImageNetV2 uses ResNetv2-50; C10-C averages all 75 corruption–severity conditions. Blue / red : best/second-best unrounded means across all methods; ties share a colour.

Appendix figures & tables47 assets

Supplementary material from the paper’s appendix.

Appendix

Summary	CIFAR- 100	Caltech- 101	Tiny ImageNet	ImageNetV2	CIFAR- 10	Fashion- MNIST	ImageNetV2 (ResNet-18)
Credal-based measures
$U_{0}$ ( $F$ )	$86.42_{\pm 0.27}$	$97.56_{\pm 0.11}$	$84.11_{\pm 0.48}$	$82.36_{\pm 0.46}$	$99.02_{\pm 0.04}$	$98.91_{\pm 0.06}$	$76.78_{\pm 0.30}$
$U_{1}$ (GH)	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{87.31}}_{\pm 0.29}$	${\color[rgb]{0.082,0.3945,0.7539}\mathbf{97.82}}_{\pm 0.11}$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{85.14}}_{\pm 0.46}$	${\color[rgb]{0.082,0.3945,0.7539}\mathbf{88.07}}_{\pm 0.32}$	$99.11_{\pm 0.04}$	$98.94_{\pm 0.06}$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{79.12}}_{\pm 0.30}$
$U_{2}$ (sat.)	$87.13_{\pm 0.31}$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{97.76}}_{\pm 0.13}$	$84.78_{\pm 0.45}$	$87.56_{\pm 0.34}$	$99.15_{\pm 0.03}$	$98.94_{\pm 0.06}$	$77.48_{\pm 0.37}$
$U_{\infty}$ (TV-MMI)	$86.29_{\pm 0.34}$	$97.69_{\pm 0.14}$	$83.46_{\pm 0.49}$	$86.17_{\pm 0.39}$	${\color[rgb]{0.082,0.3945,0.7539}\mathbf{99.18}}_{\pm 0.04}$	$98.94_{\pm 0.06}$	$74.51_{\pm 0.37}$
$I_{\mathrm{H}}$	$87.26_{\pm 0.29}$	$97.72_{\pm 0.13}$	$85.06_{\pm 0.46}$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{88.02}}_{\pm 0.33}$	$99.16_{\pm 0.03}$	$98.94_{\pm 0.06}$	$78.76_{\pm 0.31}$

Appendix

Table 2: LAC: AUARC (%) for all seven dataset/model conditions. Mean ±SE over ten paired splits; conventions as in Appendix E.2 .

Summary	CIFAR- 100	Caltech- 101	Tiny ImageNet	ImageNetV2	CIFAR- 10	Fashion- MNIST	ImageNetV2 (ResNet-18)
Credal-based measures
$U_{0}$ ( $F$ )	$86.02_{\pm 0.28}$	$93.78_{\pm 0.21}$	$83.60_{\pm 0.49}$	$79.06_{\pm 0.49}$	$98.93_{\pm 0.05}$	$96.88_{\pm 0.29}$	$76.12_{\pm 0.31}$
$U_{1}$ (GH)	$86.92_{\pm 0.29}$	$94.42_{\pm 0.20}$	$84.61_{\pm 0.46}$	$83.72_{\pm 0.43}$	$99.01_{\pm 0.05}$	$97.80_{\pm 0.22}$	$78.50_{\pm 0.31}$
$U_{2}$ (sat.)	$87.10_{\pm 0.30}$	$94.80_{\pm 0.19}$	$84.99_{\pm 0.45}$	$85.51_{\pm 0.41}$	$99.04_{\pm 0.05}$	$98.21_{\pm 0.15}$	$78.98_{\pm 0.32}$
$U_{\infty}$ (TV-MMI)	$87.08_{\pm 0.32}$	$95.16_{\pm 0.19}$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{85.11}}_{\pm 0.44}$	$86.10_{\pm 0.39}$	$99.08_{\pm 0.04}$	$98.41_{\pm 0.13}$	$79.04_{\pm 0.34}$
$I_{\mathrm{H}}$	$86.98_{\pm 0.28}$	$94.54_{\pm 0.19}$	$84.77_{\pm 0.46}$	$84.86_{\pm 0.42}$	$99.06_{\pm 0.05}$	$98.34_{\pm 0.13}$	$78.59_{\pm 0.30}$

Appendix

Table 3: APS: AUARC (%) for all seven dataset/model conditions. Mean ±SE over ten paired splits; conventions as in Appendix E.2 .

Summary	CIFAR- 100	Caltech- 101	Tiny ImageNet	ImageNetV2	CIFAR- 10	Fashion- MNIST	ImageNetV2 (ResNet-18)
Credal-based measures
$U_{0}$ ( $F$ )	$86.93_{\pm 0.28}$	$94.09_{\pm 0.20}$	$84.53_{\pm 0.47}$	$83.00_{\pm 0.43}$	$98.98_{\pm 0.05}$	$97.71_{\pm 0.24}$	$78.46_{\pm 0.31}$
$U_{1}$ (GH)	$87.08_{\pm 0.30}$	$94.53_{\pm 0.20}$	$84.87_{\pm 0.46}$	$84.88_{\pm 0.42}$	$99.02_{\pm 0.05}$	$98.09_{\pm 0.19}$	$78.86_{\pm 0.32}$
$U_{2}$ (sat.)	$87.11_{\pm 0.30}$	$94.82_{\pm 0.19}$	$85.02_{\pm 0.45}$	$85.63_{\pm 0.40}$	$99.04_{\pm 0.05}$	$98.26_{\pm 0.15}$	$79.02_{\pm 0.33}$
$U_{\infty}$ (TV-MMI)	$87.08_{\pm 0.32}$	$95.17_{\pm 0.19}$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{85.10}}_{\pm 0.44}$	$86.10_{\pm 0.39}$	$99.08_{\pm 0.04}$	$98.42_{\pm 0.13}$	$79.04_{\pm 0.34}$
$I_{\mathrm{H}}$	$87.11_{\pm 0.29}$	$94.64_{\pm 0.19}$	$84.98_{\pm 0.45}$	$85.58_{\pm 0.40}$	$99.07_{\pm 0.05}$	$98.37_{\pm 0.13}$	$78.93_{\pm 0.31}$

Appendix

Table 4: RAPS: AUARC (%) for all seven dataset/model conditions. Mean ±SE over ten paired splits; conventions as in Appendix E.2 .

Summary	CIFAR- 100	Caltech- 101	Tiny ImageNet	ImageNetV2	CIFAR- 10	Fashion- MNIST	ImageNetV2 (ResNet-18)
Credal-based measures
$U_{0}$ ( $F$ )	$86.62_{\pm 0.32}$	$96.51_{\pm 0.14}$	$84.85_{\pm 0.39}$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{86.79}}_{\pm 0.36}$	$97.58_{\pm 0.18}$	$94.76_{\pm 0.35}$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{79.29}}_{\pm 0.32}$
$U_{1}$ (GH)	$86.18_{\pm 0.33}$	$96.46_{\pm 0.14}$	$84.45_{\pm 0.37}$	$86.65_{\pm 0.34}$	$97.50_{\pm 0.18}$	$94.73_{\pm 0.34}$	$79.04_{\pm 0.33}$
$U_{2}$ (sat.)	$85.87_{\pm 0.34}$	$96.44_{\pm 0.14}$	$84.21_{\pm 0.36}$	$86.55_{\pm 0.35}$	$97.45_{\pm 0.18}$	$94.71_{\pm 0.34}$	$78.85_{\pm 0.33}$
$U_{\infty}$ (TV-MMI)	$85.51_{\pm 0.35}$	$96.39_{\pm 0.14}$	$83.95_{\pm 0.36}$	$86.42_{\pm 0.35}$	$97.37_{\pm 0.19}$	$94.68_{\pm 0.34}$	$78.58_{\pm 0.34}$
$I_{\mathrm{H}}$	$85.79_{\pm 0.34}$	$96.44_{\pm 0.14}$	$84.17_{\pm 0.35}$	$86.56_{\pm 0.35}$	$97.43_{\pm 0.18}$	$94.70_{\pm 0.35}$	$78.84_{\pm 0.33}$

Appendix

Table 5: SAPS: AUARC (%) for all seven dataset/model conditions. Mean ±SE over ten paired splits; conventions as in Appendix E.2 .

Dataset / model	LAC	APS	RAPS	SAPS
CIFAR-100	$62.693\,{\scriptstyle\pm\,0.572}$	$64.464\,{\scriptstyle\pm\,0.699}$	$64.478\,{\scriptstyle\pm\,0.704}$	$64.001\,{\scriptstyle\pm\,0.643}$
Caltech-101	$44.552\,{\scriptstyle\pm\,1.172}$	$26.380\,{\scriptstyle\pm\,1.327}$	$26.503\,{\scriptstyle\pm\,1.337}$	$32.817\,{\scriptstyle\pm\,1.575}$
Tiny ImageNet	$69.344\,{\scriptstyle\pm\,1.192}$	$75.530\,{\scriptstyle\pm\,1.084}$	$75.594\,{\scriptstyle\pm\,1.087}$	$75.604\,{\scriptstyle\pm\,1.080}$
ImageNetV2 (R50)	$60.842\,{\scriptstyle\pm\,0.528}$	$65.819\,{\scriptstyle\pm\,0.877}$	$66.018\,{\scriptstyle\pm\,0.876}$	$66.674\,{\scriptstyle\pm\,0.782}$
CIFAR-10	$44.655\,{\scriptstyle\pm\,1.790}$	$42.644\,{\scriptstyle\pm\,1.903}$	$42.648\,{\scriptstyle\pm\,1.901}$	$33.599\,{\scriptstyle\pm\,1.568}$
Fashion-MNIST	$48.419\,{\scriptstyle\pm\,1.579}$	$46.284\,{\scriptstyle\pm\,1.810}$	$46.396\,{\scriptstyle\pm\,1.808}$	$27.534\,{\scriptstyle\pm\,2.321}$

Appendix

Table 6: Error average precision (%) for

U_{2}

under the four conformal scores. Mean

\pm

one standard error over ten paired splits. ImageNetV2 uses ResNetv2-50 unless labelled ResNet-18.

Summary	CIFAR- 100	Caltech- 101	Tiny ImageNet	ImageNetV2	CIFAR- 10	Fashion- MNIST	ImageNetV2 (ResNet-18)
Credal-based measures
$U_{0}$ ( $F$ )	$81.35_{\pm 0.30}$	$87.83_{\pm 0.54}$	$83.85_{\pm 0.42}$	$74.95_{\pm 0.62}$	$90.81_{\pm 0.31}$	$91.66_{\pm 0.33}$	$79.46_{\pm 0.24}$
$U_{1}$ (GH)	${\color[rgb]{0.082,0.3945,0.7539}\mathbf{83.34}}_{\pm 0.37}$	${\color[rgb]{0.082,0.3945,0.7539}\mathbf{89.46}}_{\pm 0.32}$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{85.92}}_{\pm 0.40}$	${\color[rgb]{0.082,0.3945,0.7539}\mathbf{84.84}}_{\pm 0.42}$	$91.64_{\pm 0.32}$	$91.95_{\pm 0.33}$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{83.26}}_{\pm 0.16}$
$U_{2}$ (sat.)	$82.72_{\pm 0.41}$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{89.17}}_{\pm 0.35}$	$84.62_{\pm 0.42}$	$82.99_{\pm 0.40}$	$91.95_{\pm 0.33}$	${\color[rgb]{0.082,0.3945,0.7539}\mathbf{92.01}}_{\pm 0.33}$	$78.98_{\pm 0.30}$
$U_{\infty}$ (TV-MMI)	$80.27_{\pm 0.46}$	$88.81_{\pm 0.39}$	$80.99_{\pm 0.48}$	$79.18_{\pm 0.49}$	${\color[rgb]{0.082,0.3945,0.7539}\mathbf{91.99}}_{\pm 0.33}$	$91.92_{\pm 0.34}$	$72.77_{\pm 0.37}$
$I_{\mathrm{H}}$	$83.18_{\pm 0.38}$	$88.85_{\pm 0.34}$	$85.63_{\pm 0.41}$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{84.47}}_{\pm 0.41}$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{91.99}}_{\pm 0.33}$	$91.99_{\pm 0.33}$	$82.39_{\pm 0.19}$

Appendix

Table 7: LAC: Error AUROC (%) for all seven dataset/model conditions. Mean ±SE over ten paired splits; conventions as in Appendix E.2 .

Summary	CIFAR- 100	Caltech- 101	Tiny ImageNet	ImageNetV2	CIFAR- 10	Fashion- MNIST	ImageNetV2 (ResNet-18)
Credal-based measures
$U_{0}$ ( $F$ )	$80.64_{\pm 0.28}$	$67.92_{\pm 1.05}$	$83.09_{\pm 0.44}$	$69.32_{\pm 0.65}$	$90.08_{\pm 0.32}$	$81.31_{\pm 1.18}$	$78.61_{\pm 0.26}$
$U_{1}$ (GH)	$82.55_{\pm 0.35}$	$71.04_{\pm 0.97}$	$84.93_{\pm 0.41}$	$77.82_{\pm 0.56}$	$90.80_{\pm 0.36}$	$86.46_{\pm 0.82}$	$82.21_{\pm 0.18}$
$U_{2}$ (sat.)	$82.92_{\pm 0.40}$	$73.10_{\pm 0.92}$	$85.67_{\pm 0.41}$	$81.11_{\pm 0.49}$	$91.13_{\pm 0.40}$	$88.58_{\pm 0.63}$	$83.08_{\pm 0.20}$
$U_{\infty}$ (TV-MMI)	$82.83_{\pm 0.45}$	$75.13_{\pm 0.91}$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{85.90}}_{\pm 0.42}$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{82.10}}_{\pm 0.46}$	$91.38_{\pm 0.41}$	$89.61_{\pm 0.46}$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{83.16}}_{\pm 0.24}$
$I_{\mathrm{H}}$	$82.63_{\pm 0.36}$	$71.45_{\pm 0.91}$	$85.18_{\pm 0.41}$	$79.68_{\pm 0.54}$	$91.27_{\pm 0.40}$	$89.28_{\pm 0.53}$	$82.30_{\pm 0.18}$

Appendix

Table 8: APS: Error AUROC (%) for all seven dataset/model conditions. Mean ±SE over ten paired splits; conventions as in Appendix E.2 .

Summary	CIFAR- 100	Caltech- 101	Tiny ImageNet	ImageNetV2	CIFAR- 10	Fashion- MNIST	ImageNetV2 (ResNet-18)
Credal-based measures
$U_{0}$ ( $F$ )	$82.52_{\pm 0.35}$	$69.38_{\pm 0.99}$	$84.71_{\pm 0.41}$	$76.61_{\pm 0.59}$	$90.58_{\pm 0.35}$	$86.04_{\pm 0.92}$	$82.06_{\pm 0.17}$
$U_{1}$ (GH)	$82.86_{\pm 0.39}$	$71.63_{\pm 0.95}$	$85.40_{\pm 0.41}$	$79.96_{\pm 0.51}$	$90.95_{\pm 0.38}$	$88.04_{\pm 0.75}$	$82.82_{\pm 0.18}$
$U_{2}$ (sat.)	$82.94_{\pm 0.41}$	$73.24_{\pm 0.92}$	$85.73_{\pm 0.42}$	$81.31_{\pm 0.48}$	$91.17_{\pm 0.40}$	$88.86_{\pm 0.61}$	$83.14_{\pm 0.21}$
$U_{\infty}$ (TV-MMI)	$82.83_{\pm 0.45}$	$75.14_{\pm 0.91}$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{85.90}}_{\pm 0.42}$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{82.10}}_{\pm 0.46}$	$91.39_{\pm 0.41}$	$89.64_{\pm 0.46}$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{83.16}}_{\pm 0.24}$
$I_{\mathrm{H}}$	$82.92_{\pm 0.38}$	$72.02_{\pm 0.91}$	$85.59_{\pm 0.41}$	$81.01_{\pm 0.50}$	$91.35_{\pm 0.41}$	$89.47_{\pm 0.51}$	$82.90_{\pm 0.18}$

Appendix

Table 9: RAPS: Error AUROC (%) for all seven dataset/model conditions. Mean ±SE over ten paired splits; conventions as in Appendix E.2 .

Summary	CIFAR- 100	Caltech- 101	Tiny ImageNet	ImageNetV2	CIFAR- 10	Fashion- MNIST	ImageNetV2 (ResNet-18)
Credal-based measures
$U_{0}$ ( $F$ )	$82.53_{\pm 0.41}$	$81.42_{\pm 0.65}$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{85.85}}_{\pm 0.42}$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{83.27}}_{\pm 0.40}$	$78.12_{\pm 1.36}$	$64.84_{\pm 1.54}$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{83.65}}_{\pm 0.21}$
$U_{1}$ (GH)	$81.97_{\pm 0.43}$	$81.23_{\pm 0.67}$	$85.41_{\pm 0.43}$	$83.01_{\pm 0.38}$	$77.44_{\pm 1.38}$	$64.49_{\pm 1.50}$	$83.38_{\pm 0.24}$
$U_{2}$ (sat.)	$81.58_{\pm 0.45}$	$81.16_{\pm 0.68}$	$85.14_{\pm 0.43}$	$82.83_{\pm 0.38}$	$76.96_{\pm 1.37}$	$64.28_{\pm 1.49}$	$83.16_{\pm 0.25}$
$U_{\infty}$ (TV-MMI)	$81.05_{\pm 0.48}$	$81.04_{\pm 0.70}$	$84.79_{\pm 0.43}$	$82.59_{\pm 0.38}$	$76.17_{\pm 1.42}$	$63.92_{\pm 1.46}$	$82.80_{\pm 0.29}$
$I_{\mathrm{H}}$	$81.50_{\pm 0.44}$	$81.19_{\pm 0.68}$	$85.14_{\pm 0.44}$	$82.88_{\pm 0.39}$	$76.83_{\pm 1.40}$	$64.24_{\pm 1.50}$	$83.21_{\pm 0.26}$

Appendix

Table 10: SAPS: Error AUROC (%) for all seven dataset/model conditions. Mean ±SE over ten paired splits; conventions as in Appendix E.2 .

Summary	CIFAR- 100	Caltech- 101	Tiny ImageNet	ImageNetV2	CIFAR- 10	Fashion- MNIST	ImageNetV2 (ResNet-18)
Credal-based measures
$U_{0}$ ( $F$ )	$0.470_{\pm 0.005}$	$0.472_{\pm 0.012}$	$0.552_{\pm 0.008}$	$0.416_{\pm 0.010}$	$0.490_{\pm 0.012}$	$0.516_{\pm 0.008}$	$0.462_{\pm 0.004}$
$U_{1}$ (GH)	${\color[rgb]{0.082,0.3945,0.7539}\mathbf{0.541}}_{\pm 0.006}$	${\color[rgb]{0.082,0.3945,0.7539}\mathbf{0.492}}_{\pm 0.007}$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{0.601}}_{\pm 0.008}$	${\color[rgb]{0.082,0.3945,0.7539}\mathbf{0.566}}_{\pm 0.007}$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{0.507}}_{\pm 0.010}$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{0.527}}_{\pm 0.008}$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{0.564}}_{\pm 0.003}$
$U_{2}$ (sat.)	$0.530_{\pm 0.007}$	$0.489_{\pm 0.007}$	$0.574_{\pm 0.008}$	$0.529_{\pm 0.007}$	${\color[rgb]{0.082,0.3945,0.7539}\mathbf{0.510}}_{\pm 0.009}$	${\color[rgb]{0.082,0.3945,0.7539}\mathbf{0.529}}_{\pm 0.009}$	$0.504_{\pm 0.003}$
$U_{\infty}$ (TV-MMI)	$0.490_{\pm 0.007}$	$0.486_{\pm 0.008}$	$0.525_{\pm 0.008}$	$0.467_{\pm 0.007}$	$0.502_{\pm 0.008}$	$0.523_{\pm 0.011}$	$0.416_{\pm 0.005}$
$I_{\mathrm{H}}$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{0.533}}_{\pm 0.006}$	$0.487_{\pm 0.006}$	$0.583_{\pm 0.008}$	$0.549_{\pm 0.007}$	$0.501_{\pm 0.009}$	$0.521_{\pm 0.009}$	$0.543_{\pm 0.003}$

Appendix

Table 11: LAC: Pearson correlation with errors for all seven dataset/model conditions. Mean ±SE over ten paired splits; conventions as in Appendix E.2 .

Summary	CIFAR- 100	Caltech- 101	Tiny ImageNet	ImageNetV2	CIFAR- 10	Fashion- MNIST	ImageNetV2 (ResNet-18)
Credal-based measures
$U_{0}$ ( $F$ )	$0.504_{\pm 0.005}$	$0.414_{\pm 0.009}$	$0.563_{\pm 0.008}$	$0.399_{\pm 0.010}$	$0.329_{\pm 0.006}$	$0.374_{\pm 0.007}$	$0.505_{\pm 0.004}$
$U_{1}$ (GH)	${\color[rgb]{0.082,0.3945,0.7539}\mathbf{0.536}}_{\pm 0.006}$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{0.431}}_{\pm 0.005}$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{0.598}}_{\pm 0.008}$	${\color[rgb]{0.082,0.3945,0.7539}\mathbf{0.557}}_{\pm 0.007}$	$0.335_{\pm 0.006}$	$0.377_{\pm 0.007}$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{0.570}}_{\pm 0.003}$
$U_{2}$ (sat.)	$0.526_{\pm 0.006}$	$0.428_{\pm 0.005}$	$0.576_{\pm 0.008}$	$0.527_{\pm 0.006}$	$0.338_{\pm 0.006}$	$0.377_{\pm 0.007}$	$0.496_{\pm 0.005}$
$U_{\infty}$ (TV-MMI)	$0.487_{\pm 0.007}$	$0.425_{\pm 0.005}$	$0.516_{\pm 0.009}$	$0.466_{\pm 0.007}$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{0.339}}_{\pm 0.007}$	$0.377_{\pm 0.007}$	$0.390_{\pm 0.006}$
$I_{\mathrm{H}}$	$0.534_{\pm 0.006}$	$0.425_{\pm 0.005}$	$0.593_{\pm 0.008}$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{0.551}}_{\pm 0.007}$	$0.338_{\pm 0.006}$	$0.377_{\pm 0.007}$	$0.555_{\pm 0.003}$

Appendix

Table 12: LAC: Spearman correlation with errors for all seven dataset/model conditions. Mean ±SE over ten paired splits; conventions as in Appendix E.2 .

Summary	CIFAR- 100	Caltech- 101	Tiny ImageNet	ImageNetV2	CIFAR- 10	Fashion- MNIST	ImageNetV2 (ResNet-18)
Credal-based measures
$U_{0}$ ( $F$ )	$0.405_{\pm 0.006}$	$0.167_{\pm 0.015}$	$0.504_{\pm 0.008}$	$0.325_{\pm 0.009}$	$0.452_{\pm 0.015}$	$0.450_{\pm 0.011}$	$0.401_{\pm 0.004}$
$U_{1}$ (GH)	$0.483_{\pm 0.007}$	$0.209_{\pm 0.016}$	$0.560_{\pm 0.008}$	$0.457_{\pm 0.009}$	$0.450_{\pm 0.015}$	$0.450_{\pm 0.014}$	$0.509_{\pm 0.003}$
$U_{2}$ (sat.)	$0.510_{\pm 0.007}$	$0.244_{\pm 0.017}$	$0.589_{\pm 0.008}$	$0.514_{\pm 0.009}$	$0.445_{\pm 0.015}$	$0.437_{\pm 0.016}$	$0.554_{\pm 0.003}$
$U_{\infty}$ (TV-MMI)	${\color[rgb]{0.082,0.3945,0.7539}\mathbf{0.518}}_{\pm 0.008}$	$0.280_{\pm 0.017}$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{0.600}}_{\pm 0.008}$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{0.534}}_{\pm 0.008}$	$0.438_{\pm 0.015}$	$0.421_{\pm 0.018}$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{0.565}}_{\pm 0.004}$
$I_{\mathrm{H}}$	$0.512_{\pm 0.006}$	$0.218_{\pm 0.016}$	$0.586_{\pm 0.008}$	$0.490_{\pm 0.009}$	${\color[rgb]{0.082,0.3945,0.7539}\mathbf{0.488}}_{\pm 0.013}$	${\color[rgb]{0.082,0.3945,0.7539}\mathbf{0.497}}_{\pm 0.013}$	$0.537_{\pm 0.003}$

Appendix

Table 13: APS: Pearson correlation with errors for all seven dataset/model conditions. Mean ±SE over ten paired splits; conventions as in Appendix E.2 .

Summary	CIFAR- 100	Caltech- 101	Tiny ImageNet	ImageNetV2	CIFAR- 10	Fashion- MNIST	ImageNetV2 (ResNet-18)
Credal-based measures
$U_{0}$ ( $F$ )	$0.493_{\pm 0.005}$	$0.196_{\pm 0.012}$	$0.550_{\pm 0.008}$	$0.309_{\pm 0.010}$	$0.323_{\pm 0.006}$	$0.283_{\pm 0.011}$	$0.490_{\pm 0.005}$
$U_{1}$ (GH)	$0.523_{\pm 0.006}$	$0.230_{\pm 0.012}$	$0.581_{\pm 0.008}$	$0.445_{\pm 0.009}$	$0.329_{\pm 0.007}$	$0.330_{\pm 0.008}$	$0.552_{\pm 0.003}$
$U_{2}$ (sat.)	$0.529_{\pm 0.006}$	$0.253_{\pm 0.011}$	$0.593_{\pm 0.008}$	$0.497_{\pm 0.008}$	$0.331_{\pm 0.007}$	$0.349_{\pm 0.007}$	$0.567_{\pm 0.003}$
$U_{\infty}$ (TV-MMI)	$0.528_{\pm 0.007}$	$0.275_{\pm 0.011}$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{0.597}}_{\pm 0.008}$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{0.513}}_{\pm 0.007}$	$0.334_{\pm 0.007}$	$0.362_{\pm 0.006}$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{0.568}}_{\pm 0.004}$
$I_{\mathrm{H}}$	$0.525_{\pm 0.006}$	$0.235_{\pm 0.011}$	$0.585_{\pm 0.008}$	$0.474_{\pm 0.008}$	$0.333_{\pm 0.007}$	$0.358_{\pm 0.007}$	$0.553_{\pm 0.003}$

Appendix

Table 14: APS: Spearman correlation with errors for all seven dataset/model conditions. Mean ±SE over ten paired splits; conventions as in Appendix E.2 .

Summary	CIFAR- 100	Caltech- 101	Tiny ImageNet	ImageNetV2	CIFAR- 10	Fashion- MNIST	ImageNetV2 (ResNet-18)
Credal-based measures
$U_{0}$ ( $F$ )	$0.413_{\pm 0.007}$	$0.182_{\pm 0.015}$	$0.487_{\pm 0.008}$	$0.413_{\pm 0.009}$	$0.443_{\pm 0.015}$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{0.455}}_{\pm 0.013}$	$0.429_{\pm 0.003}$
$U_{1}$ (GH)	$0.475_{\pm 0.007}$	$0.216_{\pm 0.017}$	$0.552_{\pm 0.008}$	$0.481_{\pm 0.009}$	$0.445_{\pm 0.015}$	$0.441_{\pm 0.015}$	$0.511_{\pm 0.003}$
$U_{2}$ (sat.)	$0.499_{\pm 0.008}$	$0.246_{\pm 0.017}$	$0.580_{\pm 0.008}$	$0.515_{\pm 0.009}$	$0.442_{\pm 0.015}$	$0.430_{\pm 0.016}$	$0.545_{\pm 0.004}$
$U_{\infty}$ (TV-MMI)	$0.509_{\pm 0.008}$	$0.280_{\pm 0.017}$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{0.593}}_{\pm 0.008}$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{0.532}}_{\pm 0.008}$	$0.436_{\pm 0.015}$	$0.416_{\pm 0.018}$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{0.558}}_{\pm 0.005}$
$I_{\mathrm{H}}$	$0.515_{\pm 0.007}$	$0.227_{\pm 0.016}$	$0.587_{\pm 0.008}$	$0.512_{\pm 0.009}$	${\color[rgb]{0.082,0.3945,0.7539}\mathbf{0.486}}_{\pm 0.013}$	${\color[rgb]{0.082,0.3945,0.7539}\mathbf{0.492}}_{\pm 0.013}$	$0.548_{\pm 0.003}$

Appendix

Table 15: RAPS: Pearson correlation with errors for all seven dataset/model conditions. Mean ±SE over ten paired splits; conventions as in Appendix E.2 .

Summary	CIFAR- 100	Caltech- 101	Tiny ImageNet	ImageNetV2	CIFAR- 10	Fashion- MNIST	ImageNetV2 (ResNet-18)
Credal-based measures
$U_{0}$ ( $F$ )	$0.523_{\pm 0.005}$	$0.212_{\pm 0.012}$	$0.577_{\pm 0.008}$	$0.425_{\pm 0.009}$	$0.327_{\pm 0.006}$	$0.326_{\pm 0.008}$	$0.549_{\pm 0.003}$
$U_{1}$ (GH)	$0.529_{\pm 0.006}$	$0.237_{\pm 0.011}$	$0.589_{\pm 0.008}$	$0.479_{\pm 0.008}$	$0.330_{\pm 0.007}$	$0.344_{\pm 0.008}$	$0.562_{\pm 0.003}$
$U_{2}$ (sat.)	$0.530_{\pm 0.006}$	$0.254_{\pm 0.011}$	$0.594_{\pm 0.008}$	$0.500_{\pm 0.008}$	$0.332_{\pm 0.007}$	$0.352_{\pm 0.007}$	$0.568_{\pm 0.003}$
$U_{\infty}$ (TV-MMI)	$0.528_{\pm 0.007}$	$0.275_{\pm 0.011}$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{0.597}}_{\pm 0.008}$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{0.513}}_{\pm 0.007}$	$0.334_{\pm 0.007}$	$0.363_{\pm 0.006}$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{0.568}}_{\pm 0.004}$
$I_{\mathrm{H}}$	$0.529_{\pm 0.006}$	$0.241_{\pm 0.011}$	$0.592_{\pm 0.008}$	$0.496_{\pm 0.008}$	$0.333_{\pm 0.007}$	$0.360_{\pm 0.007}$	$0.564_{\pm 0.003}$

Appendix

Table 16: RAPS: Spearman correlation with errors for all seven dataset/model conditions. Mean ±SE over ten paired splits; conventions as in Appendix E.2 .

Summary	CIFAR- 100	Caltech- 101	Tiny ImageNet	ImageNetV2	CIFAR- 10	Fashion- MNIST	ImageNetV2 (ResNet-18)
Credal-based measures
$U_{0}$ ( $F$ )	$0.480_{\pm 0.007}$	$0.323_{\pm 0.016}$	$0.584_{\pm 0.008}$	$0.536_{\pm 0.007}$	$0.377_{\pm 0.012}$	$0.293_{\pm 0.023}$	$0.559_{\pm 0.004}$
$U_{1}$ (GH)	$0.481_{\pm 0.008}$	$0.330_{\pm 0.016}$	$0.581_{\pm 0.008}$	$0.533_{\pm 0.007}$	$0.381_{\pm 0.012}$	$0.295_{\pm 0.023}$	$0.552_{\pm 0.005}$
$U_{2}$ (sat.)	$0.485_{\pm 0.008}$	$0.334_{\pm 0.016}$	$0.583_{\pm 0.008}$	$0.534_{\pm 0.007}$	$0.383_{\pm 0.013}$	$0.295_{\pm 0.024}$	$0.552_{\pm 0.005}$
$U_{\infty}$ (TV-MMI)	$0.489_{\pm 0.008}$	$0.338_{\pm 0.015}$	$0.585_{\pm 0.008}$	$0.535_{\pm 0.007}$	$0.385_{\pm 0.014}$	$0.295_{\pm 0.024}$	$0.551_{\pm 0.005}$
$I_{\mathrm{H}}$	$0.506_{\pm 0.008}$	$0.352_{\pm 0.014}$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{0.601}}_{\pm 0.008}$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{0.552}}_{\pm 0.007}$	$0.401_{\pm 0.012}$	$0.298_{\pm 0.023}$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{0.568}}_{\pm 0.005}$

Appendix

Table 17: SAPS: Pearson correlation with errors for all seven dataset/model conditions. Mean ±SE over ten paired splits; conventions as in Appendix E.2 .

Summary	CIFAR- 100	Caltech- 101	Tiny ImageNet	ImageNetV2	CIFAR- 10	Fashion- MNIST	ImageNetV2 (ResNet-18)
Credal-based measures
$U_{0}$ ( $F$ )	$0.523_{\pm 0.006}$	$0.344_{\pm 0.009}$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{0.597}}_{\pm 0.009}$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{0.532}}_{\pm 0.006}$	$0.227_{\pm 0.012}$	$0.133_{\pm 0.013}$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{0.577}}_{\pm 0.003}$
$U_{1}$ (GH)	$0.514_{\pm 0.007}$	$0.342_{\pm 0.009}$	$0.589_{\pm 0.009}$	$0.528_{\pm 0.006}$	$0.221_{\pm 0.012}$	$0.129_{\pm 0.013}$	$0.572_{\pm 0.004}$
$U_{2}$ (sat.)	$0.508_{\pm 0.007}$	$0.341_{\pm 0.009}$	$0.585_{\pm 0.009}$	$0.525_{\pm 0.006}$	$0.217_{\pm 0.012}$	$0.128_{\pm 0.013}$	$0.568_{\pm 0.004}$
$U_{\infty}$ (TV-MMI)	$0.499_{\pm 0.007}$	$0.340_{\pm 0.010}$	$0.579_{\pm 0.009}$	$0.521_{\pm 0.006}$	$0.211_{\pm 0.013}$	$0.124_{\pm 0.013}$	$0.562_{\pm 0.005}$
$I_{\mathrm{H}}$	$0.507_{\pm 0.007}$	$0.341_{\pm 0.009}$	$0.585_{\pm 0.009}$	$0.525_{\pm 0.006}$	$0.216_{\pm 0.012}$	$0.127_{\pm 0.013}$	$0.569_{\pm 0.004}$

Appendix

Table 18: SAPS: Spearman correlation with errors for all seven dataset/model conditions. Mean ±SE over ten paired splits; conventions as in Appendix E.2 .

Summary	CIFAR- 100	Caltech- 101	Tiny ImageNet	ImageNetV2	CIFAR- 10	Fashion- MNIST	ImageNetV2 (ResNet-18)
Credal-based measures
$U_{0}$ ( $F$ )	$0.408_{\pm 0.008}$	$0.657_{\pm 0.024}$	$0.467_{\pm 0.012}$	$0.451_{\pm 0.010}$	$0.299_{\pm 0.010}$	$0.269_{\pm 0.008}$	$0.475_{\pm 0.010}$
$U_{1}$ (GH)	${\color[rgb]{0.082,0.3945,0.7539}\mathbf{0.422}}_{\pm 0.007}$	$0.495_{\pm 0.016}$	${\color[rgb]{0.082,0.3945,0.7539}\mathbf{0.484}}_{\pm 0.013}$	$0.540_{\pm 0.010}$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{0.307}}_{\pm 0.011}$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{0.271}}_{\pm 0.008}$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{0.490}}_{\pm 0.011}$
$U_{2}$ (sat.)	$0.387_{\pm 0.007}$	$0.449_{\pm 0.015}$	$0.444_{\pm 0.013}$	$0.447_{\pm 0.009}$	$0.305_{\pm 0.011}$	$0.268_{\pm 0.008}$	$0.378_{\pm 0.010}$
$U_{\infty}$ (TV-MMI)	$0.339_{\pm 0.007}$	$0.418_{\pm 0.014}$	$0.394_{\pm 0.012}$	$0.355_{\pm 0.008}$	$0.296_{\pm 0.011}$	$0.259_{\pm 0.008}$	$0.272_{\pm 0.009}$
$I_{\mathrm{H}}$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{0.415}}_{\pm 0.007}$	$0.513_{\pm 0.015}$	$0.468_{\pm 0.013}$	$0.509_{\pm 0.010}$	${\color[rgb]{0.082,0.3945,0.7539}\mathbf{0.308}}_{\pm 0.011}$	${\color[rgb]{0.082,0.3945,0.7539}\mathbf{0.275}}_{\pm 0.008}$	$0.462_{\pm 0.011}$

Appendix

Table 19: LAC: Pearson correlation with NLL for all seven dataset/model conditions. Mean ±SE over ten paired splits; conventions as in Appendix E.2 .

Summary	CIFAR- 100	Caltech- 101	Tiny ImageNet	ImageNetV2	CIFAR- 10	Fashion- MNIST	ImageNetV2 (ResNet-18)
Credal-based measures
$U_{0}$ ( $F$ )	$0.806_{\pm 0.004}$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{0.945}}_{\pm 0.007}$	$0.802_{\pm 0.007}$	$0.711_{\pm 0.007}$	$0.978_{\pm 0.002}$	$0.822_{\pm 0.002}$	$0.728_{\pm 0.004}$
$U_{1}$ (GH)	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{0.837}}_{\pm 0.005}$	$0.902_{\pm 0.005}$	${\color[rgb]{0.082,0.3945,0.7539}\mathbf{0.824}}_{\pm 0.007}$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{0.829}}_{\pm 0.006}$	$0.986_{\pm 0.001}$	$0.830_{\pm 0.002}$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{0.788}}_{\pm 0.004}$
$U_{2}$ (sat.)	$0.822_{\pm 0.006}$	$0.873_{\pm 0.004}$	$0.798_{\pm 0.008}$	$0.764_{\pm 0.006}$	$0.983_{\pm 0.001}$	$0.833_{\pm 0.002}$	$0.679_{\pm 0.006}$
$U_{\infty}$ (TV-MMI)	$0.786_{\pm 0.007}$	$0.843_{\pm 0.003}$	$0.749_{\pm 0.009}$	$0.697_{\pm 0.007}$	$0.971_{\pm 0.001}$	$0.835_{\pm 0.002}$	$0.559_{\pm 0.006}$
$I_{\mathrm{H}}$	$0.834_{\pm 0.005}$	$0.874_{\pm 0.004}$	$0.819_{\pm 0.007}$	$0.809_{\pm 0.006}$	$0.979_{\pm 0.001}$	$0.834_{\pm 0.002}$	$0.771_{\pm 0.005}$

Appendix

Table 20: LAC: Spearman correlation with NLL for all seven dataset/model conditions. Mean ±SE over ten paired splits; conventions as in Appendix E.2 .

Summary	CIFAR- 100	Caltech- 101	Tiny ImageNet	ImageNetV2	CIFAR- 10	Fashion- MNIST	ImageNetV2 (ResNet-18)
Credal-based measures
$U_{0}$ ( $F$ )	$0.365_{\pm 0.008}$	$0.615_{\pm 0.013}$	$0.432_{\pm 0.012}$	$0.358_{\pm 0.010}$	$0.240_{\pm 0.010}$	$0.167_{\pm 0.009}$	$0.425_{\pm 0.009}$
$U_{1}$ (GH)	$0.399_{\pm 0.008}$	$0.668_{\pm 0.013}$	$0.466_{\pm 0.012}$	$0.497_{\pm 0.010}$	$0.232_{\pm 0.009}$	$0.162_{\pm 0.007}$	$0.500_{\pm 0.010}$
$U_{2}$ (sat.)	$0.401_{\pm 0.008}$	$0.692_{\pm 0.012}$	$0.478_{\pm 0.013}$	${\color[rgb]{0.082,0.3945,0.7539}\mathbf{0.534}}_{\pm 0.009}$	$0.226_{\pm 0.009}$	$0.153_{\pm 0.007}$	$0.510_{\pm 0.011}$
$U_{\infty}$ (TV-MMI)	$0.391_{\pm 0.008}$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{0.703}}_{\pm 0.012}$	$0.478_{\pm 0.013}$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{0.531}}_{\pm 0.009}$	$0.218_{\pm 0.009}$	$0.141_{\pm 0.007}$	$0.498_{\pm 0.011}$
$I_{\mathrm{H}}$	${\color[rgb]{0.082,0.3945,0.7539}\mathbf{0.421}}_{\pm 0.008}$	$0.673_{\pm 0.012}$	${\color[rgb]{0.082,0.3945,0.7539}\mathbf{0.483}}_{\pm 0.013}$	$0.518_{\pm 0.010}$	$0.267_{\pm 0.011}$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{0.202}}_{\pm 0.007}$	${\color[rgb]{0.082,0.3945,0.7539}\mathbf{0.513}}_{\pm 0.011}$

Appendix

Table 21: APS: Pearson correlation with NLL for all seven dataset/model conditions. Mean ±SE over ten paired splits; conventions as in Appendix E.2 .

Summary	CIFAR- 100	Caltech- 101	Tiny ImageNet	ImageNetV2	CIFAR- 10	Fashion- MNIST	ImageNetV2 (ResNet-18)
Credal-based measures
$U_{0}$ ( $F$ )	$0.787_{\pm 0.004}$	$0.833_{\pm 0.005}$	$0.786_{\pm 0.007}$	$0.594_{\pm 0.008}$	$0.952_{\pm 0.002}$	$0.484_{\pm 0.015}$	$0.706_{\pm 0.004}$
$U_{1}$ (GH)	$0.824_{\pm 0.005}$	$0.874_{\pm 0.004}$	$0.813_{\pm 0.007}$	$0.754_{\pm 0.007}$	$0.966_{\pm 0.001}$	$0.659_{\pm 0.008}$	$0.776_{\pm 0.004}$
$U_{2}$ (sat.)	$0.830_{\pm 0.005}$	$0.892_{\pm 0.003}$	$0.821_{\pm 0.007}$	$0.809_{\pm 0.007}$	$0.973_{\pm 0.001}$	$0.763_{\pm 0.008}$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{0.785}}_{\pm 0.005}$
$U_{\infty}$ (TV-MMI)	$0.827_{\pm 0.006}$	$0.905_{\pm 0.003}$	$0.821_{\pm 0.007}$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{0.823}}_{\pm 0.007}$	$0.977_{\pm 0.001}$	$0.822_{\pm 0.007}$	$0.780_{\pm 0.005}$
$I_{\mathrm{H}}$	$0.828_{\pm 0.005}$	$0.884_{\pm 0.003}$	$0.818_{\pm 0.007}$	$0.792_{\pm 0.007}$	$0.975_{\pm 0.001}$	$0.804_{\pm 0.006}$	$0.779_{\pm 0.004}$

Appendix

Table 22: APS: Spearman correlation with NLL for all seven dataset/model conditions. Mean ±SE over ten paired splits; conventions as in Appendix E.2 .

Summary	CIFAR- 100	Caltech- 101	Tiny ImageNet	ImageNetV2	CIFAR- 10	Fashion- MNIST	ImageNetV2 (ResNet-18)
Credal-based measures
$U_{0}$ ( $F$ )	$0.357_{\pm 0.008}$	$0.635_{\pm 0.013}$	$0.415_{\pm 0.011}$	$0.472_{\pm 0.009}$	$0.230_{\pm 0.010}$	$0.169_{\pm 0.008}$	$0.447_{\pm 0.010}$
$U_{1}$ (GH)	$0.385_{\pm 0.008}$	$0.675_{\pm 0.013}$	$0.457_{\pm 0.012}$	$0.522_{\pm 0.009}$	$0.226_{\pm 0.010}$	$0.156_{\pm 0.007}$	$0.497_{\pm 0.010}$
$U_{2}$ (sat.)	$0.389_{\pm 0.008}$	$0.694_{\pm 0.012}$	$0.471_{\pm 0.012}$	${\color[rgb]{0.082,0.3945,0.7539}\mathbf{0.535}}_{\pm 0.009}$	$0.223_{\pm 0.010}$	$0.147_{\pm 0.007}$	$0.506_{\pm 0.011}$
$U_{\infty}$ (TV-MMI)	$0.383_{\pm 0.009}$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{0.703}}_{\pm 0.012}$	$0.472_{\pm 0.013}$	$0.531_{\pm 0.009}$	$0.216_{\pm 0.009}$	$0.136_{\pm 0.007}$	$0.497_{\pm 0.011}$
$I_{\mathrm{H}}$	${\color[rgb]{0.082,0.3945,0.7539}\mathbf{0.412}}_{\pm 0.008}$	$0.679_{\pm 0.011}$	${\color[rgb]{0.082,0.3945,0.7539}\mathbf{0.480}}_{\pm 0.013}$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{0.534}}_{\pm 0.010}$	$0.263_{\pm 0.011}$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{0.195}}_{\pm 0.007}$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{0.512}}_{\pm 0.011}$

Appendix

Table 23: RAPS: Pearson correlation with NLL for all seven dataset/model conditions. Mean ±SE over ten paired splits; conventions as in Appendix E.2 .

Summary	CIFAR- 100	Caltech- 101	Tiny ImageNet	ImageNetV2	CIFAR- 10	Fashion- MNIST	ImageNetV2 (ResNet-18)
Credal-based measures
$U_{0}$ ( $F$ )	$0.825_{\pm 0.005}$	$0.859_{\pm 0.004}$	$0.812_{\pm 0.007}$	$0.732_{\pm 0.007}$	$0.961_{\pm 0.001}$	$0.642_{\pm 0.009}$	$0.775_{\pm 0.004}$
$U_{1}$ (GH)	$0.830_{\pm 0.005}$	$0.881_{\pm 0.004}$	$0.819_{\pm 0.007}$	$0.792_{\pm 0.007}$	$0.969_{\pm 0.001}$	$0.725_{\pm 0.008}$	$0.784_{\pm 0.004}$
$U_{2}$ (sat.)	$0.830_{\pm 0.005}$	$0.894_{\pm 0.003}$	$0.821_{\pm 0.007}$	$0.813_{\pm 0.007}$	$0.973_{\pm 0.001}$	$0.779_{\pm 0.008}$	$0.785_{\pm 0.005}$
$U_{\infty}$ (TV-MMI)	$0.827_{\pm 0.006}$	$0.905_{\pm 0.003}$	$0.821_{\pm 0.007}$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{0.823}}_{\pm 0.007}$	$0.977_{\pm 0.001}$	$0.825_{\pm 0.009}$	$0.780_{\pm 0.005}$
$I_{\mathrm{H}}$	$0.832_{\pm 0.005}$	$0.890_{\pm 0.003}$	$0.822_{\pm 0.007}$	$0.813_{\pm 0.007}$	$0.975_{\pm 0.001}$	$0.817_{\pm 0.007}$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{0.786}}_{\pm 0.004}$

Appendix

Table 24: RAPS: Spearman correlation with NLL for all seven dataset/model conditions. Mean ±SE over ten paired splits; conventions as in Appendix E.2 .

Summary	CIFAR- 100	Caltech- 101	Tiny ImageNet	ImageNetV2	CIFAR- 10	Fashion- MNIST	ImageNetV2 (ResNet-18)
Credal-based measures
$U_{0}$ ( $F$ )	$0.344_{\pm 0.007}$	$0.716_{\pm 0.013}$	$0.459_{\pm 0.012}$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{0.522}}_{\pm 0.009}$	$0.165_{\pm 0.011}$	$0.052_{\pm 0.012}$	$0.489_{\pm 0.010}$
$U_{1}$ (GH)	$0.344_{\pm 0.008}$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{0.718}}_{\pm 0.012}$	$0.456_{\pm 0.012}$	$0.517_{\pm 0.009}$	$0.168_{\pm 0.011}$	$0.054_{\pm 0.012}$	$0.484_{\pm 0.010}$
$U_{2}$ (sat.)	$0.346_{\pm 0.008}$	$0.718_{\pm 0.012}$	$0.456_{\pm 0.012}$	$0.516_{\pm 0.009}$	$0.169_{\pm 0.011}$	$0.055_{\pm 0.012}$	$0.482_{\pm 0.010}$
$U_{\infty}$ (TV-MMI)	$0.348_{\pm 0.008}$	$0.713_{\pm 0.012}$	$0.457_{\pm 0.012}$	$0.512_{\pm 0.009}$	$0.171_{\pm 0.011}$	$0.056_{\pm 0.012}$	$0.478_{\pm 0.010}$
$I_{\mathrm{H}}$	$0.359_{\pm 0.008}$	$0.716_{\pm 0.011}$	$0.468_{\pm 0.012}$	$0.520_{\pm 0.009}$	$0.179_{\pm 0.012}$	$0.058_{\pm 0.013}$	$0.481_{\pm 0.011}$

Appendix

Table 25: SAPS: Pearson correlation with NLL for all seven dataset/model conditions. Mean ±SE over ten paired splits; conventions as in Appendix E.2 .

Summary	CIFAR- 100	Caltech- 101	Tiny ImageNet	ImageNetV2	CIFAR- 10	Fashion- MNIST	ImageNetV2 (ResNet-18)
Credal-based measures
$U_{0}$ ( $F$ )	$0.782_{\pm 0.006}$	$0.907_{\pm 0.003}$	$0.788_{\pm 0.005}$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{0.832}}_{\pm 0.006}$	$0.221_{\pm 0.009}$	$0.125_{\pm 0.013}$	$0.777_{\pm 0.005}$
$U_{1}$ (GH)	$0.756_{\pm 0.006}$	$0.891_{\pm 0.002}$	$0.764_{\pm 0.005}$	$0.815_{\pm 0.006}$	$0.212_{\pm 0.009}$	$0.120_{\pm 0.013}$	$0.766_{\pm 0.005}$
$U_{2}$ (sat.)	$0.740_{\pm 0.006}$	$0.886_{\pm 0.002}$	$0.751_{\pm 0.005}$	$0.804_{\pm 0.006}$	$0.206_{\pm 0.009}$	$0.116_{\pm 0.013}$	$0.759_{\pm 0.005}$
$U_{\infty}$ (TV-MMI)	$0.721_{\pm 0.006}$	$0.875_{\pm 0.003}$	$0.738_{\pm 0.005}$	$0.791_{\pm 0.006}$	$0.200_{\pm 0.010}$	$0.112_{\pm 0.013}$	$0.749_{\pm 0.005}$
$I_{\mathrm{H}}$	$0.736_{\pm 0.006}$	$0.884_{\pm 0.002}$	$0.749_{\pm 0.005}$	$0.802_{\pm 0.006}$	$0.206_{\pm 0.009}$	$0.117_{\pm 0.013}$	$0.758_{\pm 0.005}$

Appendix

Table 26: SAPS: Spearman correlation with NLL for all seven dataset/model conditions. Mean ±SE over ten paired splits; conventions as in Appendix E.2 .

Ranking	C100	C10.1	Tiny	SVHN	Fashion	C10-C
Credal-based measures
$U_{0}$ ( $F$ )	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{86.08}}_{\pm 0.08}$	$59.84_{\pm 0.17}$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{85.80}}_{\pm 0.09}$	${\color[rgb]{0.082,0.3945,0.7539}\mathbf{93.80}}_{\pm 0.06}$	${\color[rgb]{0.082,0.3945,0.7539}\mathbf{96.71}}_{\pm 0.03}$	$68.76_{\pm 0.08}$
$U_{1}$ (GH)	${\color[rgb]{0.082,0.3945,0.7539}\mathbf{86.24}}_{\pm 0.08}$	$60.08_{\pm 0.17}$	${\color[rgb]{0.082,0.3945,0.7539}\mathbf{85.88}}_{\pm 0.08}$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{93.34}}_{\pm 0.07}$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{95.72}}_{\pm 0.03}$	${\color[rgb]{0.082,0.3945,0.7539}\mathbf{68.93}}_{\pm 0.08}$
$U_{2}$ (sat.)	$86.01_{\pm 0.08}$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{60.15}}_{\pm 0.17}$	$85.59_{\pm 0.08}$	$92.61_{\pm 0.07}$	$94.59_{\pm 0.04}$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{68.87}}_{\pm 0.09}$
$U_{\infty}$ (TV-MMI)	$85.35_{\pm 0.09}$	$60.09_{\pm 0.17}$	$84.87_{\pm 0.09}$	$91.20_{\pm 0.08}$	$92.81_{\pm 0.06}$	$68.58_{\pm 0.09}$
$I_{\mathrm{H}}$	$85.94_{\pm 0.09}$	${\color[rgb]{0.082,0.3945,0.7539}\mathbf{60.17}}_{\pm 0.17}$	$85.51_{\pm 0.09}$	$92.39_{\pm 0.08}$	$94.24_{\pm 0.05}$	$68.85_{\pm 0.09}$

Appendix

Table 27: LAC OOD AUROC (%), mean

\pm

one standard error. C100, C10.1, Tiny, and Fashion abbreviate CIFAR-100, CIFAR-10.1, Tiny ImageNet, and Fashion-MNIST; C10-C averages all 75 corruption conditions. Five credal summaries, including

U_{2}

, and both raw and normalised cardinalities are shown. Blue / red : best/second-best unrounded means across all methods; ties share a colour.

Ranking	C100	C10.1	Tiny	SVHN	Fashion	C10-C
Credal-based measures
$U_{0}$ ( $F$ )	$84.99_{\pm 0.08}$	$59.28_{\pm 0.19}$	$84.76_{\pm 0.09}$	${\color[rgb]{0.082,0.3945,0.7539}\mathbf{93.08}}_{\pm 0.07}$	$96.26_{\pm 0.03}$	$68.04_{\pm 0.09}$
$U_{1}$ (GH)	$85.29_{\pm 0.07}$	$59.50_{\pm 0.19}$	$84.97_{\pm 0.09}$	$92.82_{\pm 0.08}$	$95.58_{\pm 0.04}$	$68.27_{\pm 0.09}$
$U_{2}$ (sat.)	$85.42_{\pm 0.07}$	$59.63_{\pm 0.19}$	$85.05_{\pm 0.09}$	$92.62_{\pm 0.08}$	$95.14_{\pm 0.04}$	$68.38_{\pm 0.09}$
$U_{\infty}$ (TV-MMI)	$85.49_{\pm 0.07}$	$59.77_{\pm 0.19}$	$85.06_{\pm 0.08}$	$92.36_{\pm 0.08}$	$94.64_{\pm 0.04}$	$68.46_{\pm 0.09}$
$I_{\mathrm{H}}$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{85.61}}_{\pm 0.07}$	$59.72_{\pm 0.19}$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{85.23}}_{\pm 0.08}$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{92.84}}_{\pm 0.08}$	$95.33_{\pm 0.04}$	$68.48_{\pm 0.09}$

Appendix

Table 28: APS OOD AUROC (%), with the same benchmarks, baselines, aggregation, and paired seeds as Table 27 . Blue / red : best/second-best unrounded means across all methods; ties share a colour.

Ranking	C100	C10.1	Tiny	SVHN	Fashion	C10-C
Credal-based measures
$U_{0}$ ( $F$ )	$85.31_{\pm 0.08}$	$59.45_{\pm 0.19}$	$85.02_{\pm 0.09}$	${\color[rgb]{0.082,0.3945,0.7539}\mathbf{93.01}}_{\pm 0.08}$	${\color[rgb]{0.082,0.3945,0.7539}\mathbf{95.87}}_{\pm 0.03}$	$68.27_{\pm 0.09}$
$U_{1}$ (GH)	$85.40_{\pm 0.08}$	$59.57_{\pm 0.19}$	$85.06_{\pm 0.09}$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{92.78}}_{\pm 0.08}$	$95.41_{\pm 0.04}$	$68.36_{\pm 0.09}$
$U_{\infty}$ (TV-MMI)	$85.49_{\pm 0.07}$	$59.76_{\pm 0.19}$	$85.06_{\pm 0.09}$	$92.36_{\pm 0.08}$	$94.64_{\pm 0.04}$	$68.46_{\pm 0.09}$
$I_{\mathrm{H}}$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{85.62}}_{\pm 0.08}$	$59.75_{\pm 0.19}$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{85.22}}_{\pm 0.09}$	$92.74_{\pm 0.08}$	$95.16_{\pm 0.04}$	$68.50_{\pm 0.09}$
Probability-vector-based measures

Appendix

Table 29: RAPS OOD AUROC (%), using fixed penalty

.01

and

k_{\rm reg}=5

, with the same benchmarks, probability and cardinality baselines, aggregation, and paired seeds as Table 27 . The four original credal summaries are evaluated. Blue / red : best/second-best unrounded means across all methods; ties share a colour.

Ranking	C100	C10.1	Tiny	SVHN	Fashion	C10-C
Credal-based measures
$U_{0}$ ( $F$ )	$71.37_{\pm 0.15}$	$53.77_{\pm 0.34}$	$72.22_{\pm 0.17}$	$81.51_{\pm 0.13}$	$84.03_{\pm 0.12}$	$59.12_{\pm 0.13}$
$U_{1}$ (GH)	$70.67_{\pm 0.15}$	$53.61_{\pm 0.37}$	$71.54_{\pm 0.16}$	$80.76_{\pm 0.12}$	$83.21_{\pm 0.13}$	$58.79_{\pm 0.14}$
$U_{\infty}$ (TV-MMI)	$69.67_{\pm 0.15}$	$53.40_{\pm 0.39}$	$70.56_{\pm 0.14}$	$79.66_{\pm 0.10}$	$81.99_{\pm 0.13}$	$58.35_{\pm 0.14}$
$I_{\mathrm{H}}$	$70.19_{\pm 0.15}$	$53.51_{\pm 0.38}$	$71.08_{\pm 0.16}$	$80.26_{\pm 0.11}$	$82.64_{\pm 0.12}$	$58.57_{\pm 0.14}$
Probability-vector-based measures

Appendix

Table 30: SAPS OOD AUROC (%), using fixed weight

.2

	Five dataset shifts		CIFAR-10-C, severity 3
	Credal-based	Set-size-based	Credal-based	Set-size-based
$n_{\mathrm{cal}}$	TV-MMI	$\|C^{\mathrm{raw}}_{0.01}\|$	TV-MMI	$\|C^{\mathrm{raw}}_{0.01}\|$
100	${{\color[rgb]{0.7773,0.1563,0.1563}\mathbf{0.8140}}_{\pm.0051}}$	${{\color[rgb]{0.082,0.3945,0.7539}\mathbf{0.8224}}_{\pm.0074}}$	${{\color[rgb]{0.7773,0.1563,0.1563}\mathbf{0.6565}}_{\pm.0059}}$	${{\color[rgb]{0.082,0.3945,0.7539}\mathbf{0.6600}}_{\pm.0064}}$
250	${{\color[rgb]{0.7773,0.1563,0.1563}\mathbf{0.8281}}_{\pm.0015}}$	${{\color[rgb]{0.082,0.3945,0.7539}\mathbf{0.8300}}_{\pm.0026}}$	${{\color[rgb]{0.082,0.3945,0.7539}\mathbf{0.6720}}_{\pm.0023}}$	${{\color[rgb]{0.7773,0.1563,0.1563}\mathbf{0.6615}}_{\pm.0039}}$
500	${{\color[rgb]{0.082,0.3945,0.7539}\mathbf{0.8300}}_{\pm.0015}}$	${{\color[rgb]{0.7773,0.1563,0.1563}\mathbf{0.8217}}_{\pm.0050}}$	${{\color[rgb]{0.082,0.3945,0.7539}\mathbf{0.6747}}_{\pm.0024}}$	${{\color[rgb]{0.7773,0.1563,0.1563}\mathbf{0.6503}}_{\pm.0059}}$
1000	${{\color[rgb]{0.082,0.3945,0.7539}\mathbf{0.8336}}_{\pm.0009}}$	${{\color[rgb]{0.7773,0.1563,0.1563}\mathbf{0.8254}}_{\pm.0040}}$	${{\color[rgb]{0.082,0.3945,0.7539}\mathbf{0.6817}}_{\pm.0010}}$	${{\color[rgb]{0.7773,0.1563,0.1563}\mathbf{0.6524}}_{\pm.0047}}$

Appendix

Table 31: Calibration-size sensitivity of distribution-shift AUROC (on a 0–1 scale). Entries are means with one standard error over 10 randomised-APS score draws. Dataset-shift results average over five datasets; CIFAR-10-C results average over all 15 severity-3 corruptions. These are supplementary APS results, separate from the main LAC comparison. Blue / red : best/second-best unrounded means across all methods; ties share a colour. Methods are compared within each row and benchmark family.

		Credal-based measures					Probability-vector-based measures
Corruption	Sev.	$U_{0}$	$U_{1}$	$U_{2}$	$U_{\infty}$	$I_{\mathrm{H}}$	$H(q)$	$1-q_{(1)}$	$q_{(2)}$
Brightness	1	${\color[rgb]{0.082,0.3945,0.7539}\mathbf{49.92}}_{\pm 0.15}$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{49.89}}_{\pm 0.14}$	$49.87_{\pm 0.14}$	$49.83_{\pm 0.14}$	$49.85_{\pm 0.14}$	$49.88_{\pm 0.14}$	$49.87_{\pm 0.14}$	$49.83_{\pm 0.14}$
Brightness	2	${\color[rgb]{0.082,0.3945,0.7539}\mathbf{50.75}}_{\pm 0.15}$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{50.71}}_{\pm 0.14}$	$50.67_{\pm 0.14}$	$50.61_{\pm 0.14}$	$50.65_{\pm 0.14}$	$50.68_{\pm 0.15}$	$50.67_{\pm 0.15}$	$50.62_{\pm 0.15}$
Brightness	3	${\color[rgb]{0.082,0.3945,0.7539}\mathbf{52.02}}_{\pm 0.15}$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{51.98}}_{\pm 0.16}$	$51.93_{\pm 0.16}$	$51.85_{\pm 0.15}$	$51.91_{\pm 0.16}$	$51.93_{\pm 0.15}$	$51.91_{\pm 0.15}$	$51.86_{\pm 0.15}$
Brightness	4	${\color[rgb]{0.082,0.3945,0.7539}\mathbf{53.71}}_{\pm 0.15}$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{53.65}}_{\pm 0.15}$	$53.59_{\pm 0.15}$	$53.47_{\pm 0.14}$	$53.56_{\pm 0.14}$	$53.59_{\pm 0.14}$	$53.57_{\pm 0.14}$	$53.48_{\pm 0.14}$
Brightness	5	${\color[rgb]{0.082,0.3945,0.7539}\mathbf{57.92}}_{\pm 0.13}$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{57.87}}_{\pm 0.13}$	$57.80_{\pm 0.13}$	$57.62_{\pm 0.14}$	$57.76_{\pm 0.13}$	$57.79_{\pm 0.13}$	$57.76_{\pm 0.13}$	$57.65_{\pm 0.13}$

Appendix

Table 32: LAC on all 75 CIFAR-10-C conditions: AUROC (%), mean

\pm

one standard error over ten paired seeds. Five core summaries, including the saturating summary, and three original-probability baselines are retained for every condition. Blue / red : best/second-best unrounded means across all methods; ties share a colour. Rankings compare methods within each row.

		Credal-based measures					Probability-vector-based measures
Corruption	Sev.	$U_{0}$	$U_{1}$	$U_{2}$	$U_{\infty}$	$I_{\mathrm{H}}$	$H(q)$	$1-q_{(1)}$	$q_{(2)}$
Brightness	1	${\color[rgb]{0.082,0.3945,0.7539}\mathbf{49.97}}_{\pm 0.14}$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{49.94}}_{\pm 0.14}$	$49.93_{\pm 0.14}$	$49.90_{\pm 0.14}$	$49.92_{\pm 0.14}$	$49.88_{\pm 0.14}$	$49.87_{\pm 0.14}$	$49.83_{\pm 0.14}$
Brightness	2	${\color[rgb]{0.082,0.3945,0.7539}\mathbf{50.75}}_{\pm 0.14}$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{50.72}}_{\pm 0.14}$	$50.71_{\pm 0.14}$	$50.70_{\pm 0.15}$	$50.71_{\pm 0.14}$	$50.68_{\pm 0.15}$	$50.67_{\pm 0.15}$	$50.62_{\pm 0.15}$
Brightness	3	${\color[rgb]{0.082,0.3945,0.7539}\mathbf{51.98}}_{\pm 0.15}$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{51.96}}_{\pm 0.15}$	$51.95_{\pm 0.15}$	$51.94_{\pm 0.15}$	$51.96_{\pm 0.15}$	$51.93_{\pm 0.15}$	$51.91_{\pm 0.15}$	$51.86_{\pm 0.15}$
Brightness	4	${\color[rgb]{0.082,0.3945,0.7539}\mathbf{53.64}}_{\pm 0.15}$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{53.63}}_{\pm 0.15}$	$53.62_{\pm 0.15}$	$53.60_{\pm 0.15}$	$53.62_{\pm 0.15}$	$53.59_{\pm 0.14}$	$53.57_{\pm 0.14}$	$53.48_{\pm 0.14}$
Brightness	5	$57.73_{\pm 0.13}$	$57.74_{\pm 0.12}$	$57.74_{\pm 0.13}$	$57.71_{\pm 0.13}$	$57.74_{\pm 0.13}$	${\color[rgb]{0.082,0.3945,0.7539}\mathbf{57.79}}_{\pm 0.13}$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{57.76}}_{\pm 0.13}$	$57.65_{\pm 0.13}$

Appendix

Table 33: APS on all 75 CIFAR-10-C conditions: AUROC (%), mean

\pm

		Credal-based measures				Probability-vector-based measures
Corruption	Sev.	$U_{0}$	$U_{1}$	$U_{\infty}$	$I_{\mathrm{H}}$	$H(q)$	$1-q_{(1)}$	$q_{(2)}$
Brightness	1	${\color[rgb]{0.082,0.3945,0.7539}\mathbf{49.95}}_{\pm 0.14}$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{49.94}}_{\pm 0.14}$	$49.91_{\pm 0.14}$	$49.92_{\pm 0.14}$	$49.88_{\pm 0.14}$	$49.87_{\pm 0.14}$	$49.83_{\pm 0.14}$
Brightness	2	${\color[rgb]{0.082,0.3945,0.7539}\mathbf{50.75}}_{\pm 0.14}$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{50.73}}_{\pm 0.14}$	$50.71_{\pm 0.15}$	$50.72_{\pm 0.15}$	$50.68_{\pm 0.15}$	$50.67_{\pm 0.15}$	$50.62_{\pm 0.15}$
Brightness	3	${\color[rgb]{0.082,0.3945,0.7539}\mathbf{51.98}}_{\pm 0.15}$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{51.97}}_{\pm 0.15}$	$51.94_{\pm 0.15}$	$51.96_{\pm 0.15}$	$51.93_{\pm 0.15}$	$51.91_{\pm 0.15}$	$51.86_{\pm 0.15}$
Brightness	4	${\color[rgb]{0.082,0.3945,0.7539}\mathbf{53.64}}_{\pm 0.14}$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{53.63}}_{\pm 0.14}$	$53.60_{\pm 0.15}$	$53.62_{\pm 0.15}$	$53.59_{\pm 0.14}$	$53.57_{\pm 0.14}$	$53.48_{\pm 0.14}$
Brightness	5	$57.74_{\pm 0.12}$	$57.74_{\pm 0.12}$	$57.71_{\pm 0.13}$	$57.74_{\pm 0.13}$	${\color[rgb]{0.082,0.3945,0.7539}\mathbf{57.79}}_{\pm 0.13}$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{57.76}}_{\pm 0.13}$	$57.65_{\pm 0.13}$

Appendix

Table 34: RAPS on all 75 CIFAR-10-C conditions: AUROC (%), mean

\pm

one standard error over ten paired seeds. The four original core summaries and three original-probability baselines are retained for every condition. Blue / red : best/second-best unrounded means across all methods; ties share a colour. Rankings compare methods within each row.

		Credal-based measures				Probability-vector-based measures
Corruption	Sev.	$U_{0}$	$U_{1}$	$U_{\infty}$	$I_{\mathrm{H}}$	$H(q)$	$1-q_{(1)}$	$q_{(2)}$
Brightness	1	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{50.01}}_{\pm 0.15}$	$50.00_{\pm 0.16}$	${\color[rgb]{0.082,0.3945,0.7539}\mathbf{50.02}}_{\pm 0.16}$	$50.00_{\pm 0.16}$	$49.88_{\pm 0.14}$	$49.87_{\pm 0.14}$	$49.83_{\pm 0.14}$
Brightness	2	$50.27_{\pm 0.15}$	$50.24_{\pm 0.15}$	$50.25_{\pm 0.16}$	$50.24_{\pm 0.16}$	${\color[rgb]{0.082,0.3945,0.7539}\mathbf{50.68}}_{\pm 0.15}$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{50.67}}_{\pm 0.15}$	$50.62_{\pm 0.15}$
Brightness	3	$50.61_{\pm 0.17}$	$50.58_{\pm 0.17}$	$50.58_{\pm 0.17}$	$50.57_{\pm 0.17}$	${\color[rgb]{0.082,0.3945,0.7539}\mathbf{51.93}}_{\pm 0.15}$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{51.91}}_{\pm 0.15}$	$51.86_{\pm 0.15}$
Brightness	4	$51.14_{\pm 0.18}$	$51.09_{\pm 0.18}$	$51.05_{\pm 0.18}$	$51.06_{\pm 0.18}$	${\color[rgb]{0.082,0.3945,0.7539}\mathbf{53.59}}_{\pm 0.14}$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{53.57}}_{\pm 0.14}$	$53.48_{\pm 0.14}$
Brightness	5	$52.49_{\pm 0.17}$	$52.40_{\pm 0.17}$	$52.30_{\pm 0.17}$	$52.34_{\pm 0.17}$	${\color[rgb]{0.082,0.3945,0.7539}\mathbf{57.79}}_{\pm 0.13}$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{57.76}}_{\pm 0.13}$	$57.65_{\pm 0.13}$

Appendix

Table 35: SAPS on all 75 CIFAR-10-C conditions: AUROC (%), mean

\pm

	LAC				APS
Dataset / model	$U_{0}$	$U_{1}$	$U_{2}$	$U_{\infty}$	$U_{0}$	$U_{1}$	$U_{2}$	$U_{\infty}$
CIFAR-100	$86.419$	$\mathbf{87.313}$	$87.128$	$86.295$	$86.016$	$86.923$	$\mathbf{87.101}$	$87.077$
Caltech-101	$97.565$	$\mathbf{97.817}$	$97.759$	$97.685$	$93.781$	$94.419$	$94.801$	$\mathbf{95.165}$
Tiny ImageNet	$84.110$	$\mathbf{85.138}$	$84.784$	$83.459$	$83.603$	$84.612$	$84.986$	$\mathbf{85.111}$
ImageNetV2 (R50)	$82.355$	$\mathbf{88.071}$	$87.559$	$86.168$	$79.061$	$83.716$	$85.509$	$\mathbf{86.103}$
CIFAR-10	$99.024$	$99.112$	$99.150$	$\mathbf{99.177}$	$98.933$	$99.005$	$99.044$	$\mathbf{99.083}$

Appendix

Table 36: Effect of the Tsallis set-size penalty on selective prediction: mean AUARC (%) over ten paired splits. The summaries

U_{0},U_{1},U_{2},U_{\infty}

correspond to normalised fuzziness, GH, saturation, and TV-MMI. Bold identifies the largest mean among these four summaries within each score construction; no setting is selected separately by dataset. All seven dataset/model conditions are shown. Complete results with standard errors and baselines appear in Appendix E ; paired differences appear below.

Dataset / model	vs $U_{1}$	vs $U_{\infty}$	vs $H(q)$	vs $1-q_{(1)}$	vs $q_{(2)}$
CIFAR-100	$-0.185\,{\scriptstyle\pm\,0.041}$	$+0.833\,{\scriptstyle\pm\,0.042}$	$-0.187\,{\scriptstyle\pm\,0.041}$	$-0.152\,{\scriptstyle\pm\,0.021}$	$+0.833\,{\scriptstyle\pm\,0.043}$
Caltech-101	$-0.058\,{\scriptstyle\pm\,0.015}$	$+0.074\,{\scriptstyle\pm\,0.014}$	$+2.660\,{\scriptstyle\pm\,0.125}$	$+1.027\,{\scriptstyle\pm\,0.086}$	$+0.077\,{\scriptstyle\pm\,0.017}$
Tiny ImageNet	$-0.353\,{\scriptstyle\pm\,0.091}$	$+1.325\,{\scriptstyle\pm\,0.067}$	$-0.261\,{\scriptstyle\pm\,0.104}$	$-0.519\,{\scriptstyle\pm\,0.072}$	$+1.325\,{\scriptstyle\pm\,0.067}$
ImageNetV2 (R50)	$-0.512\,{\scriptstyle\pm\,0.086}$	$+1.391\,{\scriptstyle\pm\,0.089}$	$+2.421\,{\scriptstyle\pm\,0.192}$	$+0.684\,{\scriptstyle\pm\,0.101}$	$+1.391\,{\scriptstyle\pm\,0.089}$
CIFAR-10	$+0.038\,{\scriptstyle\pm\,0.006}$	$-0.027\,{\scriptstyle\pm\,0.011}$	$+0.042\,{\scriptstyle\pm\,0.011}$	$+0.037\,{\scriptstyle\pm\,0.009}$	$-0.020\,{\scriptstyle\pm\,0.008}$
Fashion-MNIST	$+0.005\,{\scriptstyle\pm\,0.003}$	$+0.009\,{\scriptstyle\pm\,0.006}$	$-0.009\,{\scriptstyle\pm\,0.008}$	$+0.198\,{\scriptstyle\pm\,0.054}$	$-0.005\,{\scriptstyle\pm\,0.009}$

Appendix

Table 37: LAC: paired AUARC differences (percentage points),

U_{2}

minus each comparator. Positive values favour saturation. Mean

\pm

paired SE over ten splits.

Dataset / model	vs $U_{1}$	vs $U_{\infty}$	vs $H(q)$	vs $1-q_{(1)}$	vs $q_{(2)}$
CIFAR-100	$+0.178\,{\scriptstyle\pm\,0.026}$	$+0.024\,{\scriptstyle\pm\,0.022}$	$-0.214\,{\scriptstyle\pm\,0.031}$	$-0.179\,{\scriptstyle\pm\,0.037}$	$+0.807\,{\scriptstyle\pm\,0.070}$
Caltech-101	$+0.383\,{\scriptstyle\pm\,0.032}$	$-0.363\,{\scriptstyle\pm\,0.024}$	$-0.297\,{\scriptstyle\pm\,0.103}$	$-1.930\,{\scriptstyle\pm\,0.108}$	$-2.880\,{\scriptstyle\pm\,0.170}$
Tiny ImageNet	$+0.374\,{\scriptstyle\pm\,0.038}$	$-0.125\,{\scriptstyle\pm\,0.025}$	$-0.059\,{\scriptstyle\pm\,0.029}$	$-0.317\,{\scriptstyle\pm\,0.056}$	$+1.527\,{\scriptstyle\pm\,0.145}$
ImageNetV2 (R50)	$+1.793\,{\scriptstyle\pm\,0.077}$	$-0.594\,{\scriptstyle\pm\,0.038}$	$+0.371\,{\scriptstyle\pm\,0.060}$	$-1.366\,{\scriptstyle\pm\,0.106}$	$-0.659\,{\scriptstyle\pm\,0.231}$
CIFAR-10	$+0.039\,{\scriptstyle\pm\,0.008}$	$-0.039\,{\scriptstyle\pm\,0.009}$	$-0.063\,{\scriptstyle\pm\,0.012}$	$-0.069\,{\scriptstyle\pm\,0.013}$	$-0.126\,{\scriptstyle\pm\,0.025}$
Fashion-MNIST	$+0.409\,{\scriptstyle\pm\,0.078}$	$-0.197\,{\scriptstyle\pm\,0.045}$	$-0.743\,{\scriptstyle\pm\,0.109}$	$-0.536\,{\scriptstyle\pm\,0.075}$	$-0.739\,{\scriptstyle\pm\,0.109}$

Appendix

Table 38: APS: paired AUARC differences (percentage points),

U_{2}

minus each comparator. Positive values favour saturation. Mean

\pm

paired SE over ten splits.

Dataset / model	vs $U_{1}$	vs $U_{\infty}$	vs $H(q)$	vs $1-q_{(1)}$	vs $q_{(2)}$
CIFAR-100	$+0.034\,{\scriptstyle\pm\,0.013}$	$+0.034\,{\scriptstyle\pm\,0.017}$	$-0.205\,{\scriptstyle\pm\,0.035}$	$-0.170\,{\scriptstyle\pm\,0.037}$	$+0.815\,{\scriptstyle\pm\,0.067}$
Caltech-101	$+0.291\,{\scriptstyle\pm\,0.019}$	$-0.343\,{\scriptstyle\pm\,0.023}$	$-0.275\,{\scriptstyle\pm\,0.103}$	$-1.908\,{\scriptstyle\pm\,0.108}$	$-2.858\,{\scriptstyle\pm\,0.169}$
Tiny ImageNet	$+0.151\,{\scriptstyle\pm\,0.023}$	$-0.088\,{\scriptstyle\pm\,0.019}$	$-0.029\,{\scriptstyle\pm\,0.031}$	$-0.287\,{\scriptstyle\pm\,0.053}$	$+1.557\,{\scriptstyle\pm\,0.143}$
ImageNetV2 (R50)	$+0.746\,{\scriptstyle\pm\,0.036}$	$-0.473\,{\scriptstyle\pm\,0.030}$	$+0.491\,{\scriptstyle\pm\,0.064}$	$-1.245\,{\scriptstyle\pm\,0.100}$	$-0.538\,{\scriptstyle\pm\,0.225}$
CIFAR-10	$+0.026\,{\scriptstyle\pm\,0.006}$	$-0.039\,{\scriptstyle\pm\,0.012}$	$-0.063\,{\scriptstyle\pm\,0.014}$	$-0.069\,{\scriptstyle\pm\,0.015}$	$-0.126\,{\scriptstyle\pm\,0.027}$
Fashion-MNIST	$+0.165\,{\scriptstyle\pm\,0.054}$	$-0.161\,{\scriptstyle\pm\,0.033}$	$-0.695\,{\scriptstyle\pm\,0.101}$	$-0.488\,{\scriptstyle\pm\,0.067}$	$-0.691\,{\scriptstyle\pm\,0.101}$

Appendix

Table 39: RAPS: paired AUARC differences (percentage points),

U_{2}

minus each comparator. Positive values favour saturation. Mean

\pm

paired SE over ten splits.

Dataset / model	vs $U_{1}$	vs $U_{\infty}$	vs $H(q)$	vs $1-q_{(1)}$	vs $q_{(2)}$
CIFAR-100	$-0.310\,{\scriptstyle\pm\,0.030}$	$+0.366\,{\scriptstyle\pm\,0.028}$	$-1.444\,{\scriptstyle\pm\,0.122}$	$-1.409\,{\scriptstyle\pm\,0.122}$	$-0.423\,{\scriptstyle\pm\,0.115}$
Caltech-101	$-0.020\,{\scriptstyle\pm\,0.005}$	$+0.045\,{\scriptstyle\pm\,0.016}$	$+1.338\,{\scriptstyle\pm\,0.096}$	$-0.295\,{\scriptstyle\pm\,0.080}$	$-1.245\,{\scriptstyle\pm\,0.136}$
Tiny ImageNet	$-0.242\,{\scriptstyle\pm\,0.024}$	$+0.256\,{\scriptstyle\pm\,0.028}$	$-0.836\,{\scriptstyle\pm\,0.151}$	$-1.094\,{\scriptstyle\pm\,0.132}$	$+0.750\,{\scriptstyle\pm\,0.190}$
ImageNetV2 (R50)	$-0.098\,{\scriptstyle\pm\,0.021}$	$+0.127\,{\scriptstyle\pm\,0.029}$	$+1.412\,{\scriptstyle\pm\,0.115}$	$-0.324\,{\scriptstyle\pm\,0.066}$	$+0.383\,{\scriptstyle\pm\,0.189}$
CIFAR-10	$-0.051\,{\scriptstyle\pm\,0.012}$	$+0.077\,{\scriptstyle\pm\,0.020}$	$-1.661\,{\scriptstyle\pm\,0.170}$	$-1.667\,{\scriptstyle\pm\,0.171}$	$-1.724\,{\scriptstyle\pm\,0.165}$
Fashion-MNIST	$-0.022\,{\scriptstyle\pm\,0.009}$	$+0.030\,{\scriptstyle\pm\,0.011}$	$-4.246\,{\scriptstyle\pm\,0.315}$	$-4.039\,{\scriptstyle\pm\,0.301}$	$-4.242\,{\scriptstyle\pm\,0.314}$

Appendix

Table 40: SAPS: paired AUARC differences (percentage points),

U_{2}

minus each comparator. Positive values favour saturation. Mean

\pm

paired SE over ten splits.

Dataset / model	$\rho(U_{2},U_{1})$	$\rho(U_{2},U_{\infty})$	Rank 2 (%)	Ranks 2–5 (%)	Full minus top-5
CIFAR-100	0.971	0.968	73.82	96.06	$+0.167\,{\scriptstyle\pm\,0.011}$
Caltech-101	0.996	0.996	80.72	95.47	$+0.015\,{\scriptstyle\pm\,0.002}$
Tiny ImageNet	0.941	0.952	68.52	93.01	$+0.366\,{\scriptstyle\pm\,0.025}$
ImageNetV2 (R50)	0.934	0.962	68.59	92.99	$+0.342\,{\scriptstyle\pm\,0.020}$
CIFAR-10	0.997	0.993	72.17	95.77	$-0.007\,{\scriptstyle\pm\,0.003}$
Fashion-MNIST	0.999	0.997	73.06	95.66	$+0.002\,{\scriptstyle\pm\,0.001}$

Appendix

Table 41: LAC rank agreement and tail contributions. Correlations and contribution shares are averaged over ten splits. The last column gives the full score’s AUARC minus the top-5 truncation in percentage points, with paired SE.

Figure 2: Large-sample behaviour in a correctly specified three-class problem. Solid curves are means over 40 repetitions and ribbons show one standard error; dotted lines are oracle-model/population-calibration Monte Carlo references. The lower-right panel varies the oracle softmax temperature and includes the exact Dirac control at the origin. Increasing both sample sizes removes sampling variation but does not collapse the conformal credal set when

P(Y\mid X)

is non-degenerate.

Dataset	Features	Classes	Total	Initial	Pool	Acq. cal.	Audit cal.	Test
Digits v1	216	10	2,000	100	750	500	250	400
Digits v2	64	10	5,620	150	2,846	1,000	500	1,124
Digits v3	64	10	2,000	100	750	500	250	400
Letters	16	26	20,000	500	12,500	2,000	1,000	4,000

Appendix

Table 42: Active-learning datasets and fixed partition sizes. The last five columns partition every dataset.

Acquisition rule	Digits v1	Digits v2	Digits v3	Letters
Probability-vector-based measures
Predictive entropy	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{95.38}}\pm 0.33$	$95.23\pm 0.23$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{93.35}}\pm 0.36$	$78.89\pm 0.28$
Max-probability uncertainty $1-q_{(1)}$	${\color[rgb]{0.082,0.3945,0.7539}\mathbf{95.68}}\pm 0.31$	${\color[rgb]{0.082,0.3945,0.7539}\mathbf{96.07}}\pm 0.21$	${\color[rgb]{0.082,0.3945,0.7539}\mathbf{93.57}}\pm 0.35$	$79.56\pm 0.25$
Second probability $q_{(2)}$	$94.51\pm 0.37$	$94.37\pm 0.15$	$91.95\pm 0.35$	$76.01\pm 0.33$
Credal-based measures
$\mathrm{TV\!\text{-}\!MMI}$	$94.47\pm 0.34$	$94.36\pm 0.17$	$91.86\pm 0.33$	$76.02\pm 0.33$

Appendix

Table 43: LAC active learning: AULC (%; mean

\pm

one standard error over ten paired seeds). Higher is better. Conventions follow Appendix I .

Acquisition rule	Digits v1	Digits v2	Digits v3	Letters
Probability-vector-based measures
Predictive entropy	$95.38\pm 0.33$	$95.23\pm 0.23$	$93.35\pm 0.36$	$78.89\pm 0.28$
Max-probability uncertainty $1-q_{(1)}$	${\color[rgb]{0.082,0.3945,0.7539}\mathbf{95.68}}\pm 0.31$	${\color[rgb]{0.082,0.3945,0.7539}\mathbf{96.07}}\pm 0.21$	$93.57\pm 0.35$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{79.56}}\pm 0.25$
Second probability $q_{(2)}$	$94.51\pm 0.37$	$94.37\pm 0.15$	$91.95\pm 0.35$	$76.01\pm 0.33$
Credal-based measures
$\mathrm{TV\!\text{-}\!MMI}$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{95.59}}\pm 0.31$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{95.96}}\pm 0.22$	${\color[rgb]{0.082,0.3945,0.7539}\mathbf{93.82}}\pm 0.33$	${\color[rgb]{0.082,0.3945,0.7539}\mathbf{79.69}}\pm 0.26$

Appendix

Table 44: APS active learning: AULC (%; mean

\pm

one standard error over ten paired seeds). Higher is better. Conventions follow Appendix I .

Acquisition rule	Digits v1	Digits v2	Digits v3	Letters
Probability-vector-based measures
Predictive entropy	$95.38\pm 0.33$	$95.23\pm 0.23$	$93.35\pm 0.36$	$78.89\pm 0.28$
Max-probability uncertainty $1-q_{(1)}$	${\color[rgb]{0.082,0.3945,0.7539}\mathbf{95.68}}\pm 0.31$	${\color[rgb]{0.082,0.3945,0.7539}\mathbf{96.07}}\pm 0.21$	$93.57\pm 0.35$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{79.56}}\pm 0.25$
Second probability $q_{(2)}$	$94.51\pm 0.37$	$94.37\pm 0.15$	$91.95\pm 0.35$	$76.01\pm 0.33$
Credal-based measures
$\mathrm{TV\!\text{-}\!MMI}$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{95.60}}\pm 0.31$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{95.96}}\pm 0.22$	${\color[rgb]{0.082,0.3945,0.7539}\mathbf{93.82}}\pm 0.33$	${\color[rgb]{0.082,0.3945,0.7539}\mathbf{79.69}}\pm 0.26$

Appendix

Table 45: RAPS active learning: AULC (%; mean

\pm

one standard error over ten paired seeds). Higher is better. Conventions follow Appendix I .

Acquisition rule	Digits v1	Digits v2	Digits v3	Letters
Probability-vector-based measures
Predictive entropy	$95.38\pm 0.33$	$95.23\pm 0.23$	$93.35\pm 0.36$	$78.89\pm 0.28$
Max-probability uncertainty $1-q_{(1)}$	$95.68\pm 0.31$	$96.07\pm 0.21$	$93.57\pm 0.35$	$79.56\pm 0.25$
Second probability $q_{(2)}$	$94.51\pm 0.37$	$94.37\pm 0.15$	$91.95\pm 0.35$	$76.01\pm 0.33$
Credal-based measures
$\mathrm{TV\!\text{-}\!MMI}$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{95.69}}\pm 0.32$	${\color[rgb]{0.082,0.3945,0.7539}\mathbf{96.17}}\pm 0.20$	${\color[rgb]{0.7773,0.1563,0.1563}\mathbf{93.73}}\pm 0.36$	$79.63\pm 0.27$

Appendix

Table 46: SAPS active learning: AULC (%; mean

\pm

one standard error over ten paired seeds). Higher is better. Conventions follow Appendix I .

Paired contrast	Digits v1	Digits v2	Digits v3	Letters
LAC: $F-H(q)$	$-0.219\pm 0.215$	$-0.323\pm 0.223$	$-0.266\pm 0.144$	$+1.187\pm 0.168$
LAC: $F-(1-q_{(1)})$	$-0.519\pm 0.146$	$-1.156\pm 0.233$	$-0.487\pm 0.121$	$+0.516\pm 0.173$
LAC: TV-MMI $-q_{(2)}$	$-0.042\pm 0.095$	$-0.005\pm 0.038$	$-0.093\pm 0.066$	$+0.002\pm 0.002$
APS: TV-MMI $-H(q)$	$+0.218\pm 0.106$	$+0.727\pm 0.114$	$+0.470\pm 0.083$	$+0.808\pm 0.064$
APS: TV-MMI $-(1-q_{(1)})$	$-0.082\pm 0.082$	$-0.106\pm 0.107$	$+0.250\pm 0.062$	$+0.137\pm 0.112$
RAPS: TV-MMI $-H(q)$	$+0.218\pm 0.106$	$+0.727\pm 0.114$	$+0.470\pm 0.083$	$+0.808\pm 0.064$

Appendix

Table 47: Selected paired differences in normalised active-learning AULC (percentage points; mean

\pm

one standard error of the ten within-seed differences). Positive values favour the first acquisition rule. These descriptive contrasts explain the patterns in the complete tables; they are not multiplicity-adjusted tests.

Explore similar work

Oct 6, 2026

Conformal Prediction Sets Quantify Information Gain: A Theoretical Perspective

Conformal prediction is a popular tool for uncertainty quantification that outputs prediction sets with finite-sample coverage guarantees. While prediction set size is commonly used as a heuristic measure of uncertainty, the information-theoretic basis for this interpretation remains poorly understood. In this work, we provide such a foundation using a decision-theoretic generalization of entropy tailored to set-valued prediction. In particular, we introduce a family of generalized information measures based on the size and coverage of conformal prediction sets. Notably, Shannon mutual information admits an exact integral representation in terms of these measures. We then show that, in standard classification settings, the reduction in conformal set size from additional information (i) is sandwiched between calibration-dependent members of this family and (ii) obeys a data processing inequality, both up to finite-sample calibration and model error terms. Together, our results formally relate conformal prediction to classical information-theoretic quantities and justify using set-size reduction as an information gain metric. Empirically, we validate our theory across 11 classification settings and show that set-size reduction and Shannon mutual information can rank features differently in a greedy feature selection experiment.

Kevin Zhang, Stephen Bates

EECS, MIT, Cambridge MA, USA.

Set Prediction Conformal Prediction Mutual Information

Ask PDF

May 25, 2025

Optimal Conformal Prediction under Epistemic Uncertainty

Conformal prediction (CP) is a widely used frequentist framework to quantify uncertainty by constructing prediction sets with user-specified marginal coverage guarantees. In practice, CP is typically applied on top of probabilistic classifiers, which are able to express aleatoric but not epistemic uncertainty. In this paper, we consider the question of how to optimally employ CP on top of a more expressive formalism, namely credal sets, which can express both aleatoric and epistemic uncertainty. More specifically, we propose probabilistic Bernoulli prediction sets (BPS) and derive a variant that achieves conditional coverage for valid credal sets while remaining minimal in expected size. We then address the more realistic scenario in which the validity of the credal sets is not guaranteed. Assuming access to calibration data with ground-truth distributions over labels, we apply conformal risk control to BPS and derive a PAC-style guarantee: with high probability over the data, the achieved conditional coverage is at least the desired level. We validate our theoretical findings empirically over various datasets.

Alireza Javanmardi, Soroush H. Zargarbashi, Santo M. A. R. Thies +3

1LMU Munich · 2MCML · 3CISPA +3

Conformal Prediction Conformal Risk Control Epistemic Uncertainty

Ask PDF

Nov 19, 2025

Beyond Uncertainty Sets: Leveraging Optimal Transport to Extend Conformal Predictive Distributions to Multivariate Settings

Conformal prediction (CP) constructs uncertainty sets for model outputs with finite-sample coverage guarantees. Yet ranking scores is straightforward only when they are scalar-valued, limiting CP to real-valued scores or ad-hoc one-dimensional reductions. Vector-valued scores arise naturally in multi-output regression and model aggregation, where each predictor in an ensemble provides its own score. Optimal transport (OT) defines vector ranks and center-outward multivariate quantile regions, though generally with asymptotic coverage guarantees. Applying a fixed transport map learned from calibration data to a new point introduces an uncontrolled approximation error. We restore finite-sample, distribution-free coverage by conformalizing vector-valued OT quantile regions. Each candidate's rank is defined by transporting the calibration scores augmented with that candidate's score, preserving the symmetry needed for validity. This appears to require a continuum of OT problems. However, we prove that the optimal assignment is piecewise constant across a fixed polyhedral partition of score space. This lets us characterize the entire prediction set in

O(n^3)

time, matching the cost of a single assignment solve. It also addresses a limitation of prediction sets: they indicate which outcomes are plausible, but not their relative likelihood. In one dimension, conformal predictive distributions (CPDs) fill this gap by producing a predictive distribution with finite-sample calibration. Extending CPDs beyond one dimension remained an open problem. We construct, to our knowledge, the first multivariate CPDs with finite-sample calibration: a center-outward predictive distribution whose derived uncertainty regions have conformal coverage. We present both conservative and exact randomized versions; the latter generalizes the classical Dempster-Hill procedure.

Eugene Ndiaye

Apple

Uncertainty Quantification Conformal Prediction Optimal Transport

Ask PDF