cs.LGApr 9, 2026

Evidential Transformation Network: Turning Pretrained Models into Evidential Models for Post-hoc Uncertainty Estimation

Authors: Yongchan Chun, Chanhee Park, Jeongho Yoon, Jaehyung Seo, Heuiseok Lim

Organizations: Korea University · Konkuk University

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Pretrained models have become standard in both vision and language, yet they typically do not provide reliable measures of confidence. Existing uncertainty estimation methods, such as deep ensembles and MC dropout, are often too computationally expensive to deploy in practice. Evidential Deep Learning (EDL) offers a more efficient alternative, but it requires models to be trained to output evidential quantities from the start, which is rarely true for pretrained networks. To enable EDL-style uncertainty estimation in pretrained models, we propose the Evidential Transformation Network (ETN), a lightweight post-hoc module that converts a pretrained predictor into an evidential model. ETN operates in logit space: it learns a sample-dependent affine transformation of the logits and interprets the transformed outputs as parameters of a Dirichlet distribution for uncertainty estimation. We evaluate ETN on image classification and large language model question-answering benchmarks under both in-distribution and out-of-distribution settings. ETN consistently improves uncertainty estimation over post-hoc baselines while preserving accuracy and adding only minimal computational overhead.

Figures & tables

Figure 1: Comparison of average uncertainty estimation performance across in-distribution (ID) and out-of-distribution (OOD) settings. For ID, we score AUPR for correctly identifying confident predictions, while for OOD, AUPR for separating ID from OOD samples by confidence is scored. Plots show performance against accuracy (left) and inference runtime (right). Evidential Transformation Network achieves the best uncertainty performance with almost no additional inference cost and no accuracy drop.
Figure 2: Comparison of uncertainty estimation performance based on different dimension of transformation parameter AA .
Table 1: AUPR scores on CIFAR-10, SVHN, and CIFAR-100 (top), and on ImageNet, ImageNet-A, ImageNet-S, and ImageNet-R (bottom).
MethodRACERACE →\to MMLUOBQAOBQA →\to MMLU
ACCMPUMMPMIDEACCMPUMMPMIDE
MAPCE\mathrm{MAP}_{\text{CE}}89.54\displaystyle 89.54 0.1{0.1}97.05‾\displaystyle\underline{97.05} 0.1{0.1}–96.01\displaystyle 96.01 0.3{0.3}––87.07\displaystyle 87.07 0.2{0.2}96.94\displaystyle 96.94 0.2{0.2}–89.83‾\displaystyle\underline{89.83} 0.9{0.9}––
DeepEns85.84\displaystyle 85.84 0.1{0.1}96.66\displaystyle 96.66 0.2{0.2}–94.28\displaystyle 94.28 0.5{0.5}91.42\displaystyle 91.42 1.1{1.1}–83.47\displaystyle 83.47 0.4{0.4}94.51\displaystyle 94.51 0.2{0.2}–87.08\displaystyle 87.08 1.0{1.0}83.68‾\displaystyle\underline{83.68} 0.8{0.8}–
MCD89.66‾\displaystyle\underline{89.66} 0.0{0.0}96.94\displaystyle 96.94 0.0{0.0}–96.59‾\displaystyle\underline{96.59} 0.0{0.0}96.33 0.0{0.0}–86.40\displaystyle 86.40 0.2{0.2}97.18 0.0{0.0}–88.99\displaystyle 88.99 0.0{0.0}82.38\displaystyle 82.38 0.0{0.0}–
LL88.74\displaystyle 88.74 0.2{0.2}64.98\displaystyle 64.98 0.3{0.3}–25.24\displaystyle 25.24 0.9{0.9}27.22\displaystyle 27.22 0.8{0.8}–86.07\displaystyle 86.07 0.1{0.1}51.27\displaystyle 51.27 0.2{0.2}–24.74\displaystyle 24.74 0.6{0.6}24.63\displaystyle 24.63 0.3{0.3}–
MAPEDL\mathrm{MAP}_{\text{EDL}}84.94\displaystyle 84.94 0.2{0.2}91.70\displaystyle 91.70 0.4{0.4}87.86\displaystyle 87.86 1.5{1.5}91.19\displaystyle 91.19 0.3{0.3}87.67\displaystyle 87.67 1.0{1.0}91.18\displaystyle 91.18 0.3{0.3}80.27\displaystyle 80.27 1.2{1.2}85.62\displaystyle 85.62 1.8{1.8}76.69\displaystyle 76.69 1.7{1.7}79.86\displaystyle 79.86 1.9{1.9}70.16\displaystyle 70.16 1.8{1.8}79.95\displaystyle 79.95 1.9{1.9}
Table 2: AUPR scores on RACE and OBQA with MMLU subsets as OOD.
Figure 3: Comparison of uncertainty estimation performance based on different transformation methods.
Figure 4: Ablation studies on each AUPR scores on different parameters of prior distribution (left) and different number of MC samples (right). 95% confidence interval is shaded.
SettingVGG16ResNet50Llama-3.1-8BGemma-2-9B
Pretrain
Batch size1024–44
Learning rate2.5×10−42.5\times 10^{-4}–2.5×10−42.5\times 10^{-4}2.5×10−42.5\times 10^{-4}
Epochs200–33
Uncertainty adaptation
Batch size10246488
Table 3: Training and hyperparameter settings for each model.
Domain and Subsets of MMLU
Computer Science:
college_computer_science
computer_security
high_school_computer_science
machine_learning
Engineering:
Table 4: MMLU domains and their corresponding subsets.
MethodCIFAR10 →\to CIFAR10-OODImageNet →\to ImageNet-OODOBQA →\to MMLURACE →\to MMLU
MD45.43\displaystyle 45.43 1.2{1.2} / 56.69\displaystyle 56.69 1.06{1.06}87.54 0.16{0.16} / 77.45\displaystyle 77.45 0.24{0.24}70.5\displaystyle 70.5 0.04{0.04} / 54.42\displaystyle 54.42 0.05{0.05}87.28\displaystyle 87.28 0.01{0.01} / 54.44\displaystyle 54.44 0.02{0.02}
ODIN86.41 0.99{0.99} / 87.29 0.82{0.82}79.77\displaystyle 79.77 0.00{0.00} / 72.11\displaystyle 72.11 0.00{0.00}61.35\displaystyle 61.35 0.00{0.00} / 50.11\displaystyle 50.11 0.00{0.00}81.22\displaystyle 81.22 0.00{0.00} / 49.65\displaystyle 49.65 0.00{0.00}
ETN85.93\displaystyle 85.93 0.92{0.92} / 86.5\displaystyle 86.5 0.97{0.97}84.78\displaystyle 84.78 0.36{0.36} / 79.86 0.49{0.49}91.7 0.00{0.00} / 83.39 0.01{0.01}96.80 0.39{0.39} / 87.57 0.01{0.01}
Table 5: Comparison of ETN to OOD-detection methods. We showcase both AUPR and AUROC scores, respectively. For ETN, we outline the scores based on maximum probability.
MethodCIFAR-10→\to CIFAR-OODImageNet→\to ImageNet-OODRACE→\to MMLUOBQA→\to MMLU
ACCUEUEACCUEUEACCUEUEACCUEUE
DUQ42.25\displaystyle 42.25 9.4{9.4}42.05\displaystyle 42.05 9.3{9.3}54.86\displaystyle 54.86 8.0{8.0}0.09\displaystyle 0.09 0.0{0.0}0.18\displaystyle 0.18 0.0{0.0}69.77\displaystyle 69.77 1.9{1.9}21.52\displaystyle 21.52 0.0{0.0}22.54\displaystyle 22.54 0.2{0.2}74.31\displaystyle 74.31 11.7{11.7}27.60\displaystyle 27.60 0.0{0.0}26.74\displaystyle 26.74 1.1{1.1}50.85\displaystyle 50.85 3.0{3.0}
SNGP83.62\displaystyle 83.62 1.4{1.4}90.75\displaystyle 90.75 1.3{1.3}57.72\displaystyle 57.72 4.7{4.7}12.83\displaystyle 12.83 1.0{1.0}12.83\displaystyle 12.83 0.9{0.9}65.73\displaystyle 65.73 0.8{0.8}45.73\displaystyle 45.73 19.5{19.5}49.97\displaystyle 49.97 22.9{22.9}95.11\displaystyle 95.11 1.1{1.1}39.53\displaystyle 39.53 17.8{17.8}43.74\displaystyle 43.74 20.8{20.8}94.74 4.3{4.3}
ETN90.70 0.0{0.0}98.99 0.1{0.1}85.93 0.9{0.9}79.61 0.0{0.0}88.04 0.1{0.1}79.86 0.5{0.5}89.69 0.0{0.0}97.60 0.0{0.0}96.80 0.0{0.0}88.80 0.0{0.0}97.15 0.0{0.0}91.70\displaystyle 91.70 0.0{0.0}
Table 6: Comparison of ETN with deterministic uncertainty estimation methods in terms of accuracy (ACC) and uncertainty estimation (UE), where UE is measured by AUPR. For ETN, we report UE based on maximum probability.
Figure 5: Comparison of uncertainty estimation performance based on different transformation methods on CIFAR-10 and OBQA.
Figure 6: Comparison of uncertainty estimation performance and accuracy across different dimensionalities of the transformation parameter AA modeled by ETN on ImageNet and RACE.
Figure 7: Histograms of logit margins for models trained with EDL and CE.
Figure 8: Comparison of uncertainty estimation performance and accuracy across different dimensionalities of the transformation parameter AA modeled by static scaling.
Figure 9: Comparison of uncertainty estimation performance and accuracy across different dimensionalities of the transformation parameter AA modeled by AdaTS .
MethodRACERACE →\to MMLUOBQAOBQA →\to MMLU
ACCMPUMMPMIDEACCMPUMMPMIDE
MAPCE\mathrm{MAP}_{\text{CE}}88.81\displaystyle 88.81 0.2{0.2}98.16\displaystyle 98.16 0.1{0.1}–96.03\displaystyle 96.03 0.1{0.1}––88.87\displaystyle 88.87 0.6{0.6}97.59\displaystyle 97.59 0.2{0.2}–85.85‾\displaystyle\underline{85.85} 0.1{0.1}––
DeepEns89.09\displaystyle 89.09 0.1{0.1}98.26\displaystyle 98.26 0.0{0.0}–94.64\displaystyle 94.64 0.2{0.2}93.79\displaystyle 93.79 0.4{0.4}–89.00\displaystyle 89.00 0.0{0.0}97.84\displaystyle 97.84 0.3{0.3}–83.04\displaystyle 83.04 0.9{0.9}78.82\displaystyle 78.82 0.5{0.5}–
MCD89.38\displaystyle 89.38 0.4{0.4}98.40‾\displaystyle\underline{98.40} 0.2{0.2}–96.36‾\displaystyle\underline{96.36} 0.3{0.3}95.36 0.4{0.4}–89.60\displaystyle 89.60 0.5{0.5}97.87‾\displaystyle\underline{97.87} 0.5{0.5}–85.59\displaystyle 85.59 1.4{1.4}81.81‾\displaystyle\underline{81.81} 1.8{1.8}–
LL88.51\displaystyle 88.51 0.2{0.2}69.05\displaystyle 69.05 0.3{0.3}–27.47\displaystyle 27.47 1.4{1.4}27.03\displaystyle 27.03 1.0{1.0}–92.53\displaystyle 92.53 0.2{0.2}79.45\displaystyle 79.45 0.6{0.6}–25.32\displaystyle 25.32 0.6{0.6}26.25\displaystyle 26.25 0.8{0.8}–
MAPEDL\mathrm{MAP}_{\text{EDL}}87.14\displaystyle 87.14 0.2{0.2}92.27\displaystyle 92.27 0.7{0.7}87.60\displaystyle 87.60 1.1{1.1}93.97\displaystyle 93.97 1.5{1.5}83.89\displaystyle 83.89 1.1{1.1}90.36\displaystyle 90.36 1.2{1.2}87.53\displaystyle 87.53 0.4{0.4}94.63\displaystyle 94.63 0.9{0.9}86.32\displaystyle 86.32 3.3{3.3}80.28\displaystyle 80.28 2.7{2.7}63.50\displaystyle 63.50 1.5{1.5}79.78\displaystyle 79.78 2.6{2.6}
Table 7: AUPR scores of Gemma-2-9B on RACE and OBQA, using MMLU subsets as OOD data.
Table 8: AUROC scores on CIFAR-10, SVHN, and CIFAR-100 (top), and on ImageNet, ImageNet-A, ImageNet-S, and ImageNet-R (bottom).
Table 9: AUROC scores on on RACE and OBQA, using MMLU subsets as OOD data. Results on Llama-3.1-8B is reported (top), and Gemma-2-9B is reported (bottom).
More in this topic

Uncertainty Quantification

Uncertainty quantification estimates how reliable predictions and scientific inferences are, and how uncertainty propagates into decisions.

See all 647 papers

Explore similar work

CardsList
Apr 10, 2026

Uncertainty-Aware Transformers: Conformal Prediction for Language Models

Transformers have had a profound impact on the field of artificial intelligence, especially on large language models and their variants. However, as was the case with neural networks, their black-box nature limits trust and deployment in high-stakes settings. For models to be genuinely useful and trustworthy in critical applications, they must provide more than just predictions: they must supply users with a clear understanding of the reasoning that underpins their decisions. This article presents an uncertainty quantification framework for transformer-based language models. This framework, called CONFIDE (CONformal prediction for FIne-tuned DEep language models), applies conformal prediction to the internal embeddings of encoder-only architectures, like BERT and RoBERTa, while enabling hyperparameter tuning. CONFIDE uses either [CLS] token embeddings or flattened hidden states to construct class-conditional nonconformity scores, enabling statistically valid prediction sets with instance-level explanations. Empirically, CONFIDE improves test accuracy by up to 4.09% on BERT-tiny and achieves greater correct efficiency (i.e., the expected size of the prediction set conditioned on it containing the true label) compared to prior methods, including NM2 and VanillaNN. We show that early and intermediate transformer layers often yield better-calibrated and more semantically meaningful representations for conformal prediction. In resource-constrained models and high-stakes tasks with ambiguous labels, CONFIDE offers robustness and interpretability where softmax-based uncertainty fails. We position CONFIDE as a framework for practical diagnostic and efficiency/robustness improvement over prior conformal baselines.

Abhiram Vellore, Niraj K. Jha

Department of Electrical and Computer Engineering, Princeton University

Conformal Prediction for Language ModelsLLM Uncertainty EstimationConformal PredictionLanguage Model Calibration
AskPDF
Oct 1, 2026

Robust Evidential Learning Through Latent Consistency

Reliable uncertainty quantification is essential for deploying deep learning models in high-stakes settings, where out-of-distribution and adversarial inputs can induce confident but unreliable predictions. Evidential Deep Learning provides efficient uncertainty estimates in a single forward pass, but can still assign high evidential strength to inputs that are poorly supported by the learned representation, such as adversarial inputs. We introduce CLEAR, a lightweight, task-agnostic post-hoc method that improves evidential robustness without retraining or altering the base prediction. Using held-out calibration data, CLEAR characterises the group-conditioned geometry of the model's latent space. At inference, it efficiently generates perturbation views directly in the latent space and measures their conflict relative to the calibrated geometry of the predicted group. High latent conflict indicates unsupported evidence, which CLEAR uses to selectively reduce evidential strength while retaining evidence for latent-consistent inputs. On ImageNet→\rightarrowCUB, CLEAR improves OOD and adversarial AUROC by +8.29+8.29 and +5.01+5.01 while running 17.4×\times faster than competing post-hoc methods while preserving predictive performance across classification, regression, and object detection benchmarks.

Charmaine Barker, Daniel Bethell, Simos Gerasimou

Department of Computer Science, University of York, UK · Department of Elect. Eng., and Computer Science and Eng., Cyprus University of Technology, Cyprus

Evidential Deep LearningAdversarial RobustnessOOD DetectionAdversarial Example Detection
AskPDF
Apr 7, 2026

Ensemble-Based Dirichlet Modeling for Predictive Uncertainty and Selective Classification

Neural network classifiers trained with cross-entropy loss achieve strong predictive accuracy but lack the capability to provide inherent predictive uncertainty estimates, thus requiring external techniques to obtain these estimates. In addition, softmax scores for the true class can vary substantially across independent training runs, which limits the reliability of uncertainty-based decisions in downstream tasks. Evidential Deep Learning aims to address these limitations by producing uncertainty estimates in a single pass, but evidential training is highly sensitive to design choices including loss formulation, prior regularization, and activation functions. Therefore, this work introduces an alternative Dirichlet parameter estimation strategy by applying a method of moments estimator to ensembles of softmax outputs, with an optional maximum-likelihood refinement step. This ensemble-based construction decouples uncertainty estimation from the fragile evidential loss design while also mitigating the variability of single-run cross-entropy training, producing explicit Dirichlet predictive distributions. Across multiple datasets, we show that the improved stability and predictive uncertainty behavior of these ensemble-derived Dirichlet estimates translate into stronger performance in downstream uncertainty-guided applications such as prediction confidence scoring and selective classification.

Courtney Franzen, Farhad Pourkamali-Anaraki

Department of Mathematical and Statistical Science, University of Colorado Denver, Denver, 80208, CO, USA,. · Department of Mathematical and Statistical Science, University of Colorado Denver, Denver, 80208, CO, USA.

Uncertainty Quantification
AskPDF