Epistemic Uncertainty

Ask about this topic

Overview

Epistemic uncertainty concerns what a model does not know, often distinguished from aleatoric uncertainty caused by inherent variability or ambiguity in observations. Researchers estimate it with Bayesian methods, ensembles and dropout, evidential models, and set-valued or credal predictions, then use it for out-of-distribution detection, selective prediction, active learning, and safer decisions. Work increasingly evaluates whether uncertainty is calibrated and useful for specific decisions, including through conformal guarantees, abstention tests, and benchmarks under distribution shift. A central tension is that common signals such as ensemble disagreement or predictive entropy may not reliably track missing knowledge, and their value depends on how uncertainty is defined and tested.

Latest papers

CardsList

113 papers14 papers in the last four weeks, up 250% on the four weeks before. 0.1% of all new papers.

Mar 10, 2026

The Epistemic Support-Point Filter: Jaynesian Maximum Entropy Meets Popperian Falsification

This paper proves that the Epistemic Support-Point Filter (ESPF) is the unique optimal recursive estimator within the class of epistemically admissible evidence-only filters. Where Bayesian filters minimize mean squared error and are driven toward an assumed truth, the ESPF minimizes maximum entropy and surfaces what has not been proven impossible -- a fundamentally different epistemic commitment with fundamentally different failure modes. Two results locate this theorem within the broader landscape of estimation theory. The first is a unification: the ESPF's optimality criterion is the log-geometric mean of the alpha-cut volume family in the Holder mean hierarchy. The Popperian minimax bound and the Kalman MMSE criterion occupy the p=+inf and p=0 positions on the same curve. Possibility and probability are not competing frameworks: they are the same ignorance functional evaluated under different alpha-cut geometries. The Kalman filter is the Gaussian specialization of the ESPF's optimality criterion, not a separate invention. The second result is a diagnostic: numerical validation over a 2-day, 877-step Smolyak Level-3 orbital tracking run shows that possibilistic stress manifests through necessity saturation and surprisal escalation rather than MVEE sign change -- a direct consequence of the Holder ordering, not an empirical observation. Three lemmas establish the result: the Possibilistic Entropy Lemma decomposes the ignorance functional; the Possibilistic Cramer-Rao Bound limits entropy reduction per measurement; the Evidence-Optimality Lemma proves minimum-q selection is the unique minimizer and that any rule incorporating prior possibility risks race-to-bottom bias.

Moriba Kemessia Jah

Black Swan Research Group, GaiaVerse, Ltd. · Jah Decision Intelligence Group, Aerospace Engineering & Engineering Mechanics, The University of Texas at Austin

Epistemic UncertaintyMinimax Estimation
AskPDF
Mar 9, 2026

Efficient Credal Prediction through Decalibration

A reliable representation of uncertainty is essential for the application of modern machine learning methods in safety-critical settings. In this regard, the use of credal sets (i.e., convex sets of probability distributions) has recently been proposed as a suitable approach to representing epistemic uncertainty. However, as with other approaches to epistemic uncertainty, training credal predictors is computationally complex and usually involves (re-)training an ensemble of models. The resulting computational complexity prevents their adoption for complex models such as foundation models and multi-modal systems. To address this problem, we propose an efficient method for credal prediction that is grounded in the notion of relative likelihood and inspired by techniques for the calibration of probabilistic classifiers. For each class label, our method predicts a range of plausible probabilities in the form of an interval. To produce the lower and upper bounds of these intervals, we propose a technique that we refer to as decalibration. Extensive experiments show that our method yields credal sets with strong performance across diverse tasks, including coverage-efficiency evaluation, out-of-distribution detection, and in-context learning. Notably, we demonstrate credal prediction on models such as TabPFN and CLIP -- architectures for which the construction of credal sets was previously infeasible.

Paul Hofman, Timo Löhr, Maximilian Muschalik +2

LMU Munich · Munich Center for Machine Learning (MCML) · DFKI (DSA)

Epistemic Uncertainty
AskPDF
Mar 9, 2026

TRIAGE: Type-Routed Interventions via Aleatoric-Epistemic Gated Estimation in Robotic Manipulation and Adaptive Perception -- Don't Treat All Uncertainty the Same

Most uncertainty-aware robotic systems collapse prediction uncertainty into a single scalar score and use it to trigger uniform corrective responses. This aggregation obscures whether uncertainty arises from corrupted observations or from mismatch between the learned model and the true system dynamics. As a result, corrective actions may be applied to the wrong component of the closed loop, degrading performance relative to leaving the policy unchanged. We introduce a lightweight post hoc framework that decomposes uncertainty into aleatoric and epistemic components and uses these signals to regulate system responses at inference time. Aleatoric uncertainty is estimated from deviations in the observation distribution using a Mahalanobis density model, while epistemic uncertainty is detected using a noise robust forward dynamics ensemble that isolates model mismatch from measurement corruption. The two signals remain empirically near orthogonal during closed loop execution and enable type specific responses. High aleatoric uncertainty triggers observation recovery, while high epistemic uncertainty moderates control actions. The same signals also regulate adaptive perception by guiding model capacity selection during tracking inference. Experiments demonstrate consistent improvements across both control and perception tasks. In robotic manipulation, the decomposed controller improves task success from 59.4% to 80.4% under compound perturbations and outperforms a combined uncertainty baseline by up to 21.0%. In adaptive tracking inference on MOT17, uncertainty-guided model selection reduces average compute by 58.2% relative to a fixed high capacity detector while preserving detection quality within 0.4%. Code and demo videos are available at https://divake.github.io/uncertainty-decomposition/.

Divake Kumar, Sina Tayebati, Devashri Naik +4

University of Illinois at Chicago · Intel Labs

Robotic ControlUncertainty DecompositionRobotic ManipulationEpistemic UncertaintyAleatoric Uncertainty
AskPDF
Mar 6, 2026

CREDO: Epistemic-Aware Conformalized Credal Envelopes for Regression

Conformal prediction delivers prediction intervals with distribution-free coverage, but its intervals can look overconfident in regions where the model is extrapolating, because standard conformal scores do not explicitly represent epistemic uncertainty. Credal methods, by contrast, make epistemic effects visible by working with sets of plausible predictive distributions, but they are typically model-based and lack calibration guarantees. We introduce CREDO, a simple "credal-then-conformalize" recipe that combines both strengths. CREDO first builds an interpretable credal envelope that widens when local evidence is weak, then applies split conformal calibration on top of this envelope to guarantee marginal coverage without further assumptions. This separation of roles yields prediction intervals that are interpretable: their width can be decomposed into aleatoric noise, epistemic inflation, and a distribution-free calibration slack. We provide a fast implementation based on trimming extreme posterior predictive endpoints, prove validity, and show on benchmark regressions that CREDO maintains target coverage while improving sparsity adaptivity at competitive efficiency.

Luben M. C. Cabezas, Sabina J. Sloman, Bruno M. Resende +3

Department of Statistics, Federal University of S˜ao Carlos, S˜ao Carlos, Brazil · Institute of Mathematics and Computer Science , University of S˜ao Paulo, S˜ao Carlos, Brazil · Universit´e Grenoble Alpes, Inria, CNRS, Grenoble INP, LJK, France +1

Regression Prediction IntervalsConformal PredictionEpistemic Uncertainty
AskPDF
Mar 4, 2026

No Certificate, No Categorical Speech Act: A Brouwerian Assertibility Constraint for Public Reason

Generative AI can convert uncertainty into authoritative-seeming verdicts, displacing the justificatory work on which democratic epistemic agency depends. As a corrective, I propose a Brouwer-inspired assertibility constraint for responsible AI: in high-stakes domains, systems may assert or deny claims only if they can provide a publicly inspectable and contestable certificate of entitlement; otherwise they must return Undetermined. This constraint yields a three-status interface semantics (Asserted, Denied, Undetermined) whose statuses mark entitlement to categorical speech rather than truth values of the underlying world-claim. The framework distinguishes internal entitlement from public standing while connecting them through the certificate as a boundary object. Philosophically, it develops a form of applied intuitionism: resistance to unwarranted completeness becomes a norm of public machine speech, under which categorical commitment requires exhibited and contestable entitlement. I operationalize the constraint through structured certificates, decision-layer gates for threshold and argmax predicates, and a reason-coded output contract. A design lemma shows that, under a refutation-soundness condition on the contractually specified negative side, certificate-sound binary totality presupposes witnessed decidability on the declared scope. Under the output contract, Undetermined is therefore not a tunable reject option but a mandatory status whenever the available certificates do not uniquely license assertion or denial. By making outputs answerable to contestable warrants rather than confidence alone, the paper aims to preserve epistemic agency where automated speech enters public justification.

Michael Jülich

aihorizon R&D, Munich, Germany.

AI AccountabilitySelective PredictionAI GovernanceHuman Oversight of AIEpistemic Uncertainty
AskPDF
Mar 2, 2026

Can machines be uncertain?

The paper investigates whether and how AI systems can realize states of uncertainty. By adopting a functionalist and behavioral perspective, it examines how symbolic, connectionist and hybrid architectures make room for uncertainty. The paper distinguishes between epistemic uncertainty, or uncertainty inherent in the data or information, and subjective uncertainty, or the system's own attitude of being uncertain. It further distinguishes between distributed and discrete realizations of subjective uncertainty. A key contribution is the idea that some states of uncertainty are interrogative attitudes whose content is a question rather than a proposition.

Luis Rosa

Washington University in St. Louis

Epistemic Uncertainty
AskPDF
Mar 2, 2026

Instrumental and Proximal Causal Inference with Gaussian Processes

Instrumental variable (IV) and proximal causal learning (Proxy) methods are central frameworks for causal inference in the presence of unobserved confounding. Despite substantial methodological advances, existing approaches rarely provide reliable epistemic uncertainty (EU) quantification. We address this gap through a Deconditional Gaussian Process (DGP) framework for uncertainty-aware causal learning. Our formulation recovers popular kernel estimators as the posterior mean, ensuring predictive precision, while the posterior variance yields principled and well-calibrated EU. Moreover, the probabilistic structure enables systematic model selection via marginal log-likelihood optimization. Empirical results demonstrate strong predictive performance alongside informative EU quantification, evaluated via empirical coverage frequencies and decision-aware accuracy rejection curves. Together, our approach provides a unified, practical solution for causal inference under unobserved confounding with reliable uncertainty.

Yuqi Zhang, Krikamol Muandet, Dino Sejdinovic +2

Department of Statistics and Actuarial Science, University of Hong Kong, Hong Kong · Rational Intelligence Lab, CISPA Helmholtz Center for Information Security, Germany · Australian Institute for Machine Learning, Adelaide University, Australia +1

Proximal Causal InferenceCausal InferenceCausal Inference with Unobserved ConfoundingInstrumental Variable EstimationEpistemic UncertaintyGaussian Process Regression
AskPDF
Mar 1, 2026

Epistemic Gain, Aleatoric Cost: Uncertainty Decomposition in Multi-Agent Debate for Math Reasoning

Multi-Agent Debate (MAD) has shown promise in improving reasoning and reducing hallucinations, yet it remains unclear how information exchange shapes individual reasoning behavior. Empirically, MAD exhibits paradoxical phenomena, including rising accuracy with increasing token entropy and marked differences between homogeneous and heterogeneous agent combinations. In this paper, we introduce a Bayesian uncertainty analysis framework for MAD, which decomposes answer-level predictive uncertainty into epistemic uncertainty and aleatoric uncertainty, corresponding to the potential gain and cost of debate. Across multiple agent configurations, we find that effective debate depends on achieving high epistemic gain under controlled aleatoric cost. Building on this insight, we design an uncertainty-guided multi-agent reinforcement learning algorithm that encourages lower aleatoric cost and more effective epistemic information utilization. Experiments show that our approach simultaneously enhances each agent's accuracy and promotes a more productive debate process, providing an operational Bayesian perspective for understanding and improving MAD.

Dan Qiao, Binbin Chen, Fengyu Cai +7

School of Data Science, The Chinese University of Hong Kong, Shenzhen, China · ByteDance, Beijing, China · Technical University of Darmstadt, Darmstadt, Germany +2

Multi-Agent DebateEpistemic UncertaintyAleatoric UncertaintyMulti-Agent Reinforcement Learning
AskPDF
Feb 26, 2026

Invariant Transformation and Resampling based Epistemic-Uncertainty Reduction

An artificial intelligence (AI) model can be viewed as a function that maps inputs to outputs in high-dimensional spaces. Once designed and well trained, the AI model is applied for inference. However, even optimized AI models can produce inference errors due to aleatoric and epistemic uncertainties. Interestingly, we observed that when inferring multiple samples based on invariant transformations of an input, inference errors can show partial independences due to epistemic uncertainty. Leveraging this insight, we propose a "resampling" based inferencing that applies to a trained AI model with multiple transformed versions of an input, and aggregates inference outputs to a more accurate result. This approach has the potential to improve inference accuracy and offers a strategy for balancing model size and performance.

Sha Hu

Lund Research Center, Huawei Technologies Sweden AB, Sweden.

Inference-Time OptimizationTest-Time AugmentationEpistemic Uncertainty
AskPDF
Feb 26, 2026

Set-based v.s. Distribution-based Representations of Epistemic Uncertainty: A Comparative Study

Epistemic uncertainty in neural networks is commonly modeled using two second-order paradigms: distribution-based representations, which rely on posterior parameter distributions, and set-based representations based on credal sets (convex sets of probability distributions). These frameworks are often regarded as fundamentally non-comparable due to differing semantics, assumptions, and evaluation practices, leaving their relative merits unclear. Empirical comparisons are further confounded by variations in the underlying predictive models. To clarify this issue, we present a controlled comparative study enabling principled, like-for-like evaluation of the two paradigms. Both representations are constructed from the same finite collection of predictive distributions generated by a shared neural network, isolating representational effects from predictive accuracy. Our study evaluates each representation through the lens of 3 uncertainty measures across 8 benchmarks, including selective prediction and out-of-distribution detection, spanning 6 underlying predictive models and 10 independent runs per configuration. Our results show that meaningful comparison between these seemingly non-comparable frameworks is both feasible and informative, providing insights into how second-order representation choices impact practical uncertainty-aware performance.

Kaizheng Wang, Yunjia Wang, Fabio Cuzzolin +3

College of Computing and Data Science, Nanyang Technological University, Singapore · Department of Computer Science, KU Leuven, Belgium · School of Engineering, Computing, and Mathematics, Oxford Brookes University, U.K. +1

Uncertainty QuantificationEpistemic Uncertainty
AskPDF
Feb 24, 2026

Not Just How Much, But Where: Decomposing Epistemic Uncertainty into Per-Class Contributions

In safety-critical classification, the cost of failure is often asymmetric, yet Bayesian deep learning summarises epistemic uncertainty with a single scalar, mutual information (MI), that cannot distinguish whether a model's ignorance involves a benign or safety-critical class. We decompose MI into a per-class vector Ck(x)=σk2/(2μk)C_k(x)=σ_k^{2}/(2μ_k), with μk=E[pk]μ_k{=}\mathbb{E}[p_k] and σk2=Var[pk]σ_k^2{=}\mathrm{Var}[p_k] across posterior samples. The decomposition follows from a second-order Taylor expansion of the entropy; the 1/μk1/μ_k weighting corrects boundary suppression and makes CkC_k comparable across rare and common classes. By construction ∑kCk≈MI\sum_k C_k \approx \mathrm{MI}, and a companion skewness diagnostic flags inputs where the approximation degrades. After characterising the axiomatic properties of CkC_k, we validate it on three tasks: (i) selective prediction for diabetic retinopathy, where critical-class CkC_k reduces selective risk by 34.7% over MI and 56.2% over variance baselines; (ii) out-of-distribution detection on clinical and image benchmarks, where ∑kCk\sum_k C_k achieves the highest AUROC and the per-class view exposes asymmetric shifts invisible to MI; and (iii) a controlled label-noise study in which ∑kCk\sum_k C_k shows less sensitivity to injected aleatoric noise than MI under end-to-end Bayesian training, while both metrics degrade under transfer learning. Across all tasks, the quality of the posterior approximation shapes uncertainty at least as strongly as the choice of metric, suggesting that how uncertainty is propagated through the network matters as much as how it is measured.

Mame Diarra Toure, David A. Stephens

Department of Mathematics and Statistics , McGill University

Bayesian Neural NetworksClassificationMutual Information EstimationUncertainty DecompositionEpistemic Uncertainty
AskPDF
Feb 24, 2026

When Alpha Breaks: Two-Level Uncertainty for Safe Deployment of Cross-Sectional Stock Rankers

Cross-sectional ranking models are often deployed as if point predictions were sufficient: the model outputs scores and the portfolio follows the induced ordering. Under non-stationarity, rankers can fail during regime shifts. In the AI Stock Forecaster, a LightGBM ranker performs well overall at a 20-day horizon, yet the 2024 holdout coincides with an AI thematic rally and sector rotation that breaks the signal at longer horizons and weakens 20d. This motivates treating deployment as two decisions: (i) whether the strategy should trade at all, and (ii) how to control risk within active trades. We adapt Direct Epistemic Uncertainty Prediction (DEUP) to ranking by predicting rank displacement and defining an epistemic uncertainty signal ehat relative to a point-in-time (PIT-safe) baseline. Empirically, ehat is structurally coupled with signal strength (median correlation between ehat and absolute score is about 0.6 across 1,865 dates), so inverse-uncertainty sizing de-levers the strongest signals and degrades performance. To address this, we propose a two-level deployment policy: a strategy-level regime-trust gate G(t) that decides whether to trade (AUROC around 0.72 overall and 0.75 in FINAL) and a position-level epistemic tail-risk cap that reduces exposure only for the most uncertain predictions. The operational policy, trade only when G(t) is at least 0.2, apply volatility sizing on active dates, and cap the top epistemic tail, improves risk-adjusted performance in the 20d policy comparison and indicates DEUP adds value mainly as a tail-risk guard rather than a continuous sizing denominator.

Ursina Sanderink

Quantitative FinanceDecision-Making under UncertaintyEpistemic UncertaintyLearning to Rank
AskPDF
Feb 24, 2026

Structure-Aware Epistemic Uncertainty Quantification for Neural Operator PDE Surrogates

Neural operators (NOs) provide fast, resolution-invariant surrogates for mapping input fields to PDE solution fields, but their predictions can exhibit significant epistemic uncertainty due to finite data, imperfect optimization, and distribution shift. For practical deployment in scientific computing, uncertainty quantification (UQ) must be both computationally efficient and spatially faithful, i.e., uncertainty bands should align with the localized residual structures that matter for downstream risk management. We propose a structure-aware epistemic UQ scheme that exploits the modular anatomy common to modern NOs (lifting-propagation-recovering). Instead of applying unstructured weight perturbations (e.g., naive dropout) across the entire network, we restrict Monte Carlo sampling to a module-aligned subspace by injecting stochasticity only into the lifting module, and treat the learned solver dynamics (propagation and recovery) as deterministic. We instantiate this principle with two lightweight lifting-level perturbations, including channel-wise multiplicative feature dropout and a Gaussian feature perturbation with matched variance, followed by standard calibration to construct uncertainty bands. Experiments on challenging PDE benchmarks (including discontinuous-coefficient Darcy flow and geometry-shifted 3D car CFD surrogates) demonstrate that the proposed structure-aware design yields more reliable coverage, tighter bands, and improved residual-uncertainty alignment compared with common baselines, while remaining practical in runtime.

Haoze Song, Zhihao Li, Mengyi Deng +4

The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China · The Hong Kong University of Science and Technology, Hong Kong SAR, China

Uncertainty QuantificationEpistemic UncertaintyNeural OperatorsPDE Surrogate ModelingUncertainty Calibration
AskPDF
Feb 24, 2026

ERA: Evidence-based Reliability Alignment for Honest Retrieval-Augmented Generation

Retrieval-Augmented Generation (RAG) grounds language models in factual evidence but introduces critical challenges regarding knowledge conflicts between internalized parameters and retrieved information. However, existing reliability methods, typically relying on scalar confidence, fail to explicitly distinguish between epistemic uncertainty and inherent data ambiguity in such hybrid scenarios. In this paper, we propose a new framework called ERA (Evidence-based Reliability Alignment) to enhance abstention behavior in RAG systems by shifting confidence estimation from scalar probabilities to explicit evidence distributions. Our method consists of two main components: (1) Contextual Evidence Quantification, which models internal and external knowledge as independent belief masses via the Dirichlet distribution, and (2) Quantifying Knowledge Conflict, which leverages Dempster-Shafer Theory (DST) to rigorously measure the geometric discordance between information sources. These components are used to disentangle epistemic uncertainty from aleatoric uncertainty and modulate the optimization objective based on detected conflicts. Experiments on standard benchmarks and a curated generalization dataset demonstrate that our approach significantly outperforms baselines, optimizing the trade-off between answer coverage and abstention with superior calibration.

Sunguk Shin, Meeyoung Cha, Byung-Jun Lee +1

Korea University, Seoul, South Korea · MPI-SP, Bochum, Germany · KAIST, Daejeon, South Korea +1

RAG RobustnessLLM AbstentionKnowledge Conflicts in Language ModelsConfidence Estimation in Language ModelsEpistemic Uncertainty
AskPDF
Feb 1, 2026

Density-Informed Pseudo-Counts for Calibrated Evidential Deep Learning

Evidential Deep Learning (EDL) is a popular framework for uncertainty-aware classification that models predictive uncertainty via Dirichlet distributions parameterized by neural networks. Despite its popularity, its theoretical foundations and behavior under distributional shift remain poorly understood. In this work, we provide a principled statistical interpretation by proving that EDL training corresponds to amortized variational inference in a hierarchical Bayesian model with a tempered pseudo-likelihood. This perspective reveals a major drawback: standard EDL conflates epistemic and aleatoric uncertainty, leading to systematic overconfidence on out-of-distribution (OOD) inputs. To address this, we introduce Density-Informed Pseudo-count EDL (DIP-EDL), a new parametrization that decouples class prediction from the magnitude of uncertainty by separately estimating the conditional label distribution and the marginal covariate density. This separation preserves evidence in high-density regions while shrinking predictions toward a uniform prior for OOD data. Theoretically, we prove that DIP-EDL achieves asymptotic concentration. Empirically, we show that our method enhances interpretability and improves robustness and uncertainty calibration under distributional shift.

Pietro Carlotti, Nevena Gligić, Arya Farahi

Department of Statistics and Data Sciences University of Texas at Austin

Evidential Deep LearningOOD DetectionDensity EstimationUncertainty CalibrationEpistemic Uncertainty
AskPDF
Oct 29, 2025

Uncertainty Quantification for Regression: A Unified Framework based on kernel scores

Regression tasks, notably in safety-critical domains, require reliable uncertainty quantification, yet the literature remains largely classification-focused. To address this, we introduce a family of measures for total, aleatoric, and epistemic uncertainty in multivariate regression based on strictly proper kernel scores. The framework provides a principled recipe for designing new uncertainty measures whose behavior, such as tail sensitivity or out-of-distribution responsiveness, is governed by the choice of the underlying kernel, while also encompassing existing measures under a joint analysis. We prove explicit correspondences between properties of the kernel and behavior of resulting uncertainty measures, yielding concrete design guidelines for practitioners. Extensive experiments across structured regression tasks, including spatial and functional domains, demonstrate effectiveness on downstream tasks such as out-of-distribution detection and active learning, and reveal that different kernel choices lead to distinct trade-offs, offering practitioners guidance for task-specific selection.

Christopher Bülte, Yusuf Sale, Gitta Kutyniok +1

LMU Munich · 2Munich Center for Machine Learning (MCML) · 3DLR-German Aerospace Center +2

Uncertainty QuantificationAleatoric UncertaintyUncertainty DecompositionEpistemic Uncertainty
AskPDF
Aug 12, 2025

Rethinking Prospect Theory for LLMs: Revealing the Instability of Decision-Making under Epistemic Uncertainty

Real-world decision-making often involves uncertainty expressed in linguistic rather than numerical terms, and Prospect Theory (PT) provides a classic framework for modeling human behavior under such uncertainty. Although recent studies have developed frameworks to estimate PT parameters for Large Language Models (LLMs), few have examined whether PT itself adequately describes LLM decision-making behavior. To address these gaps, we develop a streamlined workflow grounded in a classic behavioral economics experimental paradigm. First, we estimate PT parameters and evaluate how well the resulting model captures LLM decision-making behavior. We then derive probability mappings for epistemic markers in the same context and inject them into prompts to examine the stability of PT parameters under linguistic uncertainty. Our findings suggest that PT does not consistently provide a reliable account of LLM decision-making across models, and that its application to LLMs is likely sensitive to epistemic uncertainty. The findings caution against the deployment of PT-based frameworks in real-world applications where epistemic ambiguity is prevalent, giving valuable insights in behaviour interpretation and future alignment direction for LLM decision-making.

Rui Wang, Qihan Lin, Jiayu Liu +7

Hong Kong University of Science and Technology · Huazhong University of Science and Technology · University of Illinois Urbana-Champaign

Prospect TheoryLLM Decision-MakingDecision-Making under UncertaintyEpistemic Uncertainty
AskPDF
Jul 24, 2025

Toward Efficient Uncertainty in LLMs through Evidential Knowledge Distillation

Accurate uncertainty quantification remains a key challenge for standard LLMs, prompting the adoption of Bayesian and ensemble-based methods. However, such methods typically necessitate computationally expensive sampling, involving multiple forward passes to effectively estimate predictive uncertainty. In this paper, we introduce an approach enabling uncertainty estimation in LLMs without incurring the heavy inference latency typically associated with sampling methods. Specifically, we distill uncertainty-aware teachers - originally requiring multiple forward passes - into single-pass students, fine-tuned using LoRA. We compare two distinct distillation strategies: one in which the student employs traditional softmax-based outputs, and another in which the student leverages Dirichlet-distributed outputs to explicitly model epistemic uncertainty via evidential learning. Empirical evaluation on classification tasks demonstrate that such students can achieve comparable predictive and uncertainty quantification performance relative to their teachers, while requiring only a single forward pass.

Lakshmana Sri Harsha Nemani, P. K. Srijith, Tomasz Kuśmierczyk

Department of Computer Science and Engineering, Indian Institute of Technology Hyderabad, India · Jagiellonian University, Faculty of Mathematics and Computer Science, Kraków, Poland

LLM Uncertainty EstimationUncertainty-Aware Knowledge DistillationEfficient Language Model InferenceKnowledge DistillationEpistemic Uncertainty
AskPDF
Jul 10, 2025

CLEAR: Calibrated Learning for Epistemic and Aleatoric Risk

Accurate uncertainty quantification is critical for reliable predictive modeling. Existing methods typically address either aleatoric uncertainty due to measurement noise or epistemic uncertainty resulting from limited data, but not both in a balanced manner. We propose CLEAR, a calibration method with two distinct parameters, γ1γ_1 and γ2γ_2, to combine the two uncertainty components and improve the conditional coverage of predictive intervals for regression tasks. CLEAR is compatible with any pair of aleatoric and epistemic estimators; we show how it can be used with (i) quantile regression for aleatoric uncertainty and (ii) ensembles drawn from the Predictability-Computability-Stability (PCS) framework for epistemic uncertainty. Across 17 diverse real-world datasets, CLEAR achieves an average improvement of 28.3% and 17.5% in the interval width compared to the two individually calibrated baselines while maintaining nominal coverage. Similar improvements are observed when applying CLEAR to Deep Ensembles (epistemic) and Simultaneous Quantile Regression (aleatoric). The benefits are especially evident in scenarios dominated by high aleatoric or epistemic uncertainty. Project page: https://unco3892.github.io/clear/

Ilia Azizi, Juraj Bodik, Jakob Heiss +1

Department of Operations, HEC, University of Lausanne, Switzerland · BegooAI, Switzerland · Department of Statistics, University of California, Berkeley, USA +1

Uncertainty QuantificationAleatoric UncertaintyUncertainty CalibrationRegression Prediction IntervalsEpistemic Uncertainty
AskPDF
May 25, 2025

Optimal Conformal Prediction under Epistemic Uncertainty

Conformal prediction (CP) is a widely used frequentist framework to quantify uncertainty by constructing prediction sets with user-specified marginal coverage guarantees. In practice, CP is typically applied on top of probabilistic classifiers, which are able to express aleatoric but not epistemic uncertainty. In this paper, we consider the question of how to optimally employ CP on top of a more expressive formalism, namely credal sets, which can express both aleatoric and epistemic uncertainty. More specifically, we propose probabilistic Bernoulli prediction sets (BPS) and derive a variant that achieves conditional coverage for valid credal sets while remaining minimal in expected size. We then address the more realistic scenario in which the validity of the credal sets is not guaranteed. Assuming access to calibration data with ground-truth distributions over labels, we apply conformal risk control to BPS and derive a PAC-style guarantee: with high probability over the data, the achieved conditional coverage is at least the desired level. We validate our theoretical findings empirically over various datasets.

Alireza Javanmardi, Soroush H. Zargarbashi, Santo M. A. R. Thies +3

1LMU Munich · 2MCML · 3CISPA +3

Conformal PredictionConformal Risk ControlEpistemic Uncertainty
AskPDF
Apr 25, 2025

An Axiomatic Assessment of Entropy- and Variance-based Uncertainty Quantification in Regression

Uncertainty quantification is crucial in machine learning, yet most (axiomatic) studies of uncertainty measures focus on classification, leaving a gap in regression settings with limited formal justification and evaluations. In this work, we provide a formal way of representing uncertainty in continuous space, using a general parametric formulation, allowing for tractable analysis and evaluation of uncertainty measures. Within this framework, we propose a set of axioms that enable rigorous assessment of total, aleatoric, and epistemic uncertainty measures. Together, this allows for a theoretical examination of uncertainty measures and their corresponding properties. As a specific example, we compare the widely used entropy- and variance-based measures with respect to established predictive models and analyze their limitations and challenges in uncertainty quantification. Our work provides a principled way to understand and develop uncertainty measures in supervised regression, offering theoretical insights and practical guidelines for reliable uncertainty assessment.

Christopher Bülte, Yusuf Sale, Timo Löhr +3

1LMU Munich · 2Munich Center for Machine Learning (MCML) · 3DLR-German Aerospace Center +2

Uncertainty QuantificationAleatoric UncertaintyEpistemic Uncertainty
AskPDF
Jul 20, 2024

Prompting the Unknown: Understanding Response Uncertainty in Large Language Models

Large language models (LLMs) are widely used in decision-making across diverse domains. Ensuring the generation of safe and reliable responses is critical for the effective deployment of LLM-based applications, particularly in high-stakes domains such as healthcare and finance. Most of these applications typically use carefully crafted prompts to guide response generation; however, the relationship between prompts and the reliability of LLM-generated responses is not yet fully understood. To address this gap, we propose a novel prompt-response concept model that explains the relationship between the amount of task-relevant information (informativeness) provided in the prompt and the LLM-generated response uncertainty by identifying four sources of response uncertainty: prompt underspecification, model quality, task variability, and semantic redundancy. We prove that response uncertainty decreases as prompt informativeness or model quality increases, mirroring the behavior of epistemic uncertainty in probabilistic models. Our experimental results on real-world datasets further validate our proposed model and corroborate the theoretical results.

Ze Yu Zhang, Arun Verma, Finale Doshi-Velez +1

Department of Computer Science, National University of Singapore, Republic of Singapore · Singapore-MIT Alliance for Research and Technology Centre · Harvard University

LLM PromptingLLM ReliabilityLLM Uncertainty EstimationEpistemic Uncertainty
AskPDF
May 29, 2024

Active Exploration via Autoregressive Generation of Missing Data

We pose uncertainty quantification and exploration in online decision-making as a problem of training and generation from an autoregressive sequence model, an area experiencing rapid innovation. Our approach rests on viewing uncertainty as arising from missing future outcomes that could be revealed through action choices, rather than from unobservable latent parameters of the environment. This reformulation aligns naturally with modern machine learning capabilities: we can i) train generative models through next-outcome prediction rather than fit explicit priors, ii) assess uncertainty through autoregressive generation rather than sampling latent parameters from posteriors, and iii) adapt to new information by extending the sequence model's context rather than explicit posterior updating. Our main theoretical result establishes a reduction from online decision-making to offline next-outcome prediction: Bayesian regret is controlled directly by the sequence model's offline prediction loss, without requiring an explicit latent-variable posterior. Experiments, including a semi-synthetic news recommendation task, show that autoregressive generation produces calibrated epistemic uncertainty and enables effective exploration by using article text as prior information to focus exploration on resolving remaining uncertainties.

Tiffany Tianhui Cai, Hongseok Namkoong, Daniel Russo +1

Department of Statistics Columbia University New York, NY 10025, USA · Decision, Risk, and Operations Division Columbia Business School New York, NY 10025, USA · Department of Mathematics Imperial College London London SW7 2AZ, United Kingdom

RL ExplorationAutoregressive GenerationDecision-Making under UncertaintyEpistemic UncertaintyUncertainty-Aware Exploration
AskPDF