Aleatoric Uncertainty

Follow this topic Ask about this topic

Momentum

2 papers in the last four weeks, against 1 the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28

Latest papers 29

Sep 30, 2026cs.CV

Structural Limits of the Information-Theoretic Uncertainty Decomposition

Uncertainty estimation in machine learning typically decomposes uncertainty into aleatoric uncertainty (AU) and epistemic uncertainty (EU) using the standard information-theoretic framework. However, in practice, two critical issues arise: entanglement (AU and EU are highly correlated) and epistemic collapse (EU magnitude shrinks with increasing model capacity). We analyze this framework on a functional level and discover that significant portions of the assumed AU, EU range are infeasible in finite settings, and cannot be attained with any class probabilities. We characterize how this infeasible region scales with the number of classes and Monte Carlo samples

N

(e.g., from ensembles with

N

members), revealing it is bounded by

\text{AU} \leq \log(2)/N

. Crucially, the infeasible region's boundary helps explain epistemic collapse: when model confidence is high,

\text{AU} > \text{EU}

is guaranteed by this fundamental structural limitation. Our findings show that increasing ensemble size mitigates epistemic collapse by reducing the infeasible area. Lastly, we caution against interpreting AU and EU as independent quantities in low AU regimes, since we show they are coupled when

\text{AU} \leq \log(2)/N

.

Jakob Lønborg Christensen, Christian F. Baumgartner, Morten Rieger Hannemose +2

DTU Compute Technical University of Denmark · Faculty of Health Sciences and Medicine University of Lucerne

Aleatoric Uncertainty Information-Theoretic Lower Bounds Uncertainty Decomposition Epistemic Uncertainty

Sep 7, 2026cs.AI

From Answers to Interpretations: Rethinking Ambiguity-Induced Aleatoric Uncertainty Estimation in LLMs

A key challenge in reliable LLM deployment is recognizing when uncertainty reflects irreducible variability in the task rather than limitations in the model's knowledge. In language tasks, a central source of such aleatoric uncertainty is input ambiguity or underspecification, where multiple interpretations remain plausible. Existing decomposition methods estimate aleatoric uncertainty by generating multiple clarifications of the input, querying the model for an answer under each clarification, and comparing the resulting answers. We argue that answers are not necessary for identifying ambiguity: they are often redundant, add avoidable cost, and can mislead through epistemic leakage. We support this claim theoretically, and propose a clarification-only approach that estimates this ambiguity-induced component directly from the space of plausible interpretations, without answers to the clarified inputs. Using ambiguity detection as an operational evaluation across three benchmarks, this direct approach improves AUROC (63.34 vs. 60.85), reduces computational cost by 4-26x in output tokens and 2.2-3.5x in API calls, and yields estimates with substantially lower correlation with epistemic uncertainty. Overall, our results suggest that ambiguity-induced aleatoric uncertainty is better estimated from the interpretation space than from the response space.

Omer Nahum, Niv Nayman, Jonathan Fhima +4

Data and Decision Sciences, Technion, Israel · Amazon Web Services · Computer Vision Group, University of Bern, Switzerland

LLM Uncertainty Estimation Aleatoric Uncertainty

Aug 31, 2026physics.ao-ph

Uncertainty-Aware End-to-End AI Weather Forecasting: Disentangling Observation and Model Contributions

End-to-end weather forecasting systems produce skillful global gridded and station forecasts directly from raw Earth observations, replacing the numerical weather prediction pipeline, including data assimilation, at a fraction of its cost. These systems are deterministic and issue no uncertainty. Here we render the Aardvark Weather model probabilistic by attaching one stochastic mechanism to each component: learned, input-dependent noise at the observation encoder, capturing aleatoric uncertainty inherited from the observing system, and Monte Carlo dropout in the processor, capturing epistemic uncertainty in the learned dynamics. The resulting nested ensemble attributes forecast spread to the two sources through a law-of-total-variance decomposition, cross-checked by withholding observation streams. Probabilistic finetuning significantly improves the mean forecast, by 4.2% on average across variables and lead times. The ensemble is calibrated against ERA5 through the medium range (spread-skill ratio 0.98), keeps station RMSE within 2.4% of the deterministic model while beating it in CRPS at every lead time, and trails the operational ECMWF ensemble. The encoder branch behaves as observation-driven uncertainty. Component-attributed uncertainty makes end-to-end forecasts more transparent, a step toward observation-driven digital twins of the atmosphere.

Rodrigo Almeida, Noelia Otero, Jost Arndt +3

Applied Machine Learning Group, Fraunhofer Heinrich-Hertz Institute, 10587 Berlin, Germany · Department of Electrical Engineering and Computer Science, Technische Universität Berlin, 10587 Berlin, Germany · BIFOLD - Berlin Institute for the Foundations of Learning and Data, 10587 Berlin, Germany

Probabilistic Forecasting Aleatoric Uncertainty Uncertainty Decomposition Epistemic Uncertainty Weather Forecasting

Aug 7, 2026cs.LG

Tracing sources of epistemic uncertainty in deep learning predictions: homo- and hetero-scedastic linearized estimators

We adapt two classical statistical estimators for quantifying uncertainty to modern deep learning, in order to provide clearer insights into uncertainty attributable to two sources : aleatoric uncertainty, or locally scarce data. Our approach leverages recent advances in approximate Fisher Information Matrices, to enable scaling to actual architectures. Experimental results demonstrate how each test points is differentially impacted by both sources, highlighting the practical utility of our estimators in improving the robustness of real-world applications.

Pierre Nodet, Thomas George

Orange Research, Châtillon, France

Uncertainty Quantification Aleatoric Uncertainty Uncertainty Decomposition Epistemic Uncertainty

Aug 6, 2026cs.LG

A Ground-Truth Framework for Uncertainty Disentanglement with Posterior Risk

Reliable uncertainty estimates are critical in safety-sensitive applications. For such estimates to be useful in practice, it is crucial to understand the sources underlying a model's uncertainty, motivating the disentanglement of total uncertainty into epistemic and aleatoric uncertainty. Existing notions of uncertainty differ in the sources they capture and, consequently, in their definitions of aleatoric and epistemic uncertainty, with no universally accepted definition. We define uncertainty through sample-conditional pointwise posterior risk, which is the expected loss of a predictor under the distribution of plausible ground-truth functions given the observed sample. This definition unifies probabilistic and risk-based concepts of uncertainty. To assess state-of-the-art uncertainty disentanglement methods, we develop a framework that directly compares their estimates against ground-truth uncertainty defined primarily by posterior risk, alongside commonly used alternative uncertainty definitions. We find that Spectral-normalized Neural Gaussian Processes and Variational Latent Gaussian Processes most closely recover the ground-truth uncertainty, while most methods track posterior variance more closely than posterior risk, missing the predictors' bias. Beyond method rankings, we investigate how strongly estimated aleatoric and epistemic uncertainty are entangled and how sensitive uncertainty quality is to modeling choices, yielding practical guidance for uncertainty disentanglement. To support further method development and validation, we release 13 semi-synthetic UCI/OpenML datasets with known posteriors, enabling the computation of ground-truth uncertainty. Code and data will be made publicly available upon acceptance.

Frieder Wizgall, Georg Tirpitz, Moritz Seiler +2

University of Tübingen, Tübingen, Germany · Hertie Institute for AI in Brain Health, University of Tübingen, Tübingen, Germany · Bernstein Center for Computational Neuroscience Berlin, Berlin, Germany +2

Uncertainty Quantification Aleatoric Uncertainty Uncertainty Decomposition Epistemic Uncertainty

Jul 27, 2026cs.LG

Attribution and Uncertainty Behavior of Learned Residual Gyro Correction for Gyro-Stellar Estimation

This work investigates uncertainty decomposition and explainability in a deep learning-based framework for gyroscope bias correction. A 1-D Convolutional Neural Network is trained to predict residual angular rate corrections from multi-sensor inputs, including gyroscope and star tracker measurements. The bias corrections are sent to a flight-representative Gyro-Stellar Estimator. The network produces both mean corrections and input-dependent (heteroscedastic) aleatoric uncertainty, while epistemic uncertainty is estimated via an ensemble of independently trained models. The proposed approach is trained under nominal conditions and evaluated in both nominal and structured perturbations that include additive and temporally correlated noise. Gradient-based attribution methods are applied to both the correction and uncertainty outputs, enabling a decomposition of the evidence that drives state updates and uncertainty estimates. By aggregating attribution patterns across rotational axes and regimes, we reveal axis-specific behaviors and characterize how structured perturbations influence the collaboration between aleatoric and epistemic uncertainty. Uncertainty analysis shows that aleatoric uncertainty increases with perturbation intensity, but the distributions overlap and the calibration is not consistent across regimes. On the other hand, epistemic uncertainty gives a clear signal that gets clearer as the distributional shift happens, showing that the models disagree more. These results show that aleatoric and epistemic uncertainty work well together and that epistemic uncertainty is better at distinguishing between nominal and perturbed operating conditions. The results provide insight into the behavior of hybrid learning-based state estimation components and motivate the use of uncertainty for downstream monitoring and fault detection.

Mariela De Lucas Álvarez, Melvin Laux, Arthur de Freitas Precht +4

Robotics Innovation Center, German Research Center for AI (DFKI GmbH), Bremen, Germany · Airbus Defence and Space GmbH, Immenstaad, Germany

Residual Learning Aleatoric Uncertainty Inertial Navigation Epistemic Uncertainty

Jul 17, 2026cs.CV

Disentangling Model and Human Data Uncertainty in Apparent Facial Age Estimation

Estimating the apparent age of individuals from facial images is challenging due to the subjective nature of perception and the inherent variability of the data. We investigate the role of uncertainty estimation, attributing uncertainty jointly to a lack of knowledge (epistemic) or inherent noise/chance (aleatoric). Leveraging the APPA-REAL dataset, we train Bayesian Neural Networks on datasets of varying sizes using three BNN approximations: MC-DropConnect, Flipout, and Deep Ensembles using supervision on human aleatoric uncertainty available in the APPA-REAL dataset. Each model outputs both the predicted apparent age and the amount of aleatoric and epistemic uncertainty. Our results confirm the hypothesis that the inherent aleatoric uncertainty remains stable across dataset sizes, while epistemic uncertainty increases as training data decreases. These findings demonstrate that different sources of uncertainty can be quantified in face age estimation.

Andrei Foitos, Ivo Pascal de Jong, Matias Valdenegro-Toro

Bernoulli Institute, University of Groningen, Groningen, NL

Bayesian Neural Networks Uncertainty Quantification Aleatoric Uncertainty Epistemic Uncertainty

Jul 16, 2026stat.ML

Subjective Risk Decomposition: A New View for Uncertainty Quantification

We present a novel viewpoint for uncertainty quantification. Uncertainty measures are not primitives, in need of axioms and argumentation, but instead consequences, of higher-level modelling decisions. We show how epistemic and aleatoric uncertainty measures can be derived via decomposition of a subjective risk, based on a strictly proper loss. Reverse cross entropy provides a prominent example, where decomposition recovers the classic information-theoretic uncertainty terms. The same approach recovers numerous measures previously proposed across the UQ literature, providing them a common theoretical foundation. This suggests a new approach to UQ: given a modelling scenario and strictly proper loss, the corresponding epistemic and aleatoric terms are induced by the subjective-risk decomposition. We then extend our view to learning theory: we introduce and analyse subjective risk analogues of excess risk, approximation error and estimation error, and identify the connections to UQ. We consider this a first step towards a full learning-theoretic framework for uncertainty quantification.

Raghad Alamri, Michele Caprio, Gavin Brown

Department of Computer Science, The University of Manchester · Department of Computer Science, University of Warwick

Uncertainty Quantification Aleatoric Uncertainty Uncertainty Decomposition Epistemic Uncertainty

Jul 3, 2026cs.LG

Trading Confidence: Comprehensive Uncertainty Estimation in Algorithmic Trading

Reinforcement Learning (RL) has emerged as a powerful approach in financial trading, enabling agents to learn optimal strategies through direct market interaction. However, financial markets are highly uncertain, with price fluctuations driven by stochastic volatility, model limitations, and regime shifts. Traditional RL models struggle in dynamic environments, often failing to adapt to sudden market disruptions, leading to suboptimal trading decisions. To address this challenge, we propose an uncertainty-aware RL framework that integrates distributional, epistemic, and aleatoric uncertainty estimations. Our approach enhances uncertainty estimation using SHAP-weighted reconstruction uncertainty, MC Dropout, and an LSTM-based technical indicator consensus mechanism. Experimental results on five major U.S. stock indices demonstrate that RL agents equipped with uncertainty estimation significantly outperform traditional models in return and risk management. This study advances uncertainty estimation in RL-based financial trading, with future research extending its application to other asset classes and alternative RL architectures for greater adaptability.

Lin Li, Li Rong Wang, Hsuan Fu +1

Nanyang Technological University 50 Nanyang Avenue, Singapore · Nanyang Technological University Centre for Frontier AI Research, A*STAR 50 Nanyang Avenue, Singapore · Laval University Quebec, Canada

Reinforcement Learning Uncertainty Quantification Aleatoric Uncertainty Algorithmic Trading Epistemic Uncertainty

Jun 21, 2026cs.CV

Interpretable Uncertainty Routing Separating Emotion Ambiguity from Distribution Shift in Facial Expression Recognition

Facial expression recognition (FER) is inherently ambiguous: human annotators frequently disagree, and models deployed in real environments face distribution shift. Crucially, these two conditions demand different downstream actions, as ambiguous in-distribution faces should be reported with their ambiguity whereas out-of-distribution inputs should be rejected. However, a single uncertainty score conflates the two. In this study, uncertainty decomposition into aleatoric and epistemic components for FER is investigated, and Uncertainty-Aware Routing (UAR), an inference-time routing mechanism that exploits the separation, is introduced. Specifically, aleatoric and epistemic uncertainties are obtained from a Deep Ensemble of fully fine-tuned DINOv2 models and are each validated against an independent external signal: aleatoric against human annotator disagreement, and epistemic against distribution shift induced by image corruptions. The proposed dual-validation protocol reveals that aleatoric recovers annotator disagreement with Spearman correlation 0.66 (95% CI: 0.64-0.68), and epistemic detects corruption-induced shifts, achieving average AUROC of 0.699 at the highest corruption severity. UAR retains approximately 1.8 times more ambiguous in-distribution faces than single-uncertainty routing at a matched out-of-distribution rejection rate. A strong label-distribution-learning baseline achieves comparable disagreement recovery but cannot separate ambiguity from shift and therefore cannot route, establishing that the value of decomposition lies in the separation enabling interpretable and differentiated action selection.

Keito Inoshita, Takato Ueno

Faculty of Business and Commerce, Kansai University, Suita, Japan · Data Science and AI Innovation Research Promotion Center, Shiga University, Hikone, Japan · Graduate School of Data Science, Shiga University, Hikone, Japan

Uncertainty Quantification Selective Prediction Facial Expression Recognition Aleatoric Uncertainty OOD Detection Uncertainty Decomposition+1

Jun 17, 2026cs.LG

On the QUEST for Uncertainty Quantification via Highest Density Regions

Uncertainty quantification (UQ) is essential for reliable decision-making in safety-critical applications in probabilistic machine learning. For regression problems, dominant scalar UQ approaches - notably, those based on proper scoring rules - measure uncertainty via pointwise predictive risk. This can lead to counterintuitive results when the target statistic is not the conditional expectation. We propose an alternative framework, in which uncertainty is characterised by the volume of the most probable subset of a distribution's support. QUEST (Quantifying Uncertainty via highest dEnSiTy regions) is a novel approach to UQ based on the concentration of Lebesgue measure at a distribution's peak(s), evaluated at one or more values of a robustness parameter

α

. We establish connections between our measures and classical statistics from information theory and economics. We show that, unlike popular alternatives based on proper scoring rules, QUEST measures of epistemic and aleatoric uncertainty satisfy a set of axioms adapted from the UQ literature, including monotonicity under distributional spread and invariance to location shifts. Selective prediction benchmarks confirm that QUEST performs favourably against standard measures such as variance and differential entropy.

Sam Goring, Tom Kuipers, Nicola Paoletti +1

King’s College London · Northeastern University London

Uncertainty Quantification Aleatoric Uncertainty Epistemic Uncertainty

Jun 12, 2026stat.ML

Hybrid Uncertainty Sensitivity Analysis Based on the HSIC for High-Dimensional Responses with Aleatory--Epistemic Separation

Quantifying the influence of hybrid aleatory and epistemic uncertainties on high-dimensional system responses remains a major challenge in global sensitivity analysis (GSA). Existing Hilbert--Schmidt Independence Criterion (HSIC)-based approaches are primarily restricted to single-output settings and lack a rigorous decomposition of heterogeneous uncertainty sources and their interactions. To address this limitation, a novel double-space tensor-product RKHS framework is proposed for sensitivity analysis under hybrid uncertainty. By constructing factorized kernels over both the latent input space and the multidimensional output space, a concurrent double Möbius inversion is derived to orthogonally decompose the global dependence measure into pure aleatory effects, pure epistemic effects, and their interaction contributions. The resulting dimension-wise sensitivity indices preserve the uncertainty attribution structure across all output dimensions. To satisfy the independence assumptions required by the decomposition, an auxiliary-variable representation based on the inverse probability integral transform is introduced, enabling the treatment of hierarchical uncertainties and Copula-induced correlations within a unified latent space. A fully vectorized single-loop implementation is further developed to avoid the computational burden of nested Monte Carlo simulation. Statistical significance and estimation uncertainty are quantified through permutation testing and Bootstrap confidence intervals. Numerical studies on a modified multi-output Ishigami function and an aerodynamic pressure-field problem demonstrate the accuracy, scalability, and practical applicability of the proposed framework.

Shijie Zhong, Jiangfeng Fu, Pengfei Wei

School of Power and Energy Northwestern Polytechnical University Xi’an, Shanxi 710129

Uncertainty Quantification Aleatoric Uncertainty Sensitivity Analysis Epistemic Uncertainty

Jun 10, 2026cs.LG

What Uncertainties Do We Need for Dynamical Systems?

The distinction between aleatoric and epistemic uncertainty has received considerable attention in machine learning research, mainly in the context of supervised learning but also in other settings such as generative modeling. In this paper, we offer a machine learning perspective on uncertainty modeling for dynamical systems, which has been studied much less so far. In particular, we ask: what uncertainties do we need for dynamical systems? We discuss sources of uncertainty, clarify their nature (aleatoric or epistemic), and consider how the objectives of representing and quantifying uncertainty vary across different tasks.

Yusuf Sale, Christopher Bülte, Felix Czaja +2

Institute of Computer Science, LMU Munich · Munich · Department of Mathematics, LMU Munich +1

Dynamical Systems Uncertainty Quantification Aleatoric Uncertainty Epistemic Uncertainty

May 25, 2026cs.LG

Neuronal Stochastic Attention Circuit (NSAC) for Probabilistic Representation Learning

Reliable uncertainty quantification in continuous-time (CT) representation learning remains nascent, particularly within CT attention literature. We introduce the Neuronal Stochastic Attention Circuit (NSAC), a novel biologically-inspired CT attention architecture that reformulates attention logit computation as the solution of an Ornstein-Uhlenbeck stochastic differential equation modulated by input-dependent, nonlinear interlinked gates derived from repurposed C. elegans Neuronal Circuit Policies (NCPs) wiring mechanism. It induces a Gaussian distribution over logits that propagates principled stochasticity through a logistic-normal distribution over attention weights to yield probabilistic output. A two-term objective function combining Gaussian negative log-likelihood with an epistemic-separation regularizer enforces higher predictive variance under distributional shifts and enables joint quantification of aleatoric and epistemic uncertainty. Theoretically, we provide: (i) state stability bounds; (ii) closed-form guarantees; and (iii) frozen-coefficient error approximation. Empirically, we implement NSAC in a diverse set of learning tasks including: (i) irregular CT function approximation; (ii) multivariate regression; (iii) long-range forecasting; (iv) Industry 4.0; and (v) lane-keeping of autonomous vehicles. We observe that NSAC remains competitive against several baselines in terms of accuracy and produces informative uncertainty estimates while being interpretable at the neuronal cell level.

Waleed Razzaq, Yun-Bo Zhao

Department of Automation, University of Science & Technology of China, Hefei, China · Institute of Artificial Intelligence, Hefei Comprehensive National Science Center

Representation Learning Uncertainty Quantification Aleatoric Uncertainty Attention Mechanisms

May 21, 2026cs.LG

A Posterior-Predictive Variance Decomposition for Epistemic and Aleatoric Uncertainty in Wind Power Forecasting

Accurate wind power forecasting requires reliable uncertainty quantification, yet most existing methods report a single predictive uncertainty that conflates epistemic and aleatoric sources. This paper applies the law of total variance to the joint setting of heteroscedastic neural network regression and Bayesian posterior approximation, deriving an explicit decomposition of total uncertainty (TU) into aleatoric (AU) and epistemic (EU) components. The resulting estimators are compatible with standard posterior-approximation methods and with

β

-NLL training to regulate the mean--variance learning trade-off. A wind power--specific evaluation framework is proposed to validate disentanglement without access to ground-truth uncertainty labels, comprising three modules: controlled synthetic experiments to verify responses to heteroscedastic noise and distribution shift; data-property--driven validation on a real-world wind turbine SCADA dataset; and dataset-size scaling experiments to examine the predicted asymptotic behavior of EU. Across synthetic and real-world experiments, the decomposed AU and EU components respond in theoretically consistent directions to noise structure, distributional shift, and training-scale variation, supporting the theoretical consistency and operational utility of the proposed decomposition and evaluation protocol.

Yinsong Chen, Samson S. Yu, Kashem M. Muttaqi

School of Engineering, Deakin University, Melbourne, VIC 3216, Australia · ARC Training Centre in Energy Technologies for Future Grids, School of Engineering, University of Wollongong, Wollongong, NSW 2522, Australia

Uncertainty Quantification Time Series Forecasting Probabilistic Forecasting Aleatoric Uncertainty Wind Power Forecasting Uncertainty Decomposition+1

May 13, 2026cs.LG

GeoFlowVLM: Geometry-Aware Joint Uncertainty for Frozen Vision-Language Embedding

Standard dual-encoder vision-language models that map images and text to deterministic points on a shared unit hypersphere through

\ell_2

normalization typically expose neither \emph{aleatoric} uncertainty (cross-modal ambiguity) nor \emph{epistemic} uncertainty (lack of training-distribution support). Existing post-hoc methods either recover at most one of the two uncertainty components, or ignore the hyperspherical geometry of these models' embeddings. We propose \textbf{GeoFlowVLM} as a post-hoc adapter that learns the joint distribution of paired

\ell_2

-normalised dual-encoder VLM embeddings on the product hypersphere

\mathbb{S}^{d-1} \times \mathbb{S}^{d-1}

via Riemannian flow matching with a single masked velocity field. A consistency result shows that, in the population limit, the trained network exposes the joint flow and both cross-modal conditional flows as valid Riemannian flow-matching velocity fields on their respective domains. We derive two quantities from this single model: a conditional retrieval entropy that quantifies aleatoric ambiguity with a decision-theoretic interpretation via a Fano-type bound, and a marginal-typicality epistemic score justified by an exact chain-rule decomposition of the joint NLL. This decomposition isolates a cross-modal pointwise-mutual-information term that is structurally discriminative rather than epistemic, and is empirically the only consistently uninformative standalone component. Empirically, the entropy tracks Recall@1 with near-ideal monotonic calibration across three retrieval benchmarks in both directions, and the marginal-typicality sum yields consistently calibrated selective accuracy across four zero-shot classification benchmarks.

Mayank Nautiyal, Li Ju, Andreas Hellander +2

Department of Information Technology, Uppsala University, Uppsala, Sweden · SciLifeLab, Uppsala University, Uppsala, Sweden

Cross-Modal Learning Flow Matching Vision-Language Models Riemannian Flow Matching Vision-Language Retrieval Aleatoric Uncertainty+2

May 8, 2026cs.AI

The Limits of AI-Driven Allocation: Optimal Screening under Aleatoric Uncertainty

The rise of machine learning has shifted targeted resource allocation in policy and humanitarian settings toward algorithmic targeting based on predicted risk scores. This approach is typically cheaper and faster than traditional screening procedures that directly observe the latent vulnerability status through physical verification. Yet, even access to the true conditional vulnerability probability cannot eliminate misallocation: aleatoric uncertainty over individual vulnerability status is irreducible, and probabilistic targeting inevitably misallocates some resources. In this work we study how screening and algorithmic targeting should be optimally combined in a two-stage allocation framework where a screening stage observes true outcomes for a subset of units before a final allocation stage assigns the resource under a fixed coverage budget. We show that the optimal strategy screens units at the margin of algorithmic allocation, while directly targeting the highest-risk units. Furthermore, we empirically characterize when screening and algorithmic targeting act as complements or substitutes: efficiency gains from screening grow as the aleatoric uncertainty in the population increases. We illustrate our framework with applications in income-based social protection programs and humanitarian demining in Colombia, where the tension between screening costs and allocation efficiency is operationally consequential.

Santiago Cortes-Gomez, Mateo Dulce Rubio, Carlos Patino +1

Machine Learning Department, Carnegie Mellon University · Center for Data Science, New York University · University of Amsterdam

Aleatoric Uncertainty Decision-Making under Uncertainty

Apr 29, 2026cs.LG

Uncertainty-Aware Reward Discounting for Mitigating Reward Hacking

Reinforcement learning from human feedback (RLHF) systems face a compounding alignment challenge: not only are learned reward models uncertain about unseen state-action pairs, but the human preference annotations they are trained on are themselves inconsistent, context-dependent, and noisy. Existing approaches address these uncertainty sources in isolation - epistemic uncertainty is used to guide exploration, while preference uncertainty is absorbed during reward model training but discarded during policy optimization. We introduce Uncertainty-Aware Reward Discounting (UARD), a principled framework that jointly models epistemic uncertainty in value estimation via ensemble disagreement and aleatoric uncertainty in human preference annotations via annotator variability, combining these signals through a confidence-adjusted Reliability Filter that adaptively modulates reward weighting during policy optimization. We prove that this dynamic discounting preserves the contraction property of the Bellman operator, guaranteeing convergence to a unique fixed point, and provide an information-theoretic justification grounded in the Information Bottleneck principle. Empirically, UARD reduces reward hacking incidents by up to 93.6% across discrete decision-making and continuous control benchmarks (MuJoCo) compared to nine baselines including DQN, Ensemble-DQN, CQL, CPO, TRPO, SAC, EDAC, SUNRISE, and PPO, while maintaining competitive task performance on well-specified rewards. Under annotation noise ranging from 10% to 30% Gaussian perturbation, UARD retains near-zero safety violations compared to baselines' near-linear degradation. These results demonstrate that treating uncertainty as an active component of the optimization objective - rather than a passive diagnostic signal - provides a principled pathway toward more reliable and aligned RL systems.

Disha Singha

Reward Hacking Reinforcement Learning Aleatoric Uncertainty Safe RL RL from Human Feedback

Apr 28, 2026cs.CL

Quantifying Aleatoric Uncertainty of In-Context Learning for Robust Measure of LLM Prediction Confidence

In-Context Learning (ICL) allows LLMs to adapt to new tasks from a few demonstrations, but its reliability remains a concern: predictions are highly sensitive to both prompt design and the model's ability to understand the context, obscuring whether failures arise from data properties or model limitations. Uncertainty decomposition-separating aleatoric from epistemic sources-is particularly crucial in this setting, yet existing methods, designed for standard generation tasks, fail to capture the unique dynamics of ICL. To address this, we introduce a concept of self-function vectors, built upon Bayesian views and the mechanistic interpretability of ICL. These vectors leverage internal model representations to model the latent concept learned during in-context prompting, thereby enabling a direct estimation of aleatoric uncertainty within a Bayesian framework and circumventing the reliance on brittle input or decoding manipulations. Given the lack of established benchmarks and suitable evaluation protocols, we also propose the first and rigorous evaluation protocol, in which data is manipulated in controlled ways so as to quantify aleatoric uncertainty precisely and separately from epistemic uncertainty. With this new evaluation framework, initially grounded in synthetic tasks for conceptual development and subsequently extended to real-world datasets, we show that our proposed methodology can measure uncertainty of LLM predictions made under ICL more reliably than existing alternative methods. Moreover, we show it can be used as a practical tool for trustworthy-related applications, such as hallucination detection. Our findings pave a new direction for connecting the quantitative view of uncertainty with the mechanistic understanding of model behavior.

Jinseok Chung, Minkyoung Song, Hyunji Jung +1

POSTECH

Uncertainty Quantification In-Context Learning LLM Uncertainty Estimation Aleatoric Uncertainty Uncertainty Decomposition

Apr 27, 2026cs.AI

Credal Concept Bottleneck Models for Epistemic-Aleatoric Uncertainty Decomposition

Concept Bottleneck Models (CBMs) predict through human-interpretable concepts, but they typically output point concept probabilities that conflate epistemic uncertainty (reducible model underspecification) with aleatoric uncertainty (irreducible input ambiguity). This makes concept-level uncertainty hard to interpret and, more importantly, hard to act upon. We introduce CREDENCE (Credal Ensemble Concept Estimation), a CBM framework that decomposes concept uncertainty by construction. CREDENCE represents each concept as a credal prediction (a probability interval), derives epistemic uncertainty from disagreement across diverse concept heads, and estimates aleatoric uncertainty via a dedicated ambiguity output trained to match annotator disagreement when available. The resulting signals support prescriptive decisions: automate low-uncertainty cases, prioritize data collection for high-epistemic cases, route high-aleatoric cases to human review, and abstain when both are high. Across several tasks, we show that epistemic uncertainty is positively associated with prediction errors, whereas aleatoric uncertainty closely tracks annotator disagreement, providing guidance beyond error correlation. Our implementation is available at the following link: https://github.com/Tankiit/Credal_Sets/tree/ensemble-credal-cbm

Tanmoy Mukherjee, Thomas Bailleux, Pierre Marquis +1

Univ. Artois, CNRS, CRIL, France · Institut Universitaire de France

Uncertainty Quantification Concept Bottleneck Models Aleatoric Uncertainty Decision-Making under Uncertainty Uncertainty Decomposition Epistemic Uncertainty

Apr 18, 2026cs.AI

Complementing Self-Consistency with Cross-Model Disagreement for Uncertainty Quantification

Large language models (LLMs) often produce confident yet incorrect responses, and uncertainty quantification is one potential solution to more robust usage. Recent works routinely rely on self-consistency to estimate aleatoric uncertainty (AU), yet this proxy collapses when models are overconfident and produce the same incorrect answer across samples. We analyze this regime and show that cross-model semantic disagreement is higher on incorrect answers precisely when AU is low. Motivated by this, we introduce an epistemic uncertainty (EU) term that operates in the black-box access setting: EU uses only generated text from a small, scale-matched ensemble and is computed as the gap between inter-model and intra-model sequence-semantic similarity. We then define total uncertainty (TU) as the sum of AU and EU. In a comprehensive study across five 7-9B instruction-tuned models and ten long-form tasks, TU improves ranking calibration and selective abstention relative to AU, and EU reliably flags confident failures where AU is low. We further characterize when EU is most useful via agreement and complementarity diagnostics.

Kimia Hamidieh, Veronika Thost, Walter Gerych +2

1MIT · MIT-IBM Watson AI Lab · 3WPI +1

Selective Prediction Confidence Estimation in Language Models LLM Reliability LLM Uncertainty Estimation Aleatoric Uncertainty Uncertainty Calibration+2

Apr 13, 2026cs.AI

Retrieval-Augmented Generation Must Move Beyond Factual Grounding to Represent Diverse Opinions

Retrieval-Augmented Generation (RAG) systems are built on an unexamined assumption - that queries have correct answers and retrieval should converge toward them. This position paper argues that this creates a factual bias where RAG systems optimize for reducing epistemic uncertainty while ignoring the aleatoric uncertainty, inherent in opinion-rich content. The consequences go beyond technical limitations- due to risk of minority voice erasure and risk of opinion manipulation. To address this, we formalize opinion-aware retrieval through uncertainty quantification and derive a unified objective using the Wasserstein distance. As an existence proof, we present Opinion-Aware RAG (O-RAG), which enriches documents with LLM-extracted, entity-linked opinion metadata before indexing. Across e-commerce seller forums and public hotel reviews, O-RAG reduces Wasserstein distance to corpus-level sentiment distributions by 18-48%, and human evaluators preferred its responses 79.2% of the time. We close with a research agenda for opinion-aware RAG.

Aditya Agrawal, Alwarappan Nakkiran, Aman Singh Thakur +2

Retrieval-Augmented Generation Aleatoric Uncertainty

Mar 28, 2026cs.AI

Quantification of Credal Uncertainty: A Distance-Based Approach

Credal sets, i.e., closed convex sets of probability measures, provide a natural framework to represent aleatoric and epistemic uncertainty in machine learning. Yet how to quantify these two types of uncertainty for a given credal set, particularly in multiclass classification, remains underexplored. In this paper, we propose a distance-based approach to quantify total, aleatoric, and epistemic uncertainty for credal sets. Concretely, we introduce a family of such measures within the framework of Integral Probability Metrics (IPMs). The resulting quantities admit clear semantic interpretations, satisfy natural theoretical desiderata, and remain computationally tractable for common choices of IPMs. We instantiate the framework with the total variation distance and obtain simple, efficient uncertainty measures for multiclass classification. In the binary case, this choice recovers established uncertainty measures, for which a principled multiclass generalization has so far been missing. Empirical results confirm practical usefulness, with favorable performance at low computational cost.

Xabier Gonzalez-Garcia, Siu Lun Chau, Julian Rodemann +6

Dept. of Statistics, CS & Mathematics, Public Univ. of Navarre · EPIC Lab, Nanyang Technological Univ. Singapore · Rational Intelligence Lab, CISPA Helmholtz Center for Information Security +7

Uncertainty Quantification Aleatoric Uncertainty Integral Probability Metrics Uncertainty Decomposition Epistemic Uncertainty

Mar 19, 2026cs.CV

Rethinking Uncertainty Quantification and Entanglement in Image Segmentation

Uncertainty quantification (UQ) is crucial in safety-critical applications such as medical image segmentation. Total uncertainty is typically decomposed into data-related aleatoric uncertainty (AU) and model-related epistemic uncertainty (EU). Many methods exist for modeling AU (such as Probabilistic UNet, Diffusion) and EU (such as ensembles, MC Dropout), but it is unclear how they interact when combined. Additionally, recent work has revealed substantial entanglement between AU and EU, undermining the interpretability and practical usefulness of the decomposition. We present a comprehensive empirical study covering a broad range of AU-EU model combinations, propose an entanglement proxy based on the relative performance of uncertainty measures, and evaluate model combinations across downstream uncertainty quantification tasks. Ensembles consistently show more favorable proxy values and superior performance. Softmax models usually beat other AU methods, except in calibration where the results are dataset-dependent. A softmax ensemble performs remarkably well on all tasks. Finally, we analyze potential sources of uncertainty entanglement and outline directions for mitigating this effect.

Jakob Lønborg Christensen, Vedrana Andersen Dahl, Morten Rieger Hannemose +2

Technical University of Denmark, DTU Compute · University of Lucerne, Faculty of Health Sciences and Medicine, Switzerland

Uncertainty Quantification Deep Ensembles Aleatoric Uncertainty Segmentation Uncertainty Estimation Uncertainty Calibration Uncertainty Decomposition+1

Jan 7, 2026cs.LG

Disentangling Aleatoric and Epistemic Uncertainty in Physics-Informed Neural Networks. Application to Insulation Material Degradation Prognostics

Physics-Informed Neural Networks (PINNs) provide a framework for integrating physical laws with data. However, their application to Prognostics and Health Management (PHM) remains constrained by the limited uncertainty quantification (UQ) capabilities. Most existing PINN-based prognostics approaches are deterministic or account only for epistemic uncertainty, limiting their suitability for risk-aware decision-making. This work introduces a heteroscedastic Bayesian Physics-Informed Neural Network (B-PINN) framework that jointly models epistemic and aleatoric uncertainty, yielding full predictive posteriors for spatiotemporal insulation material ageing estimation. The approach integrates Bayesian Neural Networks (BNNs) with physics-based residual enforcement and prior distributions, enabling probabilistic inference within a physics-informed learning architecture. The framework is evaluated on transformer insulation ageing application, validated with a finite-element thermal model and field measurements from a solar power plant, and benchmarked against deterministic PINNs, dropout-based PINNs (d-PINNs), and alternative B-PINN variants. Results show that the proposed B-PINN provides improved predictive accuracy and better-calibrated uncertainty estimates than competing approaches. A systematic sensitivity study further analyzes the impact of boundary-condition, initial-condition, and residual sampling strategies on accuracy, calibration, and generalization, and the influence of measurement noise on aleatoric uncertainty. Overall, the findings highlight the capability of Bayesian physics-informed learning to support uncertainty-aware prognostics and informed decision-making in transformer asset management by tracking aleatoric and epistemic sources of uncertainty.

Ibai Ramirez, Jokin Alcibar, Joel Pino +2

Department of Electronics and Computer Science Mondragon University Arrasate-Mondragon, 20500, Spain · CARTIF Technology Centre Valladolid, 47151, Spain · Department of Physical Chemistry University of the Basque Country (EHU)[cs.LG] Basque Centre for Applied Mathematics (BCAM) Leioa, 20018, Spain +1

Bayesian Neural Networks Uncertainty Quantification Predictive Maintenance Physics-Informed ML Aleatoric Uncertainty Uncertainty Calibration

Oct 29, 2025cs.LG

Uncertainty Quantification for Regression: A Unified Framework based on kernel scores

Regression tasks, notably in safety-critical domains, require reliable uncertainty quantification, yet the literature remains largely classification-focused. To address this, we introduce a family of measures for total, aleatoric, and epistemic uncertainty in multivariate regression based on strictly proper kernel scores. The framework provides a principled recipe for designing new uncertainty measures whose behavior, such as tail sensitivity or out-of-distribution responsiveness, is governed by the choice of the underlying kernel, while also encompassing existing measures under a joint analysis. We prove explicit correspondences between properties of the kernel and behavior of resulting uncertainty measures, yielding concrete design guidelines for practitioners. Extensive experiments across structured regression tasks, including spatial and functional domains, demonstrate effectiveness on downstream tasks such as out-of-distribution detection and active learning, and reveal that different kernel choices lead to distinct trade-offs, offering practitioners guidance for task-specific selection.

Christopher Bülte, Yusuf Sale, Gitta Kutyniok +1

LMU Munich · 2Munich Center for Machine Learning (MCML) · 3DLR-German Aerospace Center +2

Uncertainty Quantification Aleatoric Uncertainty Uncertainty Decomposition Epistemic Uncertainty

Jul 10, 2025stat.ML

CLEAR: Calibrated Learning for Epistemic and Aleatoric Risk

Accurate uncertainty quantification is critical for reliable predictive modeling. Existing methods typically address either aleatoric uncertainty due to measurement noise or epistemic uncertainty resulting from limited data, but not both in a balanced manner. We propose CLEAR, a calibration method with two distinct parameters,

γ_1

and

γ_2

, to combine the two uncertainty components and improve the conditional coverage of predictive intervals for regression tasks. CLEAR is compatible with any pair of aleatoric and epistemic estimators; we show how it can be used with (i) quantile regression for aleatoric uncertainty and (ii) ensembles drawn from the Predictability-Computability-Stability (PCS) framework for epistemic uncertainty. Across 17 diverse real-world datasets, CLEAR achieves an average improvement of 28.3% and 17.5% in the interval width compared to the two individually calibrated baselines while maintaining nominal coverage. Similar improvements are observed when applying CLEAR to Deep Ensembles (epistemic) and Simultaneous Quantile Regression (aleatoric). The benefits are especially evident in scenarios dominated by high aleatoric or epistemic uncertainty. Project page: https://unco3892.github.io/clear/

Ilia Azizi, Juraj Bodik, Jakob Heiss +1

Department of Operations, HEC, University of Lausanne, Switzerland · BegooAI, Switzerland · Department of Statistics, University of California, Berkeley, USA +1

Uncertainty Quantification Aleatoric Uncertainty Uncertainty Calibration Regression Prediction Intervals Epistemic Uncertainty

Apr 25, 2025cs.LG

An Axiomatic Assessment of Entropy- and Variance-based Uncertainty Quantification in Regression

Uncertainty quantification is crucial in machine learning, yet most (axiomatic) studies of uncertainty measures focus on classification, leaving a gap in regression settings with limited formal justification and evaluations. In this work, we provide a formal way of representing uncertainty in continuous space, using a general parametric formulation, allowing for tractable analysis and evaluation of uncertainty measures. Within this framework, we propose a set of axioms that enable rigorous assessment of total, aleatoric, and epistemic uncertainty measures. Together, this allows for a theoretical examination of uncertainty measures and their corresponding properties. As a specific example, we compare the widely used entropy- and variance-based measures with respect to established predictive models and analyze their limitations and challenges in uncertainty quantification. Our work provides a principled way to understand and develop uncertainty measures in supervised regression, offering theoretical insights and practical guidelines for reliable uncertainty assessment.

Christopher Bülte, Yusuf Sale, Timo Löhr +3

1LMU Munich · 2Munich Center for Machine Learning (MCML) · 3DLR-German Aerospace Center +2

Uncertainty Quantification Aleatoric Uncertainty Epistemic Uncertainty

Date pendingcs.CL

Decomposing LLM-Judge Uncertainty to Target Expert Labels

An LLM judge evaluates outputs at scale. Experts should label only where it is least sure. Its natural escalation signal conflates two uncertainties: aleatoric, real disagreement in the expert pool, which labels cannot reduce, and epistemic, the judge's ignorance, which labels do reduce. A small Bayesian model separates them: a regression on labels already collected learns how far to trust a black-box judge's prediction. Both components follow as simple formulas, with no sampling or further judge calls. The components isolate on a real LLM judge against exactly known truth, and stated confidence is no guide to its actual error. On real human disagreement (ChaosNLI) the epistemic ranking removes 83% more error than total uncertainty for the same expert labels, though simply escalating the least-labelled items does as well there. We demonstrate we can estimate where a judge is ignorant rather than where experts genuinely disagree, and propose using this to direct expert labelling. Code and data are available at https://github.com/composo-ai/judge-uncertainty-decomposition.

Ryan Lail

Composo AI

Annotator Disagreement LLM-as-a-Judge Uncertainty Quantification Human-in-the-Loop Annotation LLM Uncertainty Estimation Aleatoric Uncertainty+2