Uncertainty Quantification

Also known as UQ

Uncertainty quantification studies what is unknown in a prediction or inference, where that uncertainty comes from, and how it should affect decisions. Common approaches include Bayesian methods, ensembles and Gaussian processes, which model predictive distributions; conformal prediction, which provides distribution-free coverage guarantees; and calibration methods that adjust confidence or intervals against observed errors. Researchers apply these tools to settings from medical imaging and physical simulations to language models, evaluating calibration, coverage, error detection and performance under distribution shift. A central challenge is that well-calibrated or narrow estimates do not always identify actual failures, so methods must be tested in the conditions and for the decisions where they will be used, while balancing reliability against computational cost.

Latest papers

663 papers58 papers in the last four weeks, level with the four weeks before. 0.3% of all new papers.

The Illusion of AI Expertise Under Uncertainty: A Probabilistic Paradigm to Interpret Results Under Ground Truth Variation
Jan 9, 2026Aparna Elangovan, Lei Xu, Mahsa Elyasi +8LLM Evaluation Generative AI Evaluation
Independent Researcher · Amazon · Gazi University +6
Disentangling Aleatoric and Epistemic Uncertainty in Physics-Informed Neural Networks. Application to Insulation Material Degradation Prognostics
Jan 7, 2026Ibai Ramirez, Jokin Alcibar, Joel Pino +2Bayesian Neural Networks Uncertainty Quantification
Department of Electronics and Computer Science Mondragon University Arrasate-Mondragon, 20500, Spain · CARTIF Technology Centre Valladolid, 47151, Spain · Department of Physical Chemistry University of the Basque Country (EHU)[cs.LG] Basque Centre for Applied Mathematics (BCAM) Leioa, 20018, Spain +1
Quantitative Understanding of PDF Fits and their Uncertainties
Dec 30, 2025Amedeo Chiefa, Luigi Del Debbio, Richard KenwayUncertainty Quantification High-Energy Physics
The Higgs Centre for Theoretical Physics, School of Physics and Astronomy, The University of Edinburgh, Peter Guthrie Tait Road, Edinburgh EH9 3FD, United Kingdom
Improving Semantic Uncertainty Quantification in LVLMs with Semantic Gaussian Processes
Dec 16, 2025Joseph Hoche, Andrei Bursuc, David Brellmann +4Vision-Language Models Uncertainty Quantification
AMIAD, Pôle Recherche, Palaiseau · Safran Tech · University of Liège +3
CLAPS: Aleatoric-Epistemic Scaling via Last-Layer Laplace for Conformal Regression
Dec 1, 2025Dongseok Kim, Hyoungsun Choi, Mohamed Jismy Aashik Rasool +1Uncertainty Quantification Conformal Prediction
Department of Computer Engineering, Gachon University
Beyond Uncertainty Sets: Leveraging Optimal Transport to Extend Conformal Predictive Distributions to Multivariate Settings
Nov 19, 2025Eugene NdiayeUncertainty Quantification Conformal Prediction
Apple
QiVC-Net: Quantum-Inspired Variational Convolutional Network, with Application to Biosignal Classification
Nov 7, 2025Amin Golnari, Jamileh Yousefi, Reza Moheimani +1Uncertainty Quantification Quantum-Inspired ML
College of Engineering and Computer Science VinUniversity Hanoi, Vietnam · Shannon School of Business Cape Breton University Sydney, NS, Canada · Faculty of Computer Science Chemnitz University of Technology Chemnitz, Germany +2
Uncertainty Quantification for Regression: A Unified Framework based on kernel scores
Oct 29, 2025Christopher Bülte, Yusuf Sale, Gitta Kutyniok +1Uncertainty Quantification Aleatoric Uncertainty
LMU Munich · 2Munich Center for Machine Learning (MCML) · 3DLR-German Aerospace Center +2
Deep Jump Gaussian Processes for Surrogate Modeling of High-Dimensional Piecewise Continuous Functions
Oct 24, 2025Yang Xu, Chiwoo ParkDeep Gaussian Processes Surrogate Modeling
Industrial & Systems Engineering, University of Washington, Seattle, WA, USA
Toward Uncertainty-Aware and Generalizable Neural Decoding for Quantum LDPC Codes
Oct 5, 2025Xiangjun Mi, Frank MuellerQuantum LDPC Codes Quantum Error Correction Decoding
North Carolina State University, Raleigh, NC 27695, USA
Geometric Uncertainty for Detecting and Correcting Hallucinations in LLMs
Sep 17, 2025Edward Phillips, Sean Wu, Soheila Molaei +3Uncertainty Quantification LLM Hallucination Mitigation
Department of Engineering Science, University of Oxford · GlaxoSmithKline · Oxford Suzhou Centre for Advanced Research
Uncertainty-Aware Ankle Exoskeleton Control
Aug 28, 2025Fatima Mumtaza Tourk, Bishoy Galoaa, Sanat Shajan +3Uncertainty Quantification Ankle Exoskeleton Control
Department of Mechanical Engineering, Northeastern University, Boston, MA 02115 USA
DE-VAE: Revealing Uncertainty in Parametric and Inverse Projections with Variational Autoencoders using Differential Entropy
Aug 16, 2025Frederik L. Dennig, Daniel A. KeimVariational Autoencoders Uncertainty Quantification
University of Konstanz
A General Approach to Visualizing Uncertainty in Statistical Graphics
Jul 31, 2025Bernarda Petek, David Nabergoj, Erik ŠtrumbeljScientific Visualization Data Visualization
University of Ljubljana Faculty of Computer and Information Science Veˇcna pot 113, 1000 Ljubljana, Slovenia
Last Layer Hamiltonian Monte Carlo
Jul 11, 2025Koen Vellenga, H. Joe Steinhauer, Göran Falkman +2Bayesian Neural Networks Markov Chain Monte Carlo
University of Skövde & Volvo Car Corporation, Sweden · University of Skövde, Sweden · Volvo Car Corporation, Sweden
CLEAR: Calibrated Learning for Epistemic and Aleatoric Risk
Jul 10, 2025Ilia Azizi, Juraj Bodik, Jakob Heiss +1Uncertainty Quantification Aleatoric Uncertainty
Department of Operations, HEC, University of Lausanne, Switzerland · BegooAI, Switzerland · Department of Statistics, University of California, Berkeley, USA +1
Latent Space Analysis for Interpretable Uncertainty in Melanoma Classification
Jun 23, 2025Ciro Listone, Aniello MuranoVariational Autoencoders Skin Lesion Classification
University of Naples Federico II, Naples, Italy.
Representation Learning for Equivariant Inference with Guarantees
May 26, 2025Daniel Ordoñez-Apraez, Vladimir Kostić, Alek Fröhlich +3Disentangled Representation Learning Representation Learning
CSML, Italian Institute of Technology, Genova, Italy · Universit`a di Genova, Genova, Italy · University of Novi Sad, Novi Sad, Serbia +2
Repulsive Ensembles for Bayesian Inference in Physics-informed Neural Networks
May 22, 2025Philipp Pilar, Markus Heinonen, Niklas WahlströmUncertainty Quantification Physics-Informed ML
Department of Information Technology Uppsala University, Uppsala, Sweden · Department of Computer Science Aalto University, Espoo, Finland
An Axiomatic Assessment of Entropy- and Variance-based Uncertainty Quantification in Regression
Apr 25, 2025Christopher Bülte, Yusuf Sale, Timo Löhr +3Uncertainty Quantification Aleatoric Uncertainty
1LMU Munich · 2Munich Center for Machine Learning (MCML) · 3DLR-German Aerospace Center +2
Bifidelity Parameter Estimation Using Conditional Diffusion Models
Apr 2, 2025Caroline Tatsuoka, Minglei Yang, Dongbin Xiu +1Simulation-Based Inference Surrogate Modeling
Department of Mathematics, The Ohio State University, Columbus, OH 43210, USA · Fusion Energy Science Division, Oak Ridge National Laboratory, Oak Ridge, TN, 37831, USA · Computer Science and Mathematics Division, Oak Ridge National Laboratory, Oak Ridge, TN, 37831, USA
Explainable Bayesian deep learning through input-skip Latent Binary Bayesian Neural Networks
Mar 13, 2025Eirik Høyheim, Lars Skaaret-Lund, Solve Sæbø +1Bayesian Neural Networks Uncertainty Quantification
Bioinformatics and applied statistics (BIAS), Norwegian University of Life Sciences, Ås, Norway
Uncertainty Propagation for Echocardiography Clinical Metric Estimation via Contour Sampling
Feb 18, 2025Thierry Judge, Olivier Bernard, Nicolas Duchateau +5Uncertainty Quantification Uncertainty Propagation
Department of Computer Science, University of Sherbrooke, Sherbrooke, QC, Canada · INSA, Universite Claude Bernard Lyon 1, CNRS UMR 5220, Inserm U1206, CREATIS, Villeurbanne, France · Ultromics Ltd., Oxford, OX4 2SU, UK +1
Active Learning with Bayesian Multi-Fidelity Laplace Neural Operators for Oscillatory Parametric PDEs
Feb 1, 2025Bongseok Kim, Haoyang Zheng, Michael Penwarden +1Dynamical Systems Surrogate Modeling
School of Mechanical Engineering, Purdue University, 585 Purdue Mall, West Lafayette, 47907, IN, USA. · Department of Mathematics, Purdue University, 150 North University Street, West Lafayette, 47907, IN, USA.
Statistical Uncertainty Quantification for Aggregate Performance Metrics in Machine Learning Benchmarks
Jan 8, 2025Rachel Longjohn, Giri Gopalan, Emily CasletonUncertainty Quantification
University of California Irvine · Los Alamos National Laboratory
Label-Confidence-Aware Uncertainty Estimation in Natural Language Generation
Dec 10, 2024Qinhong Lin, Yinglun Feng, Yuhao Zhang +2Uncertainty Quantification LLM Uncertainty Estimation
School of Cyberspace Security, Beijing University of Posts and Telecommunications, Beijing, China · QuanCheng Laboratory, Jinan, China
Uncertainty measurement for complex event prediction in safety-critical systems
Nov 2, 2024Maria J. P. Peixoto, Akramul AzimUncertainty Quantification Sensitivity Analysis
Department of Electrical, Computer and Software Engineering, Ontario Tech University, Oshawa, Ontario, Canada.
Statistical Taylor Expansion: A New and Path-Independent Method for Uncertainty Analysis
Oct 2, 2024Chengpu WangUncertainty Quantification Uncertainty Propagation
Grossman Street, Melville, NY 11747, USA
Uncertainty Quantification in Machine Learning for Biosignal Applications -- A Review
Nov 15, 2023Ivo Pascal de Jong, Andreea Ioana Sburlea, Matias Valdenegro-ToroUncertainty Quantification
Department of Artificial Intelligence, Bernoulli Institute, University of Groningen, Groningen, The Netherlands
Training-Free Uncertainty Estimation for Embedding Models
May 31, 2023Young-Jin Park, Addison Kristanto Julistiono, Hao Wang +2Representation Learning Uncertainty Quantification
Massachusetts Institute of Technology · MIT-IBM Watson AI Lab

Jan 9, 2026J/K move · Enter open · S save

The Illusion of AI Expertise Under Uncertainty: A Probabilistic Paradigm to Interpret Results Under Ground Truth Variation

Aparna Elangovan, Lei Xu, Mahsa Elyasi, Ismail Akdulum+7

Independent Researcher · Amazon · Gazi University+6

LLM Evaluation Generative AI Evaluation Uncertainty Quantification

Open paper AskPDF arXiv

Benchmarking the capabilities of AI systems, including Large Language Models (LLMs) and Vision Models, typically ignores the impact of uncertainty in the underlying ground truth answers from experts. This ambiguity is not just limited to human preferences, but is also consequential even in safety critical domains such as medicine where uncertainty is pervasive. In this paper, we introduce a probabilistic paradigm to theoretically explain how high certainty in ground truth answers is almost always necessary for even an expert to achieve high scores. In contrast, in datasets with high variation in ground truth answers, the expert performance drops substantially resulting in almost no difference between a random labeller and an expert. This characteristic also manifests when comparing models, where uncertainty obfuscates differences between poor and high performing models. Therefore, ignoring uncertainty in ground truth evaluation data can result in the misleading conclusion that a non-expert has similar performance to that of an expert. Given ground truth answer variability and commonly used F1 and accuracy metrics, we bring forth a simple and explainable probabilistic coin toss model to estimate expected accuracyand expected F1 and compare the estimated scores with those that experts (human or model) achieve across multiple domains -- including radiology and models such as GPT-5 and Gemini-3. The results lead to the recommendation that stratification by the probability of the ground truth answer becomes critical when expert performance is relatively low. Under stratified evaluation, performance comparison becomes more reliable in high certainty bins, mitigating the effect of the key confounding factor --- uncertainty.