Uncertainty Quantification Method

Uncertainty quantification (UQ) methods aim to estimate the reliability of predictions made by machine learning models, particularly deep neural networks and large language models, by providing a measure of confidence or uncertainty associated with each prediction. Current research focuses on developing and benchmarking UQ techniques across diverse applications, including image analysis, natural language processing, and scientific modeling, employing methods such as Bayesian neural networks, Monte Carlo dropout, and conformal prediction. The ability to quantify uncertainty is crucial for building trustworthy and reliable AI systems, enabling informed decision-making in high-stakes applications and improving the interpretability and robustness of machine learning models in various scientific fields.

Papers

October 4, 2024

Lightning UQ Box: A Comprehensive Framework for Uncertainty Quantification in Deep Learning
Nils Lehmann, Jakob Gawlikowski, Adam J. Stewart, Vytautas Jancauskas, Stefan Depeweg, Eric Nalisnick, Nina Maria Gottschling
Deep Learning Deep Neural Network Uncertainty Quantification Neural Network Output Uncertainty Quantification Method

June 21, 2024

Benchmarking Uncertainty Quantification Methods for Large Language Models with LM-Polygraph
Roman Vashurin, Ekaterina Fadeeva, Artem Vazhentsev, Lyudmila Rvanova, Akim Tsvigun, Daniil Vasilev, Rui Xing, Abdelrahman Boda Sadallah, Kirill Grishchenkov, Sergey Petrakov, Alexander Panchenko, Timothy Baldwin, Preslav Nakov, Maxim Panov, Artem Shelmanov
Large Language Model Uncertainty Quantification Text Generation Task Lie Detection Uncertainty Quantification Method Confidence Elicitation

March 2, 2024

Extracting Usable Predictions from Quantized Networks through Uncertainty Quantification for OOD Detection
Rishi Singhal, Srinath Srinivasan
Human Prediction Pre Trained Uncertainty Quantification Confidence Estimation Low Confidence OOD Detection Uncertainty Quantification Method

September 12, 2023

Calibration in Machine Learning Uncertainty Quantification: beyond consistency to target adaptivity
Pascal Pernot
Adaptive Importance Strong Consistency Calibration Performance Validation Approach Quantitative Validation Uncertainty Quantification Method

August 18, 2023

Robust Uncertainty Quantification Using Conformalised Monte Carlo Prediction
Daniel Bethell, Simos Gerasimou, Radu Calinescu
Uncertainty Quantification Conformal Prediction Monte Carlo Robust Prediction Uncertainty Quantification Method

February 7, 2023

IB-UQ: Information bottleneck based uncertainty quantification for neural function regression and neural operator learning
Ling Guo, Hao Wu, Wenwen Zhou, Yan Wang, Tao Zhou
Uncertainty Quantification Neural Operator Information Bottleneck Deep Regression Confidence Map Poor Extrapolation Performance Uncertainty Quantification Method

September 17, 2022

Sample-based Uncertainty Quantification with a Single Deterministic Neural Network
Takuya Kanazawa, Chetan Gupta
Uncertainty Quantification Single Neural Network Predictive Distribution Deterministic Neural Network Uncertainty Sampling Point Forecasting Uncertainty Quantification Method

November 8, 2021

Uncertainty Quantification in Neural Differential Equations
Olga Graf, Pablo Flores, Pavlos Protopapas, Karim Pichara
Deep Learning Uncertainty Quantification Differential Equation Predictive Uncertainty Uncertainty Quantification Method

Uncertainty Quantification Method

Papers

Lightning UQ Box: A Comprehensive Framework for Uncertainty Quantification in Deep Learning

Benchmarking Uncertainty Quantification Methods for Large Language Models with LM-Polygraph

Extracting Usable Predictions from Quantized Networks through Uncertainty Quantification for OOD Detection

Calibration in Machine Learning Uncertainty Quantification: beyond consistency to target adaptivity

Robust Uncertainty Quantification Using Conformalised Monte Carlo Prediction

IB-UQ: Information bottleneck based uncertainty quantification for neural function regression and neural operator learning

Sample-based Uncertainty Quantification with a Single Deterministic Neural Network

Uncertainty Quantification in Neural Differential Equations