Calibrated Uncertainty

Calibrated uncertainty in machine learning focuses on generating reliable estimates of prediction uncertainty, ensuring that the model's confidence accurately reflects its prediction accuracy. Current research emphasizes developing methods to achieve well-calibrated uncertainty across diverse model architectures, including Bayesian neural networks, conformal prediction, and ensemble methods, and applying these techniques to various tasks like regression, classification, and dense prediction in domains such as robotics, medical imaging, and renewable energy forecasting. This work is crucial for building trustworthy and robust AI systems, particularly in high-stakes applications where understanding and managing uncertainty is paramount for safe and reliable decision-making.

Papers

October 30, 2024

A Monte Carlo Framework for Calibrated Uncertainty Estimation in Sequence Prediction
Qidong Yang, Weicheng Zhu, Joseph Keslin, Laure Zanna, Tim G. J. Rudner, Carlos Fernandez-Granda
Monte Carlo Sequence Prediction Calibrated Uncertainty

October 2, 2024

FairlyUncertain: A Comprehensive Benchmark of Uncertainty in Algorithmic Fairness
Lucas Rosenblatt, R. Teal Witter
High Uncertainty Anticipation Algorithmic Fairness Predictive Uncertainty Comprehensive Benchmark Fair Predictor Fairness Intervention Calibrated Uncertainty

September 30, 2024

End-to-End Conformal Calibration for Optimization Under Uncertainty
Christopher Yeh, Nicolas Christianson, Alan Wu, Adam Wierman, Yisong Yue
Optimization Purpose High Uncertainty Anticipation Robustness Guarantee Conformal Regression Calibrated Uncertainty Input Uncertainty Input Convex Neural Network

June 30, 2024

DADEE: Well-calibrated uncertainty quantification in neural networks for barriers-based robot safety
Masoud Ataei, Vikas Dhiman
Neural Network Uncertainty Quantification Uncertainty Estimation Control Barrier Function Robot Safety Calibrated Uncertainty

May 28, 2024

Evaluating Bayesian deep learning for radio galaxy classification
Devina Mohan, Anna M. M. Scaife
Bayesian Neural Network Bayesian Perspective Uncertainty Calibration Calibrated Uncertainty Radio Astronomy Radio Galaxy Classification

February 28, 2024

Efficiently Computable Safety Bounds for Gaussian Processes in Active Learning
Jörn Tebbe, Christoph Zimmer, Ansgar Steland, Markus Lange-Hegermann, Fabian Mies
Active Learning Gaussian Process Safety Constraint Safety Guarantee Safety Assurance Calibrated Uncertainty

February 2, 2024

Calibrated Uncertainty Quantification for Operator Learning via Conformal Prediction
Ziqi Ma, Kamyar Azizzadenesheli, Anima Anandkumar
Uncertainty Quantification Conformal Prediction Operator Learning Quantile Regression Calibrated Uncertainty Pressure Map

October 2, 2023

Unified Uncertainty Calibration
Kamalika Chaudhuri, David Lopez-Paz
Uncertainty Estimation Epistemic Uncertainty Uncertainty Calibration Calibrated Uncertainty

August 19, 2023

Calibrating Uncertainty for Semi-Supervised Crowd Counting
Chen Li, Xiaoling Hu, Shahira Abousamra, Chao Chen
Pseudo Label Crowd Counting Calibrated Uncertainty Semi Supervised Crowd Counting

July 19, 2023

Uncertainty-Driven Multi-Scale Feature Fusion Network for Real-time Image Deraining
Ming Tong, Xuefeng Yan, Yongzhen Wang
Multi Scale Image Deraining Rain Removal Calibrated Uncertainty Feature Fusion Block

February 21, 2023

Likelihood Annealing: Fast Calibrated Uncertainty for Regression
Uddeshya Upadhyay, Jae Myung Kim, Cordelia Schmidt, Bernhard Schölkopf, Zeynep Akata
High Dimensional Uncertainty Estimation Novel Regression Deep Regression Uncertainty Estimation Method Calibrated Uncertainty

December 14, 2022

Modeling Multimodal Aleatoric Uncertainty in Segmentation with Mixture of Stochastic Experts
Zhitong Gao, Yucong Chen, Chuyu Zhang, Xuming He
Segmentation Based Approach Mixture Component Aleatoric Uncertainty Segmentation Quality Calibrated Uncertainty Uncertainty Representation Multimodal Uncertainty

November 8, 2022

Calibrated Perception Uncertainty Across Objects and Regions in Bird's-Eye-View
Markus Kängsepp, Meelis Kull
Autonomous Vehicle Arbitrary Object Region Specific Eye View Calibrated Uncertainty Perceptual Uncertainty

November 5, 2022

Accurate and Reliable Methods for 5G UAV Jamming Identification With Calibrated Uncertainty
Hamed Farkhari, Joseanne Viana, Pedro Sebastiao, Luis Miguel Campos, Luis Bernardo, Rui Dinis, Sarang Kahvazadeh
Binary Classification Softmax Function Uncertainty Calibration Neural Network Output UAV Dataset Calibrated Uncertainty Reliable Online Method

September 29, 2022

Uncertainty Estimation for 3D Dense Prediction via Cross-Point Embeddings
Kaiwen Cai, Chris Xiaoxuan Lu, Xiaowei Huang
Uncertainty Estimation 3D Point Cloud Calibrated Uncertainty Point Embeddings

July 14, 2022

BayesCap: Bayesian Identity Cap for Calibrated Uncertainty in Frozen Neural Networks
Uddeshya Upadhyay, Shyamgopal Karthik, Yanbei Chen, Massimiliano Mancini, Zeynep Akata
Uncertainty Estimation Calibrated Uncertainty Frozen Pre Trained Identity Inference

June 15, 2022

On Calibrated Model Uncertainty in Deep Learning
Biraja Ghoshal, Allan Tucker
Deep Learning Uncertainty Estimation Bayesian Neural Network Uncertainty Calibration Calibrated Uncertainty

June 14, 2022

Bayesian neural networks for the probabilistic forecasting of wind direction and speed using ocean data
Mariana C A Clare, Matthew D Piggott
Bayesian Neural Network Probabilistic Forecast Speed Effect Wind Power Wind Pressure Calibrated Uncertainty Ocean Data

Calibrated Uncertainty

Papers

A Monte Carlo Framework for Calibrated Uncertainty Estimation in Sequence Prediction

FairlyUncertain: A Comprehensive Benchmark of Uncertainty in Algorithmic Fairness

End-to-End Conformal Calibration for Optimization Under Uncertainty

DADEE: Well-calibrated uncertainty quantification in neural networks for barriers-based robot safety

Evaluating Bayesian deep learning for radio galaxy classification

Efficiently Computable Safety Bounds for Gaussian Processes in Active Learning

Calibrated Uncertainty Quantification for Operator Learning via Conformal Prediction

Unified Uncertainty Calibration

Calibrating Uncertainty for Semi-Supervised Crowd Counting

Uncertainty-Driven Multi-Scale Feature Fusion Network for Real-time Image Deraining

Likelihood Annealing: Fast Calibrated Uncertainty for Regression

Modeling Multimodal Aleatoric Uncertainty in Segmentation with Mixture of Stochastic Experts

Calibrated Perception Uncertainty Across Objects and Regions in Bird's-Eye-View

Accurate and Reliable Methods for 5G UAV Jamming Identification With Calibrated Uncertainty

Uncertainty Estimation for 3D Dense Prediction via Cross-Point Embeddings

BayesCap: Bayesian Identity Cap for Calibrated Uncertainty in Frozen Neural Networks

On Calibrated Model Uncertainty in Deep Learning

Bayesian neural networks for the probabilistic forecasting of wind direction and speed using ocean data