Uncertainty Calibration

Follow this topic Ask about this topic

Uncertainty calibration studies whether predictions reported as confident or uncertain match how often they are correct, and whether predictive intervals achieve their stated coverage. Common approaches include post-hoc temperature or isotonic scaling, conformal prediction, and uncertainty-aware training with Bayesian models, ensembles, or evidential methods; language-model work also calibrates semantic or verbal confidence for abstention and routing. Methods are tested across classification, forecasting, medical imaging, robotics, and language tasks, often under distribution shift. A central challenge is that good average calibration may hide unreliable subgroups or extreme cases, while improving calibration can trade off against sharpness, accuracy, error ranking, or the limited supply of representative validation data.

Latest papers

265 papers31 papers in the last four weeks, up 63% on the four weeks before. 0.2% of all new papers.

An Empirical Analysis of Calibration and Selective Prediction in Multimodal Clinical Condition Classification
Mar 3, 2026L. Julián Lechuga López, Farah E. Shamout, Tim G. J. RudnerSelective Prediction Uncertainty Calibration
New York University · New York University Abu Dhabi · University of Toronto
LOCUS: A Distribution-Free Loss-Quantile Score for Risk-Aware Predictions
Mar 2, 2026Matheus Barreto, Mário de Castro, Thiago R. Ramos +2Uncertainty Calibration
Department of Statistics, Federal University of S˜ao Carlos, S˜ao Carlos, S˜ao Paulo, Brazil · Institute of Mathematics and Computer Science, University of S˜ao Paulo, S˜ao Carlos, S˜ao Paulo, Brazil · School of Forest, Fisheries, and Geomatics Sciences, University of Florida, Gainesville, Florida, United States of America
SphUnc: Hyperspherical Uncertainty Decomposition and Causal Identification via Information Geometry
Mar 1, 2026Rong Fu, Chunlei Meng, Jinshuo Liu +8Uncertainty Decomposition Uncertainty Calibration
Independent Researcher
RaUF: Learning the Spatial Uncertainty Field of Radar
Mar 1, 2026Shengpeng Wang, Kuangyu Wang, Wei WangmmWave Radar Sensing Autonomous Driving Perception
Huazhong University of Science and Technology · Wuhan University
V-ECE: Estimating General Expected Calibration Errors
Feb 27, 2026Eugène Berta, Sacha Braun, Francis Bach +2Probability Calibration Uncertainty Calibration
INRIA · Ecole Normale Sup´erieure, PSL Research University · UC Berkeley
ABM-UDE: Developing Surrogates for Epidemic Agent-Based Models via Scientific Machine Learning
Feb 25, 2026Sharv Murgai, Utkarsh Utkarsh, Kyle C. Nguyen +3Neural Surrogate Modeling Epidemic Forecasting
Monta Vista High School · MIT CSAIL · Sandia National Laboratories
Structure-Aware Epistemic Uncertainty Quantification for Neural Operator PDE Surrogates
Feb 24, 2026Haoze Song, Zhihao Li, Mengyi Deng +4Uncertainty Quantification Epistemic Uncertainty
The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China · The Hong Kong University of Science and Technology, Hong Kong SAR, China
JUCAL: Jointly Calibrating Aleatoric and Epistemic Uncertainty in Classification Tasks
Feb 23, 2026Jakob Heiss, Sören Lambrecht, Jakob Weissteiner +4Uncertainty Calibration Post-Hoc Calibration
UC Berkeley · ETH Zurich · UBS Zurich +1
How Uncertain Is the Grade? A Benchmark of Uncertainty Metrics for LLM-Based Automatic Assessment
Feb 17, 2026Hang Li, Kaiqi Yang, Xianxuan Long +9Uncertainty Quantification LLM Uncertainty Estimation
Washington State University, Pullman, USA
The Impact of Class Uncertainty Propagation in Perception-Based Motion Planning
Feb 17, 2026Jibran Iqbal Shah, Andrei Ivanovic, Kelly Zhu +4Autonomous Driving Planning Motion Planning
Department of Computer Science, University of Toronto. · Toyota Research Institute.
Uncertainty-Calibrated Spatiotemporal Field Diffusion with Sparse Supervision
Feb 17, 2026Kevin Valencia, Xihaier Luo, Shinjae Yoo +1Spatiotemporal Forecasting Diffusion-Based Inverse Problems
University of California, Los Angeles · Brookhaven National Laboratory
Complex-Valued Unitary Representations as Classification Heads for Improved Uncertainty Quantification in Deep Neural Networks
Feb 17, 2026Akbar Anbar Jafari, Cagri Ozcinar, Gholamreza AnbarjafariUncertainty Calibration Complex-Valued Neural Networks
University of Tartu Tartu, Estonia · 3S Holding OÜ Tartu 51011, Estonia
Activation-Space Uncertainty Quantification for Pretrained Networks
Feb 16, 2026Richard Bergna, Stefan Depeweg, Sergio Calvo-Ordoñez +3Uncertainty Quantification Uncertainty Calibration
Department of Engineering, University of Cambridge, Cambridge, UK · Siemens AG, Munich, Germany · Mathematical Institute and Oxford-Man Institute, University of Oxford, Oxford, UK
Confidence-Uncertainty Boundary Calibration for Bayesian Deep Learning in Medical Image Analysis
Feb 12, 2026Hua Xu, Julián D. Arias-Londoño, Juan I. Godino-LlorenteBayesian Neural Networks Post-Hoc Calibration
Escuela Técnica Superior de Ingenieros de Telecomunicación Universidad Politécnica de Madrid 28040, Madrid, Spain
Density-Informed Pseudo-Counts for Calibrated Evidential Deep Learning
Feb 1, 2026Pietro Carlotti, Nevena Gligić, Arya FarahiEvidential Deep Learning OOD Detection
Department of Statistics and Data Sciences University of Texas at Austin
Calibrated uncertainty quantification for prosumer flexibility aggregation in ancillary service markets
Jan 21, 2026Yogesh Pipada Sunil Kumar, S. Ali Pourmousavi, Jon A. R. Liisberg +1Uncertainty Quantification Uncertainty Calibration
School of Electrical and Mechanical Engineering, The University of Adelaide, Ingkarni Wardli Building, Adelaide, SA 5005, Australia · Watts A/S, Brogade 19D, Køge 4600, Denmark
When Models Know When They Do Not Know: Calibration, Cascading, and Cleaning
Jan 12, 2026Chenjie Hao, Weyl Lu, Yuko Ishiwaka +3Adaptive Inference Confidence Estimation in Language Models
University of California, Davis · SoftBank Corp. · Aizip
Disentangling Aleatoric and Epistemic Uncertainty in Physics-Informed Neural Networks. Application to Insulation Material Degradation Prognostics
Jan 7, 2026Ibai Ramirez, Jokin Alcibar, Joel Pino +2Bayesian Neural Networks Uncertainty Quantification
Department of Electronics and Computer Science Mondragon University Arrasate-Mondragon, 20500, Spain · CARTIF Technology Centre Valladolid, 47151, Spain · Department of Physical Chemistry University of the Basque Country (EHU)[cs.LG] Basque Centre for Applied Mathematics (BCAM) Leioa, 20018, Spain +1
Improving Semantic Uncertainty Quantification in LVLMs with Semantic Gaussian Processes
Dec 16, 2025Joseph Hoche, Andrei Bursuc, David Brellmann +4Vision-Language Models Uncertainty Quantification
AMIAD, Pôle Recherche, Palaiseau · Safran Tech · University of Liège +3
MedBayes-Lite: A Clinical Uncertainty Governance Layer for Risk-Aware Medical Decision Support
Nov 20, 2025Elias Hossain, Md Mehedi Hasan Nipu, Maleeha Sheikh +5Selective Prediction Healthcare
College of Engineering and Computer Science, University of Central Florida, Orlando, FL 32816, USA · Department of Computer Science and Engineering, North South University, Dhaka 1229, Bangladesh · Department of Electrical and Computer Engineering, Purdue University Fort Wayne, Fort Wayne, IN 46805, USA +4
Conformal and kNN Predictive Uncertainty Quantification Algorithms in Metric Spaces
Jul 21, 2025Gábor Lugosi, Marcos MatabuenaConformal Prediction k-Nearest Neighbors
Department of Economics and Business, Pompeu Fabra University, Barcelona, Spain · ICREA, Pg. Lluís Companys 23, 08010 Barcelona, Spain · Barcelona Graduate School of Economics +3
CLEAR: Calibrated Learning for Epistemic and Aleatoric Risk
Jul 10, 2025Ilia Azizi, Juraj Bodik, Jakob Heiss +1Uncertainty Quantification Aleatoric Uncertainty
Department of Operations, HEC, University of Lausanne, Switzerland · BegooAI, Switzerland · Department of Statistics, University of California, Berkeley, USA +1
Towards Understanding The Calibration Benefits of Sharpness-Aware Minimization
May 29, 2025Chengli Tan, Yubo Zhou, Haishan Ye +7Sharpness-Aware Minimization Model Calibration
Northwestern Polytechnical University · SGIT AI Lab, State Grid Corporation of China · Xi’an Jiaotong University +2
Understanding and Mitigating Under-Confidence in GNNs from the Final Layer
May 16, 2025Jincheng Huang, Jie Xu, Xiaoshuang Shi +3Graph Neural Networks Uncertainty Calibration
School of Computer Science and Engineering, University of Electronic Science and Technology of China · Singapore Management University · Southeast University +1
Label-Confidence-Aware Uncertainty Estimation in Natural Language Generation
Dec 10, 2024Qinhong Lin, Yinglun Feng, Yuhao Zhang +2Uncertainty Quantification LLM Uncertainty Estimation
School of Cyberspace Security, Beijing University of Posts and Telecommunications, Beijing, China · QuanCheng Laboratory, Jinan, China

Mar 3, 2026J/K move · Enter open · S save

An Empirical Analysis of Calibration and Selective Prediction in Multimodal Clinical Condition Classification

L. Julián Lechuga López, Farah E. Shamout, Tim G. J. Rudner

New York University · New York University Abu Dhabi · University of Toronto

Selective Prediction Uncertainty Calibration Multimodal Model Evaluation Multi-Label Classification Multimodal Classification

Open paper AskPDF arXiv

As artificial intelligence systems move toward clinical deployment, ensuring reliable prediction behavior is fundamental for safety-critical decision-making tasks. One proposed safeguard is selective prediction, where models can defer uncertain predictions to human experts for review. In this work, we empirically evaluate the reliability of uncertainty-based selective prediction in multilabel clinical condition classification using multimodal ICU data. Across a range of state-of-the-art unimodal and multimodal models, we find that selective prediction can substantially degrade performance despite strong standard evaluation metrics. This failure is driven by severe class-dependent miscalibration, whereby models assign high uncertainty to correct predictions and low uncertainty to incorrect ones, particularly for underrepresented clinical conditions. Our results show that commonly used aggregate metrics can obscure these effects, limiting their ability to assess selective prediction behavior in this setting. Taken together, our findings characterize a task-specific failure mode of selective prediction in multimodal clinical condition classification and highlight the need for calibration-aware evaluation to provide strong guarantees of safety and robustness in clinical AI.