Confidence Estimation

Latest papers

42 papers9 papers in the last four weeks, up 80% on the four weeks before. 0.1% of all new papers.

Beyond ECE: Calibrated Size Ratio, Risk Assessment, and Confidence-Weighted Metrics
May 3, 2026Fernando Martin-Maroto, Nabil Abderrahaman, Gonzalo G. de PolaviejaProbability Calibration Uncertainty Calibration
Mathematics of Behavior and Intelligence Lab. Champalimaud Research. Avenida de Brasília, Lisbon · Algebraic AI Calle Castelló, 117, Madrid
Cross-Entropy Is Load-Bearing: A Pre-Registered Scope Test of the K-Way Energy Probe on Bidirectional Predictive Coding
Apr 23, 2026Jon-Paul CacioliPredictive Coding Confidence Estimation
Independent Researcher, Melbourne, Australia
K-Way Energy Probes for Metacognition Reduce to Softmax in Discriminative Predictive Coding Networks
Apr 13, 2026Jon-Paul CacioliConfidence Estimation Predictive Coding
Independent Researcher, Melbourne, Australia
SELFDOUBT: Uncertainty Quantification for Reasoning LLMs via the Hedge-to-Verify Ratio
Apr 7, 2026Satwik Pandey, Suresh Raghu, Shashwat PandeyLLM Uncertainty Estimation Confidence Estimation in Language Models
Independent Researcher · Zillow Group
Identifying Influential N-grams in Confidence Calibration via Regression Analysis
Apr 7, 2026Shintaro Ozaki, Wataru Hashimoto, Hidetaka Kamigaito +2Language Model Calibration Confidence Estimation in Language Models
Nara Institute of Science and Technology (NAIST) · The University of Tokyo
Bayesian Elicitation with LLMs: Model Size Helps, Extra "Reasoning" Doesn't Always
Apr 2, 2026Luka Hobor, Mario Brcic, Mihael Kovac +1LLM Uncertainty Estimation LLM Reliability
University of Zagreb Faculty of Electrical Engineering and Computing, Zagreb, Croatia
GravCal: Single-Image Calibration of IMU Gravity Priors with Per-Sample Confidence
Mar 20, 2026Haichao Zhu, Qian ZhangConfidence Estimation
Independent Researcher · UC Riverside
How Uncertainty Estimation Scales with Sampling in Reasoning Models
Mar 19, 2026Maksym Del, Markus Kängsepp, Marharyta Domnich +4LLM Uncertainty Estimation Uncertainty Quantification
Institute of Computer Science, University of Tartu
Rescaling Confidence: What Scale Design Reveals About LLM Metacognition
Mar 10, 2026Yuyang Dai, Yuxia WangConfidence Estimation in Language Models Language Model Calibration
INSAIT
From Misclassifications to Outliers: Joint Reliability Assessment in Classification
Mar 4, 2026Yang Li, Youyang Sha, Yinzhi Wang +4Confidence Estimation Classification
Intellindust AI Lab, Shenzhen, 518000, China. · Samsung AI Center, Cambridge, CB1 2GA, United Kingdom.
On Calibration of Large Language Models: From Response To Capability
Feb 14, 2026Sin-Han Yang, Cheng-Kuang Wu, Chieh-Yen Lin +3LLM Evaluation Confidence Estimation in Language Models
Appier AI Research · National Taiwan University
Calibration Is Not Enough: Evaluating Confidence Estimation Under Language Variations
Jan 12, 2026Yuxi Xia, Dennis Ulmer, Terra Blevins +3LLM Uncertainty Estimation Confidence Estimation in Language Models
Faculty of Computer Science, UniVie Doctoral School Computer Science · Faculty of Philological and Cultural Studies, University of Vienna, Austria · ILLC, University of Amsterdam, Netherlands +2

May 3, 2026J/K move · Enter open · S save

Beyond ECE: Calibrated Size Ratio, Risk Assessment, and Confidence-Weighted Metrics

Fernando Martin-Maroto, Nabil Abderrahaman, Gonzalo G. de Polavieja

Mathematics of Behavior and Intelligence Lab. Champalimaud Research. Avenida de Brasília, Lisbon · Algebraic AI Calle Castelló, 117, Madrid

Probability Calibration Uncertainty Calibration Confidence Estimation

Open paper AskPDF arXiv

Confidence calibration has been dominated by the Expected Calibration Error (ECE), a linear metric that counts calibration offset equally regardless of the confidence level at which it occurs. We show that ECE can remain small even under arbitrarily large overconfidence risk, so we propose Calibrated Size Ratio (CSR) instead, an interpretable metric that equals 1 under perfect calibration, from which we derive the risk probability

P_{\mathrm{risk}}

that quantifies the statistical evidence for overconfidence. We further argue that overconfidence risk assessment must be complemented by a measure of discriminative value: whether the assigned confidences actively distinguish correct from incorrect predictions. We show that confidence-weighted accuracy

\mathrm{cwA}

is the natural such complement, and that confidence-weighting extends to all standard classification metrics. In particular, we prove that the confidence-weighted AUC (cwAUC) captures the information about calibration while the classical AUC cannot. We validate the proposed indicators on several synthetic confidence distributions under multiple controlled calibration profiles and find that CSR separates risky from non-risky assignments. We also test the metrics on fifteen real datasets, with and without post-hoc calibration, and find that standard methods can yield risky confidence profiles.