Overview

Probability calibration studies whether predictions made with a given confidence are correct at the corresponding rate, a property that matters when decisions depend on estimated risk rather than just the most likely class. Common approaches include post-hoc methods such as temperature scaling and isotonic regression, recalibration of full predictive distributions, and online or group-conditional methods that seek reliability across changing sequences or subpopulations. Researchers assess calibration with reliability diagnostics and proper scoring rules, and apply these ideas to classifiers, forecasts, medical systems, and language-model agents. A central challenge is that good average calibration may hide subgroup or distribution-shift failures, while improving calibration can interact with accuracy, ranking, and the costs of downstream decisions.

Latest papers

CardsList

139 papers8 papers in the last four weeks, down 79% on the four weeks before. 0.1% of all new papers.

  1. A Matching Lower Bound for Online Smooth Calibration

    Nov 11, 2026Kaixuan Ji, Quanquan GuOnline CalibrationProbability Calibration

  2. Calibrate, Don't Retrain: Decomposing the Value of Annotator Disagreement in Cyberbullying Severity

    Nov 11, 2026Arslan Bisharat, Katelyn Skees, Manuel Sandoval Madrigal +5Annotator DisagreementClassification

  3. Learning Calibrated Confidence for Multi-Agent Collaboration

    Oct 29, 2026Chenxiao Fan, Chongming Gao, Gangyi Zhang +9Multi-Agent LLM SystemsMulti-Agent Collaboration

  4. Whether or Where? Tail Calibration Signals Finite-Budget Reward Hacking in Best-of-N Selection

    Oct 25, 2026Cheng Gui, Yihua Shao, Lei Li +2Best-of-N SamplingReward Over-Optimization

  5. Native Probability Outputs for Clinical Triage: An Evaluation of Calibration and Decision Accuracy

    Oct 17, 2026Liang You, Zhiyin Zhou, Dawei Tang +2Clinical Language Model EvaluationClinical Triage

  6. Are You Sure Your LLM Is Sure? Repairing and Auditing Confidence

    Oct 15, 2026Arijit Dey, Bhartendu Kumar, Ramya Hebbalaguppe +1Confidence Estimation in Language ModelsLanguage Model Calibration

  7. Noisy-Label Calibration with Multiple Unreliable Labels

    Oct 14, 2026Keisuke Hanada, Futoshi FutamiModel CalibrationProbability Calibration

  8. Recalibrating Many Questions with Few Labels: Minimax Rates and Identification Limits

    Oct 13, 2026Xiaoyu Li, Chiwun YangProbability CalibrationMinimax Estimation

  9. Does Personalization Break Trust? Open-Set Rejection Calibration Under Few-Shot On-Device Intent Adaptation

    Oct 12, 2026Vaibhav Prajapati, Pushp GargOpen-Set RecognitionOOD Detection

  10. ProbabilityBench: Model Evaluation and Calibration with Exact Ground-Truth Uncertainty

    Oct 12, 2026Yuyang Dai, Yuting Tang, Futoshi Futami +1LLM Uncertainty EstimationProbability Calibration

  11. On the Calibration of Crowdsourcing Aggregators

    Oct 12, 2026Gabriel Singer, Reda Arab, Maïssa Frikha +4Probability CalibrationPost-Hoc Calibration

  12. QBFF: Deep Quantum Bottleneck for Automatic Speaker Verification Anti-Spoofing

    Oct 12, 2026Anton Firc, Nouhaila Innan, Vojtěch Staněk +6Audio Deepfake DetectionSpeaker Verification

  13. Agreement versus Softmax under Spatial Disambiguation

    Oct 9, 2026Michał Zakrzewski, Mariia Shpir, Taras Rumezhak +1Vision-Language Grounding3D Visual Grounding

  14. Beyond Discrimination: Evaluating Probability Calibration in Radiomics-Image Fusion for Renal Cell Carcinoma Classification

    Oct 9, 2026Yuan Liang, Morgan May, Abraham Campbell +3Medical Image ClassificationProbability Calibration

  15. TabReliBench: Evaluating Tabular Foundation Models Beyond Predictive Accuracy

    Oct 9, 2026Tianqi Zhao, Jia Guo, Yan Shuo Tan +1Tabular Foundation ModelsProbability Calibration

  16. Missing Modality-Aware Calibration for Trustworthy Brain Tumor Segmentation

    Oct 8, 2026Sol Lee, Hyunji Kim, Sungrae Hong +2Probability CalibrationBrain Tumor Segmentation

  17. Confidence Reasoning Graphs: Structured Confidence Estimation for LLM Agents

    Oct 6, 2026Brendan King, Farima Fatahi Bayat, Jean-Flavien Bussotti +2Probability CalibrationConfidence Estimation in Language Models

  18. High-Dimensional Online Calibration from Harmonic Weights

    Oct 6, 2026Maxwell Fishelson, Mehryar MohriOnline CalibrationProbability Calibration

  19. Explicit Asymptotic Bounds for Sequential Calibration Beyond T2/3T^{2/3}

    Oct 6, 2026Eric Dai, Maxwell FishelsonProbability CalibrationProbabilistic Forecasting

  20. Certification of Real Images through Calibrated Content Authentication

    Oct 5, 2026Sarim Hashmi, Abdelrahman Elsayed, Mohammed Talha Alam +2Probability CalibrationDigital Image Forensics

  21. Sharp Integrality Gaps in Calibration Distance

    Oct 5, 2026Zinan Wang, Xinhao YangProbability CalibrationModel Calibration

  22. SearchJev: A Calibrated System-1 Model for Search Agents

    Oct 4, 2026Congfeng Cao, Lipeng Zuo, Konstantinos Papakostas +7Soft-Label LearningProbability Calibration

  23. CalCErt: Bin-wise Certification of Confidence Calibration in Medical Image Classification

    Oct 2, 2026Leo Fillioux, Stergios Christodoulidis, Stergios Christodoulidis +2Model CalibrationProbability Calibration