Model Calibration
Overview
For classifiers and language models, calibration asks whether confidence matches empirical correctness and develops post-hoc methods such as temperature scaling, isotonic regression, and local corrections, alongside training-time objectives. Online and multicalibration methods extend reliability to changing data, subgroups, and selective prediction, while conformal methods provide coverage guarantees for prediction sets or intervals. A related line calibrates mechanistic models and simulators to measurements using Bayesian inference, evolutionary optimization, differentiable surrogates, and data assimilation in areas such as climate, traffic, medicine, and engineering. Across settings from CIFAR and ImageNet to language-model question answering and clinical data, a central challenge is that aggregate calibration may fail for particular groups or under distribution shift, while flexible corrections require representative validation data and may affect accuracy or decisions.
Latest papers
218 papers39 papers in the last four weeks, up 39% on the four weeks before. 0.2% of all new papers.