Post-Hoc Calibration

Post-hoc calibration studies how to make a trained model’s confidence scores or predictive distributions reflect how often its predictions are correct. Common approaches include temperature scaling, Platt scaling, isotonic regression, and more flexible class-, input-, or group-conditional calibrators fitted on held-out data. Researchers evaluate these methods with measures such as expected calibration error, Brier score, and negative log-likelihood, across settings including classification, medical prediction, segmentation, forecasting, and language models. A central tension is that flexible calibration can address shifts, ambiguity, and local reliability differences, but may overfit small calibration sets or change predictions and rankings; reliable evaluation must account for the deployment setting and the labels used.