Uncertainty Calibration

Uncertainty calibration studies whether predictions reported as confident or uncertain match how often they are correct, and whether predictive intervals achieve their stated coverage. Common approaches include post-hoc temperature or isotonic scaling, conformal prediction, and uncertainty-aware training with Bayesian models, ensembles, or evidential methods; language-model work also calibrates semantic or verbal confidence for abstention and routing. Methods are tested across classification, forecasting, medical imaging, robotics, and language tasks, often under distribution shift. A central challenge is that good average calibration may hide unreliable subgroups or extreme cases, while improving calibration can trade off against sharpness, accuracy, error ranking, or the limited supply of representative validation data.