Proper Scoring Rules

Proper scoring rules are losses for probabilistic predictions, such as the Brier score, log score, and continuous ranked probability score (CRPS); a strictly proper rule is minimized in expectation by reporting the true probabilities. Researchers use these rules to assess calibration and uncertainty, decompose forecast error, and develop guarantees for online forecasting and regret. They also tailor scores and training objectives to settings such as censored survival data, weather prediction, causal inference, and decisions with unequal costs. A key tension is that rules with the same truth-telling guarantee can produce different finite-sample model behavior and rankings, so the best score depends on both the data and the intended use of the forecast.