Paper ID: 2409.19157 • Published Sep 27, 2024
Calibrated Probabilistic Forecasts for Arbitrary Sequences
Charles Marx, Volodymyr Kuleshov, Stefano Ermon
TL;DR
Get AI-generated summaries with premium
Get AI-generated summaries with premium
Real-world data streams can change unpredictably due to distribution shifts,
feedback loops and adversarial actors, which challenges the validity of
forecasts. We present a forecasting framework ensuring valid uncertainty
estimates regardless of how data evolves. Leveraging the concept of Blackwell
approachability from game theory, we introduce a forecasting framework that
guarantees calibrated uncertainties for outcomes in any compact space (e.g.,
classification or bounded regression). We extend this framework to recalibrate
existing forecasters, guaranteeing calibration without sacrificing predictive
performance. We implement both general-purpose gradient-based algorithms and
algorithms optimized for popular special cases of our framework. Empirically,
our algorithms improve calibration and downstream decision-making for energy
systems.