Proper Scoring Rules

Follow this topic Ask about this topic

Momentum

2 papers in the last four weeks, with none the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28

Latest papers 23

All topics

Proper Scoring Rule-based Diffusion for Probabilistic Weather Forecasting
Sep 29, 2026Joonhyeong Park, Giung Nam, Hyungi Lee +3Probabilistic Forecasting Proper Scoring Rules
KAIST · Kookmin University · New York University
You Should Be Properly Scoring Your Odometry
Sep 22, 2026Ola Rønning, Usama Saqib, Andrzej WąsowskiProbability Calibration Proper Scoring Rules
Software Quality Research Group, IT University of Copenhagen, Denmark
Estimating Rare Events in Language Models with Proper Evaluation
Jul 20, 2026Nikita Y. Parulekar, Anqi LiuLanguage Model Safety Evaluation Proper Scoring Rules
Johns Hopkins University
Accuracy and Normalized Accuracy under Length Bias: Analysis, Guidelines, and a Bayesian Alternative
Jul 14, 2026Koen OostermeijerLLM Evaluation Proper Scoring Rules
Aleph Alpha Research
When do prophets profit in prediction markets?
Jul 7, 2026Anri Gu, Nicole Kagan, Alec Sun +2Probabilistic Forecasting Proper Scoring Rules
University of Chicago · Kalshi Research · New York University, Shanghai
Decision-Aware Training for Sample-Based Generative Models
Jul 1, 2026Kornelius Raeth, Nicole LudwigCost-Sensitive Learning Probabilistic Forecasting
T ubingen AI Center, University of T ubingen, Germany · University of Augsburg, Germany
Structured Proper Loss Geometries for Multiclass Classification: Theory and Controlled Empirical Evaluation
Jun 28, 2026Soumyadip SarkarProper Scoring Rules Multiclass Classification
Independent Researcher
Decision-Aligned Evaluation of Uncertainty Quantification
Jun 25, 2026Annika Schneider, Tommy Rochussen, Joshua Stiller +1Uncertainty Quantification Proper Scoring Rules
1Technical University of Munich · 2Helmholtz AI · 3MCML +3
Model selection with proper scoring rules on data sets of time series: prefer the mean scaled score
Jun 23, 2026Giorgio Corani, Stefano Damato, Dario Azzimonti +1Model Selection Time Series Forecasting
SUPSI, Istituto Dalle Molle di Studi sull’Intelligenza Artificiale (IDSIA), Lugano, Switzerland
Toward Simultaneously Optimal Regret in U-Calibration
Jun 16, 2026Rafael Frongillo, Haipeng Luo, Nishant A. Mehta +1Probabilistic Forecasting Proper Scoring Rules
University of Colorado Boulder · University of Southern California and Google Research · University of Victoria +1
Investigating Calibration Challenges in Probabilistic Electricity Price Forecasting
Jun 8, 2026Jan Niklas Lettner, Hadeer El Ashhab, Benjamin SchäferProbability Calibration Electricity Price Forecasting
Institute for Automation and Applied Informatics
Proper Scoring Rules for Right-Censored Survival Data
Jun 4, 2026Jef Jonkers, Glenn Van Wallendael, Luc Duchateau +1Survival Analysis Probabilistic Forecasting
IDLab Department of Electronics and Information Systems Ghent University, Belgium · IDLab Department of Electronics and Information Systems Ghent University - imec, Belgium · Biometrics Research Group Department of Morphology, Imaging, Orthopedics, Rehabilitation and Nutrition Ghent University, Belgium
Tailoring Strictly Proper Scoring Rules for Downstream Tasks: An Application to Causal Inference
Jun 2, 2026Roman Plaud, Alexandre Perez-Lebel, Antoine Saillenfest +4Inverse Probability Weighting Causal Effect Estimation
Institut Polytechnique de Paris · Onepoint, France · Fundamental Technologies +2
Proper Calibeating
May 26, 2026Dean P. Foster, Sergiu HartProper Scoring Rules Uncertainty Calibration
Department of Statistics, Wharton, University of Pennsylvania, Philadelphia, and Amazon, New York. · Institute of Mathematics, Department of Economics, and Federmann Center for the Study of Rationality, The Hebrew University of Jerusalem.
Proper Scoring Rules for Agentic Uncertainty Quantification
May 23, 2026Suresh Raghu, Satwik Pandey, Shashwat PandeyUncertainty Quantification Agent Evaluation
Independent Researcher
ECUAS $_n$ : A family of metrics for principled evaluation of uncertainty-augmented systems
May 19, 2026Lautaro Estienne, Erik Ernst, Matías Vera +2Uncertainty Quantification Proper Scoring Rules
School of Engineering, UBA, Argentina · ICC, CONICET-Universidad de Buenos Aires, Argentina · LISN, CNRS, Université Paris-Saclay, France +4
Calibeating for general proper losses: A Bregman divergence approach
May 17, 2026Maximilian Fichtl, Cristóbal Guzmán, Nishant A. MehtaFunctional Bregman Divergences Proper Scoring Rules
Independent researcher · Institute for Mathematical and Computational Engineering, Faculty of Mathematics and School of Engineering, Pontificia Universidad Católica de Chile · Department of Computer Science, University of Victoria
The Endogeneity of Miscalibration: Impossibility and Escape in Scored Reporting
May 8, 2026Lauri Lovén, Sasu TarkomaMechanism Design Proper Scoring Rules
Future Computing Group, University of Oulu, Finland · University of Oulu and University of Helsinki, Finland
Pandora's Regret: A Proper Scoring Rule for Evaluating Sequential Search
May 3, 2026Gerardo A. Flores, Yash Deshpande, Jannis R. Brea +1Proper Scoring Rules
MIT · Independent · NYU
Foresight Arena: An On-Chain Benchmark for Evaluating AI Forecasting Agents
May 1, 2026Maksym Nechepurenko, Pavel ShuvalovAI Agent Evaluation Forecasting Benchmarks
Director of Research, Devnull, Dubai, UAE · Chief Technology Officer, Devnull, Dubai, UAE
Distributional Regression with Tabular Foundation Models: Evaluating Probabilistic Predictions via Proper Scoring Rules
Mar 9, 2026Jonas Landsgesell, Pascal Knoll, Tizian WenzelTabular Foundation Models Proper Scoring Rules
University of Stuttgart (Stuttgart, Germany) · Ludwig Maximilian University of Munich (Munich, Germany) · Munich Center for Machine Learning (Munich, Germany)
An intuitive rearranging of the Yates covariance decomposition for probabilistic verification of forecasts with the Brier score
Mar 4, 2026Bruno Hebling VieiraProbability Calibration Probabilistic Forecasting
Methods of Plasticity Research, Department of Psychology, University of Zurich, Zurich, Switzerland · Methods of Plasticity Research, Department of Psychology, University of Zurich, Zurich,2026 Switzerland
How Proper Scoring Rules Shape LLM Forecasting
Date pendingBenjamin Turtel, Paul Wilczewski, Kris Skotheim +2Probabilistic Forecasting Proper Scoring Rules
Lightning Rod Labs · Technology and Operations Management, INSEAD, Fontainebleau, France · Wharton School and School of Arts & Sciences, University of Pennsylvania

Sep 29, 2026, cs.LGJ/K move · Enter open · S save

Proper Scoring Rule-based Diffusion for Probabilistic Weather Forecasting

Joonhyeong Park, Giung Nam, Hyungi Lee, Kyunghyun Cho+2

KAIST · Kookmin University · New York University

Probabilistic Forecasting Proper Scoring Rules Conditional Diffusion Models Weather Forecasting

Open paper AskPDF arXiv

Recent probabilistic weather forecasters train stochastic predictors with the continuous ranked probability score (CRPS) to generate each ensemble member in a single forward pass. These models learn the predictive distribution from the forecast context alone, which becomes difficult at longer forecast horizons where uncertainty is high. To learn the predictive distribution more effectively, we introduce auxiliary conditional denoising tasks that predict the same future state from the context and its corrupted version, which provides partial future information that can reduce prediction ambiguity. Building on distributional diffusion models, we learn the conditional distributions of these tasks with a single stochastic predictor by minimizing a proper scoring rule across noise levels. At inference, the predictor can still generate each ensemble member in a single forward pass at the fully corrupted endpoint. Standard CRPS training is recovered as the endpoint-only special case of our formulation, so our framework extends existing CRPS-based forecasters with only additional conditioning inputs. Controlled experiments show that the auxiliary tasks improve one-step forecasting across architectures, with larger gains at longer forecast horizons. The gains extend to high-dimensional global weather forecasting under both training from scratch and fine-tuning, along with improved calibration and potential benefits for generalization under distribution shift.

Momentum

Related topics

Latest papers 23