Cold Posterior

The "cold posterior effect" in Bayesian neural networks refers to the observation that artificially lowering the temperature of the posterior distribution—a technique called posterior tempering—often improves model performance, even though a fully Bayesian approach (temperature=1) is theoretically optimal. Current research focuses on understanding why this occurs, exploring alternative methods like confidence-inducing priors to achieve similar improvements without tempering, and investigating the role of factors such as model misspecification, data augmentation, and the effective sample size. This research is significant because it addresses inconsistencies between Bayesian theory and practice, potentially leading to more reliable and accurate Bayesian deep learning models for various applications, including medical imaging.

Papers

October 20, 2024

On Cold Posteriors of Probabilistic Neural Networks: Understanding the Cold Posterior Effect and A New Way to Learn Cold Posteriors with Tight Generalization Guarantees
Yijie Zhang
Bayesian Inference Bayesian Approach PAC Bayesian Bayesian Deep Learning Generalization Guarantee Bayesian Model Probabilistic Neural Network Cold Posterior

October 8, 2024

Temperature Optimization for Bayesian Deep Learning
Kenyon Ng, Chris van der Heide, Liam Hodgkinson, Susan Wei
Bayesian Deep Learning Thermal Management Posterior Predictive Critical Temperature Cold Posterior Predictive Density

March 2, 2024

Can a Confident Prior Replace a Cold Posterior?
Martin Marek, Brooks Paige, Pavel Izmailov
Bayesian Neural Network Bayesian Perspective Prior Distribution Prior Model Cold Posterior

October 2, 2023

If there is no underfitting, there is no Cold Posterior Effect
Yijie Zhang, Yi-Shan Wu, Luis A. Ortega, Andrés R. Masegosa
Posterior Inference Bayesian Perspective Model Overfitting Posterior Predictive Cold Posterior

June 22, 2022

Cold Posteriors through PAC-Bayes
Konstantinos Pitas, Julyan Arbel
Variational Inference PAC Bayesian PAC Bayesian Generalization Bound Cold Posterior

May 27, 2022

How Tempering Fixes Data Augmentation in Bayesian Neural Networks
Gregor Bachmann, Lorenzo Noci, Thomas Hofmann
Data Augmentation Bayesian Neural Network Group Convolution Unnormalized Statistical Model Cold Posterior

February 2, 2022

Posterior temperature optimized Bayesian models for inverse problems in medical imaging
Max-Heinrich Laves, Malte Tölle, Alexander Schlaefer, Sandy Engelhardt
Inverse Problem Medical Imaging Mean Field Bayesian Model Cold Posterior