PAC Bayesian

PAC-Bayesian theory provides a frequentist framework for analyzing the generalization performance of machine learning models by incorporating prior knowledge, offering probabilistic generalization bounds. Current research focuses on extending PAC-Bayesian methods to high-dimensional settings, unbounded loss functions, and various model architectures including deep neural networks and generalized linear models, often employing techniques like importance weighting and regularized posteriors to improve bound tightness and practical applicability. These advancements yield tighter generalization guarantees and inform the design of more robust and efficient learning algorithms, impacting both theoretical understanding and practical applications across diverse machine learning domains.

Papers

November 9, 2024

Multi-View Majority Vote Learning Algorithms: Direct Minimization of PAC-Bayesian Bounds
Mehdi Hennequin, Abdelkrim Zitouni, Khalid Benabdeslem, Haytham Elghazel, Yacine Gaci
Statistical Learning PAC Bayesian Multi View Learning PAC Bayesian Bound Minimization Problem

November 1, 2024

Statistical Guarantees for Lifelong Reinforcement Learning using PAC-Bayesian Theory
Zhi Zhang, Chris Chow, Yasi Zhang, Yanchao Sun, Haochen Zhang, Eric Hanchen Jiang, Han Liu, Furong Huang, Yuchen Cui, Oscar Hernan Madrid Padilla
PAC Bayesian Single Agent Reinforcement Learning Statistical Guarantee Lifelong Reinforcement Learning Policy Distribution

October 20, 2024

On Cold Posteriors of Probabilistic Neural Networks: Understanding the Cold Posterior Effect and A New Way to Learn Cold Posteriors with Tight Generalization Guarantees
Yijie Zhang
Bayesian Inference Bayesian Approach PAC Bayesian Bayesian Deep Learning Generalization Guarantee Bayesian Model Probabilistic Neural Network Cold Posterior

October 14, 2024

September 3, 2024

A sparse PAC-Bayesian approach for high-dimensional quantile prediction
The Tien Mai
PAC Bayesian PAC Bayes Bound Pseudo Likelihood Conditional Quantiles

July 19, 2024

A Unified Confidence Sequence for Generalized Linear Models, with Applications to Bandits
Junghyun Lee, Se-Young Yun, Kwang-Sung Jun
Financial Application Continuous Control PAC Bayesian Generalized Linear Model Confidence Sequence Logistic Bandit

June 8, 2024

Bayesian vs. PAC-Bayesian Deep Neural Network Ensembles
Nick Hauptvogel, Christian Igel
Bayesian Neural Network Deep Ensemble Bayesian Perspective PAC Bayesian Bayesian Ensemble

June 5, 2024

Unified PAC-Bayesian Study of Pessimism for Offline Policy Learning with Regularized Importance Sampling
Imad Aouali, Victor-Emmanuel Brunel, David Rohde, Anna Korba
Generalization Bound Unified Alignment Importance Sampling PAC Bayesian Offline Policy Learning Weight Regularization

May 23, 2024

Recursive PAC-Bayes: A Frequentist Approach to Sequential Prior Updates with No Information Loss
Yi-Shan Wu, Yijie Zhang, Badr-Eddine Chérief-Abdellatif, Yevgeny Seldin
PAC Bayesian Bayesian Learning Data Driven Prior Information Loss Frequentist Inference

April 26, 2024

Uniform Generalization Bounds on Data-Dependent Hypothesis Sets via PAC-Bayesian Theory on Random Sets
Benjamin Dupuis, Paul Viallard, George Deligiannidis, Umut Simsekli
Generalization Bound PAC Bayesian Upper Bound Generalization Error Bound Hypothesis Space Random Set

April 4, 2024

Learning-to-Optimize with PAC-Bayesian Guarantees: Theoretical Considerations and Practical Implementation
Michael Sucker, Jalal Fadili, Peter Ochs
Practical Implementation PAC Bayesian Generalization Guarantee Learned Optimizers PAC Bayesian Bound Theoretical Consideration

March 25, 2024

A note on generalization bounds for losses with finite moments
Borja Rodríguez-Gálvez, Omar Rivasplata, Ragnar Thobaben, Mikael Skoglund
Generalization Bound Short Note PAC Bayesian Planning Loss Norm Bounded Unbounded Loss Fractional Order Moment

February 19, 2024

Leveraging PAC-Bayes Theory and Gibbs Distributions for Generalization Bounds with Complexity Measures
Paul Viallard, Rémi Emonet, Amaury Habrard, Emilie Morvant, Valentina Zantedeschi
Strong Generalization Generalization Bound Generalization Gap PAC Bayesian Statistical Learning Theory PAC Bayes Bound Gibbs Distribution Complexity Measure

February 14, 2024

Better-than-KL PAC-Bayes Bounds
Ilja Kuzborskij, Kwang-Sung Jun, Yulian Wu, Kyoungseok Jang, Francesco Orabona
PAC Bayesian PAC Bayes Bound Concentration Inequality

February 13, 2024

A PAC-Bayesian Link Between Generalisation and Flat Minima
Maxime Haddouche, Paul Viallard, Umut Simsekli, Benjamin Guedj
Strong Generalization Generalization Performance Generalization Bound PAC Bayesian Flat Minimum Gradient Based Explanation Log Sobolev Generalisation Ability

February 7, 2024

Tighter Generalisation Bounds via Interpolation
Paul Viallard, Maxime Haddouche, Umut Şimşekli, Benjamin Guedj
Generalization Bound Interpolation Regime PAC Bayesian Beta Divergence Probability Divergence

February 6, 2024

More Flexible PAC-Bayesian Meta-Learning by Learning Learning Algorithms
Hossein Zakerinia, Amin Behjati, Christoph H. Lampert
Meta Learning Learning Algorithm PAC Bayesian

January 16, 2024

Personalized Federated Learning of Probabilistic Models: A PAC-Bayesian Approach
Mahrokh Ghoddousi Boroujeni, Andreas Krause, Giancarlo Ferrari Trecate
Differential Privacy Probabilistic Model Personalized Federated Learning PAC Bayesian

January 2, 2024

PAC-Bayes-Chernoff bounds for unbounded losses
Ioar Casado, Luis A. Ortega, Aritz Pérez, Andrés R. Masegosa
PAC Bayesian Probably Approximately Correct PAC Bayes Bound Tighter Bound Unbounded Loss

PAC Bayesian

Papers

Multi-View Majority Vote Learning Algorithms: Direct Minimization of PAC-Bayesian Bounds

Statistical Guarantees for Lifelong Reinforcement Learning using PAC-Bayesian Theory

On Cold Posteriors of Probabilistic Neural Networks: Understanding the Cold Posterior Effect and A New Way to Learn Cold Posteriors with Tight Generalization Guarantees

Embodied Active Learning of Generative Sensor-Object Models

Learning via Surrogate PAC-Bayes

A sparse PAC-Bayesian approach for high-dimensional quantile prediction

A Unified Confidence Sequence for Generalized Linear Models, with Applications to Bandits

Bayesian vs. PAC-Bayesian Deep Neural Network Ensembles

Unified PAC-Bayesian Study of Pessimism for Offline Policy Learning with Regularized Importance Sampling

Recursive PAC-Bayes: A Frequentist Approach to Sequential Prior Updates with No Information Loss

Uniform Generalization Bounds on Data-Dependent Hypothesis Sets via PAC-Bayesian Theory on Random Sets

Learning-to-Optimize with PAC-Bayesian Guarantees: Theoretical Considerations and Practical Implementation

A note on generalization bounds for losses with finite moments

Leveraging PAC-Bayes Theory and Gibbs Distributions for Generalization Bounds with Complexity Measures

Better-than-KL PAC-Bayes Bounds

A PAC-Bayesian Link Between Generalisation and Flat Minima

Tighter Generalisation Bounds via Interpolation

More Flexible PAC-Bayesian Meta-Learning by Learning Learning Algorithms

Personalized Federated Learning of Probabilistic Models: A PAC-Bayesian Approach

PAC-Bayes-Chernoff bounds for unbounded losses