PAC-Bayesian Generalization Bounds

Follow this topic Ask about this topic

PAC-Bayesian generalization bounds control the gap between training and test risk by treating a learned predictor as a distribution over models and charging for how far that distribution moves from a prior. Common approaches use KL-divergence bounds for Gibbs predictors, with tighter variants based on data-dependent priors, compression, parameter sensitivity, flatness, or the geometry of equivalent predictors. The framework is being extended to settings such as deep networks, graph and quantum models, control, and dependent time-series data, where bounds can guide training or provide deployment certificates. A central tension is making guarantees both non-vacuous and computationally useful without relying on restrictive assumptions about the data, prior, or predictor.

Latest papers

35 papers5 papers in the last four weeks, down 17% on the four weeks before. 0.0% of all new papers.

Lower expected Fisher rank implies better PAC generalization
Oct 23, 2026Marco Miani, Albert Kjøller Jacobsen, Albert Lopez i Serrano +2Neural Network Generalization PAC-Bayesian Generalization Bounds
AWS AI Labs · Technical University of Denmark
Dimension-Free PAC-Bayes Generalization for Deep Neural Networks via Dirichlet Processes
Oct 15, 2026Sumit VashishthaPAC-Bayesian Generalization Bounds Neural Network Generalization
Independent Researcher
Optimal Transport PAC-Bayes Bounds
Oct 12, 2026Armin Beck, Peter Ochs, Jalal FadiliTV Distance Computation PAC-Bayesian Generalization Bounds
Max Planck Institute for Informatics, Saarland Informatics Campus, Saarbrücken, Germany · Saarland University, Saarland Informatics Campus, Saarbrücken, Germany · ENSICAEN, CNRS GREYC, Caen, France
When Is Accuracy Evidence? A Unified Theory of Generalisation, Validation, and Information Fusion
Oct 2, 2026JM GorrizCross-Validation Generalization Bounds
Data Science and Computational Intelligence Institute University of Granada Granada, Spain · Department of Psychiatry University of Cambridge Cambridge, UK
Behavioral Capacity Certificates for Quantized Language Models
Sep 29, 2026Arian Eamaz, Mojtaba SoltanalianLLM Quantization KV-Cache Quantization
Department of Electrical and Computer Engineering University of Illinois Chicago
Certification Frontiers for Gaussian LoRA: Independent Priors, Posterior Risk, and Prediction-Preserving Balancing
Sep 26, 2026Joyanta Jyoti Mondal, Ibne Farabi ShihabBayesian LoRA PAC-Bayesian Generalization Bounds
Department of Computer and Information Sciences, University of Delaware, USA · Department of Computer Science, Iowa State University, USA
Model-Agnostic Online Certificate-Driven Calibration for Time Series Forecasting Under Distribution Shift
Sep 25, 2026Chenfeng Huang, Zixuan Ma, George MichailidisOnline Calibration Time Series Forecasting
Department of Statistics and Data Science, University of California, Los Angeles, Los Angeles, California, USA
PAC-Bayesian Meta-Learning for Few-Shot Identification of Linear Dynamical Systems
Sep 21, 2026Chenfeng Huang, George MichailidisPAC-Bayesian Generalization Bounds Meta-Learning
Department of Statistics & Data Science University of California, Los Angeles
On the disintegration of the stochastic majority vote: From PAC-Bayesian bounds to a self-bounding algorithm
Sep 15, 2026Julien Bastian, Benjamin Leblanc, Pascal Germain +4PAC-Bayesian Generalization Bounds Ensemble Learning
LabHC · LabHC, UJM, MALICE · ERIC +1
PAC-Bayesian Bounds for Learning Partially Observed Stochastic Linear Time-Invariant State-Space Systems with Inputs and Sub-Gaussian Noise
Sep 8, 2026Mihaly Petreczky, Mohamad Al Ahdab, John LethPAC-Bayesian Generalization Bounds Stochastic Linear Dynamical Systems
CRIStAL, Centrale Lille, Universit´e de Lille, France · Department of Electronic Systems, Aalborg University, Denmark
PAC-Bayesian Reconstruction Guarantees for Time Series Variational Autoencoders
Sep 4, 2026Chloé Hashimoto-Cullen, Ghislain Agoua, Benjamin Guedj +1Variational Autoencoders PAC-Bayesian Generalization Bounds
LPSM, Sorbonne Université · University College London and Inria
PAC-Bayes Beyond Parameter Space: Behavioral Equivalence, Z-Information, and Exact Complexity Decomposition
Aug 11, 2026Vasant G. Honavar, Satish Kumar Keshri, Neil Ashtekar +1PAC-Bayesian Generalization Bounds PAC-Bayesian Theory
Artificial Intelligence Research Laboratory Department of Informatics and Intelligent Systems The Pennsylvania State University · Artificial Intelligence Research Laboratory Department of Computer Science and Engineering The Pennsylvania State University
PAC-DP: PAC-Bayesian Diffusion Policy Learning
Jul 27, 2026Mohammad Hasan Yeganegi, Dian Yu, Andrea Del Prete +2PAC-Bayesian Generalization Bounds Robot Policy Learning
Department of Industrial Engineering, University of Trento, Italy. · Munich Institute of Robotics and Machine Intelligence (MIRMI), Technical University of Munich (TUM), Germany.
PAC--Bayes Bounds on Quotient Parameter Spaces: Geometry-induced Implicit-Bias Priors
Jul 20, 2026Nicola Aladrah, Fabio AnselmiPAC-Bayesian Generalization Bounds PAC-Bayesian Theory
Department of Mathematics, Informatics and Geoscience, University of Trieste, Via Valerio 12/1, 34127 Trieste, Italy · 2McGovern Institute, MIT, Main Street, Cambridge, MA 02139, USA
Non-vacuous Generalization Bounds for Reinforcement Learning with Verifiable Rewards
Jul 16, 2026Yuxuan Zhu, Rohan Alur, Daniel KangPAC-Bayesian Generalization Bounds Reinforcement Learning with Verifiable Rewards
UIUC · MIT · Bridgewater AIA Labs
Entanglement as a Structural Complexity Axis: A PAC-Bayesian View of Generalization in Quantum Policies and Value Functions
Jul 7, 2026Jian Xu, Delu Zeng, John Paisley +1PAC-Bayesian Generalization Bounds Quantum Machine Learning
1RIKEN iTHEMS · 2RIKEN AIP · 3South China University of Technology +1
PAC-Bayesian Certificates for Quadratic Closed-Loop Control
Jun 26, 2026Domagoj HercegPAC-Bayesian Generalization Bounds Data-Driven Control
Department of Mechanical Engineering Eindhoven University of Technology The Netherlands
VeriBound: PAC-Bayesian Generalization Bounds for Process Reward Models Trained with Formal Verification Tools
Jun 17, 2026Amirul Rahman, Mohammed Sabih AlsharariPAC-Bayesian Generalization Bounds Process Reward Models
University of Malaya
Smoothness-Based Derandomization of PAC-Bayes Bounds
Jun 17, 2026Alexandre Lemire Paquin, Brahim Chaib-Draa, Philippe GiguèreNeural Network Generalization PAC-Bayesian Generalization Bounds
Department of Computer Science and Software Engineering Université Laval, Québec, Canada
Uncertainty Estimation and Generalization Bounds for Modern Deep Learning
Jun 11, 2026Luis A. OrtegaBayesian Neural Networks Neural Network Generalization
Department of Computer Science Machine Learning Group Madrid
PAC-Bayesian Adversarially Robust Generalization for Message Passing Graph Neural Networks: A Sensitivity Analysis
Jun 4, 2026Ziling Liang, Xinping Yi, Qingsong Wen +1PAC-Bayesian Generalization Bounds Adversarial Robustness
School of Information Science and Engineering, Southeast University, Nanjing 210096, China · Squirrel Ai Learning, Bellevue, WA, 98004 USA
Certified Policy Optimisation for Nested Causal Bandits via PAC-Bayes Risk
May 28, 2026Tim Woydt, Paul-David ZuercherPAC-Bayesian Generalization Bounds Multi-Armed Bandits
ProdAxon · Department of Computer Science Technical University Darmstadt · Institute for Manufacturing Department of Engineering University of Cambridge
A PAC-Bayesian View of Generalisation for Physics-Informed Machine Learning
May 25, 2026Thien V. Nguyen, Amaury Habrard, Benjamin GuedjPAC-Bayesian Generalization Bounds Physics-Informed ML
Université Jean Monnet Saint-Étienne, CNRS, Institut d’Optique Graduate School, Laboratoire Hubert Curien UMR 5516, F-42023, Saint-Étienne, France · Université Jean Monnet Saint-Étienne, CNRS, Institut d’Optique Graduate School, Laboratoire Hubert Curien UMR 5516, Inria, F-42023, Saint-Étienne, France · Institut Universitaire de France +1
From Privacy to Generalization: Linear Max-Information Bounds for Differentially Private Learning Algorithms
May 25, 2026Christoph H. Lampert, Max Cairney-Leeming, Hossein ZakeriniaPAC-Bayesian Generalization Bounds Differentially Private Stochastic Gradient Descent
Institute of Science and Technology Austria (ISTA) Klosterneuburg, Austria
Certifying Whether to Adapt: Baseline-Relative PAC-Bayes-MMD Guarantees for Test-Time Adaptation
May 20, 2026Ahanaf Hasan Ariq, Hasaan Ahmad, Aayush Parekh +1Distribution Shift Robustness PAC-Bayesian Generalization Bounds
Ideal School and College
A Sharper Picture of Generalization in Transformers
May 20, 2026Paul Lintilhac, Sair ShaikhNeural Network Generalization PAC-Bayesian Generalization Bounds
Thayer School of Engineering Dartmouth College · Dartmouth College
Anytime PAC-Bayes for Constrained Density-Ratio Networks under Covariate Shift
May 17, 2026Paulo Akira F. Enabe, Rodrigo ProvasiPAC-Bayesian Generalization Bounds Domain Adaptation
Escola Politénica University of São Paulo Department of Structural and Geotechnical Engineering
A Survey on Data-Dependent Worst-Case Generalization Bounds
May 13, 2026Hubert Leroux, Jean Marcus, Julien RogerNeural Network Generalization PAC-Bayesian Generalization Bounds
´Ecole polytechnique Palaiseau, France
PAC-Bayes Bounds for Gibbs Posteriors via Singular Learning Theory
Apr 19, 2026Chenyang Wang, Yun YangNeural Network Generalization PAC-Bayesian Generalization Bounds
Department of Statistics, University of Illinois Urbana-Champaign · Department of Mathematics, University of Maryland, College Park
When Do Early-Exit Networks Generalize? A PAC-Bayesian Theory of Adaptive Depth
Apr 17, 2026Dongxin Guo, Jikun Wu, Siu Ming YiuNeural Network Generalization PAC-Bayesian Generalization Bounds
The University of Hong Kong, Hong Kong, China · Brain Investing Limited, Hong Kong, China

Oct 23, 2026J/K move · Enter open · S save

Lower expected Fisher rank implies better PAC generalization

Marco Miani, Albert Kjøller Jacobsen, Albert Lopez i Serrano, Søren Hauberg+1

AWS AI Labs · Technical University of Denmark

Neural Network Generalization PAC-Bayesian Generalization Bounds Fisher Information

Open paper AskPDF arXiv

Deep learning architectures tend to generalize well to unseen data, but explaining this capability is still an open area of research. Under the assumption of i.i.d. data, PAC-Bayes provides a bound on generalization error, and its tightness relies on a parameter distribution that covers a large volume of low-loss solutions. The Fisher information metric endows the parameter space with a geometry for which distances reflect functional changes, and whose zero-component captures functional invariances. We define and study a Fisher-aligned parameter distribution for which we can simultaneously guarantee (1) a low-loss, thanks to the exploration constrained on the functional invariances and (2) a large log-volume of parameters, proportional to the number of zero eigenvalues. This distribution results in a high-probability generalization error bound, which scales with the expected rank of the Fisher information matrix, shedding light on the relevance of this quantity.