Generalization Bounds

Ask about this topic

Overview

Generalization bounds estimate how well a model trained on finite data will perform on new examples, helping explain when learning is reliable. Common tools include VC and Rademacher complexity, covering numbers, PAC-Bayes and information-theoretic bounds, and algorithmic stability. For deep networks and modern settings, analyses increasingly use learned feature geometry, optimization dynamics, and structure in data, while extending guarantees to distribution shift, generative models, and dependent or distributed samples. A central tension is that broad worst-case guarantees can be too loose to be useful, whereas tighter, non-vacuous bounds often rely on assumptions about the algorithm, data geometry, or shift between training and test distributions.

Latest papers

CardsList

163 papers27 papers in the last four weeks, up 170% on the four weeks before. 0.2% of all new papers.

  1. Integral Probability Boundaries

    Oct 27, 2026Bogdan Kulynych, Shahab AsoodehIntegral Probability MetricsGeneralization Bounds

    Biomedical Data Science Center, Lausanne University Hospital (CHUV) · McMaster University and Vector Institute

  2. Quantitative Generalization Bounds for Multi-Layer Transformers: A Few Long Strings can Beat Many Short Ones

    Oct 26, 2026Marius Henry, Michael Rizvi-Martel, Michael HahnLanguage Model GeneralizationGeneralization Bounds

    LaCoCo, Saarland University, Saarland Informatics Campus · Zuse School ELIZA · Mila & Université de Montréal

  3. Fine-Grained Analysis of Catastrophic Forgetting in Continual Learning via Algorithmic Stability

    Oct 22, 2026Guodong Zheng, Yan Sun, Peng Wang +5Catastrophic ForgettingGeneralization Bounds

    School of Cyber Science and Technology, Shenzhen Campus of Sun Yat-sen University, Shenzhen, China. · College of Computing and Data Science and the Generative AI Lab at Nanyang Technological University, Singapore. · Tencent Hunyuan, Shenzhen, China.

  4. Random Projections for Spectral Algorithms: Optimal Rates in Sobolev Norms

    Oct 22, 2026Caixing Wang, Yue WangGeneralization BoundsSpectral Methods

    School of Statistics and Data Science, Southeast University, Nanjing, 210000, Jiangsu, China · Faculty of Business for Science and Technology, School of Management, University of Science and Technology of China, Hefei, 230000, Anhui, China

  5. Self-normalized Bernstein Inequalities for Regenerative Processes and Applications to Learning Without Independence

    Oct 21, 2026Imon Banerjee, Pierre AlquierGeneralization BoundsAgnostic Learning

    University of Chicago, Booth School of Business · ESSEC Business School, Asia-Pacific campus, Singapore

  6. Generalization Bounds via Vector-Valued LpL_p Correlation VαV_α

    Oct 21, 2026Mohammad Hossein Momeni Hamaneh, Mohammad Mahdi Mojahedian, Mohammad Hossein YassaeeGeneralization BoundsInformation-Theoretic Generalization Bounds

    Information Systems and Security Laboratory (ISSL), Sharif University of Technology, Tehran, Iran

  7. Gaussian Approximation of Besov Functions for Logistic Classification on Manifolds

    Oct 19, 2026Guanhang Lei, Lei Shi, Xiaoming YuanClassificationStatistical Learning Theory

    Department of Mathematics, The University of Hong Kong, Hong Kong, China · School of Mathematical Sciences, Shanghai Key Laboratory for Contemporary Applied Mathematics, Center for Applied Mathematics, Fudan University, Shanghai, China

  8. Margin-Based Generalization for Quantum Machine Learning with Classical Data Encoding

    Oct 17, 2026Ju-Young Ryu, Tak Hur, Daniel K. ParkQuantum Machine LearningGeneralization Bounds

    Korea Institute of Science and Technology Information, Republic of Korea · Department of Statistics and Data Science, Yonsei University, Republic of Korea · Department of Applied Mathematics, University of Waterloo, Canada +2

  9. Uniform convergence at optimal rate and scale

    Oct 17, 2026Noga Alon, Steve Hanneke, Shay Moran +1Generalization Bounds

    Department of Mathematics, Princeton University, Princeton, NJ, USA. · Department of Computer Science, Purdue University, West Lafayette, IN, USA. · Departments of Mathematics, Computer Science, and Data and Decision Sciences, Technion–Israel Institute of Technology; Google Research. +1

  10. Margin Certificates for Voting Classifiers under Mixing Dependence

    Oct 16, 2026Adria Binte Habib, Ibne Farabi ShihabStatistical Learning TheoryGeneralization Bounds

    Department of Computer Science and Engineering, BRAC University, Bangladesh · Department of Computer Science, Iowa State University, USA

  11. Escaping Parameter Space: Tight Generalization Bounds via Representation Quality

    Oct 16, 2026Niclas Göring, Shuofeng Zhang, Branton DeMoss +1Generalization BoundsNeural Representation Geometry

    Rudolf Peierls Centre for Theoretical Physics, University of Oxford, Oxford, UK · Mathematical Institute, University of Oxford, Oxford, UK

  12. "Deep lifts": Neural networks can generalize even with the dumbest optimizer

    Oct 16, 2026Sunil Madhow, Yu-Xiang WangNeural Network GeneralizationGeneralization Bounds

    UC San Diego

  13. Deterministic Bounds on Accuracy and Stability of Metric-Space Regression and Classification

    Oct 15, 2026Nina Maria Gottschling, Jakob Gawlikowski, Michele CaprioStatistical Learning TheoryGeneralization Bounds

    Computing and Computational Sciences, Oak Ridge National Laboratory, Oak Ridge, Tennessee. · German Aerospace Center (DLR), Remote Sensing Technology Institute, Wessling, Germany. · Department of Computer Science, University of Warwick, Coventry, United Kingdom.

  14. Generalization Guarantees of Randomized Block Coordinate Descent for Nonconvex Learning

    Oct 15, 2026Ruixi Hu, Liang WuAlgorithmic GeneralizationGeneralization Bounds

    Center of Statistical Research, School of Statistics and Data Science, Southwestern University of Finance and Economics, Chengdu, China

  15. Does Compression Imply Generalization? A Minimum Description Length Perspective

    Oct 15, 2026Lincen Yang, Jiayang Shi, Nan Pu +4Neural Network GeneralizationGeneralization Bounds

    Leiden Institute of Advanced Computer Science, Leiden University, The Netherlands · Centrum Wiskunde & Informatica (CWI), Amsterdam, The Netherlands · Hefei University of Technology, Hefei, China +3

  16. Dimension-Free PAC-Bayes Generalization for Deep Neural Networks via Dirichlet Processes

    Oct 15, 2026Sumit VashishthaPAC-Bayesian Generalization BoundsNeural Network Generalization

    Independent Researcher

  17. Generalization Analysis of Biased Stochastic Gradient Methods for Minimax Problems

    Oct 14, 2026Shuang Zeng, Yunwen Lei, Yiming YingMin-Max OptimizationGeneralization Bounds

    Department of Mathematics, The University of Hong Kong, Hong Kong, China · School of Mathematics and Statistics, The University of Sydney, Sydney, Australia

  18. Dimension-Free Scaling Laws for Size Generalization: An Exponential Separation

    Oct 13, 2026Behrooz Tahmasebi, Melanie WeberSample ComplexityStatistical Learning Theory

    Harvard John A. Paulson School of Engineering and Applied Sciences (SEAS), Harvard University, Cambridge, MA 02138, USA

  19. A Generalized McDiarmid Inequality: Sample Reuse and Non-Uniform Unbounded Differences

    Oct 12, 2026Jeremiah BirrellGeneralization Bounds

    Department of Mathematics, Texas State University, San Marcos, TX, USA

  20. How Many Independent Samples Does a Satellite Image Contain? Generalization Bounds for Spatially Dependent Data

    Oct 6, 2026Robin YoungSpatial GeneralizationRemote Sensing

    Department of Computer Science and Technology University of Cambridge, UK

  21. Certification-Enhanced Generalization Bounds

    Oct 5, 2026Leo Elmecker-Plakolm, Matthew WickerAlgorithmic StabilityGeneralization Bounds

    Department of Computing Imperial College London London, SW7 2AZ, UK

  22. On the Trade-off Between Information Loss and Generalization in Sparse Attention

    Oct 3, 2026Zhongqi Fan, Zheng TanSparse AttentionGeneralization Bounds

    Xi’an Jiaotong University

  23. When Is Accuracy Evidence? A Unified Theory of Generalisation, Validation, and Information Fusion

    Oct 2, 2026JM GorrizCross-ValidationGeneralization Bounds

    Data Science and Computational Intelligence Institute University of Granada Granada, Spain · Department of Psychiatry University of Cambridge Cambridge, UK

  24. Generalization Bounds for Flow-matching Generative Models for Intrinsically Low-dimensional Data

    Oct 2, 2026Saptarshi Chakraborty, Quentin Berthet, Peter L. BartlettFlow MatchingFlow-Based Generative Modeling

    Department of Statistics, University of Michigan · Google DeepMind · Department of Statistics, University of California, Berkeley +1

  25. Can Domain Generalization be Guaranteed in Small-Sample Learning?

    Sep 30, 2026Hong ZhengDomain GeneralizationSmall-Data Learning

    This work was completed during my Ph.D. period and was the result of a flash of inspiration. I decided to submit it to arXiv and leave its value to the readers.

  26. How Many Samples Are Enough for Learning Across Domains?

    Sep 30, 2026Hong ZhengSample ComplexityDomain Generalization

    This work was completed during my Ph.D. period and was the result of a flash of inspiration. I decided to submit it to arXiv and leave its value to the readers.

  27. Into the danger zone: stable extrapolation in high-dimensional function and operator learning

    Sep 29, 2026Ben Adcock, Simone Brugiapaglia, Xuemeng WangOOD GeneralizationGeneralization Bounds

    Simon Fraser University · Concordia University

  28. Preferent Compression Bounds Are Tight

    Sep 28, 2026Dario Paccagnan, Marius TirleaSample CompressionStatistical Learning Theory

    Department of Computing, Imperial College London, London, UK

  29. Information-Theoretic Analysis of Next-Token Prediction under Markovian Data

    Sep 28, 2026Masoud Kavian, Abdellatif Zaidi, Milad SefidgaranInformation-Theoretic Generalization BoundsNext-Token Prediction

    Mathematical and Algorithmic Science Laboratory, Huawei Paris Research Center, 92100 Boulogne-Billancourt, France · Laboratoire d’Informatique Gaspard Monge, Université Gustave Eiffel, 77420 Champs-sur-Marne, France

  30. Generalization behavior of OPTQ and the role of regularization

    Sep 25, 2026Erin George, Rayan SaabNeural Network QuantizationPost-Training Quantization