Neural Network Generalization

Ask about this topic

Overview

The field studies how neural networks move beyond fitting their training examples, including when overparameterized models can memorize data yet still perform well on new examples or shifted distributions. Theoretical approaches use Rademacher complexity, PAC-Bayes bounds, algorithmic stability, kernel and mean-field analyses, and high-dimensional statistics to relate model structure and training to test error. Empirical work examines implicit regularization from optimizers and architectures, representation learning, double descent, grokking, and robustness across settings such as images, language, graphs, and scientific models. A central open tension is that bounds and measures such as flatness do not reliably predict performance in every regime, which depends strongly on data structure, learned features, and distribution shift.

Latest papers

CardsList

311 papers35 papers in the last four weeks, up 35% on the four weeks before. 0.2% of all new papers.

  1. Grokking Finite-Dimensional Algebra

    Feb 23, 2026Pascal Jr Tikeng Notsawo, Guillaume Dumas, Guillaume RabusseauGrokkingNeural Network Generalization

    Université de Montréal · Mila, Montréal · CHU Sainte-Justine Research Center, Montréal +1

  2. Mapping Networks

    Feb 22, 2026Lord Sen, Shyamapada MukherjeeWeight-Space LearningNeural Network Compression

    National Institute of Technology Rourkela, Odisha, India

  3. Early-Warning Signals of Grokking via Loss-Landscape Geometry

    Feb 19, 2026Yongzhong XuGrokkingNeural Network Generalization

  4. On the Mechanism and Dynamics of Modular Addition: Fourier Features, Lottery Ticket, and Grokking

    Feb 18, 2026Jianliang He, Leda Wang, Siyu Chen +1Lottery Ticket HypothesisModular Addition

    Department of Statistics and Data Science, Yale University

  5. Conjugate Learning Theory: Uncovering the Mechanisms of Trainability and Generalization in Deep Neural Networks

    Feb 18, 2026Binchuan QiNeural Network GeneralizationNeural Network Optimization

    Tongji University, Siping Street, 200092 Shanghai, China · Zhejiang Yuying College of Vocational Technology, Sihao Street, Hangzhou, 310018, Zhejiang, China

  6. A unified theory of feature learning in RNNs and DNNs

    Feb 17, 2026Jan P. Bauer, Kirsten Fischer, Moritz Helias +1Recurrent Neural NetworksRepresentation Learning

    Gatsby Computational Neuroscience Unit, UCL, London, UK · IAS-6, Forschungszentrum Juelich, Juelich, Germany

  7. BEACONS: Bounded-Error, Algebraically-Composable Neural Solvers for Partial Differential Equations

    Feb 16, 2026Jonathan Gorard, Ammar Hakim, James JunoNeural PDE SolversPartial Differential Equations

    Princeton University Princeton, NJ 08544, USA · Princeton Plasma Physics Laboratory Princeton, NJ 08540, USA

  8. On the Dynamics of Delayed Generalization in Linear Models

    Feb 9, 2026Atreya Vedantam, Nataraj Das, Chandrashekar LakshminarayananLinear ClassificationNeural Network Generalization

    Indian Institute of Technology Madras, Chennai.

  9. Theoretical Analysis of Measure Consistency Regularization for Partially Observed Data

    Feb 1, 2026Yinsong Wang, Shahin ShahrampourNeural Network GeneralizationLearning with Missing Data

    University of Nevada Reno, Reno, NV 89557 USA · Department of Mechanical and Industrial Engineering at Northeastern University, Boston, MA 02115 USA

  10. To Grok Grokking: Provable Grokking in Ridge Regression

    Jan 27, 2026Mingyue Xu, Gal Vardi, Itay SafranNeural Network GeneralizationRidge Regression

    Department of Computer Science, Purdue University, West Lafayette, IN, USA · Department of Computer Science and Applied Mathematics, Weizmann Institute of Science, Israel · Stein Faculty of Computer and Information Science, Ben-Gurion University of the Negev, Israel

  11. The Impact of Anisotropic Covariance Structure on the Training Dynamics and Generalization Error of Linear Networks

    Jan 11, 2026Taishi Watanabe, Ryo Karakida, Jun-nosuke TeramaeNeural Network GeneralizationNeural Network Training Dynamics

    Graduate School of Informatics, Kyoto University · Artificial Intelligence Research Center, AIST · RIKEN AIP

  12. Addressing A Posteriori Performance Degradation in Neural Network Subgrid Stress Models

    Nov 21, 2025Andy Wu, Sanjiva K. LeleNeural Network GeneralizationData Augmentation

    Department of Aeronautics and Astronautics, Stanford University, Stanford, California, USA · Department of Mechanical Engineering, Stanford University, Stanford, California, USA

  13. On the Dimension-Free Approximation of Deep Neural Networks for Symmetric Korobov Functions

    Nov 16, 2025Yulong Lu, Tong Mao, Jinchao Xu +1Neural Network GeneralizationNeural Network Approximation Theory

    School of Mathematics, University of Minnesota, Minneapolis, 55455, MN, USA · Computer, Electrical and Mathematical Science and Engineering Division, King Abdullah University of Science and Technology, Thuwal 23955, Saudi Arabia · School of Mathematics, Georgia Institute of Technology, Atlanta, 30332, GA, USA

  14. DFNN: A Deep Fréchet Neural Network Framework for Learning Metric-Space-Valued Responses

    Oct 20, 2025Kyum Kim, Yaqing Chen, Paromita DubeyNeural Network GeneralizationNeural Network Approximation Theory

    Rutgers University · University of Southern California

  15. Memorizing Long-tail Data Can Help Generalization Through Composition

    Oct 18, 2025Mo Zhou, Haoyang Ma, Rong GeNeural Network GeneralizationNeural Network Memorization

    University of Washington · Duke University

  16. A Function-Centric Perspective on Flat and Sharp Minima

    Oct 14, 2025Israel Mason-Williams, Gabryel Mason-Williams, Helen YannakoudakisFlat MinimaLoss Landscape Geometry

    Department of Informatics, UKRI Safe and Trusted AI, London, United Kingdom · Queen Mary University of London, London, United Kingdom · Department of Informatics, King’s College London, London, United Kingdom

  17. Deep Neural Networks Inspired by Differential Equations

    Oct 9, 2025Yongshuai Liu, Lianfang Wang, Kuilin Qin +6Neural Network GeneralizationDynamic Neural Networks

    School of Mathematical Sciences, Beijing Normal University, Beijing, China · Department of Mathematics, The Chinese University of Hong Kong, Shatin, Hong Kong

  18. PENEX: AdaBoost-Inspired Neural Network Regularization

    Oct 2, 2025Klaus-Rudolf Kladny, Bernhard Schölkopf, Michael MuehlebachNeural Network Generalization

    MPI for Intelligent Systems, Tübingen · Tübingen AI Center · ELLIS Institute Tübingen

  19. Quantifying How Training Gradient Sparsity Affect Spiking Neural Network Accuracy And Robustness

    Sep 28, 2025Nhan Trong Luu, Duong Trung LuuNeural Network GeneralizationNeural Network Robustness

    College of Information and Communication Technology, Can Tho University, Vietnam · College of Engineering and Computer Science, VinUniversity, Hanoi, Vietnam · Center of Digital Transformation and Communication, Can Tho University, Vietnam

  20. Neural Langevin Machine: a local asymmetric learning rule can be creative

    Jun 30, 2025Zhendong Yu, Weizhong Huang, Haiping HuangNeural Network GeneralizationImage Generation

    PMI Lab, School of Physics, Sun Yat-sen University, Guangzhou 510275, People’s Republic of China · Guangdong Provincial Key Laboratory of Magnetoelectric Physics and Devices, Sun Yat-sen University, Guangzhou 510275, People’s Republic of China

  21. GeNeRT: A Physics-Informed Approach to Intelligent Wireless Channel Modeling via Generalizable Neural Ray Tracing

    Jun 23, 2025Kejia Bian, Meixia Tao, Shu Sun +2Neural Network GeneralizationWireless Communications

    School of Information Science and Electronic Engineering, Shanghai Jiao Tong University, Shanghai 200240, China · Shanghai Innovation Institute, Shanghai 200231, China

  22. Interpretability and Generalization Bounds for Learning Spatial Physics

    Jun 18, 2025Alejandro Francisco Queiruga, Theo Gutman-Solo, Shuai JiangNeural Network GeneralizationMechanistic Interpretability

    OpenAI, San Francisco, CA, USA · Google, Mountain View, CA, USA · Sandia National Laboratories, Albuquerque, NM, USA

  23. Random Matrix Theory for Deep Learning: Beyond Eigenvalues of Linear Models

    Jun 16, 2025Zhenyu Liao, Michael W. MahoneyRandom Matrix TheoryNeural Network Generalization

    School of Electronic Information and Communications, Huazhong University of Science and Technology, Wuhan, China · ICSI, LBNL, and Department of Statistics, University of California, Berkeley, USA

  24. Generalization in VAE and Diffusion Models: A Unified Information-Theoretic Analysis

    Jun 1, 2025Qi Chen, Jierui Zhu, Florian ShkurtiVariational AutoencodersNeural Network Generalization

    Department of Computer Science, University of Toronto · Data Science Institute · Vector Institute +2

  25. The Dynamics of Generalization in Deep Learning

    Apr 23, 2025Rubing Yang, Pratik ChaudhariNeural Network GeneralizationNeural Network Training Dynamics

    Department of Applied Mathematics and Computational Sciences, University of Pennsylvania Philadelphia, PA, 19104, USA · Department of Electrical and Systems Engineering, University of Pennsylvania Philadelphia, PA, 19104, USA

  26. Deep learning with missing data

    Apr 21, 2025Tianyi Ma, Tengyao Wang, Richard J. SamworthNeural Network GeneralizationLearning with Missing Data

    Statistical Laboratory, University of Cambridge · Department of Statistics, London School of Economics

  27. Unified Enhancement of the Generalization and Robustness of Language Models via Bi-Stage Optimization

    Mar 19, 2025Yudao Sun, Juan Yin, Juan Zhao +3Adversarial TrainingNeural Network Generalization

    Department of New Networks, Peng Cheng Laboratory, Shenzhen 518055, China · Department of Industrial Systems Engineering and Management, National University of Singapore, Singapore 117576 · School of Data Science and Artificial Intelligence, Chang’an University, Shaanxi, 710064, China

  28. The Mathematical Theory of Double Descent as Shrinkage, not Overfitting: Matching Bounds for Path-Regularized Multilayer Neural Networks

    Mar 3, 2025Hao YuDouble DescentNeural Network Generalization

    Department of Automation, Tsinghua University, Beijing, 100086, China

  29. Networks with Finite VC Dimension: Pro and Contra

    Feb 4, 2025Vera Kurkova, Marcello SanguinetiNeural Network GeneralizationNeural Network Approximation Theory

    Institute of Computer Science of the Czech Academy of Sciences Pod Vodárenskou věží, 2 - 18207 Prague, Czech Republic · DIBRIS University of Genova Via Opera Pia, 13 - 16145 Genova, Italy

  30. Boosting Adversarial Robustness and Generalization with Dictionary Structure

    Feb 2, 2025Zhichao Hou, Weizhi Gao, Hamid Krim +2Neural Network GeneralizationAdversarial Robustness

    Department of Computer Science, North Carolina State University, Raleigh, USA.