Neural Network Generalization

Overview

The field studies how neural networks move beyond fitting their training examples, including when overparameterized models can memorize data yet still perform well on new examples or shifted distributions. Theoretical approaches use Rademacher complexity, PAC-Bayes bounds, algorithmic stability, kernel and mean-field analyses, and high-dimensional statistics to relate model structure and training to test error. Empirical work examines implicit regularization from optimizers and architectures, representation learning, double descent, grokking, and robustness across settings such as images, language, graphs, and scientific models. A central open tension is that bounds and measures such as flatness do not reliably predict performance in every regime, which depends strongly on data structure, learned features, and distribution shift.

Latest papers

CardsList

325 papers15 papers in the last four weeks, down 64% on the four weeks before. 0.1% of all new papers.

  1. Decomposing SGD Dynamics in Neural Networks: Teacher-Induced Spikes and Variance Inflation

    Nov 18, 2026Samet Demir, Zafer DoganNeural Network GeneralizationNeural Network Training Dynamics

  2. Generalization Bounds for Neural Networks with Sparse Connectivity

    Nov 14, 2026Nong Minh Hieu, Antoine LedentNeural Network GeneralizationGeneralization Bounds

  3. Data-Free Metrics Don't Work: The Missing Invariance Under Functionally Preserving Reparameterisations

    Nov 11, 2026Gabryel Mason-Williams, Israel Mason-Williams, Fredrik DahlqvistRandom Matrix TheoryNeural Network Interpretability

  4. Sharp Covering-Number Bounds for Feedforward ReLU Networks via Parameter Symmetries

    Nov 11, 2026Ruizhi Deng, Guohao Shen, Tangrui XiongReLU Neural NetworksNeural Network Generalization

  5. Achieving Robust Channel Estimation Neural Networks on Doppler effects by Designed Training Data

    Nov 10, 2026Dianxin Luan, John ThompsonChannel EstimationNeural Network Generalization

  6. Deep Learning Theory through the Lens of Statistical Physics: A Topical Review

    Nov 6, 2026Wei Huang, Youjin DengStatistical Physics of LearningNeural Network Generalization

  7. Singularity-Induced Local Complexity for High-Dimensional Learning Models

    Nov 2, 2026Atsuyoshi Muta, Masaaki ImaizumiSingular Learning TheoryNeural Network Generalization

  8. Sharp Generalization for Nonparametric Regression in Interpolation Space by Shallow Over-Parameterized Neural Networks with Channel Attention

    Nov 1, 2026Yingzhen Yang, Ping LiNeural Network GeneralizationNonparametric Regression

  9. Formalizing the Intrinsic Dimension of Objective Landscapes by a Generalized Notion of Co-dimension

    Oct 29, 2026Ernesto Araya, Gitta Kutyniok, Weilin Li +2Intrinsic DimensionalityLoss Landscape Geometry

  10. Mapping the Solution-Set Structure of Grokking on Latin Squares

    Oct 26, 2026Zihan Gu, Lulu Zhao, Tianyi Ma +4GrokkingNeural Network Generalization

  11. Generalization in Learning: Eyring Formula and Predator-prey Model

    Oct 25, 2026S. V. Kozyrev, I. A. Lopatin, Alexander PechenNeural Network GeneralizationStatistical Physics of Learning

  12. Lower expected Fisher rank implies better PAC generalization

    Oct 23, 2026Marco Miani, Albert Kjøller Jacobsen, Albert Lopez i Serrano +2Neural Network GeneralizationPAC-Bayesian Generalization Bounds

  13. Characterizing Cross-Relay Generalization Failure in Machine Learning-Based Power System Protection

    Oct 20, 2026Nithin Pradeep Nadayil Kizhakkethil, Julian Oelhaf, Christian Bergler +2Neural Network GeneralizationPower System Fault Diagnosis

  14. Neural Expansion: A Unified Mechanism for How Deep Neural Networks Generalize

    Oct 16, 2026Chashi Mahiul Islam, Samuel Jacob Chacko, Mao Nishino +2Neural Network GeneralizationOverparameterized Neural Network

  15. The Sign Code: The Hidden Binary Nature of Deep Networks

    Oct 16, 2026Niclas Göring, Shuofeng Zhang, Roi Holtzman +2Neural Network GeneralizationRepresentation Learning

  16. Flat Loss, Evolving Features: Grokking in Modular Addition

    Oct 15, 2026Akash Kumar, Karthik SankaranarayananNeural Network Training DynamicsNeural Network Generalization

  17. BridgeAfford: Bridging Query-Target Gap for Generalizable 3D Affordance Grounding

    Oct 15, 2026Yijie Su, Yakun Zhu, Yi Bin +53D Affordance Grounding3D Visual Grounding

  18. Kolmogorov-Arnold Networks: Curse of Dimensionality and Structural Bias

    Oct 15, 2026Yunhao Ni, Yuxin Guo, Jie Luo +2Curse of DimensionalityKolmogorov-Arnold Networks

  19. Does Compression Imply Generalization? A Minimum Description Length Perspective

    Oct 15, 2026Lincen Yang, Jiayang Shi, Nan Pu +4Neural Network GeneralizationGeneralization Bounds

  20. Denoising Score Matching in Diffusion Models: The Effect of Rank Constraints on Generalization

    Oct 14, 2026Anand Jerry George, Nicolas MacrisScore-Based Generative ModelingNeural Network Generalization

  21. One Size Does Not Fit All: Curvature-Aware Adaptive Perturbation for SAM

    Oct 14, 2026Alessia Rossi, Felipe Campelo, Gabriel L. OliveiraCurvature-Aware OptimizationSharpness-Aware Minimization

  22. Statistical Vignettes on the Grokking Phenomenon

    Oct 13, 2026Baye Malick Gning, Elvis DohmatobGrokkingNeural Network Training Dynamics

  23. Tiny Recursive Message Passing

    Oct 13, 2026Tommaso Salvatori, Matteo Forasassi, Alexis Toumi +1Neural Algorithmic ReasoningMessage Passing Neural Networks

  24. Compression Through Time: An Interplay of Separability, Stationarity and Capacity

    Oct 12, 2026Nikolaus Kopp, Franz Pernkopf, Bernhard GeigerNeural Network GeneralizationRecurrent Neural Networks

  25. Fisher information and the geometry of memorization in neural networks

    Oct 12, 2026Daniel T. Bernstein, Chase Goddard, Luca Di Carlo +2Neural Network MemorizationNeural Network Generalization