Neural Network Generalization

Follow this topic Ask about this topic

Momentum

21 papers in the last four weeks, up 600% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 204

All topics

Pointwise Generalization in Deep Neural Networks
May 18, 2026Shaojie Li, Yunbei XuNeural Network Generalization Representation Learning
National University of Singapore
Temporal Task Diversity: Inductive Biases Under Non-Stationarity in Synthetic Sequence Modelling
May 18, 2026Afiq Abdillah Effiezal Aswadi, Oliver Britton, Ross Baker +1Neural Network Generalization In-Context Learning
1Independent · University of Oxford
Unveiling Memorization-Generalization Coexistence: A Case Study on Arithmetic Tasks with Label Noise
May 18, 2026Linyu Liu, Pinyan LuNeural Network Generalization Neural Network Memorization
1Taylor Lab, Huawei Technologies Co., Ltd. · 2Key Laboratory of Interdisciplinary Research of Computation and Economics, Shanghai University of Finance and Economics
Grokking as Structural Inference: Transformers Need Bayesian Lottery Tickets
May 15, 2026Kai Hidajat, Solden Stoll, Joseph AnNeural Network Generalization Transformer
Department of Computer Science University of Washington Seattle, WA 98195
Characterizing Learning in Deep Neural Networks using Tractable Algorithmic Complexity Analysis
May 15, 2026Pedram Bakhtiarifard, Sophia N. Wilson, Mahmoud Afifi +2Neural Network Generalization Neural Network Training Dynamics
Department of Computer Science, University of Copenhagen, Denmark · The American University in Cairo, Egypt
Learning with Shallow Neural Networks on Cluster-Structured Features
May 14, 2026Elisabetta Cornacchia, Laurent MassouliéShallow Neural Networks Neural Network Generalization
INRIA, DI/ENS, PSL.
Slower Generalization, Faster Memorization: A Sweet Spot in Algorithmic Learning
May 14, 2026Shin So, Kyelim Lee, Albert NoNeural Network Generalization Neural Network Training Dynamics
Yonsei University
Rethinking Generalization in Graph Neural Networks: A Structural Complexity Perspective
May 13, 2026Peiyao Wang, Liang Bai, Xian Yang +2Neural Network Generalization Graph Neural Networks
Institute of Intelligent Information Processing Shanxi University, Taiyuan, 030006, China · Alliance Manchester Business School University of Manchester, Manchester, M13 9PL, UK · Department of Mathematics Hong Kong Baptist University, Hong Kong SAR, China
MPINeuralODE: Multiple-Initial-Condition Physics-Informed Neural ODEs for Globally Consistent Dynamical System Learning
May 13, 2026Lake Yang, Antonio Malpica-Morales, Frank Ioannis Papadakis Wood +1Neural Network Generalization System Identification
Department of Chemical Engineering, Imperial College London, London SW7 2AZ, United Kingdom
Unified generalization analysis for physics informed neural networks
May 13, 2026Yuka Hashimoto, Tomoharu IwataNeural Network Generalization Physics-Informed ML
1. NTT, Inc. · 2. RIKEN AIP
A Survey on Data-Dependent Worst-Case Generalization Bounds
May 13, 2026Hubert Leroux, Jean Marcus, Julien RogerNeural Network Generalization PAC-Bayesian Generalization Bounds
´Ecole polytechnique Palaiseau, France
Understanding Generalization through Decision Pattern Shift
May 13, 2026Huiqi Deng, Yibo Li, Quanshi Zhang +3Neural Network Generalization Neural Network Interpretability
Xi’an Jiaotong University · Shanghai Jiao Tong University · Shanghai Artificial Intelligence Laboratory
The Mechanism of Weak-to-Strong Generalization: Feature Elicitation from Latent Knowledge
May 13, 2026Ryoya Awano, Taiji SuzukiNeural Network Generalization Representation Learning
University of Tokyo · Center for Advanced Intelligence Project, RIKEN
Detecting overfitting in Neural Networks during long-horizon grokking using Random Matrix Theory
May 12, 2026Hari K. Prakash, Charles H MartinNeural Network Generalization Neural Network Training Dynamics
University of California San Diego Data Science and Engineering · Calculation Consulting San Francisco, CA, USA
OUI as a Structural Observable: Towards an Activation-Centric View of Neural Network Training
May 12, 2026Alberto Fernández-Hernández, Jose I. Mestre, Cristian Pérez-Corral +3Neural Network Generalization Neural Network Training Dynamics
Universitat Politècnica de València, Valencia, Spain · Universitat Jaume I, Castelló de la Plana, Spain · Openchip & Software Technologies S.L., Spain
Multi-Narrow Transformation as a Single-Model Ensemble: Boundary Conditions, Mechanisms, and Failure Modes
May 12, 2026Tatsuhito Hasegawa, Taisei TanakaNeural Network Generalization Ensemble Learning
Graduate School of Engineering, University of Fukui, Bunkyo, Fukui, 9108507, Fukui, Japan
Sharp feature-learning transitions and Bayes-optimal neural scaling laws in extensive-width networks
May 11, 2026Minh-Toan Nguyen, Jean BarbierNeural Network Generalization Hierarchical Representation Learning
ICTP Trieste, Italy
Characterizing the Generalization Error of Random Feature Regression with Arbitrary Data-Augmentation
May 11, 2026Lucas Morisset, Alain Durmus, Adrien HardyNeural Network Generalization Data Augmentation
Qube Research & Technologies · École Polytechnique
Fix the Loss, Not the Radius: Rethinking the Adversarial Perturbation of Sharpness-Aware Minimization
May 11, 2026Jinping Wang, Qinhan Liu, Zhiwu Xie +1Neural Network Generalization Sharpness-Aware Minimization
Wenzhou-Kean University · International Frontier Interdisciplinary Research Institute, Wenzhou-Kean University
Model Capacity Determines Grokking through Competing Memorisation and Generalisation Speeds
May 10, 2026Yiding Song, Hanming YeNeural Network Generalization Neural Network Memorization
Harvard College Cambridge, MA 02138
Improving Generalization by Permutation Routing Across Model Copies
May 10, 2026Shuhei Kashiwamura, Timothee LeleuNeural Network Generalization
NTT Research, CA, USA · Stanford University, CA, USA
Learning Theory of Transformers: Local-to-Global Approximation via Softmax Partition of Unity
May 9, 2026Zhongjie Shi, Wenjing LiaoNeural Network Generalization Transformer
School of Mathematics, Georgia Institute of Technology, Atlanta, GA 30332, United States
The Propagation Field: A Geometric Substrate Theory of Deep Learning
May 8, 2026Xingrui GuNeural Network Generalization Neural Representation Geometry
1EECS, University of California, Berkeley, Berkeley, CA.
A Deep Risk Estimator for Known Operator Learning
May 8, 2026Andreas Maier, Md Hasan, Paulina Conrad +1Neural Network Generalization Neural Network Approximation Theory
Pattern Recognition Lab · Friedrich-Alexander-Universität Erlangen-Nürnberg · Erlangen, Germany
A Qualitative Test-Risk Mechanism for Scaling Behavior in Normalized Residual Networks
May 8, 2026Daning Cheng, Zeyu Liu, Jun Sun +4Neural Network Generalization Residual Learning
ICT · Zhejiang Lab · JD corporation +1
Every Feedforward Neural Network Definable in an o-Minimal Structure Has Finite Sample Complexity
May 8, 2026Anastasis Kratsios, Gregory Cousins, Haitz Sáez de Ocáriz Borde +2Neural Network Generalization PAC Learning
Department of Mathematics & Statistics, McMaster University, Hamilton, ON, Canada · University of Cambridge, United Kingdom · Graduate School of Engineering, The University of Tokyo, Tokyo, Japan +1
No Triangulation Without Representation: Generalization in Topological Deep Learning
May 7, 2026Johannes S. Schmidt, Martin Carrasco, Ernst Röell +3Neural Network Generalization Graph Neural Networks
Dept. of Math. & Statistics University of Montréal Montréal, Canada · Dept. of Informatics University of Fribourg Fribourg, Switzerland
Topological Signatures of Grokking
May 7, 2026Yifan Tang, Qiquan Wang, Inés García-Redondo +1Neural Network Generalization Neural Representation Geometry
Imperial College London · Queen Mary University of London · University of Fribourg
Do Neural Operators Forget Geometry? The Forgetting Hypothesis in Deep Operator Learning
May 7, 2026Yanming Xia, Angelica I. Aviles-RiveroNeural Network Generalization Neural Operators
Qiuzhen College, Tsinghua University, Beijing, China · Yau Mathematical Sciences Center, Tsinghua University, Beijing, China.
On the Blessing of Pre-training in Weak-to-Strong Generalization
May 7, 2026Wei Yao, Wang Zhaoyang, Gengze Xu +5Language Model Pretraining Neural Network Generalization
Renmin University of China · Shanghai Jiao Tong University · Tongji University +1

May 18, 2026, cs.LGJ/K move · Enter open · S save

Pointwise Generalization in Deep Neural Networks

Shaojie Li, Yunbei Xu

National University of Singapore

Neural Network Generalization Representation Learning Generalization Bounds

Open paper AskPDF arXiv

We address the fundamental question of why deep neural networks generalize by establishing a pointwise generalization theory for fully connected networks. This framework resolves long-standing barriers to characterizing the rich nonlinear feature-learning regime and builds a new statistical foundation for representation learning. For each trained model, we characterize the hypothesis via a pointwise Riemannian Dimension, derived from the eigenvalues of the learned feature representations across layers. This establishes a principled framework for deriving hypothesis-dependent, representation-aware generalization bounds. These bounds offer a systematic upgrade over approaches based on model size, products of norms, and infinite-width linearizations, yielding guarantees that are orders of magnitude tighter in both theory and experiment. Analytically, we identify the structural properties and mathematical principles that explain the tractability of deep networks. Empirically, the pointwise Riemannian Dimension exhibits substantial feature compression, decreases with increased over-parameterization, and captures the implicit bias of optimizers. Taken together, our results indicate that deep networks are mathematically tractable in practical regimes and that their generalization is sharply explained by pointwise, feature-spectrum-aware complexity.

Momentum

Related topics

Latest papers 204