Benign Overfitting

Ask about this topic

Overview

Benign overfitting asks when a model that interpolates every training example, including noisy labels, can still achieve low test error. Much of the theory studies minimum-norm linear regression and maximum-margin classifiers, using covariance spectra, signal-to-noise structure, and effective dimension to characterize risk; related work examines kernels, alternative norms, and neural networks. These analyses show that generalization depends not just on model size but on the geometry of the data, the alignment of signal and noise, and the learning algorithm’s implicit bias. A central tension is that interpolation can generalize under suitable conditions, while other settings—especially some generative models—may require explicit regularization or avoid harmful overfitting through different mechanisms.

Latest papers

CardsList

21 papers2 papers in the last four weeks, against 2 the four weeks before. 0.0% of all new papers.

  1. "Deep lifts": Neural networks can generalize even with the dumbest optimizer

    Oct 16, 2026Sunil Madhow, Yu-Xiang WangNeural Network GeneralizationGeneralization Bounds

    UC San Diego

  2. Benign Overfitting under Heterogeneous Input Fusion

    Oct 7, 2026Houzhen Liu, Xiaobo XiaBenign OverfittingStatistical Learning Theory

    University of Science and Technology of China

  3. Benign Overfitting for General Norms and Distributions

    Sep 27, 2026Daniel Barzilai, Ohad ShamirBenign OverfittingStatistical Learning Theory

    Weizmann Institute of Science · University of Toronto & Vector Institute

  4. Double Descent and Malign Overfitting in Diffusion Models

    Sep 22, 2026Raphaël Urfin, Tony Bonnaire, Giulio Biroli +1Double DescentBenign Overfitting

    Laboratoire de Physique de l’École normale supérieure, ENS, Université PSL, CNRS, Sorbonne Université, Université Paris Cité, F-75005 Paris, France · Université Paris-Saclay, CNRS, Institut d’Astrophysique Spatiale, 91405 Orsay, France · Department of Computing Sciences, Bocconi University, Milano, Italy

  5. Generalization, memorization, and overfitting for diffusion models trained in the lazy high-dimensional regime

    Aug 25, 2026Hugo Latourelle-Vigeant, Sinho Chewi, Aram-Alexandre Pooladian +2Generative ModelingScore-Based Generative Modeling

    Department of Statistics and Data Science, Yale University · Institute for Foundations of Data Science, Yale University · Department of Applied Physics, Yale University

  6. (Mis)Understanding Benign Overfitting in Equity Return Prediction

    Aug 24, 2026Hui Guo, Jiawei Huang, Runze Li +1Benign OverfittingQuantitative Finance

    Carl H. Lindner College of Business, University of Cincinnati · Católica Lisbon School of Business and Economics, Universidade Católica Portuguesa · Department of Statistics, Pennsylvania State University

  7. High-dimensional ridgeless least squares interpolation under spiked covariance structures

    Aug 7, 2026Zhijun Liu, Dandan JiangDouble DescentBenign Overfitting

    College of Sciences, Northeastern University, China. · School of Mathematics and Statistics, Xi’an Jiao Tong University, China.

  8. Benign interpolation and Occam's razor

    Aug 4, 2026Tom F. Sterkenburg, Daniel A. Herrmann, Jan-Willem RomeijnBenign OverfittingInductive Bias

  9. The Fourth Quadrant: A Stylized View of Benign Misfitting

    Aug 2, 2026Gireeja Ranade, Anant SahaiBenign OverfittingLinear Regression

    University of California, Berkeley

  10. Benign Overfitting Does Not Occur in Diffusion Models

    Jul 2, 2026Tyler Farghly, Benjamin Dupuis, Alain Durmus +1Double DescentNeural Network Generalization

    1INRIA - École Normale Supérieure - PSL Research University - CNRS, France · 2 École Polytechnique - CMAP - IP Paris, France

  11. How abundant are good interpolators?

    Jun 4, 2026August Y. Chen, Ahmed El AlaouiLinear ClassificationBenign Overfitting

    Cornell University, Department of Computer Science. Ithaca, NY, USA. · Cornell University, Department of Statistics and Data Science. Ithaca, NY, USA.

  12. When Does ℓ2\ell_2-Boosting Overfit Benignly? High-Dimensional Risk Asymptotics and the ℓ1\ell_1 Implicit Bias

    May 7, 2026Ye Su, Jian Li, Yong LiuBenign Overfitting

    Shenzhen Institutes of Advanced Technology Chinese Academy of Sciences Shenzhen 518055, China · School of Artificial Intelligence Beijing Normal University Beijing, 100875, China · Gaoling School of Artificial Intelligence Renmin University of China Beijing 100872, China

  13. A Theory of Generalization in Deep Learning

    May 2, 2026Elon Litman, Gabe GuoNeural Network GeneralizationBenign Overfitting

    Stanford University

  14. Investigation into In-Context Learning Capabilities of Transformers

    Apr 28, 2026Rushil Chandrupatla, Leo Bangayan, Sebastian LengNeural Network GeneralizationTransformer

  15. Learning Curves and Benign Overfitting of Spectral Algorithms in Large Dimensions

    Apr 25, 2026Weihao Lu, Qian Lin, Yingcun Xia +1Kernel Ridge RegressionBenign Overfitting

    Department of Statistics and Data Science, National University of Singapore. · Department of Statistics and Data Science, Tsinghua University.

  16. Benign Overfitting in Adversarial Training for Vision Transformers

    Apr 21, 2026Jiaming Zhang, Meng Ding, Shaopeng Fu +2Adversarial TrainingNeural Network Generalization

    Division of CEMSE, King Abdullah University of Science and Technology · School of Statistics, Renmin University of China · Department of Computer Science and Engineering, State University of New York at Buffalo +1

  17. A Transport Geometry of Benign Overfitting

    Apr 9, 2026Olaf Yunus Laitinen Imanov, Rana Irem Turhan, Isgandar JafarliBenign OverfittingStatistical Learning Theory

    School of ICT, Metropolia University of Applied Sciences, Myllypurontie 1, 00920, Helsinki, Finland

  18. Towards Initialization-dependent and Non-vacuous Generalization Bounds for Overparameterized Shallow Neural Networks

    Apr 1, 2026Yunwen Lei, Yufeng XieGeneralization BoundsOverparameterized Neural Network

    Department of Mathematics, The University of Hong Kong

  19. Benign Overfitting with Quantum Kernels

    Mar 21, 2025Joachim Tomasi, Sandrine Anthoine, Hachem KadriBenign OverfittingQuantum Machine Learning

    Aix-Marseille Univ, CNRS, LIS, Marseille, France. · Aix Marseille Univ, CNRS, I2M, Marseille, France.

  20. Universality of Benign Overfitting in Binary Linear Classification

    Jan 17, 2025Ichiro Hashimoto, Stanislav Volgushev, Piotr ZwiernikNeural Network GeneralizationLinear Classification

    Department of Statistical Sciences, University of Toronto

  21. A Farewell to the Bias-Variance Tradeoff? An Overview of the Theory of Overparameterized Machine Learning

    Sep 6, 2021Yehuda Dar, Vidya Muthukumar, Richard G. BaraniukDouble DescentNeural Network Generalization

    Faculty of Computer and Information Science, Ben-Gurion University · Schools of Electrical and Computer Engineering, and Industrial and Systems Engineering, Georgia Institute of Technology · Electrical and Computer Engineering Department, Rice University