Overview

Implicit regularization asks how gradient descent or stochastic gradient descent, learning rates, initialization, architecture, and training noise select particular solutions among many that fit the data, shaping generalization. Researchers study these effects with gradient-flow and stability analyses, minimum-norm and max-margin characterizations, spectral dynamics, and experiments on early stopping, batch size, pruning, and data sampling. The resulting biases can resemble explicit L2, sparsity, or spectral penalties, but their effects depend on data geometry and model regime: for example, minibatch noise may favor flatter solutions, while continued training or approximate optimizer updates can change or worsen generalization. A central challenge is predicting when a particular bias helps and turning theoretical insights into reliable guidance for settings such as image classification, language-model training, and matrix factorization.

Latest papers

CardsList

52 papers5 papers in the last four weeks, down 29% on the four weeks before. 0.0% of all new papers.

  1. Taming Neural Fields with a Frequency-Lipschitz Curriculum for Sparse-View CT Reconstruction

    Nov 11, 2026Songtao Hu, Jingrui Wu, Hong Ding +3Sparse-View CT ReconstructionImplicit Neural Representations

  2. Regularization Amplifies Subliminal Learning

    Nov 7, 2026Lawrence Feng, Jacob Mitchell Springer, Gaurav R. Ghosal +2Subliminal LearningImplicit Regularization

  3. The Prior-Force Family of Implicit Denoiser Priors and Its Decisions in Linear Inverse Problems

    Oct 30, 2026Weijie Liang, Chuling Wen, Jian LuLinear Inverse ProblemsImplicit Regularization

  4. Implicit Bias of (Stochastic) Gradient Descent on Deep Linear Diagonal Networks

    Oct 29, 2026Wiebke Bartolomaeus, Holger RauhutImplicit BiasImplicit Regularization

  5. Optimal Explicit and Implicit Regularization Strengths in High-Dimensional Continual Linear Regression

    Oct 14, 2026Gilad Karpel, Daniel SoudryHigh-Dimensional Linear RegressionRegularization-Based Continual Learning

  6. From Spectral to Frobenius: When Muon Loses Its Spectral Implicit Bias

    Oct 14, 2026Yongtao Wu, Mahdi Mansouri, Zhenyu ZhuMuon OptimizerImplicit Regularization

  7. A Broader Look at Model Merging: Rethinking Implicit Regularization Induced by Task Arithmetic

    Oct 6, 2026Sin-Han Yang, Shih-Cheng Huang, Chieh-Yen Lin +3Task Vector MergingMulti-Task Optimization

  8. Mind the Drift: Diagonal Linear Networks Under Large Learning Rates

    Oct 5, 2026Aniket Sanyal, Tom Jacobs, Rebekka BurkholzEdge of StabilityDeep Learning Optimization

  9. Grokking through the Lens of Minimum-Norm Interpolation

    Sep 29, 2026Gil Kur, Ileana Rugina, Clémentine Carla Juliette Dominé +1Statistical Learning TheoryImplicit Regularization

  10. Overfitting of Spectral Gradient Descent: How Matrix Geometry shapes Generalization and Implicit Bias

    Sep 26, 2026Guillaume Braun, Ichiro Hashimoto, Masaaki ImaizumiGradient DescentNeural Network Generalization

  11. Minimal-Norm Univariate Two-Layer ReLU Classification: Exact Solutions and Global Optimality with Skip Connections

    Sep 23, 2026Karolina Drabik, Ben Lewis, Antoni Puch +4ReLU Neural NetworksNeural Network Optimization

  12. A Function-Space Approach to the Statistical Mechanics of Learning Dynamics

    Sep 9, 2026Yizhou Zhang, Weichen Wu, Lun Du +1Statistical Physics of LearningRepresentation Learning

  13. How Much Regularization Survives Averaging? Update Masking in Federated Learning

    Aug 24, 2026Wenhao Yan, Fu Kuroda, Yucheng Jin +1Federated LearningImplicit Regularization

  14. As-Rigid-As-Possible Regularization for Implicit Surfaces

    Aug 16, 2026Tobias Djuren, Markus Worchel, Ugo Finnendahl +1Implicit RegularizationNeural Representation Geometry

  15. Non-asymptotic implicit bias of logistic regression at early-stage gradient descent dynamics

    Aug 5, 2026Han BaoLogistic RegressionImplicit Bias

  16. Quotient Dynamics, Effective Curvature, and Implicit Bias in Positive Quadratic Networks

    Jul 28, 2026Pengcheng ChengRiemannian OptimizationImplicit Regularization

  17. Beyond Negative-Ridge Endpoints: Mixed-Sign Spectral Regularization via Negative-Shifted Gradient Descent

    Jul 24, 2026Peng ZhaoSpectral RegularizationEarly Stopping

  18. FAIR: Feature-Augmented Implicit Regularization for AI-generated Fake Image Detection

    Jul 24, 2026Md Redwanul Haque, Manzur Murshed, Manoranjan Paul +1Domain GeneralizationAI-Generated Image Detection

  19. PN-QNN: Harnessing Physical Noise as a Native Regularizer in Photonic Hybrid Quantum Neural Networks

    Jul 22, 2026Farah Elnakhal, Alberto Marchisio, Nouhaila Innan +2Hybrid Quantum-Classical MLPhotonic Computing

  20. Stability Annealing Selects the Implicit Bias of Smoothed Sign Descent: A Rate-Indexed Barrier Path on Separable Data

    Jul 7, 2026Xiangwu Wang, Chengwei Cao, Yicheng Song +2Sign-Based OptimizationImplicit Regularization

  21. The Spectral Dynamics and Noise Geometry of Muon

    Jun 7, 2026Pierfrancesco Beneventano, Mahmoud Abdelmoneum, Tomaso PoggioMuon OptimizerMatrix Optimization

  22. Closed-Form Spectral Regularization for Multi-Task Model Merging

    Jun 5, 2026Yongxian Wei, Runxi Cheng, Xingxuan Zhang +4Spectral RegularizationMulti-Task Learning

  23. Implicit Fuzzification via Bounded Noise Injection for Robust Medical Image Segmentation

    Jun 3, 2026Bisheng Tang, Zhangfeng Ma, Chuchu Zhai +4Ultrasound Image SegmentationMedical Image Segmentation

  24. Fast Generalization after Interpolation via Critically Damped Momentum Optimization

    Jun 1, 2026Luca Muscarnera, Silas Ruhrberg Estévez, Yuanzhang Xiao +1Neural Network GeneralizationMomentum Methods

  25. Stochastic Rounding Increases Small Singular Values

    May 29, 2026Linkai Ma, Tingzhou Yu, Petros DrineasSpectral RegularizationStochastic Rounding

  26. Improving Adversarial Robustness of Attribution via Implicit Regularization

    May 28, 2026Amir Mehrpanah, Matteo Gamba, Hossein AzizpourTransformer InterpretabilityNeural Network Robustness

  27. Implicit Regularization in Perturbed Deep Matrix Factorization: Spectral Conditions and Stability

    May 27, 2026Jingzhe Wang, Hung-Hsu ChouAlgorithmic StabilityLow-Rank Matrix Decomposition