Overview

Grokking studies why a model can fit its training data while performing poorly on new examples, then later generalize, often on tasks such as modular arithmetic, parity, and formal-language recognition. Researchers explain this delay through theories of weight decay, parameter norms, optimization noise, and competing memorization and generalization timescales, alongside analyses of learned circuits and representations using Fourier features, symmetry, and spectral geometry. Experiments also test ways to predict or control the transition, including geometric diagnostics, regularization, architectural changes, and structure-specific priors. A central open question is whether these findings reflect one general mechanism or several task- and optimizer-dependent processes; signals that predict grokking do not always establish its cause.

Latest papers

CardsList

83 papers7 papers in the last four weeks, level with the four weeks before. 0.0% of all new papers.

  1. Deep Neural Networks Training Efficiency: Does Grokking Modify its Dynamics?

    Nov 13, 2026Eduardo Cueto-Mendoza, John D. KelleherGrokkingNeural Network Training Dynamics

  2. Dissecting Geometric Constraints in Grokking: Interaction-Dependent Acceleration in Modular Arithmetic

    Nov 2, 2026Zachary A. Hampton, Wasiq Khan, Bilal KhanGrokkingNeural Representation Geometry

  3. Scaling and Renormalization in Neural Networks: An Effective Renormalization-Group Description of Grokking

    Oct 26, 2026Min Chen, Junqi Wang, Zeguan Wu +2Renormalization GroupGrokking

  4. Mapping the Solution-Set Structure of Grokking on Latin Squares

    Oct 26, 2026Zihan Gu, Lulu Zhao, Tianyi Ma +4GrokkingNeural Network Generalization

  5. Generalization in Learning: Eyring Formula and Predator-prey Model

    Oct 25, 2026S. V. Kozyrev, I. A. Lopatin, Alexander PechenNeural Network GeneralizationStatistical Physics of Learning

  6. Flat Loss, Evolving Features: Grokking in Modular Addition

    Oct 15, 2026Akash Kumar, Karthik SankaranarayananNeural Network Training DynamicsNeural Network Generalization

  7. Statistical Vignettes on the Grokking Phenomenon

    Oct 13, 2026Baye Malick Gning, Elvis DohmatobGrokkingNeural Network Training Dynamics

  8. Misalignment of Low-Loss Regions Causes Grokking

    Sep 30, 2026Yongding Tian, Zaid Al-Ars, Maksim Kitsak +1Loss Landscape GeometryGrokking

  9. Grokking through the Lens of Minimum-Norm Interpolation

    Sep 29, 2026Gil Kur, Ileana Rugina, Clémentine Carla Juliette Dominé +1Statistical Learning TheoryImplicit Regularization

  10. A Spectral Theory of Grokking: Weight Decay induces Feature Learning

    Sep 22, 2026Lenz Pracher, Pascal de Jong, Oskar Lieshaus +2Neural Network GeneralizationWeight Decay

  11. An Exploratory Replica-Overlap Probe of the Grokking Transition

    Sep 22, 2026A. C. Opus, J. Q. LuStatistical Physics of LearningNeural Network Training Dynamics

  12. Beyond Embedding Transfer: Architectural Interfaces, Iso-Parameter Scaling, and Preconditioner-Governed Consolidation in Grokking

    Sep 16, 2026Zeyu JiaTransfer LearningAlgorithmic Reasoning

  13. Structured Features Overfit Where Random Features Grok

    Sep 14, 2026Chon-Fai Kam, Miloud Bessafi, Frederic CadetNeural Network GeneralizationRidge Regression

  14. Biologically Inspired Mechanisms for Facilitating Grokking in Multilayer Perceptrons

    Aug 28, 2026Florin LeonGrokkingNeural Network Generalization

  15. Mapping the Emergence of Regularization-Driven Dynamics in Grokking

    Aug 26, 2026Yiming Lin, Yuxuan WangNeural Network GeneralizationNeural Network Training Dynamics

  16. Is Grokking a Loss of Normal Hyperbolicity of the Interpolation Manifold?

    Aug 14, 2026Suvinava BasakGrokkingNeural Network Training Dynamics

  17. Algorithmic Information Dynamics of Learning: A Certified, Differentiable Complexity Controller for Grokking

    Aug 14, 2026Luan Ozelim, Abicumaran Uthamacumaran, Hector ZenilNeural Network Training DynamicsGrokking

  18. Post-Grokking Collapse at the Representation-Readout Interface in Muon-Trained Transformers

    Aug 7, 2026Ali Janati, Kaoutar El Maghraoui, Anass BelfatmiTransformerMuon Optimizer

  19. Tunneling the Loss Landscape: Bypassing Memorization with Monte Carlo Parameter Swapping

    Aug 3, 2026Lai Shun Chan, Xiaotian Zhang, Yue Shang +2Neural Network GeneralizationMarkov Chain Monte Carlo

  20. Grokking on the Weight-Decay Clock: A Rate Hierarchy from Softly Broken Symmetries

    Jul 27, 2026Taeyoung KimNeural Network GeneralizationWeight Decay

  21. Tensor Network Machine Learning for Wildfire Susceptibility Mapping: from Grokking Dynamics to Quantum Mixedness of Class Representations

    Jul 21, 2026Domenico Pomarico, Alessandra Costantino, Gabriel Ramirez Sanchez +12Tensor NetworksNeural Network Interpretability

  22. Algebraic Representability as the Limiting Regime of Grokking: An Exactly Solvable Model with Holomorphic Activations

    Jul 15, 2026Chon-Fai Kam, Xavier Cadet, Miloud Bessafi +1Feedforward Neural NetworksNeural Network Expressivity