Grokking

Ask about this topic

Momentum

4 papers in the last four weeks, against 2 the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 43

All topics
CardsList

  1. Misalignment of Low-Loss Regions Causes Grokking

    Sep 30, 2026Yongding Tian, Zaid Al-Ars, Maksim Kitsak +1GrokkingOverfitting

    Computer Engineering Lab Delft University of Technology Delft, NL · HDL TypeTech Delft, NL · Network and Architecture Service Delft University of Technology Delft, NL +1

  2. Grokking through the Lens of Minimum-Norm Interpolation

    Sep 29, 2026Gil Kur, Ileana Rugina, Clémentine Carla Juliette Dominé +1Stochastic InterpolantsInterpolation

    ETH Zürich · Institute of Science and Technology Austria · Harvard University

  3. A Spectral Theory of Grokking: Weight Decay induces Feature Learning

    Sep 22, 2026Lenz Pracher, Pascal de Jong, Oskar Lieshaus +2Neural Tangent KernelFeature Learning

    Arnold-Sommerfeld-Center for Theoretical Physics, Ludwig-Maximilians-Universität München, Munich, Germany · Department of Applied Physics, Standford University, Standford, CA, USA · Department of Mathematics, University of Cambridge, Cambridge, UK

  4. Quantifying the Memorization-to-Generalization Transition: Scaling Laws and Phase Structure in Grokking

    Sep 12, 2026Anish KatariaGrokkingOverparameterization

    Princeton University, Princeton, NJ, USA.

  5. Mapping the Emergence of Regularization-Driven Dynamics in Grokking

    Aug 26, 2026Yiming Lin, Yuxuan WangOverparameterizationWeight Decay

    School of Artificial Intelligence, University of Chinese Academy of Sciences

  6. Algorithmic Information Dynamics of Learning: A Certified, Differentiable Complexity Controller for Grokking

    Aug 14, 2026Luan Ozelim, Abicumaran Uthamacumaran, Hector ZenilLearning DynamicsGradient-Based Optimization

    Oxford Immune Algorithmics, Oxford University Innovation & London Institute for Healthcare Engineering, U.K · Department of Biomedical Computing, School of Biomedical Engineering and Imaging Sciences & King’s Institute for AI, King’s College London, U.K

  7. Post-Grokking Collapse at the Representation-Readout Interface in Muon-Trained Transformers

    Aug 7, 2026Ali Janati, Kaoutar El Maghraoui, Anass BelfatmiMuonGrokking

    Data Science Institute, Columbia University, New York, NY, USA · Department of Computer Science, Columbia University, New York, NY, USA · CentraleSupélec, Gif-sur-Yvette, France

  8. Tunneling the Loss Landscape: Bypassing Memorization with Monte Carlo Parameter Swapping

    Aug 3, 2026Lai Shun Chan, Xiaotian Zhang, Yue Shang +2GrokkingMemorization

    Department of Physics, City University of Hong Kong, Hong Kong, China · Department of Physics and Astronomy, University of Pennsylvania, Philadelphia, PA, USA · 3Innovation Campus Delaware, Air Liquide, Newark, DE, USA

  9. Grokking on the Weight-Decay Clock: A Rate Hierarchy from Softly Broken Symmetries

    Jul 27, 2026Taeyoung KimGrokkingWeight Decay

    Center for AI and Natural Sciences Korea Institute for Advanced Study Seoul 02455, Republic of Korea

  10. Guiding Language Models to Be More Empathetic: Culturally Sensitive Mental Health Advice Generation Through Human-LLM Collaboration

    Jul 26, 2026Fatema Tuj Johora Faria, Mukaffi Bin Moin, Md. Mahfuzur Rahman +3CounselingEmpathy

    Ahsanullah University of Science and Technology · Bangladesh University of Business and Technology · The University of New South Wales +2

  11. Algebraic Representability as the Limiting Regime of Grokking: An Exactly Solvable Model with Holomorphic Activations

    Jul 15, 2026Chon-Fai Kam, Xavier Cadet, Miloud Bessafi +1GrokkingTwo-Layer Neural Networks

    University Paris City & University of Reunion, Paris, France · Dipartimento di Fisica e Chimica Emilio Segrè, Università degli Studi di Palermo, Palermo, Italy · Thayer School of Engineering, Dartmouth College, Hanover, NH 03755, USA +2

  12. What Makes a Representational Prior Work? Feature Families, Label-Free Invariances, and Critical Windows in Grokking

    Jul 14, 2026Gunner Levi HowePrior Knowledge IntegrationImproved Generalization

  13. How to Tame Grokking: Representation Geometry as a Control Signal

    Jul 13, 2026Maksim A KazanskiiGrokkingNeural Network Representations

    Independent Researcher

  14. At-Grok Is Not Converged:A Measurement-Validity Audit for Grokking Representation Metrics

    Jul 7, 2026Truong Xuan KhanhGrokkingEnergy-Based Transformer

    H&K Research Studio / Clevix LLC, Hanoi, Vietnam

  15. Cross-Trajectory Chimera Interventions Reveal Dissociable Roles of Weight Magnitude and Direction in Grokking

    Jul 7, 2026Truong Xuan KhanhControlled ComparisonCircuits

    H&K Research Studio, Clevix LLC, Hanoi, Vietnam

  16. Grokking Is Conditional and Fragile: A Fully-Tractable, Multi-Seed Study at 12K Parameters

    Jul 6, 2026Yoshiyuki OotaniGrokkingLarge Language Model Training

    Independent Researcher

  17. Structure-Specific Representational Priors Causally Control the Grokking Delay

    Jul 5, 2026Gunner Levi HoweGrokkingRepresentation Learning

  18. A Stochastic--Geometric Theory of Scaling Laws in Grokking

    Jun 29, 2026Róisín Luo, Christian Gagné, Jonas Ngnawé +2GrokkingScaling Laws

    Research Ireland – Centre for Research Training in AI · University of Galway · Universit´e Laval +2

  19. Natural Ungrokking: Asymmetric Control of Which Rules Survive Pretraining

    Jun 24, 2026Juliana Li, Diya SreedharReversePretraining

    Harvard University, Cambridge, MA, USA.

  20. What Does the Weight Norm Control in Grokking? Logit-Scale Mediation under Cross-Entropy

    Jun 16, 2026Truong Xuan KhanhGrokkingWeight Decay

    H&K Research Studio, Clevix LLC · Hanoi, Vietnam

  21. Noise-Driven Escape from Metastable Phases explains Grokking in Deep Neural Networks

    Jun 15, 2026Ibrahim Talha Ersoy, Karoline WiesnerGrokkingMetastable Samples

    Complexity Science Group, Institute of Physics and Astronomy, University of Potsdam, Potsdam, Germany

  22. The Weight Norm Sets the Grokking Timescale: A Causal Delay Law

    Jun 11, 2026Truong Xuan Khanh, Doan Hoang Viet, Luu Duc Trung +1GrokkingWeight Decay

    H&K Research Studio / Clevix LLC, Hanoi, Vietnam · Bac A Bank, Hanoi · Banking Academy of Vietnam

  23. Circuit Synchronization Precedes Generalization: A Causal Precursor to Grokking

    Jun 11, 2026Achyuthan SivasankarGrokkingGeneralisation

    New York University

  24. Deciphering Two Training Clocks in Grokking via Deep Linear Network Theory with Conditional ReLU Reduction

    Jun 4, 2026Hu Tan, Kuo Gai, Shihua ZhangDeep NetworkRectified Linear Unit

    1State Key Laboratory of Mathematical Sciences, Academy of Mathematics and Systems Science, Chinese Academy of Sciences, Beijing 100190, China · School of Mathematical Sciences, University of Chinese Academy of Sciences, Beijing 100049, China · 3Shanghai Institute for Mathematics and Interdisciplinary Sciences (SIMIS), Shanghai, China +1

  25. TD-Grokking: Learning from Zero-Reward Problems by Training-Time Decomposition

    Jun 3, 2026Ningyuan Xi, Hao Xu, Hongsheng Xin +1Training TimeReasoning Skills

    Department of Data Science, City University of Hong Kong. · Hong Kong Institute of AI for Science, City University of Hong Kong. · Li Auto Inc.

  26. Low-Rank Decay for Grokking in Scale-Invariant Transformers: A Spectral-Geometric View

    Jun 3, 2026Mingyu LiLow-Rank StructureWeight Decay

    Beijing Normal University

  27. Fast Generalization after Interpolation via Critically Damped Momentum Optimization

    Jun 1, 2026Luca Muscarnera, Silas Ruhrberg Estévez, Yuanzhang Xiao +1Stochastic InterpolantsGradient Descent

  28. A Pre-Training Analogue of Grokking in Language Models: Tracing Delayed Grammatical Generalization

    May 29, 2026Sherin Muckatira, Namrata Shivagunde, Vijeta Deshpande +1Large Language Model PretrainingPretraining

    University of Massachusetts Lowell

  29. Two Speeds of Learning: A Representation-Readout Decomposition of Grokking and Double Descent

    May 26, 2026Chi-Ning Chou, Oscar Uzdelewicz, Neng-Chun Chiu +2GrokkingDescent

    Center for Computational Neuroscience, Flatiron Institute, New York, NY, USA. · Department of Physics, Harvard University, Cambridge, MA, USA. · Kempner Institute, Harvard University, Allston, MA, USA. +1

  30. Grokking as Structural Inference: Transformers Need Bayesian Lottery Tickets

    May 15, 2026Kai Hidajat, Solden Stoll, Joseph AnTransformer ArchitecturesSequence Modeling

    Department of Computer Science University of Washington Seattle, WA 98195

  31. First-Passage Prediction of Grokking Delay: ACalibrated Law under AdamW with Causal Validation

    May 13, 2026Truong Xuan Khanh, Truong Quynh Hoa, Luu Duc Trung +1GrokkingDecision Thresholds

    H&K Research Studio, Clevix LLC, Hanoi, Vietnam · Banking Academy of Vietnam, Hanoi, Vietnam

  32. Detecting overfitting in Neural Networks during long-horizon grokking using Random Matrix Theory

    May 12, 2026Hari K. Prakash, Charles H MartinOverfittingNeural Network Training

    University of California San Diego Data Science and Engineering · Calculation Consulting San Francisco, CA, USA

  33. Model Capacity Determines Grokking through Competing Memorisation and Generalisation Speeds

    May 10, 2026Yiding Song, Hanming YeGrokkingMemorization

    Harvard College Cambridge, MA 02138

  34. Grokers: Bottom-Up Inductive Comprehension and Write-Time Intelligence over Typed Knowledge Graphs

    May 7, 2026Gregory MagarshakKnowledge GraphsTraversal

    Qbix, Inc. & Intercoin, Inc. · IE University NYC · New York, NY, USA

  35. Topological Signatures of Grokking

    May 7, 2026Yifan Tang, Qiquan Wang, Inés García-Redondo +1Persistent HomologyNeural Network Representations

    Imperial College London · Queen Mary University of London · University of Fribourg

  36. Grokking or Glitching? How Low-Precision Drives Slingshot Loss Spikes

    May 7, 2026Liu Hanqing, Jianjun Cao, Yuanze Li +1Full-Precision PerformanceGrokking

    1Tsinghua University · 2The University of Tokyo

  37. Distributional Spectral Diagnostics for Localizing Grokking Transitions

    May 7, 2026Ziyue Wang, Yufeng Ying, Takafumi KanamoriGrokkingTransformer Residual Streams

    Institute of Science Tokyo · University of Science and Technology of China

  38. Grokability in five inequalities

    May 6, 2026Paata Ivanisvili, Xinyuan XieConcentration BoundsConvex Sets

    University of California, Irvine

  39. Almost-Orthogonality in Lp Spaces: A Case Study with Grok

    May 6, 2026Ziang Chen, Jaume de Dios Pont, Paata Ivanisvili +2Spectral NormConcentration Bounds

    xAI · ETH Zürich · University of California, Irvine +1

  40. Feature Repulsion and Spectral Lock-in: An Empirical Study of Two-Layer Network Grokking

    Apr 28, 2026Yongzhong XuFeature LearningGrokking

  41. ILDR: Geometric Early Detection of Grokking

    Apr 22, 2026Shreel GolwalaGrokkingIntra-Class

    Virginia Tech

  42. Grokking of Diffusion Models: Case Study on Modular Addition

    Apr 20, 2026Joon Hyeok Kim, Yong-Hyun Park, Mattis Dalsætra Østby +1Diffusion ModelsGrokking

    Department of Computer and Information Science University of Pennsylvania Philadelphia, PA 19104, USA

  43. To Grok Grokking: Provable Grokking in Ridge Regression

    Jan 27, 2026Mingyue Xu, Gal Vardi, Itay SafranOverparameterizationKernel Ridge Regression

    Department of Computer Science, Purdue University, West Lafayette, IN, USA · Department of Computer Science and Applied Mathematics, Weizmann Institute of Science, Israel · Stein Faculty of Computer and Information Science, Ben-Gurion University of the Negev, Israel