Neural Network Memorization

Follow this topic Ask about this topic

Momentum

4 papers in the last four weeks, against 2 the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28

Latest papers 49

All topics

Memorisation, convergence and generalisation in generative models
May 20, 2026Antoine Maillard, Sebastian GoldtNeural Network Memorization Memorization in Generative Models
1INRIA Paris & DI ENS, PSL University, Paris, France · 2International School of Advanced Studies (SISSA), Trieste, Italy
Weight Decay Regimes in Grokking Transformers: Cheap Online Diagnostics
May 19, 2026Lucky VermaWeight Decay Neural Network Memorization
Independent Researcher
Reconfigurable Nonlinear Photonic Networks for In-Situ Learning and Memory Formation via Driven-Dissipative Dynamics
May 19, 2026Isaac YorkeDynamical Systems Neural Network Memorization
Department of Engineering and Architecture, University of Parma, Parco Area delle Scienze 181/A I-43124 Parma, Italy
Adynamical systems view of training generativemodels and the memorization phenomenon
May 19, 2026Siva Athreya, Chiranjib Bhattacharya, Vivek S. BorkarNeural Network Memorization Memorization in Generative Models
International Institute for Theoretical Sciences, Survey No. 151, Hesarghatta Road, Shivakote Hobli, Bengaluru 560089, India · Department of Computer Science and Automation, Indian Institute of Science, Bengaluru 560012, India · Department of Electrical Engineering, Indian Institute of Technology Bombay, Powai, Mumbai 400076, India
Unveiling Memorization-Generalization Coexistence: A Case Study on Arithmetic Tasks with Label Noise
May 18, 2026Linyu Liu, Pinyan LuNeural Network Generalization Neural Network Memorization
1Taylor Lab, Huawei Technologies Co., Ltd. · 2Key Laboratory of Interdisciplinary Research of Computation and Economics, Shanghai University of Finance and Economics
Auditing Reasoning-Trace Memorization Claims after Unlearning with Head-Conditioned Canaries
May 17, 2026Yanhang Li, Zhichao Fan, Zexin ZhuangMemorization in Language Models LLM Auditing
Northeastern University, USA · University of Illinois Urbana-Champaign, USA · Southern Methodist University, USA
First-Passage Prediction of Grokking Delay: ACalibrated Law under AdamW with Causal Validation
May 13, 2026Truong Xuan Khanh, Truong Quynh Hoa, Luu Duc Trung +1Neural Network Memorization Neural Network Training Dynamics
H&K Research Studio, Clevix LLC, Hanoi, Vietnam · Banking Academy of Vietnam, Hanoi, Vietnam
Geometric Factual Recall in Transformers
May 12, 2026Shauli Ravfogel, Gilad Yehudai, Joan Bruna +1Neural Representation Geometry Memorization in Language Models
1New York University · 2Flatiron Institute
Model Capacity Determines Grokking through Competing Memorisation and Generalisation Speeds
May 10, 2026Yiding Song, Hanming YeNeural Network Generalization Neural Network Memorization
Harvard College Cambridge, MA 02138
Probe-Geometry Alignment: Erasing the Cross-Sequence Memorization Signature Below Chance
May 3, 2026Anamika Paul Rupa, Anietie AndyMemorization in Language Models Neural Network Memorization
Howard University
Language Diffusion Models are Associative Memories Capable of Retrieving Unseen Data
Apr 29, 2026Bao Pham, Mohammed J. Zaki, Luca Ambrogioni +2Memorization in Language Models Neural Network Memorization
1Rensselaer Polytechnic Institute (RPI) · 2Donders Institute for Brain, Cognition, and Behaviour, Radboud University · 3Dynamical Mind +1
Revisiting Non-Verbatim Memorization in Large Language Models: The Role of Entity Surface Forms
Apr 23, 2026Yuto Nishida, Naoki Shikoda, Yosuke Kishinami +4LLM Evaluation Memorization in Language Models
Nara Institute of Science and Technology · Future Corporation
Learning Hippo: Multi-attractor Dynamics and Stability Effects in a Biologically Detailed CA3 Extension of Hopfield Networks
Apr 22, 2026Daniele Corradetti, Renato CorradettiDynamical Systems Neural Processes
Grupo de Física Matemática Instituto Superior Técnico Av. Rovisco Pais, 1049-001 Lisboa, Portugal · Dipartimento di Neuroscienze, Psicologia, Area del Farmaco e Salute del Bambino (NEUROFARBA) Università di Firenze, Italy22
The Rules-and-Facts Model for Simultaneous Generalization and Memorization in Neural Networks
Mar 26, 2026Gabriele Farné, Fabrizio Boncoraglio, Lenka ZdeborováNeural Network Generalization Neural Network Memorization
Statistical Physics of Computation Laboratory, École Polytechnique Fédérale de Lausanne (EPFL) CH-1015 Lausanne
Implicit Bias and Invariance: How Hopfield Networks Efficiently Learn Graph Orbits
Dec 16, 2025Michael Murray, Tenzin Chan, Kedar Karhadker +1Sample Complexity Neural Network Memorization
Mathematical Sciences, University of Bath · Department of Mathematics, UCLA · Algebraic
Redundancy Maximization as a Principle of Associative Memory Learning in Hopfield Networks
Nov 4, 2025Mark Blümel, Andreas C. Schneider, Valentin Neuhaus +5Neural Network Memorization Hopfield Networks
Complex Systems Theory, Max Planck Institute for Dynamics and Self-Organization, Göttingen, Germany · Faculty of Physics, Institute for the Dynamics of Complex Systems, University of Göttingen · Göttingen Campus Institute for Dynamics of Biological Networks, University of Göttingen, Göttingen, Germany +1
Dynamical stability for dense patterns in attractor neural networks
Jul 14, 2025Uri Cohen, Máté LengyelDynamical Systems Neural Processes
Computational and Biological Learning Lab, Department of Engineering, University of Cambridge, Cambridge, UK · Center for Cognitive Computation, Department of Cognitive Science, Central European University, Budapest, Hungary
Continual Memorization of Factoids in Language Models
Nov 11, 2024Howard Chen, Jiayi Geng, Adithya Bhaskar +2Continual Learning for LLMs Memorization in Language Models
Princeton Language and Intelligence (PLI), Princeton University
Mitigating Memorization In Language Models
Oct 3, 2024Mansi Sakarvadia, Aswathy Ajith, Arham Khan +6Memorization in Language Models LLM Fine-Tuning
University of Chicago · Argonne National Laboratory · Lawrence Berkeley National Laboratory +3

May 20, 2026, stat.MLJ/K move · Enter open · S save

Memorisation, convergence and generalisation in generative models

Antoine Maillard, Sebastian Goldt

1INRIA Paris & DI ENS, PSL University, Paris, France · 2International School of Advanced Studies (SISSA), Trieste, Italy

Neural Network Memorization Memorization in Generative Models Generative Modeling Diffusion Models

Open paper AskPDF arXiv

Generative neural networks learn how to produce highly realistic images from a large, but finite number of examples - or do they simply memorise their training set? To settle this question, Kadkhodaie, Guth, Simoncelli and Mallat (ICLR '24) trained diffusion models independently on disjoint subsets of a dataset and showed that they converge to nearly the same density when the number of training images is large enough. This result raises two basic questions: how much data do you need for convergence, and what does convergence capture about learning the data distribution? Here, we address these questions by providing an exact analytical characterisation of the transition from memorisation to generalisation in linear generative models. We find that these models memorise at small load, while convergence emerges continuously when the number of samples is linear in the input dimension. Strikingly, we find that convergence is insensitive to recovery of the principal latent factors of the data, which are recovered in a sharp transition. After extending our approach to data with power-law spectra, we find the same distinction between convergence and latent recovery in our experiments with convolutional denoisers and in the data of Kadkhodaie et al. We thus show that generalisation in generative models decomposes into at least two distinct objectives: matching the bulk of the data distribution and recovering the principal latent factors. These objectives correspond to two different distances between true and learnt data distribution, and only the first one is captured by convergence.

Momentum

Related topics

Latest papers 49