Neural Network Memorization

Follow this topic Ask about this topic

Momentum

4 papers in the last four weeks, against 2 the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28

Latest papers 49

All topics

Learning Under Forgetting: Statistical Support-Selective Retention in Stochastic Training Dynamics
Sep 30, 2026Fujie Gao, Zuyue Zhang, Gang SunNeural Network Memorization Neural Network Training Dynamics
Boolean threshold functions, neuron capacity, and memory retrieval
Sep 24, 2026Xinyuan XieBoolean Function Learning Neural Network Memorization
Memorisation bias in medical AI
Sep 15, 2026Moritz A. Knolle, Martin J. Menten, Laurin Lux +5Neural Network Memorization Clinical Prediction
Chair for AI in Healthcare and Medicine, Technical University of Munich (TUM) and TUM University Hospital, Germany. · Munich Center for Machine Learning, Germany. · Department of Computing, Imperial College London, United Kingdom. +1
Quantifying the Memorization-to-Generalization Transition: Scaling Laws and Phase Structure in Grokking
Sep 12, 2026Anish KatariaMultilayer Perceptrons Neural Network Memorization
Princeton University, Princeton, NJ, USA.
PatchBench: Evaluating AI Agents for Vulnerability Patching
Sep 3, 2026Chihao Shen, Jiacheng Li, Aastha Mahajan +3Automated Program Repair Neural Network Memorization
University of Maryland
Measuring Memory and Generalization as Separable Geometric Channels: The Topo^2 Framework
Aug 31, 2026Zhanbo Zhang, Ming Liu, Qing WangNeural Network Generalization Neural Network Memorization
Anhui iFlytek Yinglian Technology Co., Ltd.
Tunneling the Loss Landscape: Bypassing Memorization with Monte Carlo Parameter Swapping
Aug 3, 2026Lai Shun Chan, Xiaotian Zhang, Yue Shang +2Neural Network Generalization Markov Chain Monte Carlo
Department of Physics, City University of Hong Kong, Hong Kong, China · Department of Physics and Astronomy, University of Pennsylvania, Philadelphia, PA, USA · 3Innovation Campus Delaware, Air Liquide, Newark, DE, USA
Free energy landscape of Dense Associative Memory
Jul 21, 2026Sumedha, Abhishek SinghFree Energy Calculation Neural Network Memorization
School of Physical Sciences, National Institute of Science Education and Research, Bhubaneswar, P.O. Jatni, 752050, India · 2Homi Bhabha National Institute, Training School Complex, Anushakti Nagar 400094, India
Extractable Memorization From First Principles
Jul 14, 2026A. Feder Cooper, Marika Swanberg, Jamie Hayes +5LLM Evaluation Memorization in Language Models
Yale, AVERI, Stanford · Google Research · Google DeepMind +2
The Geometry of Memorization: Finite-Time Spectral Sensitivity as a Diagnostic for Flow Matching Models
Jul 14, 2026Shuchan WangFlow Matching Neural Network Memorization
Fisher Rank Inflation: A Spectral Signature of Memorization under Label Noise
Jul 14, 2026Satwik Bathula, Anand A. JoshiNeural Network Memorization Noisy-Label Learning
Department of Electrical and Computer Engineering, University of Southern California
Random Label Prediction Heads for Studying Memorization in Deep Neural Networks
Jul 13, 2026Marlon Becker, Jonas Konrad, Luis Garcia Rodriguez +1Neural Network Generalization Neural Network Memorization
University of Münster, Germany
Towards Mechanistically Understanding Why Memorized Knowledge Fails to Generalize in Large Language Model Finetuning
Jul 9, 2026Lu Dai, Ziyang Rao, Yili Wang +3Memorization in Language Models LLM Fine-Tuning
HKUST · HKUST(GZ)
A law of robustness for two-layer neural networks with arbitrary weights
Jul 8, 2026Yitzchak ShmaloNeural Network Generalization Multilayer Perceptrons
Do You Remember? Toward Memory-Centric Multimodal AI
Jul 7, 2026Xuguang Yu, Weigang Zheng, Minyue YuMemory-Augmented VLMs Multimodal Memory
Train Smarter, Not Longer: Memorization-Guided Data Reuse for Efficient LLM Training
Jul 6, 2026Jingwei Zuo, Cong Zeng, Ilyas Chahed +6Memorization in Language Models Neural Network Memorization
Technology Innovation Institute Abu Dhabi, UAE
Radial Suppression Accelerates Algorithmic Generalization: A Geometric Analysis of Delayed Generalization
Jun 30, 2026Srijan Tiwari, Aditya Chauhan, Manjot SinghNeural Network Generalization Neural Representation Geometry
Indian Institute of Technology Roorkee
Probing Memorization of Tabular In-Context Learning
Jun 30, 2026Francesco Capano, Jonas BöhlerTabular Foundation Models Membership Inference Attacks
SAP SE, Walldorf, Germany.
Probe Choice Changes Canary-Memorization Verdicts: Three Post-Hoc Disagreement Case Studies in a Text-Dominant LoRA-Tuned Autoregressive Testbed
Jun 30, 2026Zhichao Fan, Zexin Zhuang, Yanhang LiMemorization in Language Models LLM Auditing
University of Illinois Urbana-Champaign · Southern Methodist University · Northeastern University
Explaining Machine Learning and Memorization with Statistical Mechanics
Jun 30, 2026Robin TheriaultStatistical Physics of Learning Neural Network Memorization
Faculty of Science
A Stochastic--Geometric Theory of Scaling Laws in Grokking
Jun 29, 2026Róisín Luo, Christian Gagné, Jonas Ngnawé +2Neural Network Generalization Neural Network Optimization
Research Ireland – Centre for Research Training in AI · University of Galway · Universit´e Laval +2
Applicability of memorization indicators for early spotting of overfitting while recalibrating sEMG-decoders on low sample sizes
Jun 26, 2026Stephan J. Lehmler, Tobias Glasmachers, Ioannis IossifidisNeural Network Generalization Neural Decoding
German Aerospace Center (DLR), Institute for AI Safety and Security, Ulm, Germany · Faculty of Electrical Engineering and Information Technology, Ruhr University Bochum, Bochum, Germany · Institute for Neural Computation, Ruhr University Bochum, Bochum, Germany +1
Output Vector Editing for Memorization Mitigation in Large Language Models
Jun 17, 2026Ahmad Dawar Hakimi, Kaiwei Lei, Isabelle Augenstein +1Memorization in Language Models Neural Network Memorization
Center for Information and Language Processing, LMU Munich, Germany · Department of Computer Science, University of Copenhagen, Denmark · 3Munich Center for Machine Learning, Germany +1
AI Engram: In Search of Memory Traces in Artificial Intelligence
Jun 12, 2026Jea Kwon, Dong-Kyum Kim, Jiwon Kim +3Neural Representation Geometry Memorization in Language Models
Max Planck Institute for Security and Privacy (MPI-SP), Bochum, Germany · Seoul National University (SNU), Seoul, Korea · KAIST, Daejeon, Korea
LLMs Can Leak Training Data But Do They Want To? A Propensity-Aware Evaluation of Memorization in LLMs
Jun 4, 2026Gianluca Barmina, Peter Schneider-Kamp, Lukas Galke PoechData Leakage Memorization in Language Models
University of Southern Denmark
Stochastic Estimation of the Layer-wise Hessian Trace for Monitoring Neural-network Training
May 25, 2026Maxim Bolshim, Alexander KugaevskikhNeural Network Memorization Stochastic Approximation
ITMO University, Saint Petersburg, Russia · ITMO University, St. Petersburg, Russia
Rethinking Federated Unlearning via the Lens of Memorization
May 23, 2026Jiaheng Wei, Yanjun Zhang, He Zhang +5Federated Unlearning Neural Network Memorization
Royal Melbourne Institute of Technology · Griffith University · Swinburne University of Technology
Batch Normalization Amplifies Memorization and Privacy Risks
May 23, 2026Ngoc Phu Doan, Chongyan Gu, Ihsen AlouaniNeural Network Memorization
Queen’s University Belfast
Memorization Dynamics of Fill-in-the-Middle Pretraining
May 21, 2026Tobias von Arx, Tanguy DieudonnéLanguage Model Pretraining Memorization in Language Models
Department of Computer Science, ETH Zurich, Zurich, Switzerland.
A mathematical theory of balancing relational generalization and memorization
May 21, 2026Luke Cheng, Samuel LipplNeural Network Generalization Logical Reasoning
Center for Theoretical Neuroscience Columbia University New York, NY, 10027
Memorisation, convergence and generalisation in generative models
May 20, 2026Antoine Maillard, Sebastian GoldtNeural Network Memorization Memorization in Generative Models
1INRIA Paris & DI ENS, PSL University, Paris, France · 2International School of Advanced Studies (SISSA), Trieste, Italy
Weight Decay Regimes in Grokking Transformers: Cheap Online Diagnostics
May 19, 2026Lucky VermaWeight Decay Neural Network Memorization
Independent Researcher
Reconfigurable Nonlinear Photonic Networks for In-Situ Learning and Memory Formation via Driven-Dissipative Dynamics
May 19, 2026Isaac YorkeDynamical Systems Neural Network Memorization
Department of Engineering and Architecture, University of Parma, Parco Area delle Scienze 181/A I-43124 Parma, Italy
Adynamical systems view of training generativemodels and the memorization phenomenon
May 19, 2026Siva Athreya, Chiranjib Bhattacharya, Vivek S. BorkarNeural Network Memorization Memorization in Generative Models
International Institute for Theoretical Sciences, Survey No. 151, Hesarghatta Road, Shivakote Hobli, Bengaluru 560089, India · Department of Computer Science and Automation, Indian Institute of Science, Bengaluru 560012, India · Department of Electrical Engineering, Indian Institute of Technology Bombay, Powai, Mumbai 400076, India
Unveiling Memorization-Generalization Coexistence: A Case Study on Arithmetic Tasks with Label Noise
May 18, 2026Linyu Liu, Pinyan LuNeural Network Generalization Neural Network Memorization
1Taylor Lab, Huawei Technologies Co., Ltd. · 2Key Laboratory of Interdisciplinary Research of Computation and Economics, Shanghai University of Finance and Economics
Auditing Reasoning-Trace Memorization Claims after Unlearning with Head-Conditioned Canaries
May 17, 2026Yanhang Li, Zhichao Fan, Zexin ZhuangMemorization in Language Models LLM Auditing
Northeastern University, USA · University of Illinois Urbana-Champaign, USA · Southern Methodist University, USA
First-Passage Prediction of Grokking Delay: ACalibrated Law under AdamW with Causal Validation
May 13, 2026Truong Xuan Khanh, Truong Quynh Hoa, Luu Duc Trung +1Neural Network Memorization Neural Network Training Dynamics
H&K Research Studio, Clevix LLC, Hanoi, Vietnam · Banking Academy of Vietnam, Hanoi, Vietnam
Geometric Factual Recall in Transformers
May 12, 2026Shauli Ravfogel, Gilad Yehudai, Joan Bruna +1Neural Representation Geometry Memorization in Language Models
1New York University · 2Flatiron Institute
Model Capacity Determines Grokking through Competing Memorisation and Generalisation Speeds
May 10, 2026Yiding Song, Hanming YeNeural Network Generalization Neural Network Memorization
Harvard College Cambridge, MA 02138
Probe-Geometry Alignment: Erasing the Cross-Sequence Memorization Signature Below Chance
May 3, 2026Anamika Paul Rupa, Anietie AndyMemorization in Language Models Neural Network Memorization
Howard University
Language Diffusion Models are Associative Memories Capable of Retrieving Unseen Data
Apr 29, 2026Bao Pham, Mohammed J. Zaki, Luca Ambrogioni +2Memorization in Language Models Neural Network Memorization
1Rensselaer Polytechnic Institute (RPI) · 2Donders Institute for Brain, Cognition, and Behaviour, Radboud University · 3Dynamical Mind +1
Revisiting Non-Verbatim Memorization in Large Language Models: The Role of Entity Surface Forms
Apr 23, 2026Yuto Nishida, Naoki Shikoda, Yosuke Kishinami +4LLM Evaluation Memorization in Language Models
Nara Institute of Science and Technology · Future Corporation
Learning Hippo: Multi-attractor Dynamics and Stability Effects in a Biologically Detailed CA3 Extension of Hopfield Networks
Apr 22, 2026Daniele Corradetti, Renato CorradettiDynamical Systems Neural Processes
Grupo de Física Matemática Instituto Superior Técnico Av. Rovisco Pais, 1049-001 Lisboa, Portugal · Dipartimento di Neuroscienze, Psicologia, Area del Farmaco e Salute del Bambino (NEUROFARBA) Università di Firenze, Italy22
The Rules-and-Facts Model for Simultaneous Generalization and Memorization in Neural Networks
Mar 26, 2026Gabriele Farné, Fabrizio Boncoraglio, Lenka ZdeborováNeural Network Generalization Neural Network Memorization
Statistical Physics of Computation Laboratory, École Polytechnique Fédérale de Lausanne (EPFL) CH-1015 Lausanne
Implicit Bias and Invariance: How Hopfield Networks Efficiently Learn Graph Orbits
Dec 16, 2025Michael Murray, Tenzin Chan, Kedar Karhadker +1Sample Complexity Neural Network Memorization
Mathematical Sciences, University of Bath · Department of Mathematics, UCLA · Algebraic
Redundancy Maximization as a Principle of Associative Memory Learning in Hopfield Networks
Nov 4, 2025Mark Blümel, Andreas C. Schneider, Valentin Neuhaus +5Neural Network Memorization Hopfield Networks
Complex Systems Theory, Max Planck Institute for Dynamics and Self-Organization, Göttingen, Germany · Faculty of Physics, Institute for the Dynamics of Complex Systems, University of Göttingen · Göttingen Campus Institute for Dynamics of Biological Networks, University of Göttingen, Göttingen, Germany +1
Dynamical stability for dense patterns in attractor neural networks
Jul 14, 2025Uri Cohen, Máté LengyelDynamical Systems Neural Processes
Computational and Biological Learning Lab, Department of Engineering, University of Cambridge, Cambridge, UK · Center for Cognitive Computation, Department of Cognitive Science, Central European University, Budapest, Hungary
Continual Memorization of Factoids in Language Models
Nov 11, 2024Howard Chen, Jiayi Geng, Adithya Bhaskar +2Continual Learning for LLMs Memorization in Language Models
Princeton Language and Intelligence (PLI), Princeton University
Mitigating Memorization In Language Models
Oct 3, 2024Mansi Sakarvadia, Aswathy Ajith, Arham Khan +6Memorization in Language Models LLM Fine-Tuning
University of Chicago · Argonne National Laboratory · Lawrence Berkeley National Laboratory +3

Sep 30, 2026, cs.LGJ/K move · Enter open · S save

Learning Under Forgetting: Statistical Support-Selective Retention in Stochastic Training Dynamics

Fujie Gao, Zuyue Zhang, Gang Sun

Neural Network Memorization Neural Network Training Dynamics

Open paper AskPDF arXiv

Prior work has shown that neural networks exhibit implicit biases toward low-complexity structure (e.g., spectral bias), memorization dynamics, and compression-like effects during training, but a unified dynamical account of selective retention remains incomplete. We propose Repeated Reinforcement with Persistent Forgetting (RPF) dynamics, a minimal framework in which repeated exposure reinforces patterns and structures that recur in the data, while persistent forgetting attenuates learned information. This view treats forgetting not merely as a failure mode, but as a selection mechanism. We build the theory in three successive layers. First, in an independent-feature model, we derive an exposure-selective survival law and a support-dependent retention boundary characterizing which patterns persist under forgetting. Second, in a shared-parameter model, we show that forgetting induces spectral filtering over covariance modes, preserving strongly supported shared components while suppressing weak ones. Third, under small-step and norm/coding approximations, we show how RPF dynamics induce an implicit trade-off between data fitting and the cost of stored information, yielding Minimum Description Length (MDL)-like compression. Controlled experiments provide evidence for this reinforcement--forgetting selection mechanism in scalar memories and a nonlinear shared network. Joint reinforcement and attenuation interventions shift conditional retention, while matched exposure counts reveal forgetting-dependent effects of reinforcement timing and changes in the composition of the retained set. Together, these results show that repeated reinforcement and persistent forgetting jointly provide a controllable source of inductive bias beyond neural architecture and scale.

Momentum

Related topics

Latest papers 49