Neural Network Memorization

Follow this topic Ask about this topic

Momentum

4 papers in the last four weeks, against 2 the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28

Latest papers 49

All topics

Learning Under Forgetting: Statistical Support-Selective Retention in Stochastic Training Dynamics
Sep 30, 2026Fujie Gao, Zuyue Zhang, Gang SunNeural Network Memorization Neural Network Training Dynamics
Boolean threshold functions, neuron capacity, and memory retrieval
Sep 24, 2026Xinyuan XieBoolean Function Learning Neural Network Memorization
Memorisation bias in medical AI
Sep 15, 2026Moritz A. Knolle, Martin J. Menten, Laurin Lux +5Neural Network Memorization Clinical Prediction
Chair for AI in Healthcare and Medicine, Technical University of Munich (TUM) and TUM University Hospital, Germany. · Munich Center for Machine Learning, Germany. · Department of Computing, Imperial College London, United Kingdom. +1
Quantifying the Memorization-to-Generalization Transition: Scaling Laws and Phase Structure in Grokking
Sep 12, 2026Anish KatariaMultilayer Perceptrons Neural Network Memorization
Princeton University, Princeton, NJ, USA.
PatchBench: Evaluating AI Agents for Vulnerability Patching
Sep 3, 2026Chihao Shen, Jiacheng Li, Aastha Mahajan +3Automated Program Repair Neural Network Memorization
University of Maryland
Measuring Memory and Generalization as Separable Geometric Channels: The Topo^2 Framework
Aug 31, 2026Zhanbo Zhang, Ming Liu, Qing WangNeural Network Generalization Neural Network Memorization
Anhui iFlytek Yinglian Technology Co., Ltd.
Tunneling the Loss Landscape: Bypassing Memorization with Monte Carlo Parameter Swapping
Aug 3, 2026Lai Shun Chan, Xiaotian Zhang, Yue Shang +2Neural Network Generalization Markov Chain Monte Carlo
Department of Physics, City University of Hong Kong, Hong Kong, China · Department of Physics and Astronomy, University of Pennsylvania, Philadelphia, PA, USA · 3Innovation Campus Delaware, Air Liquide, Newark, DE, USA
Free energy landscape of Dense Associative Memory
Jul 21, 2026Sumedha, Abhishek SinghFree Energy Calculation Neural Network Memorization
School of Physical Sciences, National Institute of Science Education and Research, Bhubaneswar, P.O. Jatni, 752050, India · 2Homi Bhabha National Institute, Training School Complex, Anushakti Nagar 400094, India
Extractable Memorization From First Principles
Jul 14, 2026A. Feder Cooper, Marika Swanberg, Jamie Hayes +5LLM Evaluation Memorization in Language Models
Yale, AVERI, Stanford · Google Research · Google DeepMind +2
The Geometry of Memorization: Finite-Time Spectral Sensitivity as a Diagnostic for Flow Matching Models
Jul 14, 2026Shuchan WangFlow Matching Neural Network Memorization
Fisher Rank Inflation: A Spectral Signature of Memorization under Label Noise
Jul 14, 2026Satwik Bathula, Anand A. JoshiNeural Network Memorization Noisy-Label Learning
Department of Electrical and Computer Engineering, University of Southern California
Random Label Prediction Heads for Studying Memorization in Deep Neural Networks
Jul 13, 2026Marlon Becker, Jonas Konrad, Luis Garcia Rodriguez +1Neural Network Generalization Neural Network Memorization
University of Münster, Germany
Towards Mechanistically Understanding Why Memorized Knowledge Fails to Generalize in Large Language Model Finetuning
Jul 9, 2026Lu Dai, Ziyang Rao, Yili Wang +3Memorization in Language Models LLM Fine-Tuning
HKUST · HKUST(GZ)
A law of robustness for two-layer neural networks with arbitrary weights
Jul 8, 2026Yitzchak ShmaloNeural Network Generalization Multilayer Perceptrons
Do You Remember? Toward Memory-Centric Multimodal AI
Jul 7, 2026Xuguang Yu, Weigang Zheng, Minyue YuMemory-Augmented VLMs Multimodal Memory
Train Smarter, Not Longer: Memorization-Guided Data Reuse for Efficient LLM Training
Jul 6, 2026Jingwei Zuo, Cong Zeng, Ilyas Chahed +6Memorization in Language Models Neural Network Memorization
Technology Innovation Institute Abu Dhabi, UAE
Radial Suppression Accelerates Algorithmic Generalization: A Geometric Analysis of Delayed Generalization
Jun 30, 2026Srijan Tiwari, Aditya Chauhan, Manjot SinghNeural Network Generalization Neural Representation Geometry
Indian Institute of Technology Roorkee
Probing Memorization of Tabular In-Context Learning
Jun 30, 2026Francesco Capano, Jonas BöhlerTabular Foundation Models Membership Inference Attacks
SAP SE, Walldorf, Germany.
Probe Choice Changes Canary-Memorization Verdicts: Three Post-Hoc Disagreement Case Studies in a Text-Dominant LoRA-Tuned Autoregressive Testbed
Jun 30, 2026Zhichao Fan, Zexin Zhuang, Yanhang LiMemorization in Language Models LLM Auditing
University of Illinois Urbana-Champaign · Southern Methodist University · Northeastern University
Explaining Machine Learning and Memorization with Statistical Mechanics
Jun 30, 2026Robin TheriaultStatistical Physics of Learning Neural Network Memorization
Faculty of Science
A Stochastic--Geometric Theory of Scaling Laws in Grokking
Jun 29, 2026Róisín Luo, Christian Gagné, Jonas Ngnawé +2Neural Network Generalization Neural Network Optimization
Research Ireland – Centre for Research Training in AI · University of Galway · Universit´e Laval +2
Applicability of memorization indicators for early spotting of overfitting while recalibrating sEMG-decoders on low sample sizes
Jun 26, 2026Stephan J. Lehmler, Tobias Glasmachers, Ioannis IossifidisNeural Network Generalization Neural Decoding
German Aerospace Center (DLR), Institute for AI Safety and Security, Ulm, Germany · Faculty of Electrical Engineering and Information Technology, Ruhr University Bochum, Bochum, Germany · Institute for Neural Computation, Ruhr University Bochum, Bochum, Germany +1
Output Vector Editing for Memorization Mitigation in Large Language Models
Jun 17, 2026Ahmad Dawar Hakimi, Kaiwei Lei, Isabelle Augenstein +1Memorization in Language Models Neural Network Memorization
Center for Information and Language Processing, LMU Munich, Germany · Department of Computer Science, University of Copenhagen, Denmark · 3Munich Center for Machine Learning, Germany +1
AI Engram: In Search of Memory Traces in Artificial Intelligence
Jun 12, 2026Jea Kwon, Dong-Kyum Kim, Jiwon Kim +3Neural Representation Geometry Memorization in Language Models
Max Planck Institute for Security and Privacy (MPI-SP), Bochum, Germany · Seoul National University (SNU), Seoul, Korea · KAIST, Daejeon, Korea
LLMs Can Leak Training Data But Do They Want To? A Propensity-Aware Evaluation of Memorization in LLMs
Jun 4, 2026Gianluca Barmina, Peter Schneider-Kamp, Lukas Galke PoechData Leakage Memorization in Language Models
University of Southern Denmark
Stochastic Estimation of the Layer-wise Hessian Trace for Monitoring Neural-network Training
May 25, 2026Maxim Bolshim, Alexander KugaevskikhNeural Network Memorization Stochastic Approximation
ITMO University, Saint Petersburg, Russia · ITMO University, St. Petersburg, Russia
Rethinking Federated Unlearning via the Lens of Memorization
May 23, 2026Jiaheng Wei, Yanjun Zhang, He Zhang +5Federated Unlearning Neural Network Memorization
Royal Melbourne Institute of Technology · Griffith University · Swinburne University of Technology
Batch Normalization Amplifies Memorization and Privacy Risks
May 23, 2026Ngoc Phu Doan, Chongyan Gu, Ihsen AlouaniNeural Network Memorization
Queen’s University Belfast
Memorization Dynamics of Fill-in-the-Middle Pretraining
May 21, 2026Tobias von Arx, Tanguy DieudonnéLanguage Model Pretraining Memorization in Language Models
Department of Computer Science, ETH Zurich, Zurich, Switzerland.
A mathematical theory of balancing relational generalization and memorization
May 21, 2026Luke Cheng, Samuel LipplNeural Network Generalization Logical Reasoning
Center for Theoretical Neuroscience Columbia University New York, NY, 10027

Sep 30, 2026, cs.LGJ/K move · Enter open · S save

Learning Under Forgetting: Statistical Support-Selective Retention in Stochastic Training Dynamics

Fujie Gao, Zuyue Zhang, Gang Sun

Neural Network Memorization Neural Network Training Dynamics

Open paper AskPDF arXiv

Prior work has shown that neural networks exhibit implicit biases toward low-complexity structure (e.g., spectral bias), memorization dynamics, and compression-like effects during training, but a unified dynamical account of selective retention remains incomplete. We propose Repeated Reinforcement with Persistent Forgetting (RPF) dynamics, a minimal framework in which repeated exposure reinforces patterns and structures that recur in the data, while persistent forgetting attenuates learned information. This view treats forgetting not merely as a failure mode, but as a selection mechanism. We build the theory in three successive layers. First, in an independent-feature model, we derive an exposure-selective survival law and a support-dependent retention boundary characterizing which patterns persist under forgetting. Second, in a shared-parameter model, we show that forgetting induces spectral filtering over covariance modes, preserving strongly supported shared components while suppressing weak ones. Third, under small-step and norm/coding approximations, we show how RPF dynamics induce an implicit trade-off between data fitting and the cost of stored information, yielding Minimum Description Length (MDL)-like compression. Controlled experiments provide evidence for this reinforcement--forgetting selection mechanism in scalar memories and a nonlinear shared network. Joint reinforcement and attenuation interventions shift conditional retention, while matched exposure counts reveal forgetting-dependent effects of reinforcement timing and changes in the composition of the retained set. Together, these results show that repeated reinforcement and persistent forgetting jointly provide a controllable source of inductive bias beyond neural architecture and scale.

Momentum

Related topics

Latest papers 49