Memorization in Generative Models

Follow this topic Ask about this topic

Momentum

8 papers in the last four weeks, against 1 the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 70

All topics

Divergence Decoding: Inference-Time Unlearning via Auxiliary Models
May 29, 2026Humzah Merchant, Bradford LevyPrivacy Leakage in Language Models Memorization in Generative Models
University of Chicago, Chicago, IL 60637.
Diffusion Models Preferentially Memorize Prototypical Examples or: Why Does My Diffusion Model Love Slop?
May 28, 2026Marta Aparicio Rodriguez, Anastasia Borovykh, Grigorios A. Pavliotis +1Privacy Leakage in Language Models Memorization in Generative Models
Department of Mathematics, Imperial College London, UK · ML Lab, Capital Fund Management, France · Department of Physics, École Polytechnique Fédérale de Lausanne (EPFL), Switzerland
How LoRA Remembers? A Parametric Memory Law for LLM Finetuning
May 28, 2026Ziwen Xu, Haiwen Hong, Linsong Yu +4Memorization in Language Models LLM Fine-Tuning
Zhejiang University · Alibaba Group
NumLeak: Public Numeric Benchmarks as Latent Labels in Foundation Models
May 28, 2026Anany KotawalaData Leakage LLM Evaluation
Princeton University, Princeton, NJ, USA.
The Attribution Blind Spot: Detecting When Language Models Rely on Memory Rather Than Retrieved Context
May 26, 2026Zhe Yu, Wenpeng Xing, Yunzhao Wei +4Memorization in Language Models LLM Interpretability
Binjiang Institute of Zhejiang University · Zhejiang University · National Fintech Evaluation Center +2
Localizing Memorized Regions in Diffusion Models via Coordinate-Wise Curvature Differences
May 26, 2026Gwangho Kim, Sungyoon LeeMemorization in Generative Models Diffusion Models
Department of Computer Science, Hanyang University, Seoul, South Korea.
Simulating Human Memory with Language Models
May 25, 2026Qihan Wang, Nicholas Tomlin, Michael Hu +2LLM Evaluation Memory-Augmented Language Models
NYU · UMass Amherst
Teaching Video Generators to Remember: Eliciting Dynamic Memory for Out-of-Sight State Evolution
May 25, 2026Tianshuo Xu, Yichen Xie, Depu Meng +5Video Diffusion Models Long-Horizon Video Generation
Applied Intuition · University of California, Berkeley
Extracting Training Data from Diffusion Language Models via Infilling
May 22, 2026Yihan Wang, N. AsokanData Leakage Masked Diffusion Models
University of Waterloo · KTH Royal Institute of Technology
Memorization Dynamics of Fill-in-the-Middle Pretraining
May 21, 2026Tobias von Arx, Tanguy DieudonnéLanguage Model Pretraining Memorization in Language Models
Department of Computer Science, ETH Zurich, Zurich, Switzerland.
Broken Memories: Detecting and Mitigating Memorization in Diffusion Models with Degraded Generations
May 21, 2026Yuanmin Huang, Mi Zhang, Chen Chen +4Memorization in Generative Models Diffusion Models
Fudan University Shanghai, China · East China University of Science and Technology Shanghai, China
Memorisation, convergence and generalisation in generative models
May 20, 2026Antoine Maillard, Sebastian GoldtNeural Network Memorization Memorization in Generative Models
1INRIA Paris & DI ENS, PSL University, Paris, France · 2International School of Advanced Studies (SISSA), Trieste, Italy
Adynamical systems view of training generativemodels and the memorization phenomenon
May 19, 2026Siva Athreya, Chiranjib Bhattacharya, Vivek S. BorkarNeural Network Memorization Memorization in Generative Models
International Institute for Theoretical Sciences, Survey No. 151, Hesarghatta Road, Shivakote Hobli, Bengaluru 560089, India · Department of Computer Science and Automation, Indian Institute of Science, Bengaluru 560012, India · Department of Electrical Engineering, Indian Institute of Technology Bombay, Powai, Mumbai 400076, India
Reducing Diffusion Model Memorization with Higher Order Langevin Dynamics
May 18, 2026Benjamin Sterling, Mónica F. Bugallo, Tom TirerLangevin Dynamics Memorization in Generative Models
Department of Applied Math & Statistics Stony Brook University Stony Brook, NY 11790 · Department of Electrical and Computer Engineering Stony Brook University Stony Brook, NY 11790 · Faculty of Engineering Bar-Ilan University Ramat-Gan 5290002, Israel
Auditing Reasoning-Trace Memorization Claims after Unlearning with Head-Conditioned Canaries
May 17, 2026Yanhang Li, Zhichao Fan, Zexin ZhuangMemorization in Language Models LLM Auditing
Northeastern University, USA · University of Illinois Urbana-Champaign, USA · Southern Methodist University, USA
Phasor Memory Networks: Stable Backpropagation Through Time for Scalable Explicit Memory
May 13, 2026Sungwoo Goo, Hwi-yeol Yun, Sangkeun JungMemory-Augmented Neural Networks Memory-Augmented Language Models
College of Pharmacy Chungnam National University Daejeon, South Korea · Department of Computer Science & Engineering Chungnam National University Daejeon, South Korea
Geometric Factual Recall in Transformers
May 12, 2026Shauli Ravfogel, Gilad Yehudai, Joan Bruna +1Neural Representation Geometry Memorization in Language Models
1New York University · 2Flatiron Institute
$δ$ -mem: Efficient Online Memory for Large Language Models
May 12, 2026Jingdi Lei, Di Zhang, Junxian Li +7Memory-Augmented Language Models Memorization in Language Models
Nanyang Technological University · Mind Lab · Fudan University +3
diffGHOST: Diffusion based Generative Hedged Oblivious Synthetic Trajectories
May 11, 2026Florent Guépin, Cheick Tidiani Cisse, Denis Renaud +2Trajectory Generation Synthetic Data Generation
Orange Research, Belfort, France · Orange Business, Belfort, France
Filtering Memorization from Parameter-Space in Diffusion Models
May 11, 2026Yu Zhe, Yang Jiayan, Wei Junhao +2Low-Rank Adaptation Memorization in Generative Models
RIKEN AIP · Institute of Science Tokyo · University of California, Berkeley +1
The two clocks and the innovation window: When and how generative models learn rules
May 11, 2026Binxu Wang, Emma Lucia Byrnes Finn, Bingbin LiuMemorization in Generative Models Generative Modeling
Kempner Institute at Harvard University
When Graph Language Models Go Beyond Memorization
May 7, 2026Masatsugu Yamada, Mahito SugiyamaGraph Generation Memorization in Language Models
National Institute of Informatics Tokyo, Japan · National Institute of Informatics, Tokyo, Japan · SOKENDAI, Kanagawa, Japan
Understanding diffusion models requires rethinking (again) generalization
May 7, 2026Pierre Marion, Yu-Han WuMemorization in Generative Models Diffusion Models
Inria, École Normale Supérieure, PSL Research University · LPSM, Sorbonne University & Google DeepMind
Attractor Geometry of Transformer Memory: From Conflict Arbitration to Confident Hallucination
May 7, 2026Qiyao Liang, Risto Miikkulainen, Ila FieteMemorization in Language Models Knowledge Conflicts in Language Models
Massachusetts Institute of Technology · University of Texas Austin, Cognizant · Cognizant
Memory as a Markov Matrix: Sample Efficient Knowledge Expansion via Token-to-Dictionary Mapping
May 5, 2026Kaustubh Pethkar, Ziyang Xiong, Zuofeng Shang +1Continual Learning for LLMs Memorization in Language Models
New Jersey Institute of Technology · University of California, Berkeley
Before Forgetting, Learn to Remember: Revisiting Foundational Learning Failures in LVLM Unlearning Benchmarks
May 5, 2026JuneHyoung Kwon, MiHyeon Kim, Eunju Lee +3VLM Evaluation VLM Unlearning
Department of Artificial Intelligence, Chung-Ang University · KT Corporation · Graduate School of Advanced Imaging Sciences, Multimedia and Film, Chung-Ang University
Probe-Geometry Alignment: Erasing the Cross-Sequence Memorization Signature Below Chance
May 3, 2026Anamika Paul Rupa, Anietie AndyMemorization in Language Models Neural Network Memorization
Howard University
Learning How and What to Memorize: Cognition-Inspired Two-Stage Optimization for Evolving Memory
May 1, 2026Derong Xu, Shuochen Liu, Pengfei Luo +8Memorization in Language Models Memorization in Generative Models
University of Science and Technology of China & State Key Laboratory of Cognitive Intelligence · City University of Hong Kong · Dalian University of Technology +1
Language Diffusion Models are Associative Memories Capable of Retrieving Unseen Data
Apr 29, 2026Bao Pham, Mohammed J. Zaki, Luca Ambrogioni +2Memorization in Language Models Neural Network Memorization
1Rensselaer Polytechnic Institute (RPI) · 2Donders Institute for Brain, Cognition, and Behaviour, Radboud University · 3Dynamical Mind +1
On the Memorization of Consistency Distillation for Diffusion Models
Apr 26, 2026Bingqing Jiang, Difan ZouMemorization in Generative Models Consistency Distillation
School of Computing & Data Science, The University of Hong Kong. · School of Computing & Data Science and Institute of Data Science, The University of Hong Kong.

May 29, 2026, cs.CLJ/K move · Enter open · S save

Divergence Decoding: Inference-Time Unlearning via Auxiliary Models

Humzah Merchant, Bradford Levy

University of Chicago, Chicago, IL 60637.

Privacy Leakage in Language Models Memorization in Generative Models Machine Unlearning LLM Unlearning

Open paper AskPDF arXiv

Large Language Models (LLMs) frequently memorize sensitive training data thereby creating significant privacy and copyright risks. Addressing these risks, i.e., removing such knowledge from an existing model checkpoint, has proven challenging as many unlearning methods lead to catastrophic utility loss or are ineffective for complex queries. We introduce Divergence Decoding (DD), a mechanism that uses small auxiliary models to steer the logits of the LLM away from specific data during inference. Training these models is straight forward, i.e., we use standard pre-training and fine-tuning setups. We find the method decisively outperforms state-of-the-art (SOTA) baselines on unlearning benchmarks across a variety of model and training dataset scales consistent with DD being an effective and inexpensive solution to unlearning. We then demonstrate that this steered distribution can be trivially distilled back into the base model. Since the method is generally applicable to any probabilistic model, we explore its efficacy outside of text generation and find evidence of generalization to the domain of images.

Momentum

Related topics

Latest papers 70