Memorization in Generative Models

Follow this topic Ask about this topic

Momentum

8 papers in the last four weeks, against 1 the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 70

All topics

Disentangling Paradigm, Identifier, and Decoding in Generative Retrieval
Oct 6, 2026Hicham Randrianarivo, Logan Renaud, Alexia AllalGenerative Retrieval Memorization in Generative Models
Artefact Research Center France
Rethinking Memorization Mitigation in Diffusion Models: Reinforcing Text Conditioning
Oct 1, 2026Hyungjun Joo, Sehwan Kim, Hyeonggeun Han +2Diffusion Model Alignment Attention Control in Diffusion Models
Seoul National University · Konkuk University
From Modes to Memories: Characterizing the Scale-Space Dynamics of Diffusion Models
Sep 30, 2026Cristina López Amado, Marco Fumero, Francesco LocatelloMemorization in Generative Models Diffusion Models
Institute of Science and Technology Austria (ISTA)
PEG-Tab: Sampling-Time Record Repair and Release Control for Tabular Synthesis
Sep 30, 2026Pengfei Li, QinYi Liu, Mohammad KhalilSynthetic Data Generation Synthetic Tabular Data Generation
Ghost in the Encoder: Decodable Artist Identity Representations in Lyrics-to-Song Generation
Sep 30, 2026Arhan Vohra, Choenden Kyirong, Laura Ibáñez-Martínez +1Text-to-Music Generation Audio Diffusion Models
Music Technology Group, Universitat Pompeu Fabra
First Learn, Then Memorize: The Spectral Bias of Diffusion Models
Sep 28, 2026Raphaël Urfin, Tony Bonnaire, Giulio Biroli +1Neural Network Generalization Memorization in Generative Models
Laboratoire de Physique de l’École normale supérieure, ENS, Université PSL, CNRS, Sorbonne Université, Université Paris Cité, F-75005 Paris, France · Université Paris-Saclay, CNRS, Institut d’Astrophysique Spatiale, 91405 Orsay, France · Department of Computing Sciences, Bocconi University, Milano, Italy
Double Descent and Malign Overfitting in Diffusion Models
Sep 22, 2026Raphaël Urfin, Tony Bonnaire, Giulio Biroli +1Double Descent Benign Overfitting
Laboratoire de Physique de l’École normale supérieure, ENS, Université PSL, CNRS, Sorbonne Université, Université Paris Cité, F-75005 Paris, France · Université Paris-Saclay, CNRS, Institut d’Astrophysique Spatiale, 91405 Orsay, France · Department of Computing Sciences, Bocconi University, Milano, Italy
Mitigating Sequential Reappearance in Diffusion Data-Point Unlearning
Sep 21, 2026Donghyun Kim, Taehyuk Lee, Jinyeong Kim +4Memorization in Generative Models Diffusion Model Unlearning
Department of Computer Science and Engineering, Konkuk University Trustworthy Machine Learning Lab Seoul, Republic of Korea
Copying Versus Randomization in Lempel-Ziv Music Synthesis
Sep 11, 2026Nadav Mishan, Ram ZamirControllable Music Generation Symbolic Music Generation
Department of Electrical Engineering Tel Aviv University Tel Aviv, Israel
Auditing Patient Privacy in Medical Generative Models: Scalable Memorization Detection with DeepSSIM++
Sep 3, 2026Antonio Scardace, Francesco Guarnera, Sebastiano Battiato +1Privacy Auditing Memorization in Generative Models
University of Catania, Department of Mathematics and Computer Science, Catania, Italy
Zero Gap Is Not Restoration: Stratified Per-Question Probability Evaluation and Step-wise Mitigation of Benchmark Contamination
Aug 7, 2026Ruijie Hou, Yueyang Jiao, Zhao Wang +1LLM Evaluation Memorization in Generative Models
Zhejiang University
The Geometry of Memorization: Finite-Time Spectral Sensitivity as a Diagnostic for Flow Matching Models
Jul 14, 2026Shuchan WangFlow Matching Neural Network Memorization
An exact information theory of generalization phase transitions in Bayesian diffusion models
Jul 9, 2026Henry Hunt, Mason Kamb, Surya GanguliMemorization in Generative Models Diffusion Models
Department of Applied Physics Stanford University Stanford, CA 94305
Anchored or Drifting: What Recursive Self-Generation Reveals About Training Data
Jun 30, 2026Wojciech Łapacz, Stanisław PawlakMembership Inference Attacks Memorization in Generative Models
Warsaw University of Technology
Explaining Machine Learning and Memorization with Statistical Mechanics
Jun 30, 2026Robin TheriaultStatistical Physics of Learning Neural Network Memorization
Faculty of Science
Cyclic Denoising Reveals Ultrastable Memories in Diffusion Models
Jun 22, 2026Rishabh Sharma, Stefano MartinianiDiffusion Model Sampling Memorization in Generative Models
Simons Center for Computational Physical Chemistry, New York University · Center for Soft Matter Research, Department of Physics, New York University · Center for Neural Science, New York University +1
MGI: Member vs Generated Inference
Jun 22, 2026Bihe Zhao, Michel Meintz, Juangui Xu +2Image Generation Memorization in Generative Models
CISPA Helmholtz Center for Information Security
Substitution-Based Analysis of Structural Novelty for Generative Models of Materials
Jun 22, 2026Masahiro Negishi, Aron WalshMemorization in Generative Models Crystal Structure Generation
Department of Materials, Imperial College London, Exhibition Road, South Kensington, SW7 2AZ, London, United Kingdom.
NAMESAKES: Probing Identity Memorization in Text-to-Image Models
Jun 18, 2026Morris Alper, Vasudha Varadarajan, Moran Yanuka +2Memorization in Generative Models T2I Generation Evaluation
Carnegie Mellon University · Tel Aviv University · Cornell University
Learning What to Remember: A Cognitively Grounded Multi-Factor Value Model for Agentic Memory
Jun 11, 2026Zhibao Chen, Qian ChengLLM Agent Memory Persistent Memory for Language Models
Huatai Securities, Nanjing, China
Detecting Functional Memorization in Code Language Models
Jun 11, 2026Matthieu Meeus, Anil Ramakrishna, Matthew Grange +2Memorization in Language Models LLM Auditing
Meta · Imperial College London
Substrate Asymmetry in User-Side Memory: A Diagnostic Framework
Jun 10, 2026Youwang DengRetrieval-Augmented Generation LLM Personalization
EpistemicaLab — Independent Research
Deployment-Time Memorization in Foundation-Model Agents
Jun 8, 2026Lei, Chen, Guilin Zhang +8Persistent Memory for Language Models Privacy Leakage in Language Models
Rachel
Echo-Memory: A Controlled Study of Memory in Action World Models
Jun 8, 2026Wayne King, Zeyue Xue, Yuxuan Bian +13Action-Conditioned Video Generation Video Diffusion Models
The University of Hong Kong · Joy Future Academy, JD · The Chinese University of Hong Kong +5
Evaluating the Representation Space of Diffusion Models via Self-Supervised Principles
Jun 8, 2026Xiao Li, Yixuan Jia, Zekai Zhang +6Self-Supervised Learning Memorization in Generative Models
University of Michigan · 1Ohio State University
A Theoretical Analysis of Memory and Overfitting Phenomena in Stochastic Interpolation Models
Jun 7, 2026Yunchen Li, Shaohui Lin, Zhou YuStochastic Interpolants Memorization in Generative Models
East China Normal University
Where Rectified Flows Leak: Characterising Membership Signals Along the Interpolation Path
Jun 5, 2026Thomas Sesmat, Gabriel Meseguer-Brocal, Geoffroy PeetersData Leakage Rectified Flow
LTCI, Télécom Paris, Institut Polytechnique de Paris, Palaiseau, France · Deezer Research, Paris, France
LLMs Can Leak Training Data But Do They Want To? A Propensity-Aware Evaluation of Memorization in LLMs
Jun 4, 2026Gianluca Barmina, Peter Schneider-Kamp, Lukas Galke PoechData Leakage Memorization in Language Models
University of Southern Denmark
SaliMory: Orchestrating Cognitive Memory for Conversational Agents
Jun 2, 2026Kai Zhang, Xinyuan Zhang, Hongda Jiang +11Persistent Memory for Language Models Memorization in Generative Models
Meta Reality Labs
DecMem: Towards Minute-Long Consistent World Generation with Decoupled Memory
May 29, 2026Zhenhao Yang, Xiaoshi Wu, Zhengyao Lv +5Long-Horizon Video Generation Memory-Augmented Video Generation
1The University of Hong Kong · *Work done during an internship at Kling Team, Kuaishou Tech. · 2Kling Team, Kuaishou Technology
Divergence Decoding: Inference-Time Unlearning via Auxiliary Models
May 29, 2026Humzah Merchant, Bradford LevyPrivacy Leakage in Language Models Memorization in Generative Models
University of Chicago, Chicago, IL 60637.
Diffusion Models Preferentially Memorize Prototypical Examples or: Why Does My Diffusion Model Love Slop?
May 28, 2026Marta Aparicio Rodriguez, Anastasia Borovykh, Grigorios A. Pavliotis +1Privacy Leakage in Language Models Memorization in Generative Models
Department of Mathematics, Imperial College London, UK · ML Lab, Capital Fund Management, France · Department of Physics, École Polytechnique Fédérale de Lausanne (EPFL), Switzerland
How LoRA Remembers? A Parametric Memory Law for LLM Finetuning
May 28, 2026Ziwen Xu, Haiwen Hong, Linsong Yu +4Memorization in Language Models LLM Fine-Tuning
Zhejiang University · Alibaba Group
NumLeak: Public Numeric Benchmarks as Latent Labels in Foundation Models
May 28, 2026Anany KotawalaData Leakage LLM Evaluation
Princeton University, Princeton, NJ, USA.
The Attribution Blind Spot: Detecting When Language Models Rely on Memory Rather Than Retrieved Context
May 26, 2026Zhe Yu, Wenpeng Xing, Yunzhao Wei +4Memorization in Language Models LLM Interpretability
Binjiang Institute of Zhejiang University · Zhejiang University · National Fintech Evaluation Center +2
Localizing Memorized Regions in Diffusion Models via Coordinate-Wise Curvature Differences
May 26, 2026Gwangho Kim, Sungyoon LeeMemorization in Generative Models Diffusion Models
Department of Computer Science, Hanyang University, Seoul, South Korea.
Simulating Human Memory with Language Models
May 25, 2026Qihan Wang, Nicholas Tomlin, Michael Hu +2LLM Evaluation Memory-Augmented Language Models
NYU · UMass Amherst
Teaching Video Generators to Remember: Eliciting Dynamic Memory for Out-of-Sight State Evolution
May 25, 2026Tianshuo Xu, Yichen Xie, Depu Meng +5Video Diffusion Models Long-Horizon Video Generation
Applied Intuition · University of California, Berkeley
Extracting Training Data from Diffusion Language Models via Infilling
May 22, 2026Yihan Wang, N. AsokanData Leakage Masked Diffusion Models
University of Waterloo · KTH Royal Institute of Technology
Memorization Dynamics of Fill-in-the-Middle Pretraining
May 21, 2026Tobias von Arx, Tanguy DieudonnéLanguage Model Pretraining Memorization in Language Models
Department of Computer Science, ETH Zurich, Zurich, Switzerland.
Broken Memories: Detecting and Mitigating Memorization in Diffusion Models with Degraded Generations
May 21, 2026Yuanmin Huang, Mi Zhang, Chen Chen +4Memorization in Generative Models Diffusion Models
Fudan University Shanghai, China · East China University of Science and Technology Shanghai, China
Memorisation, convergence and generalisation in generative models
May 20, 2026Antoine Maillard, Sebastian GoldtNeural Network Memorization Memorization in Generative Models
1INRIA Paris & DI ENS, PSL University, Paris, France · 2International School of Advanced Studies (SISSA), Trieste, Italy
Adynamical systems view of training generativemodels and the memorization phenomenon
May 19, 2026Siva Athreya, Chiranjib Bhattacharya, Vivek S. BorkarNeural Network Memorization Memorization in Generative Models
International Institute for Theoretical Sciences, Survey No. 151, Hesarghatta Road, Shivakote Hobli, Bengaluru 560089, India · Department of Computer Science and Automation, Indian Institute of Science, Bengaluru 560012, India · Department of Electrical Engineering, Indian Institute of Technology Bombay, Powai, Mumbai 400076, India
Reducing Diffusion Model Memorization with Higher Order Langevin Dynamics
May 18, 2026Benjamin Sterling, Mónica F. Bugallo, Tom TirerLangevin Dynamics Memorization in Generative Models
Department of Applied Math & Statistics Stony Brook University Stony Brook, NY 11790 · Department of Electrical and Computer Engineering Stony Brook University Stony Brook, NY 11790 · Faculty of Engineering Bar-Ilan University Ramat-Gan 5290002, Israel
Auditing Reasoning-Trace Memorization Claims after Unlearning with Head-Conditioned Canaries
May 17, 2026Yanhang Li, Zhichao Fan, Zexin ZhuangMemorization in Language Models LLM Auditing
Northeastern University, USA · University of Illinois Urbana-Champaign, USA · Southern Methodist University, USA
Phasor Memory Networks: Stable Backpropagation Through Time for Scalable Explicit Memory
May 13, 2026Sungwoo Goo, Hwi-yeol Yun, Sangkeun JungMemory-Augmented Neural Networks Memory-Augmented Language Models
College of Pharmacy Chungnam National University Daejeon, South Korea · Department of Computer Science & Engineering Chungnam National University Daejeon, South Korea
Geometric Factual Recall in Transformers
May 12, 2026Shauli Ravfogel, Gilad Yehudai, Joan Bruna +1Neural Representation Geometry Memorization in Language Models
1New York University · 2Flatiron Institute
$δ$ -mem: Efficient Online Memory for Large Language Models
May 12, 2026Jingdi Lei, Di Zhang, Junxian Li +7Memory-Augmented Language Models Memorization in Language Models
Nanyang Technological University · Mind Lab · Fudan University +3
diffGHOST: Diffusion based Generative Hedged Oblivious Synthetic Trajectories
May 11, 2026Florent Guépin, Cheick Tidiani Cisse, Denis Renaud +2Trajectory Generation Synthetic Data Generation
Orange Research, Belfort, France · Orange Business, Belfort, France
Filtering Memorization from Parameter-Space in Diffusion Models
May 11, 2026Yu Zhe, Yang Jiayan, Wei Junhao +2Low-Rank Adaptation Memorization in Generative Models
RIKEN AIP · Institute of Science Tokyo · University of California, Berkeley +1
The two clocks and the innovation window: When and how generative models learn rules
May 11, 2026Binxu Wang, Emma Lucia Byrnes Finn, Bingbin LiuMemorization in Generative Models Generative Modeling
Kempner Institute at Harvard University
When Graph Language Models Go Beyond Memorization
May 7, 2026Masatsugu Yamada, Mahito SugiyamaGraph Generation Memorization in Language Models
National Institute of Informatics Tokyo, Japan · National Institute of Informatics, Tokyo, Japan · SOKENDAI, Kanagawa, Japan
Understanding diffusion models requires rethinking (again) generalization
May 7, 2026Pierre Marion, Yu-Han WuMemorization in Generative Models Diffusion Models
Inria, École Normale Supérieure, PSL Research University · LPSM, Sorbonne University & Google DeepMind
Attractor Geometry of Transformer Memory: From Conflict Arbitration to Confident Hallucination
May 7, 2026Qiyao Liang, Risto Miikkulainen, Ila FieteMemorization in Language Models Knowledge Conflicts in Language Models
Massachusetts Institute of Technology · University of Texas Austin, Cognizant · Cognizant
Memory as a Markov Matrix: Sample Efficient Knowledge Expansion via Token-to-Dictionary Mapping
May 5, 2026Kaustubh Pethkar, Ziyang Xiong, Zuofeng Shang +1Continual Learning for LLMs Memorization in Language Models
New Jersey Institute of Technology · University of California, Berkeley
Before Forgetting, Learn to Remember: Revisiting Foundational Learning Failures in LVLM Unlearning Benchmarks
May 5, 2026JuneHyoung Kwon, MiHyeon Kim, Eunju Lee +3VLM Evaluation VLM Unlearning
Department of Artificial Intelligence, Chung-Ang University · KT Corporation · Graduate School of Advanced Imaging Sciences, Multimedia and Film, Chung-Ang University
Probe-Geometry Alignment: Erasing the Cross-Sequence Memorization Signature Below Chance
May 3, 2026Anamika Paul Rupa, Anietie AndyMemorization in Language Models Neural Network Memorization
Howard University
Learning How and What to Memorize: Cognition-Inspired Two-Stage Optimization for Evolving Memory
May 1, 2026Derong Xu, Shuochen Liu, Pengfei Luo +8Memorization in Language Models Memorization in Generative Models
University of Science and Technology of China & State Key Laboratory of Cognitive Intelligence · City University of Hong Kong · Dalian University of Technology +1
Language Diffusion Models are Associative Memories Capable of Retrieving Unseen Data
Apr 29, 2026Bao Pham, Mohammed J. Zaki, Luca Ambrogioni +2Memorization in Language Models Neural Network Memorization
1Rensselaer Polytechnic Institute (RPI) · 2Donders Institute for Brain, Cognition, and Behaviour, Radboud University · 3Dynamical Mind +1
On the Memorization of Consistency Distillation for Diffusion Models
Apr 26, 2026Bingqing Jiang, Difan ZouMemorization in Generative Models Consistency Distillation
School of Computing & Data Science, The University of Hong Kong. · School of Computing & Data Science and Institute of Data Science, The University of Hong Kong.

Oct 6, 2026, cs.IRJ/K move · Enter open · S save

Disentangling Paradigm, Identifier, and Decoding in Generative Retrieval

Hicham Randrianarivo, Logan Renaud, Alexia Allal

Artefact Research Center France

Generative Retrieval Memorization in Generative Models Diffusion Language Model Decoding

Open paper AskPDF arXiv

Generative retrieval trains a language model to generate the identifier of a relevant document. Recent work replaces the autoregressive decoder with diffusion, but changes identifiers, training recipe and decoding at once, so differences cannot be credited to the paradigm. On NQ320K and MS300K, we train autoregressive, masked-diffusion and block-diffusion models with residual-quantised, product-quantised and random identifiers. With identifier length and training budget fixed, we decode each model in several ways. Decoding alone moves a diffusion model's Hit@1 by 6.6 to 13.7 points. Our reference diffusion decoding, generate-and-match, generates an identifier, then retrieves the closest corpus identifiers. The generated identifier is right for 14-21% of NQ320K queries. We test one-pass scoring to decode diffusion retrievers: the model reads a fully masked identifier once, and each document is scored by its codes' probabilities. It matches or beats generate-and-match in 11 of 12 settings. Autoregressive models still lead in Hit@1; on NQ320K, the lead comes from the model, not beam search. Starting from one sampled identifier, one-pass scoring removes 46-83% of masked diffusion's deficit to beam search; from generate-and-match, at most a quarter. On NQ320K, every paradigm largely memorises which identifier answers which query: random identifiers keep 83-90% of the Hit@1 of residual-quantised ones. There, product-quantised identifiers lead residual-quantised ones by 3.4 points in the autoregressive model and by -0.7 to +3.6 in diffusion models; across decodings, AR's gap exceeds diffusion's by 1.5-2.3 points, around our 2-point threshold. Paradigm comparisons must report each paradigm at its own recipe and best decoding.

Momentum

Related topics

Latest papers 70