Memorization in Generative Models

Overview

Memorization research asks when generative models reproduce training data rather than learn reusable patterns, raising concerns about privacy, copyright, and trustworthy evaluation. Researchers test language models with extraction prompts and canaries, and use membership inference, similarity measures, and functional comparisons; for diffusion models, they also study score geometry, denoising trajectories, and noise schedules. Mitigation approaches include training-time regularization, unlearning and weight filtering, and inference-time steering or sampling changes, evaluated against both memorization and retained generation quality. A central challenge is distinguishing harmful recall from legitimate generalization, since near-duplicates and dataset structure can confound audits, while reducing copying may also weaken fidelity or utility.

Latest papers

CardsList

94 papers7 papers in the last four weeks, down 50% on the four weeks before. 0.1% of all new papers.

  1. How Diffusion Models Memorize

    Sep 30, 2025Juyeop Kim, Songkuk Kim, Jong-Seok LeeMemorization in Generative ModelsT2I Diffusion Models

  2. Continual Memorization of Factoids in Language Models

    Nov 11, 2024Howard Chen, Jiayi Geng, Adithya Bhaskar +2Continual Learning for LLMsMemorization in Language Models

  3. Mitigating Memorization In Language Models

    Oct 3, 2024Mansi Sakarvadia, Aswathy Ajith, Arham Khan +6Memorization in Language ModelsLLM Fine-Tuning