How Diffusion Models Memorize
Organizations: Yonsei University, Korea
Abstract
Despite their success in image generation, diffusion models can unintentionally memorize and reproduce training data, raising serious privacy and copyright concerns. Although prior work has studied memorization from different perspectives, a unified account of how this reproduction occurs during generation remains incomplete. In this paper, we develop a mathematical account of this process in text-to-image latent diffusion models with classifier-free guidance. We trace reproduction across denoising steps through predictions of the final denoised latent made with and without the prompt. Assuming these predictions are Bayes-optimal and uniformly bounded, we show that sufficiently low loss on a training prompt--image pair enables the prediction made with the prompt to reproduce the paired training latent with high probability, even at the first denoising step. At this first step, we bound the difference between the prediction made without the prompt and the empirical mean of the training latents, showing how guidance amplifies the reproduced training latent's deviation from that mean. We then quantify how the latent passed between denoising steps carries prompt influence to subsequent predictions made without the prompt. When both predictions are sufficiently close to the paired training latent at the final step, their guided combination also reproduces it in the final generated latent. We also analyze how their difference evolves, showing that closer agreement alone does not establish reproduction. Finally, our account provides a unified explanation of existing detection and mitigation strategies.
Figures & tables
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
| MV | RV | TV | N | Total | |
|---|---|---|---|---|---|
| SDv1 | 86/86 | 30/30 | 229/229 | 101/155 | 446/500 |
| SDv2 | 2/4 | 0/0 | 208/215 | 180/281 | 390/500 |
| RealVis | 89/90 | 34/34 | 229/230 | 94/146 | 446/500 |
Memorization in Generative Models
Memorization in generative models studies when models retain training examples, how to detect their recall, and how to limit it without losing utility.
See all 94 papers