cs.CVSep 30, 2025

How Diffusion Models Memorize

Authors: Juyeop Kim, Songkuk Kim, Jong-Seok Lee

Organizations: Yonsei University, Korea

Abstract

Despite their success in image generation, diffusion models can unintentionally memorize and reproduce training data, raising serious privacy and copyright concerns. Although prior work has studied memorization from different perspectives, a unified account of how this reproduction occurs during generation remains incomplete. In this paper, we develop a mathematical account of this process in text-to-image latent diffusion models with classifier-free guidance. We trace reproduction across denoising steps through predictions of the final denoised latent made with and without the prompt. Assuming these predictions are Bayes-optimal and uniformly bounded, we show that sufficiently low loss on a training prompt--image pair enables the prediction made with the prompt to reproduce the paired training latent with high probability, even at the first denoising step. At this first step, we bound the difference between the prediction made without the prompt and the empirical mean of the training latents, showing how guidance amplifies the reproduced training latent's deviation from that mean. We then quantify how the latent passed between denoising steps carries prompt influence to subsequent predictions made without the prompt. When both predictions are sufficiently close to the paired training latent at the final step, their guided combination also reproduces it in the final generated latent. We also analyze how their difference evolves, showing that closer agreement alone does not establish reproduction. Finally, our account provides a unified explanation of existing detection and mitigation strategies.

Figures & tables

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Broken Memories: Detecting and Mitigating Memorization in Diffusion Models with Degraded Generations

    May 21, 2026Yuanmin Huang, Mi Zhang, Chen Chen +4Memorization in Generative ModelsDiffusion Models

  2. You Don't Need All That Attention: Surgical Memorization Mitigation in Text-to-Image Diffusion Models

    Feb 23, 2026Kairan Zhao, Eleni Triantafillou, Peter TriantafillouMemorization in Generative ModelsT2I Diffusion Models

  3. Steering Away from Memorization: Reachability-Constrained Reinforcement Learning for Text-to-Image Diffusion

    Feb 24, 2026Sathwik Karnik, Juyeop Kim, Sanmi Koyejo +2Memorization in Generative ModelsT2I Diffusion Models