Despite their success in image generation, diffusion models can unintentionally memorize and reproduce training data, raising serious privacy and copyright concerns. Although prior work has studied memorization from different perspectives, a unified account of how this reproduction occurs during generation remains incomplete. In this paper, we develop a mathematical account of this process in text-to-image latent diffusion models with classifier-free guidance. We trace reproduction across denoising steps through predictions of the final denoised latent made with and without the prompt. Assuming these predictions are Bayes-optimal and uniformly bounded, we show that sufficiently low loss on a training prompt--image pair enables the prediction made with the prompt to reproduce the paired training latent with high probability, even at the first denoising step. At this first step, we bound the difference between the prediction made without the prompt and the empirical mean of the training latents, showing how guidance amplifies the reproduced training latent's deviation from that mean. We then quantify how the latent passed between denoising steps carries prompt influence to subsequent predictions made without the prompt. When both predictions are sufficiently close to the paired training latent at the final step, their guided combination also reproduces it in the final generated latent. We also analyze how their difference evolves, showing that closer agreement alone does not establish reproduction. Finally, our account provides a unified explanation of existing detection and mitigation strategies.
Figures & tables
Figure 1: Denoising measurements on SDv1 (DDIM). Coral and purple in (a,c,d,f) denote memorized and normal prompt groups, respectively ( Sec. 4.1 ). In (a), dashed and solid curves correspond to forward latents and Gaussian initializations, respectively. In (a–c,f), curves show means across prompts, and shaded bands indicate ±1 standard deviation. Seeds are first averaged within each prompt in (b,f). In (d), curves and shaded bands show the median and interquartile range, respectively, with each prompt receiving equal total weight. In (e), point colors indicate each generated image’s final SSCD, and the dotted diagonal denotes equality.
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
MV
RV
TV
N
Total
SDv1
86/86
30/30
229/229
101/155
446/500
SDv2
2/4
0/0
208/215
180/281
390/500
RealVis
89/90
34/34
229/230
94/146
446/500
Appendix
Table 1: Dataset. Original prompt categories and counts across different diffusion models. Fractions indicate the number of prompts with available paired target images over the total number of prompts originally provided by Webster [2023] .
Figure 2: Denoising measurements on SDv1 (DDPM). Coral and purple in (a,c,d,f) denote memorized and normal prompt groups, respectively ( Sec. 4.1 ). In (a), dashed and solid curves correspond to forward latents and Gaussian initializations, respectively. In (a–c,f), curves show means across prompts, and shaded bands indicate ±1 standard deviation. Seeds are first averaged within each prompt in (b,f). In (d), curves and shaded bands show the median and interquartile range, respectively, with each prompt receiving equal total weight. In (e), point colors indicate each generated image’s final SSCD, and the dotted diagonal denotes equality.
Figure 3: Denoising measurements on SDv2 (DDIM). Coral and purple in (a,c,d,f) denote memorized and normal prompt groups, respectively ( Sec. 4.1 ). In (a), dashed and solid curves correspond to forward latents and Gaussian initializations, respectively. In (a–c,f), curves show means across prompts, and shaded bands indicate ±1 standard deviation. Seeds are first averaged within each prompt in (b,f). In (d), curves and shaded bands show the median and interquartile range, respectively, with each prompt receiving equal total weight. In (e), point colors indicate each generated image’s final SSCD, and the dotted diagonal denotes equality.
Figure 4: Denoising measurements on RealVis (DDIM). Coral and purple in (a,c,d,f) denote memorized and normal prompt groups, respectively ( Sec. 4.1 ). In (a), dashed and solid curves correspond to forward latents and Gaussian initializations, respectively. In (a–c,f), curves show means across prompts, and shaded bands indicate ±1 standard deviation. Seeds are first averaged within each prompt in (b,f). In (d), curves and shaded bands show the median and interquartile range, respectively, with each prompt receiving equal total weight. In (e), point colors indicate each generated image’s final SSCD, and the dotted diagonal denotes equality.
Memorization in generative models studies when models retain training examples, how to detect their recall, and how to limit it without losing utility.
While diffusion models excel at generating high-quality images, their tendency to memorize training data poses significant privacy and copyright risks. In this work, we for the first time identify that memorization induces internal numerical instability, often manifesting as visually ``broken'' artifacts. Inspired by stability analysis in numerical methods, we introduce empirical stability regions based on latent update norms to quantitatively characterize stable behavior during generation. Leveraging this, we propose a principled, on-the-fly framework for step-wise detection and adaptive mitigation. Our approach suppresses memorization without altering prompts or guidance, thereby preserving semantic fidelity and image quality. Extensive experiments on Stable Diffusion 1.4 demonstrate that our method achieves an AUC >0.999 detection performance and a 0.0% memorization rate after mitigation with negligible overhead (≈0.01s per image).
Yuanmin Huang, Mi Zhang, Chen Chen +4
Fudan University Shanghai, China · East China University of Science and Technology Shanghai, China
Generative models have been shown to "memorize" certain training data, leading to verbatim or near-verbatim generating images, which may cause privacy concerns or copyright infringement. We introduce Guidance Using Attractive-Repulsive Dynamics (GUARD), a novel framework for memorization mitigation in text-to-image diffusion models. GUARD adjusts the image denoising process to guide the generation away from an original training image and towards one that is distinct from training data while remaining aligned with the prompt, guarding against reproducing training data, without hurting image generation quality. We propose a concrete instantiation of this framework, where the positive target that we steer towards is given by a novel method for (cross) attention attenuation based on (i) a novel statistical mechanism that automatically identifies the prompt positions where cross attention must be attenuated and (ii) attenuating cross-attention in these per-prompt locations. The resulting GUARD offers a surgical, dynamic per-prompt inference-time approach that, we find, is by far the most robust method in terms of consistently producing state-of-the-art results for memorization mitigation across two architectures and for both verbatim and template memorization, while also improving upon or yielding comparable results in terms of image quality.
Kairan Zhao, Eleni Triantafillou, Peter Triantafillou
Text-to-image diffusion models are susceptible to memorization, revealing a fundamental failure to generalize beyond the training set. Current mitigation approaches often sacrifice image quality or prompt alignment to reduce memorization. To address this, we propose Reachability-Aware Diffusion Steering (RADS), an offline-trained, sampling-time control framework that mitigates memorization while maintaining generation fidelity. RADS models the diffusion denoising process as a dynamical system and learns a policy-conditioned reachability value function that estimates future memorization risk along the denoising trajectory. This motivates a constrained reinforcement learning (RL) formulation, where a policy learns to steer the trajectory away from memorization via perturbations in the caption embedding space. Empirical evaluations show that RADS achieves a stronger empirical Pareto frontier between generation diversity (SSCD), quality (FID), and alignment (CLIP) compared to state-of-the-art inference-time baselines. Crucially, RADS applies the learned controller during sampling time, separating memorization mitigation from the pretrained model. Our website is available at: https://s-karnik.github.io/rads-memorization-project-page/.
Sathwik Karnik, Juyeop Kim, Sanmi Koyejo +2
Stanford University, Stanford, CA, USA · Yonsei University, Seoul, Korea.