Understanding how textual embeddings contribute to memorization in text-to-image diffusion models is crucial for both interpretability and safety. This paper investigates an unexpected behavior of CLIP embeddings in Stable Diffusion, revealing that the model disproportionately relies on specific embeddings. We categorize input tokens as <startoftext>, <prompt>, <endoftext> and <pad> with corresponding embeddings vsot,vpr,veot,vpad. We discover that vpr contribute minimally to generation in memorized cases. In contrast, vpad strongly affect memorization due to their structural duplication of veot, the only embedding explicitly optimized during CLIP training. This duplication unintentionally amplifies the influence of veot, causing the model to over-rely on it, thereby driving memorization. Based on these observations, we propose two simple yet effective inference-time mitigation strategies: (1) Replacing the tokenizer's default <pad> from <eot> to the ! token before embedding, and masking the veot; (2) Partial masking of vpad. Both suppress memorization without degrading quality, and are readily deployable without prior detection.
Figures & tables
Token (Category: Sequence)
Embedding
<startoftext>: <sot>
vsot
<prompt>: <pr> 1 ,…,<pr> n
v1pr,…,vnpr
<endoftext>: <eot>
veot
<pad>: <pad> 1 ,…,<pad> d
v1pad,…,vdpad
Table 1: Text embedding vectors consist of four types.
Figure 1: Mitigating memorization via <pad> replacement and veot masking. We identify <pad> embeddings ( vpad ) as a major contributor to memorization, as they are implicit duplications of the <endoftext> embedding ( veot ). To mitigate this, we replace the tokenizer’s default <pad> ( <eot> ) with the ! token, and mask veot . This method effectively suppresses memorization while preserving image quality and prompt alignment. Each row shows results from the same prompt across different seeds: the first column (Original) is generated with default embeddings and consistently reproduces the same image regardless of the seeds. The remaining five columns (Ours) are generated with our proposed mitigation under five different seeds. Additional results are provided in Appendix A .
Figure 2: A text prompt is first processed by the tokenizer into discrete tokens, which are then transformed by the CLIPText into embeddings. All embeddings, including vipr and vipad , serve as conditioning inputs for the model.
Figure 3: vipr are not important. (a) shows that vipr alone cannot guide generation, as the resulting images collapse when they are the only embeddings retained. In contrast, (b) replacing vipr with veot or (c) masking vipr preserve most of the original structure.
Method
SSCD
CLIPScore
Aesthetic
(a)
0.04 ± 0.05
0.25 ± 0.06
5.02 ± 0.45
(b)
0.55 ± 0.14
0.29 ± 0.02
5.10 ± 0.23
(c)
0.42 ± 0.18
0.29 ± 0.03
5.15 ± 0.30
Table 2: The role of vipr in memorized image generation. (a) veot,vipad→0 : only vipr remain while veot and vipad are masked. (b) vipr→veot : vipr are replaced with veot . (c) vipr→0 : vipr are masked.
Figure 4: vipad are important. (d) to (g) show cases where vipad are preserved, either by replacing vipad with veot , remaining only veot , masking veot , or remaining only vipad . All four preserve the structure and semantics of the original image, whereas (h) masking vipad generates images that deviate significantly from it.
Method
SSCD
CLIPScore
Aesthetic
(d)
0.85 ± 0.07
0.32 ± 0.01
5.28 ± 0.07
(e)
0.49 ± 0.13
0.29 ± 0.02
5.05 ± 0.24
(f)
0.95 ± 0.05
0.32 ± 0.01
5.31 ± 0.07
(g)
0.50 ± 0.17
0.29 ± 0.02
5.09 ± 0.28
(h)
0.07 ± 0.06
0.29 ± 0.04
5.10 ± 0.42
Table 3: The role of veot and vipad in memorized image generation. (d) vipad→veot : vipad are replaced with veot . (e) vipr,vipad→veot : vipr and vipad are replaced with veot . (f) veot→0 : veot is masked. (g) vipr,veot→vmeanpad : vipr and veot are replaced with the mean of vipad . (h) vipad→0 : vipad are masked, resulting in collapse.
Figure 5: While Chen et al. [3] reported a “Bright Ending”, an attention spike on <eot> (specifically on veot ) when memorization occurs, we observe that this effect is not limited to veot . Multiple <pad> ( vpad ) adjacent to <eot> also exhibit elevated attention, indicating that they play a substantial role in the memorization process rather than serving as placeholders.
Figure 6: Comparison of mitigation results across five methods using the same prompt and seed. Our method (second column) mitigates memorization while preserving structure and prompt alignment. Ren et al. [24] reduce memorization but introduce distortion, while Wen et al. [36] , RTA and RNA [ 31 ] often fail to mitigate memorization. Additional examples are in Appendix A.2 .
Category
Method
SSCD
CLIPScore
Aesthetic
LPIPS
Replace <pad> Token (Ours)
<pad> → ! & veot→0
0.08 ± 0.06
0.31 ± 0.03
5.09 ± 0.44
0.67 ± 0.05
<pad> → !
0.14 ± 0.08
0.32 ± 0.03
5.21 ± 0.38
0.64 ± 0.09
Mask vipad Embedding (Ours)
vipad→0 (70%)
0.10 ± 0.07
0.31 ± 0.04
5.13 ± 0.41
0.66 ± 0.06
Prior Works
Ren et al. [24]
0.09 ± 0.06
0.30 ± 0.03
5.18 ± 0.39
0.65 ± 0.05
Wen et al. [36]
0.51 ± 0.35
0.32 ± 0.03
5.22 ± 0.33
0.49 ± 0.19
RTA [ 31 ]
0.58 ± 0.32
0.31 ± 0.03
5.23 ± 0.31
0.42 ± 0.22
Table 4: Simply changing the <pad> token mitigates a substantial portion of memorization, while additionally masking veot further suppresses leakage. We also introduce partial masking of vipad as a flexible alternative that enables fine-grained control. Compared to prior works (bottom), our method achieves low SSCD while maintaining prompt alignment, image quality, and perceptual diversity.
Figure 7: Partial Masking vipad to mitigate memorization. (b) and (c) show generations where 70% and 100% of the vipad adjacent to veot are masked, respectively. Masking 70% empirically strikes a well-balanced tradeoff, substantially reducing memorization with negligible quality degradation, whereas full masking (100%) can often cause collapse or noticeable degradation.
Method
CLIPScore
Aesthetic
Original
0.35 ± 0.05
5.47 ± 0.51
<pad> → ! & veot→0
0.34 ± 0.06
5.42 ± 0.51
<pad> → !
0.35 ± 0.05
5.49 ± 0.52
vipad→0 (70%)
0.34 ± 0.06
5.41 ± 0.51
Table 5: Our methods preserve image quality: on 1,500 non-memorized prompts, CLIPScore and Aesthetic Score remain within the noise of the original, indicating no meaningful degradation or substantial deviation.
Figure 8: Using our mitigation, which replaces <pad> with the ! token and masks veot , we observe that prompt region before veot remains largely unchanged, thereby preserving vpr utility. The mitigation consistently suppresses attention on veot and vpad , effectively deactivating both embeddings. Overall, the mitigation removes the root cause without compromising the intended prompt semantics.
Appendix figures & tables12 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 9: <pad> replacement and veot masking. The first column (Original) shows the memorized image consistently reproduced from the original embedding regardless of seeds. The remaining five columns (Ours) are generated using our mitigation method with five different random seeds.
Figure 10: Each row shows images generated from the same prompt using different mitigation methods. Visually and quantitatively, our method produces a more favorable quality-preserving trade-off.
Method
Inference Time (s)
Ours
3.17 ± 0.01
Ren et al. [24]
3.37 ± 0.02
Wen et al. [36]
3.55 ± 0.16
RNA [ 31 ]
3.26 ± 0.02
RTA [ 31 ]
3.31 ± 0.02
Appendix
Table 6: Average inference time per image for each method.
Method
SSCD
CLIPScore
Aesthetic
Ours
0.39 ± 0.20
0.29 ± 0.03
5.33 ± 0.26
Wen
0.65 ± 0.22
0.31 ± 0.03
5.35 ± 0.23
Ren
0.20 ± 0.15
0.26 ± 0.03
4.90 ± 0.31
RTA
0.70 ± 0.23
0.31 ± 0.03
5.36 ± 0.23
RNA
0.70 ± 0.23
0.32 ± 0.03
5.33 ± 0.25
Appendix
Table 7: Comparison of mitigation methods on all 500 candidate prompts from Webster [ 35 ] .
Method
SSCD
CLIPScore
Aesthetic
Ours
0.16 ± 0.23
0.30 ± 0.07
5.14 ± 0.53
Ren
0.15 ± 0.19
0.29 ± 0.06
5.17 ± 0.51
Wen
0.79 ± 0.37
0.31 ± 0.06
5.22 ± 0.48
Appendix
Table 8: Comparison of mitigation methods on Membench [ 13 ] .
Figure 11: (a) Original generation without swapping. (b) Generation after swapping both veot and vipad , showing that the output changes to reflect the swapped prompt.
Figure 12: “Original” is generated using the default embeddings, while “Ours” applies our mitigation strategy ( <pad> → ! & veot→0 ).
Figure 13: Each left column shows the reference image, and each right column shows the corresponding model-generated image.
Metric
SD1
SD2
Webster [35]
Hong et al. [13]
Webster [35]
Number of Prompts
345
3000
219
Mean Length
19.74
15.31
21.51
Max / Min Length
38 / 4
38 / 5
35 / 6
Appendix
Table 9: Prompt Token Length Statistics for Stable Diffusion v1.4 (SD1) and Stable Diffusion v2.1 (SD2)
Figure 14: Text embeddings of “Living in the Light with Ann Graham Lotz” visualized using PCA and t-SNE for (a) CLIP and (b) OpenCLIP models. The two projection methods reveal distinct clustering behaviors.
Figure 15: Cross Attention maps of “Mothers influence on her young hippo”
Figure 16: Cross Attention maps of “Living in the Light with Ann Graham”
Memorization in generative models studies when models retain training examples, how to detect their recall, and how to limit it without losing utility.
While diffusion models excel at generating high-quality images, their tendency to memorize training data poses significant privacy and copyright risks. In this work, we for the first time identify that memorization induces internal numerical instability, often manifesting as visually ``broken'' artifacts. Inspired by stability analysis in numerical methods, we introduce empirical stability regions based on latent update norms to quantitatively characterize stable behavior during generation. Leveraging this, we propose a principled, on-the-fly framework for step-wise detection and adaptive mitigation. Our approach suppresses memorization without altering prompts or guidance, thereby preserving semantic fidelity and image quality. Extensive experiments on Stable Diffusion 1.4 demonstrate that our method achieves an AUC >0.999 detection performance and a 0.0% memorization rate after mitigation with negligible overhead (≈0.01s per image).
Yuanmin Huang, Mi Zhang, Chen Chen +4
Fudan University Shanghai, China · East China University of Science and Technology Shanghai, China
Text-to-image diffusion models often memorize training data, revealing a fundamental failure to generalize beyond the training set. Current mitigation strategies typically sacrifice image quality or prompt alignment to reduce memorization. To address this, we propose Reachability-Aware Diffusion Steering (RADS), an inference-time framework that prevents memorization while preserving generation fidelity. RADS models the diffusion denoising process as a dynamical system and applies concepts from reachability analysis to approximate the "backward reachable tube"--the set of intermediate states that inevitably evolve into memorized samples. We then formulate mitigation as a constrained reinforcement learning (RL) problem, where a policy learns to steer the trajectory away from memorization via minimal perturbations in the caption embedding space. Empirical evaluations show that RADS achieves a superior Pareto frontier between generation diversity (SSCD), quality (FID), and alignment (CLIP) compared to state-of-the-art baselines. Crucially, RADS provides robust mitigation without modifying the diffusion backbone, offering a plug-and-play solution for safe generation. Our website is available at: https://s-karnik.github.io/rads-memorization-project-page/.
Sathwik Karnik, Juyeop Kim, Sanmi Koyejo +2
Stanford University, Stanford, CA, USA · Yonsei University, Seoul, Korea
Text-to-image diffusion models have achieved remarkable progress in image synthesis, yet can exhibit memorization by closely reproducing individual training examples. Effective mitigation must preserve useful prompt information to guide alternative depictions. We introduce a training-free method that redistributes cross-attention with Gaussian smoothing before reinforcing content-token contributions and attenuating padding contributions, without additional denoiser evaluations. With this intervention, stronger content conditioning can improve prompt alignment at comparable training-image similarity. A local analysis identifies when reinforcement preserves shared value information while redistribution reduces localized attention mass. On Stable Diffusion v1.4 and v2.0, all evaluated smoothing widths lie on the empirical Pareto frontiers for training-image similarity versus both prompt alignment and image preference. A configuration selected on Stable Diffusion reduces template reproduction in DeepFloyd IF without further tuning. These findings support jointly controlling conditioning allocation and strength to generate prompt-consistent alternatives.