Memorization In Stable Diffusion Is Unexpectedly Driven by CLIP Embeddings

Bumjun Kim; Albert No

cs.CVApr 6, 2026

Memorization In Stable Diffusion Is Unexpectedly Driven by CLIP Embeddings

Authors: Bumjun Kim, Albert No

Organizations: Department of Artificial Intelligence, Yonsei University

Abstract

Understanding how textual embeddings contribute to memorization in text-to-image diffusion models is crucial for both interpretability and safety. This paper investigates an unexpected behavior of CLIP embeddings in Stable Diffusion, revealing that the model disproportionately relies on specific embeddings. We categorize input tokens as <startoftext>, <prompt>, <endoftext> and <pad> with corresponding embeddings $\mathbf{v}^{\mathbf{sot}}, \mathbf{v}^{\mathbf{pr}}, \mathbf{v}^{\mathbf{eot}}, \mathbf{v}^{\mathbf{pad}}$ . We discover that $\mathbf{v}^{\mathbf{pr}}$ contribute minimally to generation in memorized cases. In contrast, $\mathbf{v}^{\mathbf{pad}}$ strongly affect memorization due to their structural duplication of $\mathbf{v}^{\mathbf{eot}}$ , the only embedding explicitly optimized during CLIP training. This duplication unintentionally amplifies the influence of $\mathbf{v}^{\mathbf{eot}}$ , causing the model to over-rely on it, thereby driving memorization. Based on these observations, we propose two simple yet effective inference-time mitigation strategies: (1) Replacing the tokenizer's default <pad> from <eot> to the ! token before embedding, and masking the $\mathbf{v}^{\mathbf{eot}}$ ; (2) Partial masking of $\mathbf{v}^{\mathbf{pad}}$ . Both suppress memorization without degrading quality, and are readily deployable without prior detection.

Figures & tables

Token (Category: Sequence)	Embedding
<startoftext>: <sot>	$\mathbf{v}^{\mathbf{sot}}$
<prompt>: <pr> 1 ,…,<pr> n	$\mathbf{v}^{\mathbf{pr}}_{1},…,\mathbf{v}^{\mathbf{pr}}_{n}$
<endoftext>: <eot>	$\mathbf{v}^{\mathbf{eot}}$
<pad>: <pad> 1 ,…,<pad> d	$\mathbf{v}^{\mathbf{pad}}_{1},…,\mathbf{v}^{\mathbf{pad}}_{d}$

Table 1: Text embedding vectors consist of four types.

Figure 1: Mitigating memorization via <pad> replacement and $\mathbf{v}^{\mathbf{eot}}$ masking. We identify <pad> embeddings ( $\mathbf{v}^{\mathbf{pad}}$ ) as a major contributor to memorization, as they are implicit duplications of the <endoftext> embedding ( $\mathbf{v}^{\mathbf{eot}}$ ). To mitigate this, we replace the tokenizer’s default <pad> ( <eot> ) with the ! token, and mask $\mathbf{v}^{\mathbf{eot}}$ . This method effectively suppresses memorization while preserving image quality and prompt alignment. Each row shows results from the same prompt across different seeds: the first column (Original) is generated with default embeddings and consistently reproduces the same image regardless of the seeds. The remaining five columns (Ours) are generated with our proposed mitigation under five different seeds. Additional results are provided in Appendix A .

Figure 2: A text prompt is first processed by the tokenizer into discrete tokens, which are then transformed by the CLIPText into embeddings. All embeddings, including $\mathbf{v}^{\mathbf{pr}}_{i}$ and $\mathbf{v}^{\mathbf{pad}}_{i}$ , serve as conditioning inputs for the model.

Figure 3: $\mathbf{v}^{\mathbf{pr}}_{i}$ are not important. (a) shows that $\mathbf{v}^{\mathbf{pr}}_{i}$ alone cannot guide generation, as the resulting images collapse when they are the only embeddings retained. In contrast, (b) replacing $\mathbf{v}^{\mathbf{pr}}_{i}$ with $\mathbf{v}^{\mathbf{eot}}$ or (c) masking $\mathbf{v}^{\mathbf{pr}}_{i}$ preserve most of the original structure.

Method	SSCD	CLIPScore	Aesthetic
(a)	0.04 $\pm$ 0.05	0.25 $\pm$ 0.06	5.02 $\pm$ 0.45
(b)	0.55 $\pm$ 0.14	0.29 $\pm$ 0.02	5.10 $\pm$ 0.23
(c)	0.42 $\pm$ 0.18	0.29 $\pm$ 0.03	5.15 $\pm$ 0.30

Table 2: The role of

\mathbf{v}^{\mathbf{pr}}_{i}

in memorized image generation. (a)

\mathbf{v}^{\mathbf{eot}},\mathbf{v}^{\mathbf{pad}}_{i}\rightarrow\mathbf{0}

: only

\mathbf{v}^{\mathbf{pr}}_{i}

remain while

\mathbf{v}^{\mathbf{eot}}

and

\mathbf{v}^{\mathbf{pad}}_{i}

are masked. (b)

\mathbf{v}^{\mathbf{pr}}_{i}\rightarrow\mathbf{v}^{\mathbf{eot}}

\mathbf{v}^{\mathbf{pr}}_{i}

are replaced with

\mathbf{v}^{\mathbf{eot}}

. (c)

\mathbf{v}^{\mathbf{pr}}_{i}\rightarrow\mathbf{0}

\mathbf{v}^{\mathbf{pr}}_{i}

are masked.

Figure 4: $\mathbf{v}^{\mathbf{pad}}_{i}$ are important. (d) to (g) show cases where $\mathbf{v}^{\mathbf{pad}}_{i}$ are preserved, either by replacing $\mathbf{v}^{\mathbf{pad}}_{i}$ with $\mathbf{v}^{\mathbf{eot}}$ , remaining only $\mathbf{v}^{\mathbf{eot}}$ , masking $\mathbf{v}^{\mathbf{eot}}$ , or remaining only $\mathbf{v}^{\mathbf{pad}}_{i}$ . All four preserve the structure and semantics of the original image, whereas (h) masking $\mathbf{v}^{\mathbf{pad}}_{i}$ generates images that deviate significantly from it.

Method	SSCD	CLIPScore	Aesthetic
(d)	0.85 $\pm$ 0.07	0.32 $\pm$ 0.01	5.28 $\pm$ 0.07
(e)	0.49 $\pm$ 0.13	0.29 $\pm$ 0.02	5.05 $\pm$ 0.24
(f)	0.95 $\pm$ 0.05	0.32 $\pm$ 0.01	5.31 $\pm$ 0.07
(g)	0.50 $\pm$ 0.17	0.29 $\pm$ 0.02	5.09 $\pm$ 0.28
(h)	0.07 $\pm$ 0.06	0.29 $\pm$ 0.04	5.10 $\pm$ 0.42

Table 3: The role of

\mathbf{v}^{\mathbf{eot}}

and

\mathbf{v}^{\mathbf{pad}}_{i}

in memorized image generation. (d)

\mathbf{v}^{\mathbf{pad}}_{i}\rightarrow\mathbf{v}^{\mathbf{eot}}

\mathbf{v}^{\mathbf{pad}}_{i}

are replaced with

\mathbf{v}^{\mathbf{eot}}

. (e)

\mathbf{v}^{\mathbf{pr}}_{i},\mathbf{v}^{\mathbf{pad}}_{i}\rightarrow\mathbf{v}^{\mathbf{eot}}

\mathbf{v}^{\mathbf{pr}}_{i}

and

\mathbf{v}^{\mathbf{pad}}_{i}

are replaced with

\mathbf{v}^{\mathbf{eot}}

. (f)

\mathbf{v}^{\mathbf{eot}}\rightarrow\mathbf{0}

\mathbf{v}^{\mathbf{eot}}

is masked. (g)

\mathbf{v}^{\mathbf{pr}}_{i},\mathbf{v}^{\mathbf{eot}}\rightarrow\mathbf{v}^{\mathbf{pad}}_{\text{mean}}

\mathbf{v}^{\mathbf{pr}}_{i}

and

\mathbf{v}^{\mathbf{eot}}

are replaced with the mean of

\mathbf{v}^{\mathbf{pad}}_{i}

. (h)

\mathbf{v}^{\mathbf{pad}}_{i}\rightarrow\mathbf{0}

\mathbf{v}^{\mathbf{pad}}_{i}

are masked, resulting in collapse.

Figure 5: While Chen et al. [3] reported a “Bright Ending”, an attention spike on <eot> (specifically on $\mathbf{v}^{\mathbf{eot}}$ ) when memorization occurs, we observe that this effect is not limited to $\mathbf{v}^{\mathbf{eot}}$ . Multiple <pad> ( $\mathbf{v}^{\mathbf{pad}}$ ) adjacent to <eot> also exhibit elevated attention, indicating that they play a substantial role in the memorization process rather than serving as placeholders.

Figure 6: Comparison of mitigation results across five methods using the same prompt and seed. Our method (second column) mitigates memorization while preserving structure and prompt alignment. Ren et al. [24] reduce memorization but introduce distortion, while Wen et al. [36] , RTA and RNA [ 31 ] often fail to mitigate memorization. Additional examples are in Appendix A.2 .

Category	Method	SSCD	CLIPScore	Aesthetic	LPIPS
Replace <pad> Token (Ours)	<pad> $\rightarrow$ ! & $\mathbf{v}^{\mathbf{eot}}\rightarrow\mathbf{0}$	0.08 $\pm$ 0.06	0.31 $\pm$ 0.03	5.09 $\pm$ 0.44	0.67 $\pm$ 0.05
Replace <pad> Token (Ours)	<pad> $\rightarrow$ !	0.14 $\pm$ 0.08	0.32 $\pm$ 0.03	5.21 $\pm$ 0.38	0.64 $\pm$ 0.09
Mask $\mathbf{v}^{\mathbf{pad}}_{i}$ Embedding (Ours)	$\mathbf{v}^{\mathbf{pad}}_{i}\rightarrow\mathbf{0}$ (70%)	0.10 $\pm$ 0.07	0.31 $\pm$ 0.04	5.13 $\pm$ 0.41	0.66 $\pm$ 0.06
Prior Works	Ren et al. [24]	0.09 $\pm$ 0.06	0.30 $\pm$ 0.03	5.18 $\pm$ 0.39	0.65 $\pm$ 0.05
	Wen et al. [36]	0.51 $\pm$ 0.35	0.32 $\pm$ 0.03	5.22 $\pm$ 0.33	0.49 $\pm$ 0.19
	RTA [ 31 ]	0.58 $\pm$ 0.32	0.31 $\pm$ 0.03	5.23 $\pm$ 0.31	0.42 $\pm$ 0.22

Table 4: Simply changing the <pad> token mitigates a substantial portion of memorization, while additionally masking

\mathbf{v}^{\mathbf{eot}}

further suppresses leakage. We also introduce partial masking of

\mathbf{v}^{\mathbf{pad}}_{i}

as a flexible alternative that enables fine-grained control. Compared to prior works (bottom), our method achieves low SSCD while maintaining prompt alignment, image quality, and perceptual diversity.

Panel of Figure 7 — Figure 7: Partial Masking $\mathbf{v}^{\text{pad}}_{i}$ to mitigate memorization. (b) and (c) show generations where 70% and 100% of the $\mathbf{v}^{\mathbf{pad}}_{i}$ adjacent to $\mathbf{v}^{\mathbf{eot}}$ are masked, respectively. Masking 70% empirically strikes a well-balanced tradeoff, substantially reducing memorization with negligible quality degradation, whereas full masking (100%) can often cause collapse or noticeable degradation.

Method	CLIPScore	Aesthetic
Original	0.35 $\pm$ 0.05	5.47 $\pm$ 0.51
<pad> $\rightarrow$ ! & $\mathbf{v}^{\mathbf{eot}}\rightarrow\mathbf{0}$	0.34 $\pm$ 0.06	5.42 $\pm$ 0.51
<pad> $\rightarrow$ !	0.35 $\pm$ 0.05	5.49 $\pm$ 0.52
$\mathbf{v}^{\mathbf{pad}}_{i}\rightarrow\mathbf{0}$ (70%)	0.34 $\pm$ 0.06	5.41 $\pm$ 0.51

Table 5: Our methods preserve image quality: on 1,500 non-memorized prompts, CLIPScore and Aesthetic Score remain within the noise of the original, indicating no meaningful degradation or substantial deviation.

Figure 8: Using our mitigation, which replaces <pad> with the ! token and masks

\mathbf{v}^{\mathbf{eot}}

, we observe that prompt region before

\mathbf{v}^{\mathbf{eot}}

remains largely unchanged, thereby preserving

\mathbf{v}^{\mathbf{pr}}

utility. The mitigation consistently suppresses attention on

\mathbf{v}^{\mathbf{eot}}

and

\mathbf{v}^{\mathbf{pad}}

, effectively deactivating both embeddings. Overall, the mitigation removes the root cause without compromising the intended prompt semantics.

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 9: <pad> replacement and $\mathbf{v}^{\mathbf{eot}}$ masking. The first column (Original) shows the memorized image consistently reproduced from the original embedding regardless of seeds. The remaining five columns (Ours) are generated using our mitigation method with five different random seeds.

Figure 10: Each row shows images generated from the same prompt using different mitigation methods. Visually and quantitatively, our method produces a more favorable quality-preserving trade-off.

Method	Inference Time (s)
Ours	3.17 $\pm$ 0.01
Ren et al. [24]	3.37 $\pm$ 0.02
Wen et al. [36]	3.55 $\pm$ 0.16
RNA [ 31 ]	3.26 $\pm$ 0.02
RTA [ 31 ]	3.31 $\pm$ 0.02

Appendix

Table 6: Average inference time per image for each method.

Method	SSCD	CLIPScore	Aesthetic
Ours	0.39 $\pm$ 0.20	0.29 $\pm$ 0.03	5.33 $\pm$ 0.26
Wen	0.65 $\pm$ 0.22	0.31 $\pm$ 0.03	5.35 $\pm$ 0.23
Ren	0.20 $\pm$ 0.15	0.26 $\pm$ 0.03	4.90 $\pm$ 0.31
RTA	0.70 $\pm$ 0.23	0.31 $\pm$ 0.03	5.36 $\pm$ 0.23
RNA	0.70 $\pm$ 0.23	0.32 $\pm$ 0.03	5.33 $\pm$ 0.25

Appendix

Table 7: Comparison of mitigation methods on all 500 candidate prompts from Webster [ 35 ] .

Method	SSCD	CLIPScore	Aesthetic
Ours	0.16 $\pm$ 0.23	0.30 $\pm$ 0.07	5.14 $\pm$ 0.53
Ren	0.15 $\pm$ 0.19	0.29 $\pm$ 0.06	5.17 $\pm$ 0.51
Wen	0.79 $\pm$ 0.37	0.31 $\pm$ 0.06	5.22 $\pm$ 0.48

Appendix

Table 8: Comparison of mitigation methods on Membench [ 13 ] .

Panel of Figure 11 — Figure 11: (a) Original generation without swapping. (b) Generation after swapping both $\mathbf{v}^{\mathbf{eot}}$ and $\mathbf{v}^{\mathbf{pad}}_{i}$ , showing that the output changes to reflect the swapped prompt.

Panel of Figure 12 — Figure 12: “Original” is generated using the default embeddings, while “Ours” applies our mitigation strategy ( <pad> $\rightarrow$ ! & $\mathbf{v}^{\mathbf{eot}}\rightarrow\mathbf{0}$ ).

Panel of Figure 13 — Figure 13: Each left column shows the reference image, and each right column shows the corresponding model-generated image.

Metric	SD1		SD2
	Webster [35]	Hong et al. [13]	Webster [35]
Number of Prompts	345	3000	219
Mean Length	19.74	15.31	21.51
Max / Min Length	38 / 4	38 / 5	35 / 6

Appendix

Table 9: Prompt Token Length Statistics for Stable Diffusion v1.4 (SD1) and Stable Diffusion v2.1 (SD2)

Panel of Figure 14 — Figure 14: Text embeddings of “Living in the Light with Ann Graham Lotz” visualized using PCA and t-SNE for (a) CLIP and (b) OpenCLIP models. The two projection methods reveal distinct clustering behaviors.

Figure 15: Cross Attention maps of “Mothers influence on her young hippo”

Figure 16: Cross Attention maps of “Living in the Light with Ann Graham”

Memorization in Generative Models

Memorization in generative models studies when models retain training examples, how to detect their recall, and how to limit it without losing utility.

See all 86 papers

Broken Memories: Detecting and Mitigating Memorization in Diffusion Models with Degraded Generations
May 21, 2026Yuanmin Huang, Mi Zhang, Chen Chen +4Memorization in Generative Models Diffusion Models
Fudan University Shanghai, China · East China University of Science and Technology Shanghai, China
Steering Away from Memorization: Reachability-Constrained Reinforcement Learning for Text-to-Image Diffusion
Feb 24, 2026Sathwik Karnik, Juyeop Kim, Sanmi Koyejo +2Memorization in Generative Models T2I Diffusion Models
Stanford University, Stanford, CA, USA · Yonsei University, Seoul, Korea
Rethinking Memorization Mitigation in Diffusion Models: Reinforcing Text Conditioning
Oct 1, 2026Hyungjun Joo, Sehwan Kim, Hyeonggeun Han +2Diffusion Model Alignment Attention Control in Diffusion Models
Seoul National University · Konkuk University

May 21, 2026J/K move · Enter open · S save

Broken Memories: Detecting and Mitigating Memorization in Diffusion Models with Degraded Generations

Yuanmin Huang, Mi Zhang, Chen Chen, Feifei Li+3

Fudan University Shanghai, China · East China University of Science and Technology Shanghai, China

Memorization in Generative Models Diffusion Models

Open paper AskPDF arXiv

While diffusion models excel at generating high-quality images, their tendency to memorize training data poses significant privacy and copyright risks. In this work, we for the first time identify that memorization induces internal numerical instability, often manifesting as visually ``broken'' artifacts. Inspired by stability analysis in numerical methods, we introduce empirical stability regions based on latent update norms to quantitatively characterize stable behavior during generation. Leveraging this, we propose a principled, on-the-fly framework for step-wise detection and adaptive mitigation. Our approach suppresses memorization without altering prompts or guidance, thereby preserving semantic fidelity and image quality. Extensive experiments on Stable Diffusion 1.4 demonstrate that our method achieves an AUC

>0.999

detection performance and a

0.0\%

memorization rate after mitigation with negligible overhead (

\approx0.01

s per image).

Abstract

Memorization in Generative Models

Explore similar work

Broken Memories: Detecting and Mitigating Memorization in Diffusion Models with Degraded Generations

Steering Away from Memorization: Reachability-Constrained Reinforcement Learning for Text-to-Image Diffusion

Rethinking Memorization Mitigation in Diffusion Models: Reinforcing Text Conditioning