Organizations: Southeast University · Engineering Research Center of Blockchain Application, Supervision and Management (Southeast University), Ministry of Education · Beijing Institute of Technology
Diffusion models can memorize and reproduce training samples during inference, raising serious privacy and copyright concerns. Existing inference-time mitigation methods often suppress conditional information globally, which may reduce memorization but can also harm semantic alignment and image quality. In this paper, we revisit memorization from the perspective of cross-attention heads. Through systematic analysis and targeted ablation, we find that memorization is mainly driven by a small subset of shallow-layer attention heads with abnormally strong semantic control, rather than by the entire cross-attention mechanism. Motivated by this finding, we propose Projection-based Top-k Attention Head Suppression, a targeted inference-time mitigation method. Our method first identifies the top-k attention heads most related to memorization using a semantic-control score, and then removes their condition-specific components through directional projection between conditional and unconditional representations. This design selectively weakens memorization-related features while preserving useful semantic information. Extensive experiments on different diffusion models and datasets show that our method effectively reduces memorization and achieves a better trade-off between memorization mitigation and image quality than existing approaches.
Figures & tables
Figure 1: Examples of semantic-control scores across attention heads. Memorized images tend to activate a few distinctly prominent attention heads, whose semantic response strengths are substantially higher than those of other attention heads. In contrast, non-memorized images exhibit a much smoother and more uniform distribution of semantic strength across attention heads.
Figure 2: Overview of our framework. The left panel illustrates the computation of multi-head attention. Based on this process, we introduce two modifications on the right panel : (1) Identifying attention heads with strong semantic control through attention map analysis. (2) Applying projection-based suppression to mitigate memorization-related components in the selected heads.
Figure 3: Layer-wise analysis of attention heads with strong semantic control. Memorized images exhibit a larger number of high semantic-control attention heads in the shallow layers, especially layers 0 ∼ 3. In deeper layers, the statistics of memorized and non-memorized samples become similar.
Figure 4: Ablation study on attention head groups. We evaluate different suppression factors ϵ∈{0.0,0.2,0.4,0.6} . Suppressing the Top-3 heads produces the largest reduction in SSCD Score, while suppressing the Bottom-3 heads has little effect.
Figure 5: SSCD-CLIP trade-off curve comparison with baselines. Our method consistently lies in the lower-right region compared with all baselines. This demonstrates that, under the same SSCD level, our approach achieves higher CLIP scores. In other words, given comparable semantic alignment, our method provides the most effective suppression of memorization.
Figure 6: Qualitative comparison with baselines. The first column shows the memorized training images, while the remaining seven columns present the results of baselines and our method. Our method achieves a superior balance between memorization mitigation, semantic consistency, and image quality compared with existing baselines.
Layers
SD v1.4
SD v2.0
SSCD ↓
CLIP ↑
SSCD ↓
CLIP ↑
None
0.5122
0.3009
0.6306
0.2947
0–1
0.3742
0.2887
0.5975
0.2973
0–3
0.1578
0.2590
0.2662
0.2730
0–5
0.1301
0.2446
0.2067
0.2564
0–7
0.1098
0.2264
0.1602
0.2465
Table 1: Progressive Layers ablation study for mitigation. Suppressing layers 0–3 provides the most substantial mitigation, while extending suppression to deeper layers yields diminishing benefits at the cost of semantic fidelity.
Layers
SD v1.4
SD v2.0
SSCD ↓
CLIP ↑
SSCD ↓
CLIP ↑
None
0.5122
0.3009
0.6306
0.2947
0 ∼ 3
0.1578
0.2590
0.2662
0.2730
4 ∼ 7
0.4260
0.2690
0.3978
0.2667
8 ∼ 11
0.4412
0.2934
0.6165
0.2588
12 ∼ 15
0.3962
0.2969
0.6264
0.2922
Table 2: Segmented layers ablation study for mitigation. Early layers (0 ∼ 3) contribute most significantly to mitigation, while suppression in deeper layers yields limited benefits.
λ
k=2
k=3
k=4
SSCD ↓
CLIP ↑
SSCD ↓
CLIP ↑
SSCD ↓
CLIP ↑
1.00
0.1692
0.2610
0.1578
0.2590
0.1534
0.2576
0.95
0.1697
0.2603
0.1590
0.2592
0.1554
0.2581
0.90
0.1718
0.2607
0.1580
0.2593
0.1558
0.2589
0.85
0.1752
0.2611
0.1602
0.2606
0.1566
0.2592
0.80
0.1768
0.2609
0.1626
0.2613
0.1594
0.2604
Table 3: Ablation study of k and λ on SD v1.4. The stable performance across different settings demonstrates the robustness of our method to hyperparameter choices.
Method
RTA
IND
DOG
PEA
CALS
SALI
Ours
Time (s) ↓
2765
2385
5420
1490
1090
2075
1115
Table 4: Runtime comparison of memorization mitigation methods on SD v1.4 for 500 samples. Our method achieves the second-fastest runtime, with only a minor overhead compared with CALS.
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 7: SSCD-CLIP trade-off on MS-COCO fine-tuned SD v1.4. Our method achieves a better balance between memorization mitigation and semantic alignment than baselines.
Method
SD v1.4
SD v2.0
AUC ↑
T@F=1% ↑
AUC ↑
T@F=1% ↑
Wen et al.
94.32
34.32
91.43
20.63
Ren et al.
95.67
42.66
84.82
10.50
Jeon et al.
99.01
82.80
96.88
50.20
Ours
98.41
78.12
96.23
54.21
Appendix
Table 5: Memorization detection results on SD v1.4 and SD v2.0. The competitive performance further supports our finding that memorization is associated with shallow-layer attention heads exhibiting large conditional–unconditional attention discrepancies.
Figure 8: Temporal distribution of outlier semantic-control attention heads across denoising timesteps. Each row corresponds to a layer, and each column corresponds to one generated sample. In each subplot, the x -axis denotes the denoising timestep, and the y -axis denotes the attention head index. Each cyan point indicates that the corresponding attention head is identified as an outlier semantic-control head at that timestep. The horizontal and temporally continuous patterns show that memorization-related semantic control is not random or transient, but is mainly carried by a stable subset of attention heads that persist across most denoising timesteps.
Table 6: Comparison between naive suppression and our proposed suppression strategy on SD v1.4 and SD v2.0. Our method achieves comparable memorization mitigation while better preserving semantic fidelity on both models.
Method
SD v1.4
SD v2.0
CLIP ( ↑ )
FID ( ↓ )
CLIP ( ↑ )
FID ( ↓ )
No Mitigation
0.3386
85.47
0.3509
86.73
Ours
0.3390
85.49
0.3503
86.20
Appendix
Table 7: Generation quality on non-memorized prompts. Our method preserves semantic alignment and distributional quality on non-memorized inputs.
Method
SD 3
SSCD ( ↓ )
CLIP ( ↑ )
No Mitigation
0.3372
0.2625
RTA
0.2252
0.2367
PEA
0.2292
0.2438
IND
0.2442
0.2451
DOG
0.2289
0.2219
Appendix
Table 8: Memorization mitigation results on Stable Diffusion 3. Our method achieves the best balance between memorization mitigation and semantic alignment.
Method
SDv1.4 – AM
SDv1.4 – VM
SDv1.4 – TM
SDv2.0 – TM
SSCD ( ↓ )
CLIP ( ↑ )
SSCD ( ↓ )
CLIP ( ↑ )
SSCD ( ↓ )
CLIP ( ↑ )
SSCD ( ↓ )
CLIP ( ↑ )
No Mitigation
0.5122
0.3009
0.7895
0.3371
0.4322
0.2904
0.6306
0.2947
RTA
0.1045
0.1959
0.0888
0.2053
0.1090
0.1932
0.1583
0.2093
0.1906
0.2394
0.1849
0.2552
0.1923
0.2348
0.2152
0.2463
0.2407
0.2487
0.2660
0.2727
0.2334
0.2418
0.2866
0.2633
0.2873
0.2545
0.3173
0.2516
0.2786
0.2553
0.4547
0.2824
Appendix
Table 9: Mitigation results on different memorization types. AM denotes all memorized samples, VM denotes verbatim memorization, and TM denotes template memorization. We report SSCD and CLIP scores under SD v1.4 and SD v2.0, with four hyperparameter settings for each method. Lower SSCD indicates better memorization mitigation, while higher CLIP indicates better prompt-image alignment.
Memorization in generative models studies when models retain training examples, how to detect their recall, and how to limit it without losing utility.
Generative models have been shown to "memorize" certain training data, leading to verbatim or near-verbatim generating images, which may cause privacy concerns or copyright infringement. We introduce Guidance Using Attractive-Repulsive Dynamics (GUARD), a novel framework for memorization mitigation in text-to-image diffusion models. GUARD adjusts the image denoising process to guide the generation away from an original training image and towards one that is distinct from training data while remaining aligned with the prompt, guarding against reproducing training data, without hurting image generation quality. We propose a concrete instantiation of this framework, where the positive target that we steer towards is given by a novel method for (cross) attention attenuation based on (i) a novel statistical mechanism that automatically identifies the prompt positions where cross attention must be attenuated and (ii) attenuating cross-attention in these per-prompt locations. The resulting GUARD offers a surgical, dynamic per-prompt inference-time approach that, we find, is by far the most robust method in terms of consistently producing state-of-the-art results for memorization mitigation across two architectures and for both verbatim and template memorization, while also improving upon or yielding comparable results in terms of image quality.
Kairan Zhao, Eleni Triantafillou, Peter Triantafillou
Diffusion models can unintentionally memorize training samples, raising concerns about privacy and copyright. While recent methods can detect memorization, they often rely on global or model-specific signals and provide limited insight into where memorization appears within a generated image. We provide a geometric characterization of local memorization as a coordinate-wise variance collapse. However, such collapse can also arise from intrinsic data constraints rather than overfitting. To isolate overfitting-driven memorization, we propose curvature-difference methods that subtract the curvature of an underfitted baseline, either the unconditional model or a less-trained version of itself. We further derive a score-difference proxy that provides a geometric explanation for the widely used score-difference-based detection metric. Experiments on Stable Diffusion, evaluated against ground-truth memorization masks, show that our method outperforms the prior attention-based localization method. Code is available at https://github.com/Gwangho99/mem-curv-diff.
Gwangho Kim, Sungyoon Lee
Department of Computer Science, Hanyang University, Seoul, South Korea.
While diffusion models excel at generating high-quality images, their tendency to memorize training data poses significant privacy and copyright risks. In this work, we for the first time identify that memorization induces internal numerical instability, often manifesting as visually ``broken'' artifacts. Inspired by stability analysis in numerical methods, we introduce empirical stability regions based on latent update norms to quantitatively characterize stable behavior during generation. Leveraging this, we propose a principled, on-the-fly framework for step-wise detection and adaptive mitigation. Our approach suppresses memorization without altering prompts or guidance, thereby preserving semantic fidelity and image quality. Extensive experiments on Stable Diffusion 1.4 demonstrate that our method achieves an AUC >0.999 detection performance and a 0.0% memorization rate after mitigation with negligible overhead (≈0.01s per image).
Yuanmin Huang, Mi Zhang, Chen Chen +4
Fudan University Shanghai, China · East China University of Science and Technology Shanghai, China