Valid Early Detection of Memorization in Diffusion Models

Yixiao Jing; Chia-Yu Hsu; Milad Hoseinpour; Zekai Zhang

cs.LGOct 25, 2026

Valid Early Detection of Memorization in Diffusion Models

Authors: Yixiao Jing, Chia-Yu Hsu, Milad Hoseinpour, Zekai Zhang

Organizations: Department of Electrical Engineering and Computer Science, University of Michigan, Ann Arbor, MI, USA

Abstract

Diffusion models can sometimes reproduce examples from their training data, a phenomenon known as memorization. Existing memorization detection methods for unconditional diffusion models typically operate at a pre-selected timestep and therefore cannot adapt the detection time to the evolving denoising trajectory. Sequential monitoring enables adaptive detection but repeated testing across timesteps complicates trajectory-level false-alarm control. We study two ways of combining split conformal calibration with sequential monitoring for prompt-free memorization detection: per-step Bonferroni calibration and direct trajectory-level calibration of the maximum memorization score. The former controls repeated testing by splitting the false-alarm budget across timesteps, whereas the latter calibrates the event of ever crossing a threshold directly through the trajectory-level maximum. We evaluate this formulation on an unconditional CIFAR-10 EDM using two complementary per-step statistics based on geometry and representation spikiness. Experiments show that sequential monitoring improves the power-timing tradeoff over fixed-time detection. Compared with Bonferroni calibration, trajectory-level calibration achieves comparable detection power while using the false-alarm budget less conservatively and enabling earlier detection at higher noise levels.

Figures & tables

Figure 1: Early warning before generation finishes. Top shows schematic comparison of fixed-time checks at a preselected step and adaptive alarms at trajectory-dependent steps. Intermediate panels show the model’s denoised predictions at selected sampling steps. The memorized trajectory triggers its first alarm at step 8 of 18, while the generalized trajectory triggers no alarm. Final outputs are shown beside their nearest training images, with SSCD similarity scores.

Figure 2: Power–timing tradeoff across statistics and false-alarm levels. Curves show fixed-time detectors across pre-selected noise levels

\sigma_{t^{\star}}

; square and diamond markers denote sequential detection under per-step Bonferroni calibration and trajectory-level max calibration, respectively, positioned at their mean decision noise levels

\bar{\sigma}_{\mathrm{dec}}

under

H_{1}

. The horizontal axis is logarithmic, with higher noise levels corresponding to earlier stages of denoising.

Figure 3: Cumulative detection probability for StD at

\alpha=0.05

. Fixed-time procedures use pre-selected noise levels

\sigma\in\{5.32,3.26,1.92\}

, corresponding to

t^{\star}\in\{8,9,10\}

for the

T=18

sampler; sequential procedures use per-step Bonferroni calibration or trajectory-level max calibration.

Statistic	Calibration	FPR	TPR	$\bar{\sigma}_{\mathrm{dec}}\;(\uparrow)$
trF	Bonferroni	$0.021\pm 0.010$	$0.994\pm 0.001$	$5.329\pm 0.210$
trF	Trajectory-level max	$0.044\pm 0.008$	$\mathbf{0.998\pm 0.000}$	$\mathbf{5.408\pm 0.069}$
StD	Bonferroni	$0.017\pm 0.008$	$0.994\pm 0.003$	$4.900\pm 0.336$
StD	Trajectory-level max	$0.050\pm 0.015$	$\mathbf{0.996\pm 0.000}$	$\mathbf{7.678\pm 0.431}$

Table 1: Comparison of sequential calibration strategies at

\alpha=0.05

. FPR, TPR, and mean decision noise level with standard deviation over five random splits are reported. Larger

\bar{\sigma}_{\mathrm{dec}}

corresponds to earlier detection. The empirical FPR may exceed

\alpha

on a finite evaluation set.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

$\alpha$	Statistic	Calibration	FPR	TPR	$\bar{\sigma}_{\mathrm{dec}}\;(\uparrow)$
$0.10$	trF	Bonferroni	$0.045\pm 0.010$	$0.998\pm 0.002$	$5.920\pm 0.337$
	trF	Trajectory-level max	$0.084\pm 0.017$	$\mathbf{1.000\pm 0.000}$	$\mathbf{6.337\pm 0.212}$
	StD	Bonferroni	$0.030\pm 0.002$	$\mathbf{1.000\pm 0.000}$	$6.140\pm 0.292$
	StD	Trajectory-level max	$0.091\pm 0.021$	$\mathbf{1.000\pm 0.000}$	$\mathbf{9.638\pm 0.463}$
$0.20$	trF	Bonferroni	$0.076\pm 0.004$	$\mathbf{1.000\pm 0.000}$	$6.692\pm 0.168$
	trF	Trajectory-level max	$0.172\pm 0.014$	$\mathbf{1.000\pm 0.000}$	$\mathbf{8.603\pm 0.060}$

Appendix

Table 2: Comparison of sequential calibration strategies at

\alpha=0.10

and

\alpha=0.20

. FPR, TPR, and mean decision noise level with standard deviation over five random splits are reported. Larger

\bar{\sigma}_{\mathrm{dec}}

corresponds to earlier detection.

Memorization in Generative Models

Memorization in generative models studies when models retain training examples, how to detect their recall, and how to limit it without losing utility.

See all 88 papers

Early Signatures of Memorization in Diffusion Models via Basin Geometry and Cyclic Denoising
Oct 8, 2026Nikhil Verma, Siddharthan Dileep, Anoop Singh +4Memorization in Generative Models Diffusion Models
Yardi School of Artificial Intelligence, Indian Institute of Technology Delhi · Department of Computer Science and Engineering, Indian Institute of Technology Delhi · Jawaharlal Nehru Centre for Advanced Scientific Research +2
Localizing Memorized Regions in Diffusion Models via Coordinate-Wise Curvature Differences
May 26, 2026Gwangho Kim, Sungyoon LeeMemorization in Generative Models Diffusion Models
Department of Computer Science, Hanyang University, Seoul, South Korea.
Broken Memories: Detecting and Mitigating Memorization in Diffusion Models with Degraded Generations
May 21, 2026Yuanmin Huang, Mi Zhang, Chen Chen +4Memorization in Generative Models Diffusion Models
Fudan University Shanghai, China · East China University of Science and Technology Shanghai, China

Oct 8, 2026J/K move · Enter open · S save

Early Signatures of Memorization in Diffusion Models via Basin Geometry and Cyclic Denoising

Nikhil Verma, Siddharthan Dileep, Anoop Singh, Srikanth Sastry+3

Yardi School of Artificial Intelligence, Indian Institute of Technology Delhi · Department of Computer Science and Engineering, Indian Institute of Technology Delhi · Jawaharlal Nehru Centre for Advanced Scientific Research+2

Memorization in Generative Models Diffusion Models

Open paper AskPDF arXiv

Diffusion models generalize early in training and later reproduce individual training samples. Standard tests detect memorization only once one-shot generation produces near-copies, leaving a released model unaudited until its outputs fail. We show that memorization is encoded in the geometry of the learned energy landscape before it appears in generated samples, a state we call latent memorization. Using score divergence and basin volume, we find that localized basins form around training samples and separate them from held-out samples before the first memorized sample appears, with an onset that follows the same

O(n)

scaling as the memorization time. We probe these basins with cyclic denoising, which repeatedly applies partial noising and denoising. Under the exact empirical score, we prove that cycling started near an isolated training sample recovers it and returns to it over any finite number of cycles with high probability. In trained models, cycling recovers training images from CelebA and CIFAR-10 checkpoints whose one-shot samples contain no copies, and at a CelebA checkpoint with 0.1% one-shot copies, 500 cycles raise the memorized fraction above 30%. Cycling also reveals degenerate attractors that match no single training image and fade as training proceeds, so residence in a basin does not by itself imply memorization. These findings hold on a Gaussian mixture, CelebA, and CIFAR-10 across optimizers, architectures, noise schedules, and training-set sizes, and extend to off-the-shelf Stable Diffusion v1.4, where the cycled conditional-unconditional divergence gap separates memorized from non-memorized prompts with an AUC of 0.944 and a TPR of 0.866 at 1% FPR. More broadly, what a diffusion model has memorized is a property of the geometry and stability of its learned distribution, and assessing it requires examining this structure rather than generated outputs alone.

Abstract

Memorization in Generative Models

Explore similar work

Early Signatures of Memorization in Diffusion Models via Basin Geometry and Cyclic Denoising

Localizing Memorized Regions in Diffusion Models via Coordinate-Wise Curvature Differences

Broken Memories: Detecting and Mitigating Memorization in Diffusion Models with Degraded Generations