Diffusion models are known to reproduce individual examples from their training data as a form of memorization with implications for generalization, privacy, and downstream use. Recent work demonstrated that memorized training samples reside inside contiguous regions of the data/latent space known as basins of memorization -- once a denoising trajectory enters a basin, under finite perturbations it irreversibly reproduces a memorized example. Prior empirical work on such basins in trained models has primarily focused on classifier-free guidance based detection and mitigation strategies in text-to-image models, leaving basin entry and their formation largely uncharacterized. In this paper, we present a geometric characterization of memorization basins for conditional, unconditional, and rectified-flow-based diffusion models. We observe a characteristic signature in the local spectrum of the function geometry that distinguishes basin entry: the emergence of a dominant singular mode followed by a sharp spectral recovery. This signature appears at two timescales: along denoising trajectories during inference and over the course of training. We further investigate whether these geometric signatures can be used to intervene on memorization. During training, we suppress memorization by acting on the dominant mode. At inference, we develop methods to identify trajectories entering memorization basins and steer them away. We find that these interventions remain effective even with low-dimensional approximations of the underlying geometry, and demonstrate that geometry-informed steering extends to large-scale text-to-image diffusion models. Together, our results corroborate that local spectral geometry provides a common basis for characterizing memorization and for suppressing, detecting, and steering away from it across training and generation.
Figures & tables
Figure 1: Spectral and behavioral signatures of memorization during inference. DDIM trajectories on CelebA are aligned at τ=t−t∗ , with t∗ defined by the steepest increase in the leading spectral gap. (a) Both groups exhibit spectral concentration around τ=0 , with stronger concentration and subsequent recovery for memorized trajectories. (b) Memorized trajectories become less sensitive to perturbations after the transition. (c) Their predicted clean samples approach the final output most rapidly around the same region.
Figure 2: Training-time evolution of spectral concentration. Local Jacobian metrics for different training checkpoints colored by memorization rate. (a) The relative leading spectral gap Δ12 rises sharply at intermediate noise, and falls again at low noise. Its peak grows with memorization rate and shifts toward higher noise in later checkpoints. (b) Entropy effective rank reveals matching intermediate concentration followed by low-noise rank recovery in later checkpoints.
Gaussian80
Mixture32
SwissRoll8D128
CelebA32
Method
Mem ↓
Prec. ↑
Rec. ↑
Mem ↓
Prec. ↑
Rec. ↑
Mem ↓
Prec. ↑
Rec. ↑
Mem ↓
Mean r↑
FID ↓
Standard
31.85
0.978
0.807
46.85
0.997
0.757
16.10
0.928
0.816
8.80
0.766
54.89
Tikhonov
25.15
0.981
0.921
43.25
0.998
0.812
13.90
0.918
0.839
6.50
0.780
49.03
Reduced LR
21.15
0.986
0.880
41.35
0.992
0.771
10.30
0.877
0.899
6.80
0.785
52.21
Ambient-style
24.35
0.985
0.858
47.00
0.993
0.600
17.30
0.917
0.789
4.90
0.808
52.15
GeoReg (Ours)
15.85
0.983
0.979
19.25
0.980
0.963
10.20
0.904
0.878
1.56
0.846
59.77
Table 1: GeoReg reduces memorization across synthetic populations and CelebA32. Results are averaged across five independent seeds for each setting. For synthetic populations, memorization is measured against the fixed training set, while precision and recall are evaluated against independent population samples. For CelebA32, r denotes the nearest-neighbor gap ratio. Full baseline definitions, training settings, and additional metrics are given in Appendix E.1 .
Sampler
Method
AUROC ↑
AUPRC ↑
Recall ↑
Spec. ↑
MCC ↑
Time ↓
DDIM
FLIPD Ross et al. (2025)
0.845
0.722
0.437
0.932
0.443
42.05 ms
Normal bundle Stanczuk et al. (2024)
0.902
0.773
0.452
0.952
0.493
922.19 ms
Unconditional score magnitude
0.860
0.785
0.542
0.934
0.537
12.05 ms
Spectral temporal recovery
0.823
0.698
0.305
0.956
0.366
40.88 s
JER (ours)
0.956
0.895
0.898
0.886
0.763
71.26 ms
RF
FLIPD Ross et al. (2025)
0.720
0.733
0.310
0.894
0.248
42.48 ms
Table 2: Efficient geometry-based memorization detection. Results on held-out CelebA trajectories. AUROC and AUPRC denote the areas under the receiver-operating-characteristic and precision–recall curves, respectively; MCC denotes the Matthews correlation coefficient. Recall, specificity, and MCC are computed at operating points selected on a disjoint calibration set and frozen before evaluation. For the unconditional score-magnitude baseline, classifier orientation is also selected on the calibration split and then frozen for held-out evaluation.
Figure 3: GeoSteer-DM suppresses memorization across training checkpoints and geometric resolutions. Memorization rate of unmodified and steered sampling for (a) DDIM and (b) rectified flow. Full-Jacobian steering gives the strongest suppression. Low-dimensional Jacobian sketches retain much of this effect, with suppression generally strengthening as k increases.
Figure 4: GeoSteer-DM improves memorization–quality trade-offs across mitigation families. Mean SSCD to memorized training references versus CLIP prompt alignment (left) and FID (right) on memorized prompts. Connected markers show each method, including unmodified sampling, before and after adding GeoSteer-DM .
Appendix figures & tables25 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 5: Symmetry of the learned Jacobian and robustness of the spectral transition. (A) Relative Frobenius norm of the antisymmetric component of the oriented curvature operator Jt=−Jt=−∂ϵθ/∂xt . (B–C) Stable rank and effective rank of the full operator Jt (solid) and its symmetric component St (dashed) along memorized and non-memorized trajectories. The learned operator remains close to symmetric, while symmetrization preserves the characteristic spectral collapse and subsequent recovery.
Figure 6: Control for trajectory alignment. (a) Stable-rank evolution of memorized and non-memorized DDIM trajectories in absolute sampling time, showing distinct late-stage evolution without alignment at t∗ . (b) The same trajectories aligned at their individual t∗ ; dashed curves show the shuffled-anchor control obtained by randomly reassigning t∗ values within each group. The transition becomes easier to resolve after alignment, while the distinct post-transition recovery is not explained by trajectory-specific alignment.
Jti(i)=WJti(i),i∈I.
Appendix
Algorithm 2 GeoReg-DM : geometry-regularized training step
Setting
Value
Batch size
512
Adam learning rate
10−4
Geometry timestep window
t∈{90,…,500}
Selected states per update
m=32
Sketch rank
k=8
Initial interpolation
α0=0.25
Appendix
Table 3: CelebA GeoReg training and stabilization settings.
Component
Time (ms)
Fraction
Batch setup, base forward + loss
88.66
8.48%
Base backward
176.11
16.84%
State selection + geometry-mode entry
1.98
0.19%
Packed left-Jacobian sketch
237.09
22.67%
Exact Gram eigensolve + spectral objective
1.02
0.10%
Geometry backward
519.54
49.67%
Appendix
Table 4: Computational cost of GeoReg-DM . CUDA-synchronized median breakdown over ten warmed DDIM training updates on one NVIDIA A100-SXM4-40GB, using B=512 , m=32 selected states, and a fixed rank- 8 left Jacobian sketch.
Figure 7: Adversarial evidence for memorization basin entry. Escape probability under bounded interventions relative to t∗ . Escapeability decreases sharply through the transition, with larger perturbation bounds remaining effective later in denoising. End-to-end and greedy optimization yield similar profiles, while matched random perturbations almost never escape. Shaded regions show 95% Wilson confidence intervals.
Figure 8: Singular-value dynamics of the spectral transition. Exact full-SVD Jacobians along DDIM trajectories at the 160k checkpoint in absolute sampling time. (a–b) Both cohorts exhibit collapse–recovery dynamics in stable and entropy effective rank, with substantially stronger low-noise recovery for memorized trajectories. (c–d) Spectral collapse is driven by σ1 separating from lower singular modes. At low noise, secondary singular values approach σ1 for memorized trajectories, whereas σ1 remains more strongly separated for non-memorized trajectories.
Figure 9: Singular-value dynamics for rectified flow. Exact full-SVD Jacobians along rectified-flow trajectories at the 160k checkpoint in absolute sampling time. (a–b) Both cohorts exhibit spectral collapse, with substantially stronger recovery for memorized trajectories. (c–d) The collapse is driven by σ1 separating from lower modes; for memorized trajectories, σ1 subsequently declines toward the secondary singular values, whereas for non-memorized trajectories it continues to grow.
Figure 10: Generation quality under GeoReg. Representative samples from standard training and the final GeoReg configuration, together with their pixel- L2 nearest training examples. GeoReg substantially reduces copying while retaining coherent CelebA generations.
Measure
Standard
GeoReg
Copy rate ( r<1/3 )
8.80%
1.56%
Mean r
0.766
0.846
Median r
0.861
0.910
r q05
0.273
0.470
Pixel NN
3.087
3.833
LPIPS NN
0.0216
0.0238
Appendix
Table 5: Training-set proximity under GeoReg.
Figure 11: GeoReg improves population coverage on synthetic data. Rows: Gaussian80, Mixture32, and SwissRoll8D128 (shown in a 2D projection). Columns: samples from the underlying population (left), generations from standard training (middle), and generations from GeoReg (right); orange crosses mark the fixed training points. Standard training concentrates mass around individual training points, whereas GeoReg spreads generations over the population while preserving its structure.
Figure 12: Entropy effective rank under GeoReg-DM . Local entropy effective rank ν(Jt) across diffusion time for GeoReg-DM (solid) and standard training (dashed) across checkpoints. Profiles align at higher noise and diverge near low-noise endpoints, where standard training exhibits stronger rank recovery. Intermediate-noise changes are reported in Table 6 .
Δ SR
Δν
ΔΔ12
Ckpt
t=100
t=200
t=300
t=400
t=100
t=200
28k
+77.4
+122.2
+95.8
+15.9
−0.73
−0.058
[63.7, 92.1]
[93.0, 153.1]
[62.0, 127.1]
[10.3, 22.2]
[ −0.86 , −0.60 ]
[ −0.090 , −0.028 ]
30k
+14.2
+25.4
+34.6
+2.3
−0.46
−0.005
[3.3, 25.0]
[9.2, 42.4]
[18.1, 52.2]
[ −3.6 , 8.5]
[ −0.52 , −0.39 ]
[ −0.025 , 0.014]
40k
+3.3
+25.6
+43.3
+3.7
−0.45
−0.003
Appendix
Table 6: Paired spectral differences under GeoReg-DM . Mean paired differences ( GeoReg−Standard ) evaluated on 32 fixed noised training states per cell; brackets report 95% bootstrap intervals. Bold entries denote intervals excluding zero. SR denotes stable rank, ν entropy effective rank, and Δ12 the relative leading spectral gap.
Figure 13: Validation of the spectral proxy underlying JER. Exact Rényi-2 effective rank (horizontal axis) versus the deterministic oriented trace-to-Frobenius proxy (vertical axis) on frozen DDIM and rectified-flow query states. Colors distinguish memorized and non-memorized generations, and the dashed line denotes equality. The proxy is conservative in magnitude but closely preserves the ordering of the exact spectral quantity across states.
Ckpt
Prevalence
JER
FLIPD
Score magnitude
60k
7.7%
0.940 / 0.504
0.802 / 0.252
0.834 / 0.371
100k
21.9%
0.943 / 0.793
0.819 / 0.571
0.811 / 0.591
160k
33.4%
0.956 / 0.895
0.845 / 0.722
0.860 / 0.785
260k
52.9%
0.957 / 0.951
0.922 / 0.921
0.907 / 0.923
Appendix
Table 7: DDIM memorization detection across training. Held-out AUROC / AUPRC at successive model checkpoints. Prevalence is the fraction of memorized generations in the held-out set; 160k corresponds to the DDIM evaluation in Table 2.
Masked
No mask
Detector
AUROC (95% CI)
TPR@1%
AUROC (95% CI)
TPR@1%
Conditional JER (ours)
0.712 [0.677, 0.747]
30.3%
0.974 [0.967, 0.979]
75.5%
Unconditional JER (ours)
0.671 [0.633, 0.706]
23.8%
0.914 [0.908, 0.921]
51.8%
Conditional score norm
0.513 [0.474, 0.555]
1.8%
0.102 [0.080, 0.125]
0.0%
Unconditional score norm
0.701 [0.695, 0.708]
16.3%
0.807 [0.777, 0.837]
10.3%
FLIPD Ross et al. (2025)
0.494 [0.454, 0.534]
1.0%
0.802 [0.770, 0.832]
6.5%
Appendix
Table 8: Stable Diffusion prompt-level memorization detection. Results are evaluated on a frozen balanced set of 800 prompts. Confidence intervals are highlighted in the brackets. TPR@1% is the maximum TPR at FPR ≤1% . In the masked setting, padding positions are excluded from the CLIP text encoding, whereas in the no-mask setting they remain visible, allowing the post-EOS token tail to influence the conditioning representation.
Sampler
Ckpt
Copy (%) ↓
Mean r↑
Median r↑
FID ↓
DDIM
30k
1.4 → 1.0
0.85 → 0.89
0.91 → 0.94
64.9 → 74.0
60k
8.4 → 3.2
0.76 → 0.85
0.84 → 0.92
60.9 → 66.6
100k
24.6 → 6.0
0.63 → 0.83
0.69 → 0.90
56.5 → 64.6
160k
36.4 → 6.8
0.55 → 0.83
0.54 → 0.91
52.1 → 65.5
200k
42.0 → 4.4
0.52 → 0.84
0.47 → 0.91
52.1 → 67.0
300k
51.6 → 7.6
0.45 → 0.82
0.31 → 0.90
47.2 → 61.3
Appendix
Table 9: GeoSteer on CelebA across training checkpoints. Each row reports unmodified sampling → GeoSteer on 500 paired generations (five seeds, 100 samples each) sharing initial noise. FID is computed from 500 generations per arm against a fixed 1,024-image CelebA reference set (Appendix E.4).
Figure 14: Qualitative examples of geometric steering on CelebA. Matched generations at checkpoint 160k. Columns, left to right: DDIM, DDIM + GeoSteer, rectified flow, rectified flow + GeoSteer. Each steered sample starts from the same initial noise as the unsteered sample to its left. All unsteered samples are memorized ( r<1/3 ), and all steered samples are not ( r>1/3 ). Steering redirects the trajectory to a distinct, plausible CelebA generation, which can change identity attributes.
Geometry
DDIM mem. ↓
RF mem. ↓
DDIM time (s) ↓
RF time (s) ↓
k=8
18.0%
32.7%
3.50
5.96
k=16
19.7%
24.7%
3.47
6.29
k=32
19.0%
18.0%
3.52
6.21
k=64
10.0%
14.0%
3.82
6.84
Full spectrum
3.0%
6.3%
–
–
Appendix
Table 10: Effect of geometric resolution on steering. Memorization rates are pooled across the six controlled-model checkpoints. Runtime denotes the median wall-clock time per evaluated steering arm across checkpoints. Full-spectrum timing was not recorded in this benchmark.
Sampler
Method
Copy rate ↓
Mean r↑
Median r↑
DDIM
Baseline
51.6%
0.454
0.314
Matched random rotation
50.8%
0.460
0.326
GeoSteer-DM
7.6%
0.821
0.897
RF
Baseline
60.2%
0.397
0.257
Matched random rotation
58.6%
0.401
0.262
GeoSteer-DM
8.0%
0.778
0.851
Appendix
Table 11: Direction-specificity of geometric steering. Matched comparison at the 300k checkpoint across 500 trajectories per sampler. The random-plane control matches the instantaneous displacement norm and rotation angle of GeoSteer-DM relative to the unmodified update while randomizing the orientation plane.
Method
Mean SSCD ↓
SSCD ≥0.50↓
CLIP ↑
FID ↓
No mitigation
0.362
30.30%
0.279
119.42
JF GeoSteer
0.259
14.15%
0.270
109.15
Jg GeoSteer
0.229
9.60%
0.262
102.74
Appendix
Table 12: Stable Diffusion steering with alternative geometric operators. Results on 500 memorized prompts with four paired generations per prompt. JF denotes the conditional mid-block feature Jacobian, while Jg denotes the Jacobian of the classifier-free guidance residual. Lower SSCD indicates lower similarity to memorized training references.
Method
Mean SSCD ↓
SSCD ≥0.50↓
CLIP ↑
FID ↓
No mitigation
0.362
30.30%
0.279
119.42
Jg GeoSteer
0.229
9.60%
0.262
102.74
Composition with existing mitigation methods
Ren et al. Ren et al. (2024)
0.298→0.217
13.80%→3.80%
0.295→0.281
93.75→79.67
RTA Somepalli et al. (2023b)
0.334→0.213
23.95%→5.50%
0.279→0.263
95.59→82.12
Wen et al. Wen et al. (2024)
0.338→0.210
26.80%→6.60%
0.278→0.260
117.03→102.75
Appendix
Table 13: Guidance-residual GeoSteer complements existing Stable Diffusion mitigation methods. Results on the memorized-prompt cohort. The first block compares unmodified generation with Jg -based GeoSteer. In the composition block, each entry reports the corresponding mitigation method alone → the same method combined with GeoSteer. Lower mean SSCD and a smaller fraction of generations with SSCD≥0.50 indicate lower training-reference similarity.
Method
Mean SSCD
SSCD q0.90
CLIP
No mitigation
0.191
0.262
0.268
Jg GeoSteer
0.167
0.214
0.251
RTA
0.204
0.298
0.281
RTA + GeoSteer
0.183
0.238
0.271
Wen et al.
0.184
0.271
0.268
Wen et al. + GeoSteer
0.178
0.228
0.264
Appendix
Table 14: Control experiment on non-memorized prompts. Results on 100 prompts that are not memorized by the unmodified Stable Diffusion model.
Method
Mean SSCD ↓
Copy rate ↓
CLIP ↑
Baseline
0.847
93.3%
0.325
Attenuation only
0.870
93.3%
0.324
Reflection only
0.593
60.0%
0.302
Reflection + attenuation
0.649
66.7%
0.308
Appendix
Table 15: Reflection versus attenuation on Stable Diffusion. Paired ablation on memorized prompt families.
Figure 15: Qualitative Stable Diffusion examples. Representative generations from the memorized-prompt evaluation. For each example, the training reference, unmodified generation, and GeoSteer generation are shown for the same prompt and initial noise. GeoSteer reduces training-reference similarity while retaining the principal semantic content of the prompt.
Memorization in generative models studies when models retain training examples, how to detect their recall, and how to limit it without losing utility.
Diffusion models generalize early in training and later reproduce individual training samples. Standard tests detect memorization only once one-shot generation produces near-copies, leaving a released model unaudited until its outputs fail. We show that memorization is encoded in the geometry of the learned energy landscape before it appears in generated samples, a state we call latent memorization. Using score divergence and basin volume, we find that localized basins form around training samples and separate them from held-out samples before the first memorized sample appears, with an onset that follows the same O(n) scaling as the memorization time. We probe these basins with cyclic denoising, which repeatedly applies partial noising and denoising. Under the exact empirical score, we prove that cycling started near an isolated training sample recovers it and returns to it over any finite number of cycles with high probability. In trained models, cycling recovers training images from CelebA and CIFAR-10 checkpoints whose one-shot samples contain no copies, and at a CelebA checkpoint with 0.1% one-shot copies, 500 cycles raise the memorized fraction above 30%. Cycling also reveals degenerate attractors that match no single training image and fade as training proceeds, so residence in a basin does not by itself imply memorization. These findings hold on a Gaussian mixture, CelebA, and CIFAR-10 across optimizers, architectures, noise schedules, and training-set sizes, and extend to off-the-shelf Stable Diffusion v1.4, where the cycled conditional-unconditional divergence gap separates memorized from non-memorized prompts with an AUC of 0.944 and a TPR of 0.866 at 1% FPR. More broadly, what a diffusion model has memorized is a property of the geometry and stability of its learned distribution, and assessing it requires examining this structure rather than generated outputs alone.
Nikhil Verma, Siddharthan Dileep, Anoop Singh +4
Yardi School of Artificial Intelligence, Indian Institute of Technology Delhi · Department of Computer Science and Engineering, Indian Institute of Technology Delhi · Jawaharlal Nehru Centre for Advanced Scientific Research +2
Diffusion models can unintentionally memorize training samples, raising concerns about privacy and copyright. While recent methods can detect memorization, they often rely on global or model-specific signals and provide limited insight into where memorization appears within a generated image. We provide a geometric characterization of local memorization as a coordinate-wise variance collapse. However, such collapse can also arise from intrinsic data constraints rather than overfitting. To isolate overfitting-driven memorization, we propose curvature-difference methods that subtract the curvature of an underfitted baseline, either the unconditional model or a less-trained version of itself. We further derive a score-difference proxy that provides a geometric explanation for the widely used score-difference-based detection metric. Experiments on Stable Diffusion, evaluated against ground-truth memorization masks, show that our method outperforms the prior attention-based localization method. Code is available at https://github.com/Gwangho99/mem-curv-diff.
Gwangho Kim, Sungyoon Lee
Department of Computer Science, Hanyang University, Seoul, South Korea.
Diffusion models generate high-quality samples but can also memorize training data, raising serious privacy concerns. Understanding the mechanisms governing when memorization versus generalization occurs remains an active area of research. In particular, it is unclear where along the noise schedule memorization is induced, how data geometry influences it, and how phenomena at different noise scales interact. We introduce a geometric framework that partitions the noise schedule into three regimes based on the coverage properties of training data by Gaussian shells and the concentration behavior of the posterior, which we argue are two fundamental objects governing memorization and generalization in diffusion models. This perspective reveals that memorization risk is highly non-uniform across noise levels. We further identify a danger zone at medium noise levels where memorization is most pronounced. In contrast, both the small and large noise regimes resist memorization, but through fundamentally different mechanisms: small noise avoids memorization due to limited training coverage, while large noise exhibits low posterior concentration and admits a provably near linear Gaussian denoising behavior. For the medium noise regime, we identify geometric conditions through which we propose a geometry-informed targeted intervention that mitigates memorization.
Nick Dodson, Xinyu Gao, Qingsong Wang +2
Department of Mathematics, University of Missouri, Columbia, Missouri, USA · Halıcıo˘glu Data Science Institute, University of California San Diego, La Jolla, California, USA