Controlling Polar Exposure to Delay Memorization in Diffusion Models

Xuanchen Wang; Heng Wang; Weidong Cai

cs.LGOct 2, 2026

Controlling Polar Exposure to Delay Memorization in Diffusion Models

Authors: Xuanchen Wang, Heng Wang, Weidong Cai

Organizations: School of Computer Science The University of Sydney Sydney, NSW, Australia

Abstract

Diffusion models can reach useful sample quality before copying training examples, but fast optimization can compress this generalization window by accelerating sample-specific fitting. We investigate this effect through update geometry and propose Quality-Gated De-whitening (QGD), a controller that retains a fast polar-update prefix and progressively restores fixed-gain momentum. Our random-feature analysis separates covariance-controlled, curvature-equalized and amplitude-controlled memorization clocks. Under aligned spectral assumptions, it establishes a finite-exposure condition under which a fixed-gain tail recovers a delay proportional to dataset size. QGD implements this principle with a confirmed quality gate, a bounded decay envelope and causal copy feedback. Immediate switching is the conservative limit; gradual control balances delayed copying against continued quality improvement. We pair QGD with Copy-Budgeted Selection (CBS), which applies simultaneous binomial calibration to a frozen checkpoint family, followed by a fresh evaluation of the released checkpoint. On 2,000-image CIFAR-10 subsets, QGD preserves the polar baseline's quality-arrival time while expanding its useful interval by 8.32x and reducing common-checkpoint copying by 75.9%. With identical calibration and independent quality evaluation, QGD achieves FID 75.56 versus 79.37 for SGD with the same selector. Exposure-matched controls, independent detector audits and transfer to flow matching and dance generation support adaptive exposure control as a practical way to improve the quality-copying tradeoff.

Figures & tables

Figure 1: Quality-gated training and calibrated checkpoint release. QGD preserves a fast polar prefix, then uses development copying feedback to reduce polar exposure within a fixed envelope. CBS evaluates a frozen checkpoint family with fresh calibration inputs. A separate final audit evaluates the selected checkpoint without changing the decision.

CIFAR-10, $n=2,000$ ; audit checkpoint: 60k
Method	Quality arrival $\downarrow$	Copy onset $\uparrow$	Window $\Delta t_{H}\uparrow$	FID $\downarrow$	Copy (%) $\downarrow$
SGD	$16.0\!\pm\!1.4$	NR	$184.0\!\pm\!1.4$	$85.4\!\pm\!0.3$	$1.26\!\pm\!0.11$
AdamW	$14.0\!\pm\!1.4$	$124.0\!\pm\!6.2$	$110.0\!\pm\!5.3$	$82.9\!\pm\!0.3$	$2.51\!\pm\!0.10$
Muon	$8.0\!\pm\!1.4$	$28.0\!\pm\!1.4$	$20.0\!\pm\!1.4$	$81.3\!\pm\!0.3$	$16.10\!\pm\!0.47$
QGD (ours)	$8.0\!\pm\!1.4$	$174.4\!\pm\!10.1$	$166.4\!\pm\!11.3$	$77.8\!\pm\!0.3$	$3.88\!\pm\!0.31$
CIFAR-10, $n=10,000$ ; audit checkpoint: 120k

Table 1: Training-method comparison on CIFAR-10. Times are thousands of updates. Mean

\pm

sample SD over five paired trajectories. FID and copy fractions use independent audits at the stated checkpoint; clocks use confirmed development probes. NR denotes no confirmed copy crossing by the observation horizon

H=200

k/400k. Observed windows end at

\min(t_{\rm copy},H)

Figure 2: Quality, copying and exposure during training. The primary CIFAR-10 trajectory (pair 0) compares QGD with continued Muon, AdamW and the same-gate hard switch. Development curves share a 0–200k time axis, with FID target 88 and copy budget 5%. The 0–28k inset resolves the transition after quality confirmation at 10k. The hard switch is more conservative in copying; gradual QGD retains stronger quality. The coefficient for each block is fixed using the preceding development probe.

Method	Polar exposure	Observed window $\Delta t_{H}\uparrow$	FID ${}_{60}\downarrow$	Copy 60 (%) $\downarrow$
MiMuon (norm proxy)	–	$84.4\!\pm\!5.0$	$80.6\!\pm\!0.3$	$4.30\!\pm\!0.30$
Same-gate hard switch	$0.00\!\pm\!0.00$	$186.0\!\pm\!2.4$	$81.5\!\pm\!0.3$	$2.47\!\pm\!0.03$
Pilot-fixed open loop	$4.26\!\pm\!0.00$	$124.0\!\pm\!50.0$	$79.0\!\pm\!1.0$	$4.56\!\pm\!0.60$
QGD w/o quality gate	$7.03\!\pm\!0.00$	$160.8\!\pm\!10.4$	$82.9\!\pm\!0.3$	$2.95\!\pm\!0.20$
QGD w/o feedback	$7.04\!\pm\!0.00$	$70.0\!\pm\!7.1$	$79.2\!\pm\!0.3$	$4.77\!\pm\!0.21$
QGD + per-step RMS	$4.51\!\pm\!0.51$	$30.0\!\pm\!2.4$	$80.8\!\pm\!0.3$	$13.51\!\pm\!0.16$

Table 2: Switching controls and component ablations. CIFAR-10,

n=2{,}000

; mean

\pm

SD over five paired trajectories. Exposure and window are in k steps; quality and copying use independent audits at 60k. The open-loop schedule is fixed by an independent pilot. The no-gate exposure starts at initialization; MiMuon has no common post-gate mixture exposure.

Copy budget: 5%
Selection rule	Final FID $\downarrow$	Copy (%) $\downarrow$	Release $\uparrow$	Joint error $\downarrow$	Calib. outputs (k) $\downarrow$
Empirical filter	$73.97\!\pm\!0.30$	5.15	500/500	459/500	500.0
Preselect + CP	$74.09\!\pm\!0.43$	5.11	20/500	3/500	5.0
Margin preselect + CP	$77.89\!\pm\!2.28$	4.15	463/500	0/500	5.0
Fixed sequence	$77.86\!\pm\!0.64$	4.08	500/500	0/500	152.6
CBS (ours)	$75.56\!\pm\!0.51$	4.37	500/500	1/500	500.0

Table 3: Checkpoint-selection rules under copy budgets. Five QGD archives, 100 independent calibration banks each;

\delta=0.05

. Final FID and fresh audit copying condition on release; FID is summarized over defined trajectory means. At 5%, preselection never releases on one trajectory. Joint errors use the retained underlying detector rates. Calibration outputs exclude the common development archive and final audit.

Training + selection	Selected step (k)	Develop. FID $\downarrow$	Final FID $\downarrow$	Copy (%) $\downarrow$
SGD + CBS	200.0	79.00	$79.37\!\pm\!0.31$	2.67
AdamW + CBS	103.8	81.58	$81.94\!\pm\!0.29$	4.24
Muon + CBS	18.5	85.81	$86.14\!\pm\!0.12$	3.99
MiMuon (norm proxy) + CBS	64.2	80.40	$80.68\!\pm\!0.40$	4.25
Same-gate hard switch + CBS	160.0	77.46	$77.82\!\pm\!0.27$	4.30
Pilot-fixed open loop + CBS	77.8	79.00	$79.30\!\pm\!3.08$	4.37

Table 4: Calibrated selection across training strategies. CIFAR-10,

n=2{,}000

; 5% copy budget and

\delta=0.05

. All methods use a 200k archive and identical CBS allocation. Final FID uses a fresh 50k-output bank; copying uses a 10k subset. Mean

\pm

SD is across five trajectories after averaging 100 calibration decisions each. All 500 decisions per row release.

Flow matching, $n=2{,}000$
Method	FID $\downarrow$	Quality arrival $\downarrow$	Window $\Delta t_{H}\uparrow$	Copy (%) $\downarrow$
AdamW	$84.5\!\pm\!0.3$	$10.0\!\pm\!1.4$	$106.0\!\pm\!2.8$	$2.08\!\pm\!0.22$
Muon	$86.2\!\pm\!0.3$	$8.0\!\pm\!1.4$	$18.0\!\pm\!2.8$	$14.61\!\pm\!0.07$
Same-gate hard switch	$82.8\!\pm\!0.3$	$8.0\!\pm\!1.4$	$120.0\!\pm\!2.8$	$2.48\!\pm\!0.15$
QGD (ours)	$79.8\!\pm\!0.3$	$8.0\!\pm\!1.4$	$100.0\!\pm\!2.8$	$4.14\!\pm\!0.25$
AIST++ dance generation

Table 5: Transfer across generation tasks. Mean

\pm

SD over five paired trajectories. Flow times are k updates; quality is audited at the 60k checkpoint. Dance quality at 120k separates beat synchrony, kinetic-feature FID and diversity relative to held-out motions; one is the diversity target. Copying uses 10k fresh audit outputs.

Appendix figures & tables20 assets

Supplementary material from the paper’s appendix.

Appendix

$n$	Finite lower bound	Hard	Finite	Excess	Floor	Polar
64	12.55	27.95	20.85	5.25	9.62	4.00
128	23.10	53.90	39.09	5.40	10.65	4.00
256	44.19	105.80	75.56	5.48	11.28	4.00
512	86.38	209.60	148.49	5.52	11.62	4.00

Appendix

Table 6: Fluctuation arrival in the aligned spectral diagnostic. Exact mixed-flow threshold times

\tau_{1/2}

for the family in Equation 16 . “Finite” and “Excess” denote the

T=0.8

and

T=3

exponential mixtures. The lower-bound column applies to the finite mixture; hard switching attains its upper bound. All gains and the gate state are fixed across

n

Figure 3: Exposure, delay and early quality in the aligned model. (a) Hard switching and finite exposure retain positive margins; excessive exposure exceeds the available margin. (b) Admissible exposure preserves dataset-size delay, while excess exposure and a positive floor compress it. (c) At

n=256

, finite exposure improves early full population risk, while hard switching reaches lower risk later. Curves use Equation 16 ; Table 6 and Appendix D specify the exact solver and parameters.

$\sigma$	optimizer	$\tau_{\mathrm{mem}}$ slope	$\tau_{\mathrm{gen}}$ slope	$R/R_{\rm GF}$ at $512$
0.25	gradient flow	1.00 [0.99, 1.01]	0.10	1.000
0.25	Adam	0.65 [0.64, 0.66]	0.08	0.275
0.25	Muon	0.20 [0.19, 0.21]	0.05	0.047
0.25	Newton	0.00 [0.00, 0.00]	0.00	0.008
0.5	gradient flow	1.05 [1.04, 1.06]	0.10	1.000
0.5	Adam	0.70 [0.69, 0.71]	0.08	0.270

Appendix

Table 7: Quality and memorization scaling in the random-feature model. Five draws, ReLU,

d=64

unless indicated. Brackets are bootstrap ranges across draws. Slopes use common fit supports and

R=\tau_{\mathrm{mem}}/\tau_{\mathrm{gen}}

; Newton quality slopes use

n\geq 128

$n$	pop. index	$\lambda_{\rm pop}$	fluct. index	$\lambda_{\rm fl}$	energy ratio
16	4	$1.40\times 10^{-1}$	40	$5.20\times 10^{-3}$	0.200
32	4	$1.38\times 10^{-1}$	72	$2.60\times 10^{-3}$	0.184
64	4	$1.36\times 10^{-1}$	136	$1.30\times 10^{-3}$	0.169
128	4	$1.34\times 10^{-1}$	264	$6.51\times 10^{-4}$	0.156
256	4	$1.32\times 10^{-1}$	520	$3.25\times 10^{-4}$	0.143
512	4	$1.31\times 10^{-1}$	1032	$1.63\times 10^{-4}$	0.132

Appendix

Table 8: Spectral diagnostics. ReLU,

\sigma=0.25

. Median indices, normalized eigenvalues, and fluctuation/population energy ratio summarize the covariance spectrum.

$\sigma$	optimizer	$\tau_{\mathrm{mem}}$ slope	$\tau_{\mathrm{gen}}$ slope	$R/R_{\rm GF}$ at $512$
0.25	gradient flow	1.15 [1.14, 1.16]	0.12	1.000
0.25	Adam	0.80 [0.79, 0.81]	0.10	0.255
0.25	Muon	0.30 [0.29, 0.31]	0.06	0.036
0.5	gradient flow	1.10 [1.09, 1.11]	0.10	1.000
0.5	Adam	0.75 [0.74, 0.76]	0.08	0.270
0.5	Muon	0.24 [0.23, 0.25]	0.05	0.035

Appendix

Table 9: Quality and memorization scaling with

\tanh

features. Same five-draw and bootstrap-range conventions as Table 7 .

$\sigma$	quantity	$m=1024$	$m=512$
0.25	gradient flow $\beta_{\rm mem}$	1.00 [0.99, 1.01]	1.01 [1.00, 1.02]
0.25	Muon $\beta_{\rm mem}$	0.20 [0.19, 0.21]	0.21 [0.20, 0.22]
0.25	Muon–GF $\beta_{R}$ gap	-0.75 [-0.75, -0.75]	-0.75 [-0.75, -0.74]
0.5	gradient flow $\beta_{\rm mem}$	1.05 [1.04, 1.06]	1.06 [1.05, 1.07]
0.5	Muon $\beta_{\rm mem}$	0.18 [0.17, 0.19]	0.19 [0.18, 0.20]
0.5	Muon–GF $\beta_{R}$ gap	-0.82 [-0.82, -0.82]	-0.82 [-0.82, -0.81]

Appendix

Table 10: Sensitivity to the number of noise draws. Comparison of

m=512

and

m=1024

; brackets are bootstrap ranges.

	$\sigma=0.25$				$\sigma=0.5$
$n$	$R_{\rm GF}$	Muon	Adam	Newton	$R_{\rm GF}$	Muon	Adam	Newton
16	2.20	0.636	0.864	–	2.60	0.577	0.846	–
32	4.11	0.378	0.687	–	5.02	0.327	0.673	–
64	7.66	0.225	0.547	–	9.70	0.185	0.536	–
128	14.30	0.134	0.435	0.028	18.75	0.105	0.426	0.021
256	26.68	0.080	0.346	0.015	36.21	0.059	0.339	0.011

Appendix

Table 11: Window-ratio comparisons. Within each noise group, gradient-flow

R

is absolute and the other optimizers are normalized to it. Newton is – where it never reaches the quality gate.

optimizer	inverse fluct. eigenvalue	fluct. nuclear norm
gradient flow	0.99	1.00
gradient descent	0.99	1.00
Adam	0.99	1.00
Muon	0.14	0.22
Newton	–	–

Appendix

Table 12: Clock and spectral-descriptor correlations. Spearman correlations of memorization time with spectral descriptors, pooling

60

draw/size/noise cells per optimizer. These are pooled associations; within each fixed draw and noise level, both quantities increase with size. A constant clock has undefined correlation (–).

Figure 4: Geometry and random-feature evidence. Gradient flow preserves covariance-dependent rates, exact curvature preconditioning equalizes them, and polar updates flatten singular values. In the displayed comparison, the window ordering is Newton

<

Muon

<

Adam

<

gradient flow wherever all quality clocks exist. Newton points are omitted where the quality gate is not reached. Full assumptions and draw-level results appear in Appendices A – E .

$\lambda$	$\tau_{\mathrm{gen}}$	$\tau_{\mathrm{mem}}$	$R$
0.00	16	164	10.25
0.25	14	112	8.00
0.50	12	70	5.83
0.75	10	44	4.40
1.00	8	28	3.50

Appendix

Table 13: Momentum–polar interpolation at matched update RMS. Equal-RMS intervention; times in thousands of steps.

iterations	$\tau_{\mathrm{gen}}$	$\tau_{\mathrm{mem}}$	$R$
1	12	84	7.00
2	10	50	5.00
5	8	28	3.50

Appendix

Table 14: Sensitivity to Newton–Schulz iteration count. Times in thousands of steps.

Method	Pixel copying (%) $\downarrow$	SSCD copying (%) $\downarrow$
SGD	$1.30\!\pm\!0.10$	$1.18\!\pm\!0.07$
AdamW	$2.44\!\pm\!0.17$	$2.26\!\pm\!0.11$
Muon	$16.02\!\pm\!0.35$	$13.52\!\pm\!0.27$
Same-gate hard switch	$2.51\!\pm\!0.22$	$2.19\!\pm\!0.15$
Pilot-fixed open loop	$4.34\!\pm\!0.51$	$3.97\!\pm\!0.42$
QGD (ours)	$3.96\!\pm\!0.40$	$3.55\!\pm\!0.22$

Appendix

Table 15: Independent copying audits under two detectors. CIFAR-10,

n=2{,}000

, 60k checkpoint. Mean

\pm

SD over five trajectories; each detector uses a fresh 10k-output audit. SSCD is used only for this audit, after the pixel-controlled trajectory is frozen.

Method	Nine-configuration search	Selected setting
SGD	$\eta\in\{1,2,4\}10^{-4}$ ; $\beta\in\{.85,.90,.95\}$	$\eta=2\cdot 10^{-4}$ , $\beta=.9$
AdamW	$\eta\in\{1,2,4\}10^{-4}$ ; $\beta_{1}\in\{.85,.90,.95\}$	$\eta=2\cdot 10^{-4}$ , $\beta=.9$
Muon	$\eta\in\{1,2,4\}10^{-4}$ ; $\beta\in\{.85,.90,.95\}$	$\eta=2\cdot 10^{-4}$ , $\beta=.9$
MiMuon (norm proxy)	$\eta\in\{1,2,4\}10^{-4}$ ; $\rho\in\{.5,1,2\}$	$\eta=2\cdot 10^{-4}$ , $\rho=1$
QGD (ours)	$T\in\{4,8,12\}$ k; $\kappa\in\{.25,.5,.75\}$	$T=8$ k, $\kappa=.5$
Pilot-fixed open loop	$T\in\{3.8,4.5,5.2,5.9,6.6,7.3,8,8.7,9.4\}$ k	$T=5.2$ k

Appendix

Table 16: Independent pilot allocation and selected settings. Each method uses nine prespecified pilot configurations. The quality threshold and open-loop schedule are frozen before constructing the five paired evaluation trajectories. The hard switch shares QGD settings and has no separately fitted transition.

Method	Quality (k)	Copy (k)	Window (k)	FID 60	Copy 60 (%)
SGD	$16.0\!\pm\!1.4$	NR	$184.0\!\pm\!1.4$	$85.4\!\pm\!0.3$	$1.26\!\pm\!0.11$
AdamW	$14.0\!\pm\!1.4$	$124.0\!\pm\!6.2$	$110.0\!\pm\!5.3$	$82.9\!\pm\!0.3$	$2.51\!\pm\!0.10$
Muon	$8.0\!\pm\!1.4$	$28.0\!\pm\!1.4$	$20.0\!\pm\!1.4$	$81.3\!\pm\!0.3$	$16.10\!\pm\!0.47$
MiMuon (norm proxy)	$10.0\!\pm\!1.4$	$94.4\!\pm\!6.2$	$84.4\!\pm\!5.0$	$80.6\!\pm\!0.3$	$4.30\!\pm\!0.30$
Same-gate hard switch	$8.0\!\pm\!1.4$	$194.0\!\pm\!3.2$	$186.0\!\pm\!2.4$	$81.5\!\pm\!0.3$	$2.47\!\pm\!0.03$
Pilot-fixed open loop	$8.0\!\pm\!1.4$	$132.0\!\pm\!50.5$	$124.0\!\pm\!50.0$	$79.0\!\pm\!1.0$	$4.56\!\pm\!0.60$

Appendix

Table 17: Complete training-controller comparisons. Five paired CIFAR-10 trajectories,

n=2{,}000

; mean

\pm

SD. Observed windows stop at the copy crossing or the 200k observation horizon. FID and copying use independent audits at 60k.

Setting	Exposure (k)	FID ${}_{60}\downarrow$	Copy 60 (%) $\downarrow$
Default	$4.21\!\pm\!0.64$	$77.8\!\pm\!0.3$	$3.88\!\pm\!0.31$
Decay T=4k	$2.41\!\pm\!0.20$	$78.5\!\pm\!0.3$	$3.65\!\pm\!0.30$
Decay T=12k	$5.17\!\pm\!1.04$	$77.7\!\pm\!0.3$	$4.57\!\pm\!0.34$
Feedback kappa=0.25	$5.24\!\pm\!0.33$	$78.0\!\pm\!0.3$	$4.40\!\pm\!0.33$
Feedback kappa=0.75	$3.58\!\pm\!0.81$	$78.3\!\pm\!0.3$	$3.84\!\pm\!0.21$
Warning=2Warning=3Probe cadence=1k	$3.43\!\pm\!0.96$	$77.9\!\pm\!0.3$	$3.78\!\pm\!0.24$

Appendix

Table 18: Sensitivity to controller settings and probe cadence. CIFAR-10,

n=2{,}000

; 60k quality/copying readouts. Each row changes one setting from

T=8

\kappa=0.5

, warning 2.5%, cadence 2k. Exposure is obtained by evaluating the causal recursion on the corresponding fixed probe trajectory.

Baseline	Final FID difference
SGD	$3.81\!\pm\!0.33$
AdamW	$6.38\!\pm\!0.31$
Muon	$10.57\!\pm\!0.51$
MiMuon (norm proxy)	$5.11\!\pm\!0.19$
Same-gate hard switch	$2.25\!\pm\!0.31$
Pilot-fixed open loop	$3.74\!\pm\!2.89$

Appendix

Table 19: Paired differences in selected checkpoint quality. Baseline final FID minus QGD final FID under CBS at a 5% copy budget. Mean

\pm

sample SD is over the five paired trajectory differences after averaging each trajectory over its 100 calibration decisions.

Trajectory	Quality	Confirm.	$\alpha=0$	Exposure	Copy onset	FID 60
0	8	10	20	4.450	174	77.83
1	8	10	18	3.837	170	77.38
2	10	12	22	4.685	160	78.16
3	6	8	14	3.273	184	77.67
4	8	10	20	4.805	184	77.91

Appendix

Table 20: Quality gates and adaptive transition schedules. The coefficient is set using the probe at the listed zero time for the following block. Times are k steps. Every schedule obeys the 7.0416k exposure envelope.

Candidates $M$	Cadence (k)	Final FID	Release	Joint error	Outputs (k)
20	10	$75.98\!\pm\!0.59$	500/500	0/500	100
50	4	$75.75\!\pm\!0.59$	500/500	3/500	250
100	2	$75.56\!\pm\!0.51$	500/500	1/500	500

Appendix

Table 21: Sensitivity to checkpoint-family size. QGD archives at a 5% copy budget,

\delta=0.05

. Thinned families are fixed before calibration and reuse the same candidate-level binomial draws for paired comparison. Each candidate receives 5k outputs; CBS uses

\delta/M

Method	Train	Develop.	CBS	Final audit	Total
SGD	3.33	2.64	1.19	0.25	7.42
AdamW	4.00	2.64	1.19	0.25	8.08
Muon	4.56	2.64	1.19	0.25	8.64
MiMuon (norm proxy)	4.17	2.64	1.19	0.25	8.25
Same-gate hard switch	3.50	2.64	1.19	0.25	7.58
Pilot-fixed open loop	4.66	2.64	1.19	0.25	8.74

Appendix

Table 22: End-to-end cost accounting for a 200k checkpoint archive. Hours from the operation-time model in Appendix K . Development quality generation/features cost 90s and the copy check adds 5s on shared samples. CBS uses 500k outputs; the independent final quality audit uses 50k. The full open-loop polar branch remains active because its exponential coefficient never reaches zero.

Flow matching
Method	Quality (k)	Copy (k)	Window (k)	Quality	Copy (%)
SGD	$16.0\!\pm\!1.4$	$200.0\!\pm\!3.7$	$184.0\!\pm\!2.8$	$86.4\!\pm\!0.3$	$1.17\!\pm\!0.13$
AdamW	$10.0\!\pm\!1.4$	$116.0\!\pm\!3.7$	$106.0\!\pm\!2.8$	$84.5\!\pm\!0.3$	$2.08\!\pm\!0.22$
Muon	$8.0\!\pm\!1.4$	$26.0\!\pm\!3.7$	$18.0\!\pm\!2.8$	$86.2\!\pm\!0.3$	$14.61\!\pm\!0.07$
Same-gate hard switch	$8.0\!\pm\!1.4$	$128.0\!\pm\!3.7$	$120.0\!\pm\!2.8$	$82.8\!\pm\!0.3$	$2.48\!\pm\!0.15$
QGD (ours)	$8.0\!\pm\!1.4$	$108.0\!\pm\!3.7$	$100.0\!\pm\!2.8$	$79.8\!\pm\!0.3$	$4.14\!\pm\!0.25$

Appendix

Table 23: Transfer clocks and complete common-checkpoint readouts. Five paired trajectories per setting. Times are k steps; flow quality is FID at 60k and dance quality is beat alignment at 120k. Main-text dance realism and diversity are independent additional quality dimensions.

Diffusion Models

Diffusion models learn to reverse gradual data corruption, enabling flexible generation, editing, forecasting, and inference.

See all 1,192 papers

Feb 23, 2026

You Don't Need All That Attention: Surgical Memorization Mitigation in Text-to-Image Diffusion Models

Generative models have been shown to "memorize" certain training data, leading to verbatim or near-verbatim generating images, which may cause privacy concerns or copyright infringement. We introduce Guidance Using Attractive-Repulsive Dynamics (GUARD), a novel framework for memorization mitigation in text-to-image diffusion models. GUARD adjusts the image denoising process to guide the generation away from an original training image and towards one that is distinct from training data while remaining aligned with the prompt, guarding against reproducing training data, without hurting image generation quality. We propose a concrete instantiation of this framework, where the positive target that we steer towards is given by a novel method for (cross) attention attenuation based on (i) a novel statistical mechanism that automatically identifies the prompt positions where cross attention must be attenuated and (ii) attenuating cross-attention in these per-prompt locations. The resulting GUARD offers a surgical, dynamic per-prompt inference-time approach that, we find, is by far the most robust method in terms of consistently producing state-of-the-art results for memorization mitigation across two architectures and for both verbatim and template memorization, while also improving upon or yielding comparable results in terms of image quality.

Kairan Zhao, Eleni Triantafillou, Peter Triantafillou

University of Warwick, UK · Google DeepMind, UK

Memorization in Generative Models T2I Diffusion Models Diffusion Model Guidance Attention Control in Diffusion Models

Ask PDF

Mar 18, 2026

Q-Drift: Quantization-Aware Drift Correction for Diffusion Model Sampling

Post-training quantization (PTQ) is a practical path to deploy large diffusion models, but quantization noise can accumulate over the denoising trajectory and degrade generation quality. We propose Q-Drift, a sampler-side correction that aims to preserve the intended sampling marginals through a deterministic drift adjustment motivated by generalized marginal-preserving SDEs. Q-Drift uses the calibrated conditional residual variance of quantization error to determine a correction factor at each step. Our SDXL study shows that calibration with as few as 10 paired full-precision/quantized runs remains effective. The resulting sampler correction is plug-and-play with common samplers, diffusion models, and PTQ methods, while incurring negligible overhead at inference. Across six diverse text-to-image models (spanning DiT and U-Net), three samplers (Euler, flow-matching, DPM-Solver++), and two PTQ methods (SVDQuant, MixDQ), Q-Drift improves FID over the corresponding quantized baseline in all seven settings of our main evaluation, with up to 4.79 FID reduction on PixArt-Sigma (SVDQuant W3A4), while preserving CLIP scores. Code is available at https://github.com/sooyoung-ryu/Q-Drift.

Sooyoung Ryu, Mathieu Salzmann, Saqib Javed

Department of Computer Science and Engineering, Seoul National University, Seoul, South Korea · School of Computer and Communication Sciences, EPFL, Lausanne, Switzerland · Meta Reality Labs, Redmond, USA

Diffusion Model Quantization Diffusion Model Sampling Post-Training Quantization Neural Network Quantization T2I Diffusion Models

Ask PDF

Mar 7, 2026

Diffusion Controller: Framework, Algorithms and Parameterization

Controllable diffusion generation often relies on various heuristics that are seemingly disconnected without a unified understanding. We bridge this gap with Diffusion Controller (DiffCon), a unified control-theoretic view that casts reverse diffusion sampling as state-only stochastic control within (generalized) linearly-solvable Markov Decision Processes (LS-MDPs). Under this framework, control acts by reweighting the pretrained reverse-time transition kernels, balancing terminal objectives against an

f

-divergence cost. From the resulting optimality conditions, we derive practical reinforcement learning methods for diffusion fine-tuning: (i) f-divergence-regularized policy-gradient updates, including a PPO-style rule, and (ii) a regularizer-determined reward-weighted regression objective with a minimizer-preservation guarantee under the Kullback-Leibler (KL) divergence. The LS-MDP framework further implies a principled model form: the optimal score decomposes into a fixed pretrained baseline plus a lightweight control correction, motivating a side-network parameterization conditioned on exposed intermediate denoising outputs, enabling effective gray-box adaptation with a frozen backbone. Experiments on Stable Diffusion v1.4 across supervised and reward-driven finetuning show consistent gains in preference-alignment win rates and improved quality-efficiency trade-offs versus gray-box baselines and even the parameter-efficient white-box adapter LoRA.

Tong Yang, Moonkyung Ryu, Chih-Wei Hsu +4

Carnegie Mellon University; Intern at Google Research · Google · Yale University

Diffusion Model Adaptation Diffusion Model Fine-Tuning Diffusion Models Diffusion Model Alignment RL for Diffusion Models Diffusion-Based Denoising

Ask PDF

Abstract

Diffusion Models

Explore similar work

You Don't Need All That Attention: Surgical Memorization Mitigation in Text-to-Image Diffusion Models

Q-Drift: Quantization-Aware Drift Correction for Diffusion Model Sampling

Diffusion Controller: Framework, Algorithms and Parameterization