Annealed Lévy Diffusion via Gaussian Surrogate Reverse Dynamics

Yuling Jiao; Lican Kang; Jerry Zhijian Yang; Cheng Yuan; Ruizhe Zhang

cs.LGOct 14, 2026

Annealed Lévy Diffusion via Gaussian Surrogate Reverse Dynamics

Authors: Yuling Jiao, Lican Kang, Jerry Zhijian Yang, Cheng Yuan, Ruizhe Zhang

Organizations: School of Artificial Intelligence, Wuhan University, Wuhan, Hubei 430072, China. · Institute for Math and AI, Hubei Key Laboratory of Computational Science, and School of Artificial Intelligence, Wuhan University, Wuhan, 430072, China. · School of Mathematics and Statistics, Wuhan University, Wuhan, Hubei 430072, China.

Abstract

Gaussian diffusion models offer a practical framework for generative modeling. Denoising Lévy Probabilistic Models (DLPM) introduce heavy-tailed perturbations with a fixed tail index throughout denoising. We introduce Annealed Lévy Diffusion (ALD). ALD varies the stability index $α(t)$ to combine heavy-tailed exploration with Gaussian refinement. Our marginal-first formulation represents stable noise as a scalar Gaussian scale mixture. This formulation enables standard Gaussian noise-prediction training with one additional scale input. We study two sampling processes. ALD-renoise estimates the clean sample and injects fresh marginal noise. ALD-posterior preserves dependence between successive states through an accumulated conditional variance and a Gaussian bridge update. Both samplers reuse the same trained predictor with different scalar inputs. We evaluate ALD on four 2D benchmarks and an imbalanced nine-mode mixture. We also examine training efficiency and the effects of prediction perturbations on clipped Holtsmark data. Additional experiments evaluate MNIST and CelebA64 generation and CIFAR-10LT augmentation. ALD preserves local geometry on the evaluated light-tailed and manifold-structured targets.

Figures & tables

Symbol	Description
$\mathbf{x}_{t}$	State vector at diffusion time $t\in[0,1]$
$\mathbf{x}_{0}$	Data sample from $p_{\mathrm{data}}(\mathbf{x})$
$\alpha(t)$	Time-dependent stability index, $\alpha(t)\in(1,2]$
$S_{t}$	Positive stable variance multiplier, $S_{t}\sim\mathcal{A}_{\alpha(t)/2}$ ; $S_{t}\equiv 1$ if $\alpha(t)=2$
$\tilde{S}_{t}$	Clamped impulse, $\tilde{S}_{t}=\min(S_{t},S_{\max})$
$\mathbf{n}$	Standard Gaussian random vector, $\mathbf{n}\sim\mathcal{N}(\mathbf{0},\mathbf{I})$

Table 1: Summary of Notation

Dataset	Model	$W_{2}$ ( $\downarrow$ )	$\mathrm{MMD}^{2}$ ( $\downarrow$ )	SHS ( $\approx 1$ )
Heavy-Tailed Cross	DDPM	$0.0833\pm 0.0195$	$0.000852\pm 0.001177$	$0.8857\pm 0.2411$
	DLPM	$0.1772\pm 0.0529$	$0.005861\pm 0.008071$	$0.5817\pm 0.2223$
	ALD-renoise	$\mathbf{0.0830\pm 0.0374}$	$\mathbf{0.000384\pm 0.000361}$	$1.1131\pm 0.3757$
	ALD-posterior	$0.0884\pm 0.0113$	$0.000435\pm 0.000763$	$\mathbf{0.9398\pm 0.2344}$
Swiss Roll	DDPM	$0.0268\pm 0.0081$	$0.000267\pm 0.000489$	$\mathbf{1.2473\pm 0.3656}$
	DLPM	$0.0971\pm 0.0207$	$0.003356\pm 0.002302$	$0.6516\pm 0.3785$

Table 2: 2D benchmarks (mean

\pm

sample standard deviation over five seeds). Lower is better for

W_{2}

and

\mathrm{MMD}^{2}

; SHS is ranked by distance to one. Swiss Roll SHS uses the two valid seeds shared by all methods. Bold marks the best mean. Full protocol details are in Appendix A .

Figure 2: Representative samples on the four main 2D benchmarks. Rows correspond to HTC, Swiss Roll, MoG40, and Holtsmark; columns show the reference data, DDPM, DLPM, ALD-renoise, and ALD-posterior. The quantitative metrics are reported separately in Table 2 .

Model	$P_{\rm PRD}$ ( $\uparrow$ )	$R_{\rm PRD}$ ( $\uparrow$ )	$F^{\rm pr}_{1}$ ( $\uparrow$ )
DDPM	$0.9307\pm 0.0177$	$0.9805\pm 0.0039$	$0.9549\pm 0.0104$
DLPM	$0.9600\pm 0.0064$	$0.9835\pm 0.0014$	$0.9716\pm 0.0039$
DLPM5	$0.9605\pm 0.0031$	$0.9820\pm 0.0031$	$0.9711\pm 0.0011$
ALD-renoise	$0.9897\pm 0.0006$	$0.9863\pm 0.0026$	$0.9880\pm 0.0016$
ALD-posterior	$\mathbf{0.9909\pm 0.0012}$	$\mathbf{0.9935\pm 0.0004}$	$\mathbf{0.9922\pm 0.0004}$

Table 3: Additional imbalanced nine-mode benchmark (mean

\pm

std over three seeds; higher is better).

Figure 3: Additional nine-mode benchmark under unequal mixture weights.

Figure 4: Training-efficiency diagnostics on clipped Holtsmark (mean

\pm

std over three seeds):

W_{2}

versus (a) hidden width and (b) optimizer updates. ALD-renoise and ALD-posterior share checkpoints and sampling parameters; lower is better. Full settings are in Appendix A .

Method / Digit	0	1	2	3	4	5	6	7	8	9
DDPM	72.12	85.93	72.55	67.21	67.78	67.17	75.20	73.48	62.15	74.89
DLPM	86.83	91.48	88.42	85.76	88.51	86.48	86.25	87.79	81.40	89.37
ALD-renoise	74.23	85.56	66.95	60.13	62.43	64.06	70.96	70.77	61.35	74.02
ALD-posterior	89.20	93.43	91.53	87.76	88.51	88.79	89.67	90.40	84.38	89.26

Table 4: Classwise MNIST kNN coverage (%). Bold marks the highest coverage in each digit column, including ties. Values average two classifier diagnostics and two sampling seeds from one generator training seed; full settings are in Appendix A .

Training data	Accuracy
Teacher model	93.43
DDPM + Real	91.61
DLPM + Real	91.42
ALD-posterior + Real	91.72

Table 5: CIFAR-10 test accuracy (%). Bold marks the best result among the three synthetic-augmentation arms; the teacher model is shown as a real-data-only reference.

Method / Class	Airplane	Auto.	Bird	Cat	Deer	Dog	Frog	Horse	Ship	Truck	Macro
Teacher model	94.11	96.59	91.32	85.97	93.78	90.14	95.24	95.65	95.96	95.63	93.44
DDPM + Real	91.37	96.11	88.53	82.28	92.37	86.59	94.67	94.71	94.68	94.77	91.61
DLPM + Real	92.55	96.17	88.07	81.80	91.55	86.15	93.97	93.49	95.39	94.98	91.41
ALD-posterior + Real	92.14	96.57	88.69	82.50	91.83	87.03	93.96	94.35	94.85	95.18	91.71

Table 6: Classwise CIFAR-10 F 1 (%). Bold marks the best result in each column among the three synthetic-augmentation arms; the teacher model is excluded from this comparison.

Model	FID $\downarrow$	KID $\times 10^{3}\downarrow$	Precision $\uparrow$	Recall $\uparrow$	Density $\uparrow$	Coverage $\uparrow$
DDPM	$2.5763\pm 0.0197$	$1.7926\pm 0.4276$	0.7607	0.5819	1.2721	0.9680
DLPM	$2.9660\pm 0.0221$	$1.9828\pm 0.3872$	0.7681	0.5687	1.3417	0.9697
ALD-posterior	$\mathbf{2.1090\pm 0.0155}$	$\mathbf{1.1419\pm 0.2892}$	0.7536	0.5940	1.2417	0.9683

Table 7: Formal CelebA64 generation results (single training and sampling seed; 50,000 generated images and 1,000 NFE per method). Lower is better for FID and KID, higher for the four PRDC metrics. KID values and their subset standard deviations are multiplied by

10^{3}

. Bold marks the best reported value in each column, not statistical significance.

Figure 5: Uncurated ALD-posterior CelebA64 cases.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	Hidden dimension	Training steps	Learning rate
Heavy-Tailed Cross	128	15,000	$5\times 10^{-4}$
Swiss Roll	128	15,000	$10^{-3}$
MoG40	256	20,000	$5\times 10^{-4}$
Holtsmark Field	256	1,000	$5\times 10^{-4}$

Appendix

Table 8: Training settings for the main 2D benchmarks.

Figure 6: Prediction-output perturbation on Holtsmark (mean

\pm

sample standard deviation over nine runs). Lower is better for

W_{2}

and

\mathrm{MMD}^{2}

; SHS is ranked by distance to one. Both samplers reuse the same ALD checkpoints and differ only in the reverse transition.

Candidate	$\eta$	$p$	FID $\downarrow$	KID $\times 10^{3}\downarrow$	Prec. $\uparrow$	Rec. $\uparrow$	Dens. $\uparrow$	Cov. $\uparrow$
p100_e000	0	1.0	4.6452	1.2289	0.7302	0.6216	1.1331	0.9566
p100_e050	0.5	1.0	4.6430	1.3127	0.7384	0.6172	1.1388	0.9554
p100_e100	1	1.0	5.0132	1.6877	0.7418	0.6090	1.1601	0.9608
p150_e000	0	1.5	4.5631	1.0842	0.7338	0.6324	1.1390	0.9606
p150_e050	0.5	1.5	4.5266	1.1273	0.7452	0.6224	1.1823	0.9618
p150_e100	1	1.5	4.7560	1.2562	0.7612	0.6074	1.2142	0.9626

Appendix

Table 9: Validation-only ALD sampler sweep. Every candidate uses 5,000 generated images and 1,000 NFE; FID compares against all 19,867 validation images, KID uses 100 subsets of 1,000, and PRDC uses fixed 5,000/5,000 subsets. KID entries are multiplied by

10^{3}

. Bold identifies the frozen choice, not a winner on every metric. These values are not directly comparable with the formal train-reference scores in Table 7 .

Diffusion Sampling

Diffusion sampling studies how to turn a trained diffusion model into high-quality samples efficiently and reliably.

See all 264 papers

Slowly Annealed Langevin Dynamics: Theory and Applications to Training-Free Guided Generation
May 8, 2026Atsushi Nitanda, Dake Bu, Yueming Lyu +1Diffusion Model Guidance Langevin Dynamics
Agency for Science, Technology and Research (A⋆STAR) · Nanyang Technological University · City University of Hong Kong
A Stochastic Analysis Perspective on Diffusion Models
Oct 9, 2026Stanislas StrasmanScore-Based Generative Modeling Diffusion Sampling
SU, LPSM · Laboratoire de Probabilités, Statistique et Modélisation, Sorbonne Université
Generator-Guided Inverse Sampling for Lévy-Driven Generative Models
Aug 11, 2026Tianfu Qi, Jun Wang, Jun ZhangPosterior Sampling Generative Modeling
National Key Laboratory of Wireless Communications, University of Electronic Science and Technology of China, Chengdu 611731, China · Department of Electronic and Computer Engineering, Hong Kong University of Science and Technology, Hong Kong, China

May 8, 2026J/K move · Enter open · S save

Slowly Annealed Langevin Dynamics: Theory and Applications to Training-Free Guided Generation

Atsushi Nitanda, Dake Bu, Yueming Lyu, Tanya Veeravalli

Agency for Science, Technology and Research (A⋆STAR) · Nanyang Technological University · City University of Hong Kong

Diffusion Model Guidance Langevin Dynamics Score-Based Generative Modeling

Open paper AskPDF arXiv

We study Slowly Annealed Langevin Dynamics (SALD), a sampler for tracking a path of moving target distributions and approximating the terminal target through time slowdown. We establish non-asymptotic convergence guarantees via a KL differential inequality, showing that slowdown improves tracking through contraction of intermediate targets and the complexity of the path. Motivated by training-free guided generation with pretrained score-based generative models, we further introduce Velocity-Aware SALD (VA-SALD), which explicitly incorporates the underlying marginal distributions of the pretrained model and uses slowdown to correct the additional deviation induced by guidance. This yields a principled framework for training-free guided generation for diffusion-based and related generative model families, together with convergence guarantees that clarify the roles of intermediate functional inequalities and guidance bias. Code is available at https://github.com/anitan0925/sald.

Abstract

Diffusion Sampling

Explore similar work

Slowly Annealed Langevin Dynamics: Theory and Applications to Training-Free Guided Generation

A Stochastic Analysis Perspective on Diffusion Models

Generator-Guided Inverse Sampling for Lévy-Driven Generative Models