Generative Drifting is Secretly Score Matching: a Spectral and Variational Perspective

Erkan Turan; Nicolas Dufour; Maks Ovsjanikov

cs.LGMar 10, 2026

Generative Drifting is Secretly Score Matching: a Spectral and Variational Perspective

Authors: Erkan Turan, Nicolas Dufour, Maks Ovsjanikov

Organizations: LIX, Ecole Polytechnique, IP Paris · Kyutai

Abstract

Generative Modeling via Drifting~\citep{deng2026drifting} has recently achieved state-of-the-art one-step image generation through a kernel-based drift operator, yet its success is largely empirical and its theoretical foundations remain poorly understood. We observe that under a Gaussian kernel, the drift operator is exactly a score difference on smoothed distributions. This answers three questions left open in the original work: (1) whether a vanishing drift guarantees equality of distributions ( $V_{p,q}=0\Rightarrow p=q$ ), (2) how to choose between kernels, and (3) why the stop-gradient operator is indispensable for stable training. Our observations position drifting within the score-matching family. By linearizing the McKean-Vlasov dynamics and probing them in Fourier space, we reveal frequency-dependent convergence timescales comparable to Landau damping in plasma kinetic theory: the Gaussian kernel suffers an exponential high-frequency bottleneck, potentially explaining the empirical preference for the Laplacian kernel. This suggests a fix: an exponential bandwidth annealing schedule $σ(t)=σ_0 e^{-rt}$ that reduces convergence time from $\exp(O(K_{\max}^2))$ to $O(\log K_{\max})$ . Finally, by formalizing drifting as a Wasserstein gradient flow of the smoothed KL divergence, we prove that the stop-gradient operator is not a heuristic but is derived from the frozen-field discretization mandated by the Jordan-Kinderlehrer-Otto (JKO) scheme, and removing it severs training from any gradient-flow guarantee. This variational perspective further provides a general template for constructing novel drift operators, which we demonstrate with a Sinkhorn divergence drift. We validate our analysis on toy datasets and scale it up to ImageNet.

Figures & tables

Figure 1: Numerical confirmation of Theorem 4.1 on a 4-mode Gaussian mixture. (a) Empirical kernel mean-shift drift ( $N=50$ k samples). (b) Analytical score-difference form $\sigma^{2}\nabla\log(p_{\sigma}/q_{\sigma})$ . (c) Overlay: the two fields are visually indistinguishable. (d) Pointwise $\ell_{2}$ error heatmap (mean $4.9\times 10^{-3}$ ). Details are provided in Section F.1 .

Figure 2: Spectral validation. (a) Convergence time

T(k)

vs. frequency: lines are analytical predictions from Theorem 5.2 , markers are measured decay times. The fixed-bandwidth Gaussian exhibits exponential slowdown (Landau damping); the Laplacian kernel yields polynomial scaling; the annealed Gaussian eliminates the bottleneck entirely. (b) Annealing schedules

\sigma(t)

. (c) Total spectral error under different schedules. Details in F.2 .

Figure 3: Loss landscapes with and without stop-gradient, projected onto the top two principal gradient-variation directions. (a,b) Training loss $\|V\|^{2}$ : without SG the minimum is ${\sim}100\times$ deeper. (c,d) Sliced Wasserstein distance: with SG the loss minimum coincides with low distributional error; without SG the deep minimum corresponds to poor sample quality. Red star: trained solution. Details in F.3 .

Figure 4: Drift norm vs. distributional distance during training on synthetic 2D targets. (a) Mean drift norm $\|V\|$ . (b) Sliced Wasserstein distance. (c) Log–log scatter across training steps and seeds. With stop-gradient (solid), the two quantities are strongly correlated ( $r>0.95$ ) and jointly decay. Without stop-gradient (dashed), the drift norm collapses to ${\sim}10^{-8}$ while the Wasserstein distance remains at $0.389$ —a direct demonstration of drift collapse (Theorem 5.6 (ii) ). Details in F.3 .

Figure 5: Sinkhorn-derived drift (bottom row) vs. Laplacian-kernel drift (top row) on the checkerboard distribution. Training snapshots with drift vectors overlaid; rightmost panels show sliced Wasserstein distance over training. Both converge successfully (final SW $1.42\times 10^{-2}$ and $2.07\times 10^{-2}$ respectively), demonstrating that the gradient-flow template of § 6.3 yields practical operators beyond the original kernel family. Details are provided in F.4 .

Drift operator	Bandwidth	NFE	FID $\downarrow$	IS $\uparrow$
Laplacian ( Deng et al., 2026 )	$\tau$ fixed	1	8.55	148.0
Gaussian (ours)	$\sigma$ fixed	1	8.69	157.0
Gaussian (ours)	$\sigma(t)=\sigma_{0}e^{-rt}$	1	8.36	154.2
Sinkhorn (ours)	$\varepsilon$ fixed	1	8.81	135.48

Table 1: Ablation on ImageNet

256{\times}256

class-conditional generation. All variants share the same B/2 architecture, latent-MAE SSL representation, and training schedule of Deng et al. (2026) ; only the drift operator differs. We report FID (

\downarrow

) and IS (

\uparrow

) under one-step (NFE

=

1) generation. Best in bold , second-best underlined .

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Optimization
Optimizer	AdamW ( $\beta_{1}{=}0.9$ , $\beta_{2}{=}0.95$ , $\epsilon{=}10^{-8}$ )
Weight decay	$0.01$
Peak learning rate	$2\times 10^{-4}$
LR schedule	linear warmup ( $5{,}000$ steps), then constant
Gradient clipping	$\\|g\\|_{2}\leq 2.0$
Total steps	$30{,}000$

Appendix

Table 2: Training hyperparameters for the image-generation ablations ( latent_ablation configuration).

Panel of Figure 6 — Figure 6: Uncurated samples from the exponential baseline ( Deng et al., 2026 ) at $30{,}000$ steps, CFG $w{=}2.2$ . Each cell is an independently generated $256{\times}256$ image; samples from $10$ ImageNet classes (macaw, hummingbird, flamingo, golden retriever, tiger, monarch butterfly, African elephant, sports car, cheeseburger, volcano) are shuffled across the grid.

Panel of Figure 7 — Figure 7: Uncurated samples from the Gaussian-drift model at $30{,}000$ steps, CFG $w{=}1.833$ . Each cell is an independently generated $256{\times}256$ image; samples from $10$ ImageNet classes (macaw, hummingbird, flamingo, golden retriever, tiger, monarch butterfly, African elephant, sports car, cheeseburger, volcano) are shuffled across the grid.

Panel of Figure 8 — Figure 8: Uncurated samples from the Gaussian-drift model with annealing at $30{,}000$ steps, CFG $w{=}1.944$ . Each cell is an independently generated $256{\times}256$ image; samples from $10$ ImageNet classes (macaw, hummingbird, flamingo, golden retriever, tiger, monarch butterfly, African elephant, sports car, cheeseburger, volcano) are shuffled across the grid.

Panel of Figure 9 — Figure 9: Uncurated samples from the Sinkhorn-drift model at $30{,}000$ steps, CFG $w{=}1.167$ . Each cell is an independently generated $256{\times}256$ image; samples from $10$ ImageNet classes (macaw, hummingbird, flamingo, golden retriever, tiger, monarch butterfly, African elephant, sports car, cheeseburger, volcano) are shuffled across the grid.

Shape		Wall-clock per step (ms)
$B$	$d$	softmax	Gaussian	annealed (3R)	Sinkhorn
$64$	$256$	$\mathbf{0.761{\scriptstyle\,\pm\,0.020}}$	$0.775{\scriptstyle\,\pm\,0.012}$	$0.780{\scriptstyle\,\pm\,0.014}$	$1.004{\scriptstyle\,\pm\,0.105}$
$256$	$256$	$1.214{\scriptstyle\,\pm\,0.004}$	$1.265{\scriptstyle\,\pm\,0.005}$	$1.261{\scriptstyle\,\pm\,0.004}$	$\mathbf{1.042{\scriptstyle\,\pm\,0.011}}$
$1024$	$256$	$4.211{\scriptstyle\,\pm\,0.005}$	$4.343{\scriptstyle\,\pm\,0.005}$	$4.341{\scriptstyle\,\pm\,0.005}$	$\mathbf{2.552{\scriptstyle\,\pm\,0.011}}$
$64$	$768$	$\mathbf{0.899{\scriptstyle\,\pm\,0.005}}$	$0.930{\scriptstyle\,\pm\,0.005}$	$0.935{\scriptstyle\,\pm\,0.006}$	$1.040{\scriptstyle\,\pm\,0.088}$

Appendix

Table 3: Per-step drift wall-clock on a single H100, in milliseconds (mean

\pm

std over

100

iterations after

10

warm-ups). The “softmax” column is the Laplacian-kernel softmax baseline; “annealed (3R)” is the Gaussian drift with the

3R

exponential bandwidth schedule of Section 6.2 . Bold marks the fastest variant in each row.

Drifting Models

Drifting models learn transport fields that move generated samples toward a target distribution, often enabling one-step generation.

See all 53 papers

Sinkhorn-Drifting Generative Models
Mar 12, 2026Ping He, Om Khangaonkar, Hamed Pirsiavash +2Sinkhorn Divergence Drifting Models
Department of Computer Science, Vanderbilt University · Department of Computer Science, University of California, Davis · Department of Computer Science, Purdue University
A Unified View of Score-Based and Drifting Models
Mar 8, 2026Chieh-Hsin Lai, Bac Nguyen, Naoki Murata +5Generative Modeling Drifting Models
Sony AI · Sony Group Corporation · Stanford University +1
Kernel-Gradient Drifting Models
May 11, 2026Maria Esteban-Casadevall, Jorge Carrasco-Pollo, Max Welling +3Synthetic Data Generation One-Step Generative Modeling
AMLab · University of Amsterdam · CuspAI +1

Mar 12, 2026J/K move · Enter open · S save

Sinkhorn-Drifting Generative Models

Ping He, Om Khangaonkar, Hamed Pirsiavash, Yikun Bai+1

Department of Computer Science, Vanderbilt University · Department of Computer Science, University of California, Davis · Department of Computer Science, Purdue University

Sinkhorn Divergence Drifting Models Flow-Based Generative Modeling Optimal Transport One-Step Generative Modeling

Open paper AskPDF arXiv

We establish a theoretical link between the recently proposed "drifting" generative dynamics and gradient flows induced by the Sinkhorn divergence. In a particle discretization, the drift field admits a cross-minus-self decomposition: an attractive term toward the target distribution and a repulsive/self-correction term toward the current model, both expressed via one-sided normalized Gibbs kernels. We show that Sinkhorn divergence yields an analogous cross-minus-self structure, but with each term defined by entropic optimal-transport couplings obtained through two-sided Sinkhorn scaling (i.e., enforcing both marginals). This provides a precise sense in which drifting acts as a surrogate for a Sinkhorn-divergence gradient flow, interpolating between one-sided normalization and full two-sided Sinkhorn scaling. Crucially, this connection resolves an identifiability gap in prior drifting formulations: leveraging the definiteness of the Sinkhorn divergence, we show that zero drift (equilibrium of the dynamics) implies that the model and target measures match. Experiments show that Sinkhorn drifting reduces sensitivity to kernel temperature and improves one-step generative quality, trading off additional training time for a more stable optimization, without altering the inference procedure used by drift methods. These theoretical gains translate to strong low-temperature improvements in practice: on FFHQ-ALAE at the lowest temperature setting we evaluate, Sinkhorn drifting reduces mean FID from 187.7 to 37.1 and mean latent EMD from 453.3 to 144.4, while on MNIST it preserves full class coverage across the temperature sweep. Project page: https://mint-vu.github.io/SinkhornDrifting/

Abstract

Drifting Models

Explore similar work

Sinkhorn-Drifting Generative Models

A Unified View of Score-Based and Drifting Models

Kernel-Gradient Drifting Models