Drifting Fields are not Conservative

Leonard T. Franz; Sebastian Hoffmann; Tim Weiland; Bernhard Schölkopf; Georg Martius

cs.LGApr 7, 2026

Drifting Fields are not Conservative

Authors: Leonard T. Franz, Sebastian Hoffmann, Tim Weiland, Bernhard Schölkopf, Georg Martius

Organizations: Eberhard Karls Universität Tübingen · Max Planck Institute for Biogeochemistry · Max Planck Institute for Intelligent Systems

Abstract

Drifting models have recently gained attention for generating high-quality samples in a single forward pass. During training, they learn a push-forward map by following a vector-valued field, the drift field. We ask whether this procedure is equivalent to optimizing a scalar loss and find that, in general, it is not: drift fields are not conservative and cannot be written as the gradient of any scalar potential. We identify the position-dependent normalization as the source of non-conservatism, with the Gaussian kernel as the unique radial exception. Guided by this, we introduce the sharp kernel $k^\#$ and a sharp-normalized drift field that is conservative for general radial kernels. The resulting vector field is the gradient of a scalar potential that can be optimized directly using stochastic gradient descent. Moreover, the field has the form of a score difference of kernel density estimates, and gives exact equilibrium identifiability. Thus, sharp normalization closes the gap to related literature, such as Wasserstein gradient-flows and denoising score matching, also for non-Gaussian kernels. Empirically, sharp normalization preserves the performance of the original drifting objective, suggesting that the non-conservative flexibility is not required for high-quality generation.

Figures & tables

Figure 1: Sharp normalization. Drift fields are generally not conservative, as seen by the non-vanishing curl field for the Laplacian and rational quadratic kernel (left panels). We propose a new normalization scheme via a sharp kernel density estimate (KDE) that restores the field’s conservatism (right). This allows us to reformulate the drift field as the gradient of a scalar potential, the log-KDE potential, yielding a simplified and interpretable objective.

	Gaussian	Laplacian	Matérn ( $\nu=p+\tfrac{1}{2}$ )	Rational quadratic
Kernel $k(\mathbf{x},\mathbf{y})$	$\exp(-r^{2}/2\sigma^{2})$	$\exp(-r/\sigma)$	$k_{\nu,\sigma}(\mathbf{x},\mathbf{y}):=Q_{p}(cr)\,e^{-cr}$	$(1+r^{2}/\sigma^{2})^{-2}$
Sharp $k^{\#}(\mathbf{x},\mathbf{y})$	$\sigma^{2}\,k(\mathbf{x},\mathbf{y})$	$\sigma(r+\sigma)\,k(\mathbf{x},\mathbf{y})$	$\sigma^{2}\,k_{\nu+1,\,\sigma_{\#}}(\mathbf{x},\mathbf{y})$	$\frac{\sigma^{2}}{2}\,k(\mathbf{x},\mathbf{y})^{1/2}$
Flat $k^{\flat}(\mathbf{x},\mathbf{y})$	$\sigma^{-2}\,k(\mathbf{x},\mathbf{y})$	$k(\mathbf{x},\mathbf{y})/(\sigma r)$	$\frac{\nu}{(\nu-1)\sigma^{2}}\,k_{\nu-1,\,\sigma_{\flat}}(\mathbf{x},\mathbf{y})$ ( $\nu\geq\tfrac{3}{2}$ )	$\frac{4}{\sigma^{2}}\,k(\mathbf{x},\mathbf{y})^{3/2}$

Table 1: Sharp and flat kernels for common radial kernels. Here,

r:=\|\mathbf{x}-\mathbf{y}\|

. The Laplacian is the

\nu=\tfrac{1}{2}

boundary case of the half-integer Matérn family; Matérn-specific notation (

c

\sigma_{\#}

\sigma_{\flat}

Q_{p}

) and derivations are given in Section B.4 .

Figure 2: Normalized radial profiles of flat and sharp kernels . For the Laplacian, Matérn-

7/2

, and rational quadratic kernels, the sharp and flat profiles have distinct shapes, while for the Gaussian all agree up to scale.

Figure 3: Magnitudes of positive drift fields. Given a fixed target distribution (gray shading), we show the magnitude of the original drift field (black), sharp-normalized drift field (dashed), and the MMD gradient (dash-dotted) as function of

x

. The MMD gradient decays fast in the tails while the drift field grows unboundedly. The sharp-normalized field lies between these extremes, with Matérn-

7/2

showing intermediate behavior between the Laplacian and Gaussian/RQ panels.

Figure 4: Kernel width ablation. The Fréchet Inception Distance (FID-50k, lower is better) as a function of kernel width for MNIST (left), Fashion-MNIST (center), and ImageNet (right). Solid lines indicate median, shaded areas indicate minimum and maximum over three seeds (ImageNet: one seed). The black dashed line indicates the best FID (8.67) on ImageNet reported by Deng et al. using a single scale.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

	MNIST	Fashion-MNIST	ImageNet (Ablation)
Architecture
model	DiT-S/2		DiT-B/2
input size	28 $\times$ 28 $\times$ 1		32 $\times$ 32 $\times$ 4
patch size	2 $\times$ 2		2 $\times$ 2
feature dim	384		768
mlp dim	1536		3072

Appendix

Table 3: Experiment configurations used in this work.

Figure 5: Uncurated images generated on ImageNet. Results after training for 30,000 steps with the $\text{log-KDE-Laplace}^{\#}$ objective with a kernel width of $0.3$ (FID-50k $=9.35$ ).

Panel of Figure 6 — Figure 6: Uncurated generated images on MNIST and Fashion-MNIST. Real and generated samples for MNIST (top) and Fashion-MNIST (bottom) across different drift variants and log-KDE. Images were generated using the kernel width yielding the lowest FID score for each variant (see Figs. 4 and 2 ).

Drifting Models

Drifting models learn transport fields that move generated samples toward a target distribution, often enabling one-step generation.

See all 53 papers

Sinkhorn-Drifting Generative Models
Mar 12, 2026Ping He, Om Khangaonkar, Hamed Pirsiavash +2Sinkhorn Divergence Drifting Models
Department of Computer Science, Vanderbilt University · Department of Computer Science, University of California, Davis · Department of Computer Science, Purdue University
Finite-Particle Convergence Rates for Conservative and Non-Conservative Drifting Models
May 21, 2026Krishnakumar BalasubramanianOne-Step Generative Modeling Drifting Models
Department of Statistics, University of California, Davis
Generative Drifting is Secretly Score Matching: a Spectral and Variational Perspective
Mar 10, 2026Erkan Turan, Nicolas Dufour, Maks OvsjanikovDrifting Models Score Matching
LIX, Ecole Polytechnique, IP Paris · Kyutai

Mar 12, 2026J/K move · Enter open · S save

Sinkhorn-Drifting Generative Models

Ping He, Om Khangaonkar, Hamed Pirsiavash, Yikun Bai+1

Department of Computer Science, Vanderbilt University · Department of Computer Science, University of California, Davis · Department of Computer Science, Purdue University

Sinkhorn Divergence Drifting Models Flow-Based Generative Modeling Optimal Transport One-Step Generative Modeling

Open paper AskPDF arXiv

We establish a theoretical link between the recently proposed "drifting" generative dynamics and gradient flows induced by the Sinkhorn divergence. In a particle discretization, the drift field admits a cross-minus-self decomposition: an attractive term toward the target distribution and a repulsive/self-correction term toward the current model, both expressed via one-sided normalized Gibbs kernels. We show that Sinkhorn divergence yields an analogous cross-minus-self structure, but with each term defined by entropic optimal-transport couplings obtained through two-sided Sinkhorn scaling (i.e., enforcing both marginals). This provides a precise sense in which drifting acts as a surrogate for a Sinkhorn-divergence gradient flow, interpolating between one-sided normalization and full two-sided Sinkhorn scaling. Crucially, this connection resolves an identifiability gap in prior drifting formulations: leveraging the definiteness of the Sinkhorn divergence, we show that zero drift (equilibrium of the dynamics) implies that the model and target measures match. Experiments show that Sinkhorn drifting reduces sensitivity to kernel temperature and improves one-step generative quality, trading off additional training time for a more stable optimization, without altering the inference procedure used by drift methods. These theoretical gains translate to strong low-temperature improvements in practice: on FFHQ-ALAE at the lowest temperature setting we evaluate, Sinkhorn drifting reduces mean FID from 187.7 to 37.1 and mean latent EMD from 453.3 to 144.4, while on MNIST it preserves full class coverage across the temperature sweep. Project page: https://mint-vu.github.io/SinkhornDrifting/

Abstract

Drifting Models

Explore similar work

Sinkhorn-Drifting Generative Models

Finite-Particle Convergence Rates for Conservative and Non-Conservative Drifting Models

Generative Drifting is Secretly Score Matching: a Spectral and Variational Perspective