MGD: Moment Guided Diffusion for Maximum Entropy Generation

Etienne Lempereur; Nathanaël Cuvelle--Magar; Florentin Coeurdoux; Stéphane Mallat; Eric Vanden-Eijnden

stat.MLFeb 19, 2026

MGD: Moment Guided Diffusion for Maximum Entropy Generation

Authors: Etienne Lempereur, Nathanaël Cuvelle--Magar, Florentin Coeurdoux, Stéphane Mallat, Eric Vanden-Eijnden

Organizations: D´epartement d’informatique, ENS, Universit´e PSL, Paris, France · Capital Fund Management, Paris, France · Coll`ege de France, Paris, France · Flatiron Institute, New York, USA ·

Abstract

Generating samples from limited information is a fundamental problem across scientific domains. Classical maximum entropy methods provide principled uncertainty quantification from moment constraints but require sampling via MCMC or Langevin dynamics, which typically exhibit exponential slowdown in high dimensions. In contrast, generative models based on diffusion and flow matching efficiently transport noise to data but offer limited theoretical guarantees and can overfit when data is scarce. We introduce Moment Guided Diffusion (MGD), which combines elements of both approaches. Building on the stochastic interpolant framework, MGD samples maximum entropy distributions by solving a stochastic differential equation that guides moments toward prescribed values in finite time, thereby avoiding slow mixing in equilibrium-based methods. We formally obtain, in the large-volatility limit, convergence of MGD to the maximum entropy distribution and derive a tractable estimator of the resulting entropy computed directly from the dynamics. Applications to financial time series, turbulent flows, and cosmological fields using wavelet scattering moments yield estimates of negentropy for high-dimensional multiscale processes.

Figures & tables

Approach	Input	Max-ent guarantee	Moment control	Sampling
Maximum entropy (classical)	Moments $m$	✓	✓	Equilibrium (MCMC)
Diffusion models	Dataset $(x^{i})$	×	×	Non-equilibrium
Moment Guided Diffusion	Dataset $(x^{i})$	✓	✓	Non-equilibrium (guided)

Table 1: Comparison of sampling approaches for complex distributions.

Figure 1: Illustration of trajectories ( in blue or red) of

X_{t}

satisfying Equation ( 9 ) for an interpolant

I_{t}

defined with

\alpha_{t}=\pi t/2

between white noise

Z

and a bimodal unbalanced Gaussian mixture

X

, for

\sigma=1

. We display in gray in the background the density of

I_{t}

. When

t

goes to

0

, the modes progressively disappear. At early times

t

, particles evolve freely in space, but they become trapped in the modes when the density

p_{t}

becomes bimodal. Red particles are confined in the upper mode and blues in the lower one.

Figure 2: Convergence of MGD towards the maximum entropy bimodal distribution

p_{*}(x)=\mathcal{Z}^{-1}e^{-\tfrac{4}{5}(x^{4}-5x^{2}-x/2)}

for

X\sim p=p_{*}

. Left column: moment function

\phi(x)=(x,x^{2},x^{3},x^{4})

. Right column:

\phi(x)=(x^{2},\log p(x))

. (a,d) Log-density

\log p_{*}(x)

(dashed) and

\log p_{1}^{\sigma}(x)

for increasing

\sigma

(blue to red). (b,e) Maximum entropy

H(p_{*})

(red line), sampled entropy

H(p_{1}^{\sigma})

(blue dots), and lower bound

H_{*}^{\sigma}

from ( 37 ) (black dots) versus

\sigma^{2}

. (c,f) Entropy gaps

H(p_{*})-H(p_{1}^{\sigma})

(blue) and

H(p_{*})-H_{*}^{\sigma}

(black) versus

\sigma^{2}

; the dashed line shows

\sigma^{-2}

decay.

Figure 3: Convergence of MGD towards the Laplacian maximum entropy distribution

p_{*}(x)=\tfrac{1}{2}e^{-|x|}

for

X\sim p=p_{*}

. (a) Log-density

\log p_{*}(x)

(dashed) and

\log p_{1}^{\sigma}(x)

for increasing

\sigma

(blue to red). (b, top) Maximum entropy

H(p_{*})

(red line), sampled entropy

H(p_{1}^{\sigma})

(blue dots), and lower bound

H_{*}^{\sigma}

from ( 37 ) (black dots) versus

\sigma^{2}

. (b, bottom) Entropy gaps

H(p_{*})-H(p_{1}^{\sigma})

(blue) and

H(p_{*})-H_{*}^{\sigma}

(black) versus

\sigma^{2}

; the dashed line shows

\sigma^{-2}

decay.

Figure 4: (a) Log-density

\log p_{*}(x)

for

p_{*}(x)=\mathcal{Z}_{\beta}^{-1}e^{-\beta(x^{4}-5x^{2}-x/2)}

with increasing

\beta

(blue to red). The two modes are separated by a barrier of height proportional to

\beta

. (b) Number of discretization steps

n_{\rm steps}

required to reach a fixed Kullback–Leibler divergence from

p_{*}

, for MALA (red) and MGD (green), as a function of

\beta

. For MALA,

n_{\rm steps}

grows exponentially with

\beta

; for MGD, it remains nearly constant.

Panel of Figure 5 — Figure 5: (a): One-dimensional Morlet wavelet $\psi$ . The wavelet is a complex function whose real and imaginary parts are respectively in blue and red. (b): real (left) and imaginary (right) parts of a two-dimensional Morlet wavelet.

Figure 6: (a) S&P 500 daily log-returns (

d=6060

) from January 2000 to February 2024. (b) Sample generated by MGD with

\sigma^{2}=5.5

, using

r=217

empirical wavelet scattering moments ( 49 ) computed from (a). Intermittency is reproduced.

Panel of Figure 7 — Figure 7: (a) Negentropy estimate $\Delta H_{*}^{\sigma}$ from ( 43 ) versus $\sigma^{2}$ for S&P 500 log-returns. (b) Convergence of $H_{*}^{\sigma}$ , measured by $d^{-1}(H_{*}^{\sigma_{\max}}-H_{*}^{\sigma})$ with $\sigma_{\max}^{2}=5.5$ , versus $\sigma^{2}$ . The plain line shows $\sigma^{-2}$ decay.

Figure 8: Histograms of rolling volatility

\mathrm{vol}(u)

computed over

w=5

days. Dashed: S&P log-returns

X

. Red to green: MGD samples

X_{1}^{\sigma}

for increasing

\sigma^{2}

. Black: Gaussian process with the same covariance. At

\sigma^{2}=5.5

, the histogram of

X_{1}^{\sigma}

matches the S&P and differs from the Gaussian.

Panel of Figure 9 — Figure 9: (a) Cosmic web field: 2D slice from a 3D dark matter simulation. (b) Turbulent vorticity field from 2D incompressible Navier–Stokes. Both images are $128\times 128$ pixels. (c,d) MGD samples with wavelet scattering moments at $\sigma^{2}=5.5$ .

Panel of Figure 10 — Figure 10: (a) Negentropy estimate $\Delta H_{*}^{\sigma}$ from ( 43 ) versus $\sigma^{2}$ for cosmic web (blue) and turbulence (red). (b) Convergence of $H_{*}^{\sigma}$ , measured by $d^{-1}(H_{*}^{\sigma_{\max}}-H_{*}^{\sigma})$ with $\sigma_{\max}^{2}=5.5$ , versus $\sigma^{2}$ . Plain lines show $\sigma^{-2}$ decay.

Dataset	Estimated Normalized Negentropy
Laplacian	0.07
S&P 500	0.05
Cosmic Web	0.07
2D Turbulence	0.34

Table 2: Negentropy estimate

\Delta H_{*}^{\sigma}

\sigma=\sigma_{\max}

Figure 11: Histograms of finest-scale wavelet coefficients

\mathrm{Re}(X_{1}^{\lambda})

for

\lambda=\xi

(

j=0

\ell=0

) of cosmic web samples from the scattering MGD model, with

\sigma^{2}\in\{0,0.1,5.5\}

. Dashed black: original data. All histograms are computed over

500

samples. Larger

\sigma^{2}

yields better tail reproduction; small

\sigma^{2}

produce more regular samples which have too many small wavelet coefficients, as in Figure 2 (a).

Panel of Figure 12 — Figure 12: MGD with large model error on CelebA faces ( $64\times 64$ ). (a) Original sample. (b) MGD sample with wavelet scattering moments at $\sigma^{2}=1$ . (c) Negentropy estimate $\Delta H_{*}^{\sigma}$ from ( 43 ) versus $\sigma^{2}$ . (d) Convergence of $H_{*}^{\sigma}$ , measured by $d^{-1}(H_{*}^{\sigma_{\max}}-H_{*}^{\sigma})$ with $\sigma_{\max}^{2}=8.5$ ; the line shows $\sigma^{-2}$ decay.

Generative Modeling

Generative modeling learns data distributions to create new samples, from images and text to molecules, simulations, and structured records.

See all 487 papers

Training-Free Generative Sampling via Moment-Matched Score Smoothing
May 14, 2026Zhenyu Yao, Daniel PaulinLangevin Dynamics Diffusion Sampling
College of Computing and Data Science Nanyang Technological University
When Diffusion Model Can Ignore Dimension: An Entropy-Based Theory
May 8, 2026Ahmad Aghapour, Erhan BayraktarDiffusion Sampling
Department of Mathematics, University of Michigan, Ann Arbor, 48109
Adaptive Moments are Surprisingly Effective for Plug-and-Play Diffusion Sampling
Mar 17, 2026Christian Belardi, Justin Lovelace, Kilian Q. Weinberger +1Diffusion Sampling Diffusion Model Guidance
Cornell University

May 14, 2026J/K move · Enter open · S save

Training-Free Generative Sampling via Moment-Matched Score Smoothing

Zhenyu Yao, Daniel Paulin

College of Computing and Data Science Nanyang Technological University

Langevin Dynamics Diffusion Sampling

Open paper AskPDF arXiv

Diffusion models generate samples by denoising along the score of a perturbed target distribution. In practice, one trains a neural diffusion model, which is computationally expensive. Recent work suggests that score matching implicitly smooths the empirical score, and that this smoothing bias promotes generalization by capturing low-dimensional data geometry. We propose moment-matched score-smoothed overdamped Langevin dynamics (MM-SOLD), a training-free interacting particle sampler that enforces the target moments throughout the sampling trajectory. We prove that, in the large-particle limit, the empirical particle density converges to a deterministic limit whose one-particle stationary marginal is a Gibbs--Boltzmann density obtained by exponentially tilting a naive score-smoothed diffusion target. The mean and covariance of this distribution agree with the empirical moments of the training data. Experiments on 2D distributions and latent-space image generation show that MM-SOLD enables fast, robust, training-free sampling on CPUs, with sample fidelity and diversity competitive with neural diffusion baselines.

Abstract

Generative Modeling

Explore similar work

Training-Free Generative Sampling via Moment-Matched Score Smoothing

When Diffusion Model Can Ignore Dimension: An Entropy-Based Theory

Adaptive Moments are Surprisingly Effective for Plug-and-Play Diffusion Sampling