Overview

Diffusion model sampling turns a trained model’s noisy predictions into data by approximating a reverse-time stochastic process or probability-flow ODE. Common approaches include DDIM and higher-order ODE solvers, adaptive timestep schedules, covariance-aware updates, and parallel or adaptive unmasking for discrete text models. Guidance, sequential Monte Carlo, and constrained or reward-aware sampling steer generations toward desired properties, while convergence analyses and image, text, molecular, and physical benchmarks assess accuracy and cost. A central tension is reducing steps and computation without amplifying score-model error, losing diversity, or sacrificing fidelity to constraints and rewards.

Latest papers

92 papers12 papers in the last four weeks, down 20% on the four weeks before. 0.1% of all new papers.

Jun 16, 2025

Fast Convergence for High-Order ODE Solvers in Diffusion Probabilistic Models

Diffusion probabilistic models generate samples by learning to reverse a noise-injection process that transforms data into noise. A key development is the reformulation of the reverse sampling process as a deterministic probability flow ordinary differential equation (ODE), which allows for efficient sampling using high-order numerical solvers. Unlike traditional time integrator analysis, the accuracy of this sampling procedure depends not only on numerical integration errors but also on the approximation quality and regularity of the learned score function, as well as their interaction. In this work, we present a rigorous convergence analysis of deterministic samplers derived from probability flow ODEs for general forward processes with arbitrary variance schedules. Specifically, we develop and analyze pp-th order (exponential) Runge-Kutta schemes, under the practical assumption that the first and second derivatives of the learned score function are bounded. We prove that the total variation distance between the generated and target distributions can be bounded as \begin{align*} O\bigl(d^{\frac{7}{4}}\varepsilon_{\text{score}}^{\frac{1}{2}} +d(dH_{\max})^p\bigr), \end{align*} where εscore2\varepsilon^2_{\text{score}} denotes the L2L^2 error in the score function approximation, dd is the data dimension, and Hmax⁡H_{\max} represents the maximum solver step size. Numerical experiments on benchmark datasets further confirm that the derivatives of the learned score function are bounded in practice.
Feb 12, 2025

Disentangling Total-Variance and Signal-to-Noise-Ratio Improves Diffusion Models

Diffusion models have been widely adopted as a powerful generative framework across numerous scientific domains, yet their inherently iterative sampling process remains a significant computational bottleneck. While this is typically mitigated by reducing the number of reverse diffusion steps, sample quality becomes highly sensitive to the chosen noise schedule, i.e. the manner in which noise is introduced and the signal is reduced at each step. Although prior work has improved on the original variance-preserving and variance-exploding schedules, these approaches passively adjust the total variance, without direct independent control over it. In this work, we propose a novel total-variance/signal-to-noise-ratio disentangled (TV/SNR) framework, where TV and SNR can be controlled independently. Our approach reveals that schedules with exponentially exploding TV can be consistently improved by adopting a constant TV schedule while preserving the underlying SNR schedule. Furthermore, we demonstrate that generalizing the SNR schedule of optimal transport flow matching significantly enhances generation efficiency. Crucially, these gains remain consistent across various reverse diffusion solvers and reduce the search space to two independent and interpretable parameters, enabling fast and robust sampling for complex scientific applications, such as molecular structure generation.