cs.LGNov 17, 2026

Error accumulation in reverse diffusion with an imperfect score: beyond path-space analyses

Authors: George Giapitzakis, Ankur Moitra, Andrej Risteski, Dhruv Rohatgi

Organizations: CMU · MIT · UC Berkeley

Abstract

Diffusion modeling is a popular method of sampling from a complex, high-dimensional distribution pp using learned approximations of its (noised) score functions. Generating a single sample requires simulating the reverse diffusion process, which requires many queries to the learned scores. Thus, a priori, the learning errors can easily compound. Indeed, in existing theoretical analyses, the sampling error (as measured in total variation distance to pp) accumulates polynomially with the horizon of the stochastic process. This is a consequence of bounding the error of the output law by the error of the law of the entire reverse diffusion path. In this paper, we demonstrate that by circumventing the "path-space" analysis, the error accumulation of the diffusion sampler can be mitigated when pp satisfies a Poincaré inequality. Moreover, we achieve this result as a special case of a broader framework that applies to any distribution pp, but relaxes the error metric from total variation distance to a weaker metric capturing all sample statistics that do not "witness" the multimodality of pp. We then zoom out from analyzing the diffusion sampler to the information-theoretic question: given a learned score with some error bound, how accurately can we identify pp? We show that, for any given bound on the Poincaré constant of pp, the diffusion sampler already nearly achieves optimal total variation error.

Explore similar work

Jun 4, 2026

Diffusion Models Observe Only Gradients: A Geometric Perspective on Score Matching Errors

Score-based diffusion models are typically trained by minimizing the L2L^2 score matching error, and standard theoretical analyses rely on this quantity to bound the sampling discrepancy between the learned and target distributions. Recent work has shown that the score admits an orthogonal decomposition into gradient and solenoidal components in which only the gradient part matters for exact sampling. We apply this decomposition to the score error of trained models and study its quantitative consequences. Since only the gradient component of the error enters the marginal Fokker-Planck dynamics, a model can incur arbitrarily large L2L^2 score error while perfectly matching the target. Starting from this observation, we present three results. First, we derive an upper bound on the Kullback-Leibler divergence that depends only on the observable gradient component of the error. It tightens the standard Girsanov bound for generic score networks and identifies that bound's looseness as the cost of working with path-space rather than marginal-space dynamics. Second, we prove that no non-trivial function of the L2L^2 score error can uniformly lower bound any divergence between the learned and target distributions. Third, we give an estimator of the gradient component, computable from denoising score matching residuals via a dual Sobolev identity, which empirically correlates substantially better with sample quality than the full L2L^2 error.
Apr 12, 2026

Query Lower Bounds for Diffusion Sampling

Diffusion models generate samples by iteratively querying learned score estimates. A rapidly growing literature focuses on accelerating sampling by minimizing the number of score evaluations, yet the information-theoretic limits of such acceleration remain unclear. In this work, we establish the first score query lower bounds for diffusion sampling. We prove that for dd-dimensional distributions, given access to score estimates with polynomial accuracy ε=d−O(1)\varepsilon=d^{-O(1)} (in any LpL^p sense), any sampling algorithm requires Ω~(d)\widetildeΩ(\sqrt{d}) adaptive score queries. In particular, our proof shows that, within any polynomial total-query budget, successful sampling requires searching over Ω~(d)\widetildeΩ(\sqrt{d}) distinct noise levels, providing a formal explanation for why multiscale noise schedules are necessary in practice.
Jul 9, 2026

Score Accuracy Along the Forward Diffusion Does Not Certify Numerical Stability in Diffusion Sampling

Score matching controls average error under the forward marginals, but a discretized reverse-time sampler evaluates the learned score along its own trajectory. We show that small forward-marginal error does not guarantee numerical stability. We construct a single smooth score field with arbitrarily small forward-marginal L2L^2 error. The learned reverse-time process is nonexplosive, has moments of every order, and can be arbitrarily close to the exact reverse-time process in path-space total variation. Yet its Euler--Maruyama discretizations converge in probability while every positive moment diverges. Thus weak convergence can hold even though every Wasserstein distance WpW_p, p≥1p\ge1, diverges. The same failure can occur within one fixed finite neural architecture. We construct a family of bounded, globally Lipschitz denoisers for which both the forward-marginal error and the path-space total variation distance tend to zero, while their Euler--Maruyama endpoints diverge in every WpW_p. For compactly supported data, we also give a simple positive result. Projecting the learned denoiser onto a known bounded closed convex set containing the support preserves pointwise accuracy, gives grid-uniform moment bounds, and yields Wasserstein convergence under mild local regularity. Experiments with a small fixed DiT-style network show large growth along rare numerical trajectories and its suppression by denoiser projection, while overall trajectory errors remain small.