The Geometry of Noise: Why Diffusion Models Don't Need Noise Conditioning

Mojtaba Sahraee-Ardakan; Mauricio Delbracio; Peyman Milanfar

cs.LGFeb 20, 2026

The Geometry of Noise: Why Diffusion Models Don't Need Noise Conditioning

Authors: Mojtaba Sahraee-Ardakan, Mauricio Delbracio, Peyman Milanfar

Organizations: Google

Abstract

Autonomous (noise-agnostic) generative models, such as Equilibrium Matching and blind diffusion, challenge the standard paradigm by learning a single, time-invariant vector field that operates without explicit noise-level conditioning. While recent work suggests that high-dimensional concentration allows these models to implicitly estimate noise levels from corrupted observations, a fundamental paradox remains: what is the underlying landscape being optimized when the noise level is treated as a random variable, and how can a bounded, noise-agnostic network remain stable near the data manifold where gradients typically diverge? We resolve this paradox by formalizing Marginal Energy, $E_{\text{marg}}(\mathbf{u}) = -\log p(\mathbf{u})$ , where $p(\mathbf{u}) = \int p(\mathbf{u}|t)p(t)dt$ is the marginal density of the noisy data integrated over a prior distribution of unknown noise levels. We prove that generation using autonomous models is not merely blind denoising, but a specific form of Riemannian gradient flow on this Marginal Energy. Through a novel relative energy decomposition, we demonstrate that while the raw Marginal Energy landscape possesses a $1/t^p$ singularity normal to the data manifold, the learned time-invariant field implicitly incorporates a local conformal metric that perfectly counteracts the geometric singularity, converting an infinitely deep potential well into a stable attractor. We also establish the structural stability conditions for sampling with autonomous models. We identify a ``Jensen Gap'' in noise-prediction parameterizations that acts as a high-gain amplifier for estimation errors, explaining the catastrophic failure observed in deterministic blind models. Conversely, we prove that velocity-based parameterizations are inherently stable because they satisfy a bounded-gain condition that absorbs posterior uncertainty into a smooth geometric drift.

Figures & tables

Model	$a(t)$	$b(t)$	$c(t)$	$d(t)$	Autonomous field $f^{}(\mathbf{u})=\mathbb{E}_{t\|\mathbf{u}}\left[f^{}_{t}(\mathbf{u})\right]$
DDPM	$\sqrt{\bar{\alpha}_{t}}$	$\sqrt{1-\bar{\alpha}_{t}}$	0	1	$\mathbb{E}_{t\|\mathbf{u}}\left[\frac{\mathbf{u}-\sqrt{\overline{\alpha}_{t}}D_{t}^{}(\mathbf{u})}{\sqrt{1-\overline{\alpha}_{t}}}\right]=\mathbb{E}_{t\|\mathbf{u}}[\epsilon_{t}^{}(\mathbf{u})]$
EDM	1	$\sigma_{t}$	1	0	$\mathbb{E}_{t\|u}\left[D_{t}^{*}(\mathbf{u})\right]$
FM	$1-t$	$t$	$-1$	$1$	$\mathbb{E}_{t\|\mathbf{u}}\left[\frac{\mathbf{u}-D_{t}^{*}(\mathbf{u})}{t}\right]$
EqM	$1-t$	$t$	$-t$	$t$	$\mathbb{E}_{t\|u}\left[\mathbf{u}-D_{t}^{*}(\mathbf{u})\right]$

Table 1: Unified coefficients and simplification of the general autonomous field for common generative models: DDPM [ 13 ] , EDM [ 16 ] , Flow Matching (FM) [ 20 ] , and Equilibrium Matching (EqM) [ 33 ] .

Figure 1: The Singular Geometry of the Marginal Energy Landscape. (Left) 3D Energy Landscape: A visualization of the marginal energy

E_{marg}(u)=-\log p(u)

. The landscape reveals an infinitely deep potential well at the data manifold, where the energy diverges to

-\infty

. (Right) Contour View: Top-down perspective showing the energy concentration around discrete data points (stars). While the raw gradient

\nabla_{u}E_{marg}(u)

becomes singular as

u

approaches the clean data, in this work we prove that autonomous models remain stable by implicitly implementing a Riemannian gradient flow. In this framework, the posterior noise variance acts as a local conformal metric that preconditions and perfectly counteracts the geometric singularity.

Parameterization	Effective Gain $\nu(t)$	Error Mechanism	Stability
Noise ( $\boldsymbol{\epsilon}$ )	$O(1/b(t))$	Amplified Jensen Gap	Unstable
Signal ( $\mathbf{x}$ )	$O(1/b(t)^{2})$	Exp. Decay vs. Poly. Div.	Stable
Velocity ( $\mathbf{v}$ )	$\mathbf{1}$ (Bounded)	Bounded Drift	Inherently Stable

Table 2: Summary of Stability Analysis for Autonomous Models. The Drift Perturbation Error is the product of the Effective Gain

\nu(t)

and the estimation error. Detailed derivations are provided in Appendix F .

Panel of Figure 2 — Figure 2: Generative performance on CIFAR-10. Top: DDPM Blind exhibits structural instability and noise. Bottom: Flow Matching Blind achieves stable generation, matching the performance of conditioned models.

Panel of Figure 3 — Figure 3: Generative performance on SVHN (Street View House Numbers) . Top: DDPM Blind exhibits structural instability and noise. Bottom: Flow Matching Blind achieves stable generation, matching the performance of conditioned models.

Panel of Figure 4 — Figure 4: Generative performance on Fashion MNIST. . Top: DDPM Blind exhibits structural instability and noise. Bottom: Flow Matching Blind achieves stable generation, matching the performance of conditioned models.

Figure 5: Generative performance on a 2D concentric circles dataset embedded in

\mathbb{R}^{D}

. Conditional models receive explicit time embeddings, while autonomous (blind) models must implicitly infer the noise scale. At low dimensions (

D=2

, top), blind models fail due to overlapping noise distributions. At moderate dimensions (

D=8,32

, middle), Flow Matching achieves stable generation due to its bounded velocity parameterization, while DDPM Blind exhibits severe noise artifacts due to its singular gain

O(1/b(t))

. At extreme dimensions (

D=128

, bottom), absolute geometric concentration forces the estimation error to zero, allowing both blind models to converge.

Model	Parameterization	Singularity	FID (w/o $t$ )
DDIM [ 29 ]	Noise ( $\epsilon$ )	$O(1/b(t))$	40.90
Flow Matching [ 20 ]	Velocity ( $v$ )	Bounded	2.61
uEDM [ 30 ]	Velocity ( $v$ )	Bounded	2.23

Table 3: Generative performance on CIFAR-10 reported by Sun et al. [30] . Stability correlates perfectly with bounded parameterization.

Diffusion Models

Diffusion models learn to reverse gradual data corruption, enabling flexible generation, editing, forecasting, and inference.

See all 1,163 papers

May 12, 2026

Gradient-Free Noise Optimization for Reward Alignment in Generative Models

Existing reward alignment methods for diffusion and flow models rely on multi-step stochastic trajectories, making them difficult to extend to deterministic generators. A natural alternative is noise-space optimization, but existing approaches require backpropagation through the generator and reward pipeline, limiting applicability to differentiable settings. To address this, here we present ZeNO (Zeroth-order Noise Optimization), a gradient-free framework that formulates noise optimization as a path-integral control problem, estimable from zeroth-order reward evaluations. When instantiated with an Ornstein--Uhlenbeck reference process and a one-step receding horizon, the repeated update connects to Langevin dynamics targeting a reward-tilted distribution. ZeNO enables effective inference-time scaling and demonstrates strong performance across diverse generators and reward functions, including a protein structure generation task where backpropagation is infeasible.

Jeongsol Kim, Hongeun Kim, Jian Wang +1

KAIST AI · Snap Inc.

Diffusion Model Alignment Stochastic Optimal Control Zeroth-Order Optimization Generative Modeling Derivative-Free Optimization Gradient-Free Optimization

Ask PDF

May 21, 2026

Noise Schedule Design for Diffusion Models: An Optimal Control Perspective

We develop a principled framework for analyzing and designing noise schedules in diffusion models. We show that one can recast this design problem as an optimal control problem, whose state is the Fisher information of the diffusion process, which evolves according to an ODE, and whose control input is the noise schedule. The objective of the optimal control problem is a functional involving the Fisher information, which is shown to be an upper bound on the Kullback-Leibler sampling error. By solving this optimal control problem, we obtain sufficient conditions on noise schedules under which

\tilde{\mathcal{O}}(d/n)

sampling error is achievable, where

d

is the data dimension and

n

is the number of discretization steps. While existing theoretical works also prove that

\tilde{\mathcal{O}}(d/n)

sampling error bounds are achievable, these results hold for specific noise schedules that differ from those used in practice. Under additional assumptions on the data distribution, we derive a family of noise schedules that recovers the functional forms of schedules widely used in practice. We show that these schedules achieve the

\tilde{\mathcal{O}}(d/n)

error bound when their parameters are chosen according to our analysis. Our theory offers a principled approach to noise schedule design, yielding explicit functional forms and guidance for selecting their parameters.

Seo Taek Kong, Weina Wang, R. Srikant

ECE & CSL, University of Illinois Urbana-Champaign · Computer Science Department, Carnegie Mellon University · ECE, CSL & NCSA, University of Illinois Urbana-Champaign

Diffusion Sampling Diffusion-Based Image Generation Diffusion Timestep Scheduling Optimal Control Diffusion Models

Ask PDF

May 13, 2026

Do Heavy Tails Help Diffusion? On the Subtle Trade-off Between Initialization and Training

Recent works have proposed incorporating heavy-tailed (HT) noise into diffusion- and flow-based generative models, with the goals of better recovering the tails of target distributions and improving generative diversity. This motivation is intuitive: if the data are heavy-tailed, HT noise may appear better matched than light-tailed (LT) Gaussian noise. However, replacing Gaussian noise by HT noise also changes the underlying estimation problem. In this paper, we revisit this paradigm through a combined theoretical and empirical study, establishing sampling-error bounds for two representative diffusion models driven by HT and LT noise. We show that HT noise makes the statistical estimation problem harder, leading to less favorable sampling-error bounds. We support these findings with experiments on synthetic and real-world datasets, empirically recovering the predicted error trade-off. Our results call into question a growing design trend in generative modeling and challenge the use of HT noise to improve rare-region exploration.

Hamza Cherkaoui, Hélène Halconruy, Antonio Ocello

SAMOVAR, Télécom SudParis, Institut Polytechnique de Paris, Palaiseau, France · Modal’X, Université Paris Nanterre, Nanterre, France · CREST, ENSAE Paris, Institut Polytechnique de Paris, Palaiseau, France

Diffusion Model Sampling Heavy-Tailed Noise Heavy-Tailed Distributions Diffusion Models

Ask PDF

Abstract

Diffusion Models

Explore similar work

Gradient-Free Noise Optimization for Reward Alignment in Generative Models

Noise Schedule Design for Diffusion Models: An Optimal Control Perspective

Do Heavy Tails Help Diffusion? On the Subtle Trade-off Between Initialization and Training