Diffusion Sampling

Follow this topic Ask about this topic

Diffusion sampling starts with noise and repeatedly denoises it to generate data, using reverse stochastic processes such as DDPM or deterministic flows such as DDIM and probability-flow ODEs. Work improves this process with higher-order or adaptive solvers, learned timestep schedules, and methods that reuse or predict model computations; discrete and masked diffusion models also require choices about which tokens to reveal and when. Guidance, particle methods, and search can steer samples toward rewards, constraints, diversity, or solutions to inverse problems, with applications spanning image and text generation, molecules, and scientific modeling. A central tension is reducing steps and computation without losing fidelity, diversity, or correctness, while theory seeks sharper guarantees for error, stability, and sampling cost.

Latest papers

264 papers49 papers in the last four weeks, up 104% on the four weeks before. 0.3% of all new papers.

May 7, 2026

Why DDIM Hallucinates More Than DDPM: A Theoretical Analysis of Reverse Dynamics

We theoretically study the hallucination phenomena in two canonical diffusion samplers: the stochastic Denoising Diffusion Probabilistic Model (DDPM) and the deterministic Denoising Diffusion Implicit Model (DDIM). We analyze the reverse ODE (DDIM) and SDE (DDPM) for a Gaussian mixture target, proving that after a critical time

τ

, (a) DDIM can become stuck on the segment connecting the two nearest modes and (b) DDPM stochasticity helps it become unstuck from this region, thus avoiding hallucination. Our empirical validation verifies that DDPM has a significantly lower hallucination rate than DDIM when this region is entered. Building on our observations, we exhibit how using additional stochastic steps can help DDIM avoid hallucinations and offer new insights on how to design improved samplers.

Muhammad H. Ashiq, Samanyu Arora, Abhinav N. Harish +3

University of Wisconsin-Madison, Madison, WI, USA.

Diffusion Sampling Diffusion Models

May 7, 2026

Diverse Sampling in Diffusion Models with Divergence-Free Particle Guidance

Modern generative models can produce high-quality samples, but independent samples under the same condition often yields highly similar outputs. Particle-based methods increase diversity by letting the samples in a batch interact, typically through a repulsive force. These forces, however, also push each sample away from the data distribution and produce visible artifacts. We introduce EDDY, a training-free particle guidance method designed to avoid this failure mode. Instead of a repulsive gradient, EDDY couples particles through anti-symmetric matrix fields passed through a Stein operator, a family of drift perturbations that leaves the Fokker--Planck equation invariant. Unlike repulsive guidance, these interactions do not alter a particle's distribution while the batch is independent. For the same reason they cannot create diversity on their own, so EDDY pairs them with a negatively correlated initialization that keeps each particle's prior exact. To use EDDY with perceptual kernels such as DINOv2, we approximate its second-order terms with finite differences and Hutchinson estimates. Across FLUX.1-dev, FLUX.2-klein and SDXL, EDDY achieves higher image quality and prompt alignment than existing particle guidance methods at matched diversity.

Gal Vinograd, Idan Achituve, Ethan Fetaya

Faculty of Engineering Bar-Ilan University Ramat Gan, Israel · Independent Researcher

Diffusion Model Guidance Diffusion Sampling Diversity-Aware Sampling

May 7, 2026

Diffusion-Based Posterior Sampling: A Feynman-Kac Analysis of Bias and Stability

Diffusion-based posterior samplers draw from measurement- or reward-conditioned posteriors using pretrained diffusion priors. Though widely used for inverse problems, their output is biased even with exact prior scores. First, we quantify this bias for the DPS algorithm, the base method for a family of samplers, by introducing a surrogate curve of probability densities which interpolates between the true posterior and the standard Gaussian. The ratio of sampler to surrogate densities solves a parabolic PDE, and its Feynman--Kac formula yields an explicit representation of this ratio as the expectation of a path functional. This representation reveals which posterior regions are over- or under-sampled. This framework also provides a design principle for bias reduction that includes STSL, a related sampler. Second, in hard-constraint problems (e.g. the low measurement noise regime) these samplers are known to be numerically unstable. We find that this instability of discretized DPS arises from an effective guidance update whose strength scales inversely with step size, inevitably violating the forward-Euler stability criterion near the constraint. This finding motivates implicit (backward-Euler) guidance updates. Finally, we show that the standard practice of early guidance-stopping, used to avoid oscillations, introduces a secondary bias which we characterize through our framework.

William Porteous, Advait Parulekar, Sebastien Motsch +2

Department of Mathematics, UT Austin. · Department of Electrical and Computer Engineering, UT Austin. · School of Mathematical and Statistical Sciences, Arizona State University.

Stochastic Differential Equations Bayesian Inverse Problems Diffusion Guidance Diffusion Sampling Diffusion Posterior Sampling

May 7, 2026

Diffusion model for SU(N) gauge theories

Implicit score matching provides a computationally efficient approach for training diffusion models and generating high-quality samples from complex distributions. In this work, we develop a score-matching framework for SU(N) lattice gauge theories, which can be extended to other Lie groups. We apply the method to SU(3) gauge configurations with the Wilson gauge action in two and four dimensions and assess the quality of the generated samples by comparison with Hybrid Monte Carlo (HMC) simulations. We show that the diffusion models can be successfully trained and applied for sampling the Wilson gauge action. For large values of inverse coupling, accurate reverse-time integration requires predictor-corrector schemes, for which we introduce a corrector based on Hamiltonian molecular dynamics. While the corrector significantly improves sampling quality, it also increases the computational cost. We outline several strategies for improving sampling efficiency.

Javad Komijani, Marina K. Marinkovic, Lara Turgut

Institute for Theoretical Physics, ETH Zurich, 8093 Zurich, Switzerland

Score Matching Diffusion Sampling High-Energy Physics Diffusion Models Lattice Field Theory

May 7, 2026

DBMSolver: A Training-free Diffusion Bridge Sampler for High-Quality Image-to-Image Translation

Diffusion-based image-to-image (I2I) translation excels in high-fidelity generation but suffers from slow sampling in state-of-the-art Diffusion Bridge Models (DBMs), often requiring dozens of function evaluations (NFEs). We introduce DBMSolver, a training-free sampler that exploits the semi-linear structure of DBM's underlying SDE and ODE via exponential integrators, yielding highly-efficient 1st- and 2nd-order solutions. This reduces NFEs by up to 5x while boosting quality (e.g., FID drops 53% on DIODE at 20 NFEs vs. 2nd-order baseline). Experiments on inpainting, stylization, and semantics-to-image tasks across resolutions up to 256x256 show DBMSolver sets new SOTA efficiency-quality tradeoffs, enabling real-world applicability. Our code is publicly available at https://github.com/snumprlab/dbmsolver.

Sankarshana Venugopal, Mohammad Mostafavi, Jonghyun Choi

Seoul National University · ECE, IPAI, ASRI in SNU

Image-to-Image Translation Diffusion Sampling Diffusion Model Inference Acceleration Diffusion Models

May 5, 2026

Conditional Diffusion Sampling

Sampling from unnormalized multimodal distributions with limited density evaluations remains a fundamental challenge in machine learning and natural sciences. Successful approaches construct a bridge between a tractable reference and the target distribution. Parallel Tempering (PT) serves as the gold standard, while recent diffusion-based approaches offer a continuous alternative at the cost of neural training. In this work, we introduce Conditional Diffusion Sampling (CDS), a framework that combines these two paradigms. To this end, we derive Conditional Interpolants, a class of stochastic processes whose transport dynamics are governed by an exact, closed-form stochastic differential equation (SDE), requiring no neural approximation. Although these dynamics require sampling from a non-trivial initialization distribution, we show both theoretically and empirically that the cost of this initialization diminishes for sufficiently short diffusion times. CDS leverages this by a two-stage procedure: (1) PT is used to efficiently sample the initial distribution, and then (2) samples are transported via the transport SDE. This combination couples the robust global exploration of PT with efficient local transport. Experiments suggest that CDS has the potential to achieve a superior trade-off between sample quality and density evaluation cost compared to state-of-the-art samplers.

Francisco M. Castro-Macías, Pablo Morales-Álvarez, Saifuddin Syed +3

University of Granada · Work done while visiting the University of Cambridge. · University of British Columbia +2

Markov Chain Monte Carlo Sampling from Unnormalized Distributions Diffusion Sampling

May 5, 2026

Flow Sampling: Learning to Sample from Unnormalized Densities via Denoising Conditional Processes

Sampling from unnormalized densities is analogous to the generative modeling problem, but the target distribution is defined by a known energy function instead of data samples. Because evaluating the energy function is often costly, a primary challenge is to learn an efficient sampler. We introduce Flow Sampling, a framework built on diffusion models and flow matching for the data-free setting. Our training objective is conditioned on a noise sample and regresses onto a denoising diffusion drift constructed from the energy function. In contrast, diffusion models' objective is conditioned on a data sample and regresses onto a noising diffusion drift. We utilize the interpolant process to minimize the number of energy function evaluations during training, resulting in an efficient and scalable method for sampling unnormalized densities. Furthermore, our formulation naturally extends to Riemannian manifolds, enabling diffusion-based sampling in geometries beyond Euclidean space. We derive a closed-form formula for the conditional drift on constant curvature manifolds, including hyperspheres and hyperbolic spaces. We evaluate Flow Sampling on synthetic energy benchmarks, small peptides, large-scale amortized molecular conformer generation, and distributions supported on the sphere, demonstrating strong empirical performance.

Aaron Havens, Brian Karrer, Neta Shaul

FAIR at Meta · Weizmann Institute of Science

Sampling from Unnormalized Distributions Diffusion Sampling Energy-Based Models Diffusion Models

May 4, 2026

SpecEdit: Training-Free Acceleration for Diffusion based Image Editing via Semantic Locking

Diffusion-based image editing offers strong semantic controllability, but remains computationally expensive due to iterative high-resolution denoising over all spatial tokens. Dynamic-resolution sampling reduces this cost by performing early steps at reduced resolution. However, existing approaches prioritize upsampling using low-level heuristics such as edge detection or channel variance, which are weakly aligned with editing semantics and may lead to structural inconsistency. Moreover, spatial regions are often upsampled without verifying whether semantic modification is actually required, resulting in redundant high-resolution computation and accumulated errors. Therefore, we propose SpecEdit, a training-free dynamic-resolution framework tailored for diffusion-based image editing. SpecEdit follows a draft-and-verify scheme: a low-resolution draft first estimates the semantic outcome, after which token-level discrepancies are used to identify edit-relevant tokens for high-resolution denoising, while the remaining tokens stay at a coarse resolution. Experiments on Qwen-Image-Edit and FLUX.1-Kontext-dev demonstrate up to 10x and 7x acceleration, while maintaining strong quality. SpecEdit is complementary to step distillation and other acceleration techniques, achieving up to 13x speedup when combined with existing methods. Our code is in supplementary material and will be released on GitHub.

Zhengan Yan, Shikang Zheng, Haoran Qin +9

EPIC Lab , Shanghai Jiao Tong University · Shandong University · Alibaba Group +2

Diffusion Sampling Diffusion-Based Image Editing Diffusion Model Inference Acceleration Training-Free Image Editing Diffusion-Based Denoising

May 1, 2026

IdentiFace: Multi-Modal Iterative Diffusion Framework for Identifiable Suspect Face Generation in Crime Investigations

Suspect face generation remains a technical challenge in crime investigations. Traditional sketch-drawing workflows suffer from low efficiency and quality, while diffusion-based approaches still face intrinsic limitations on conditional ambiguity for text-to-image models and sampling variance for one-shot generation. We proposed IdentiFace, a novel diffusion-based framework for identifiable suspect face generation, which addressed these issues through (1) multi-modal input design to strengthen conditional control, and (2) an iterative generation pipeline enabling identifiable feature adjustment. We additionally contributed a facial identity loss and two task-specific datasets. Comprehensive experiments on synthetic datasets and in real-world scenarios indicate that IdentiFace achieves superior performance over existing methods, especially in terms of identity retrieval, and shows strong potential for practical applications.

Weichen Liu, Yixin Yang, Changsheng Chen +1

Southeast University · Nanjing, China · Shenzhen MSU-BIT University +1

Diffusion Sampling Image Generation Identity-Preserving Image Generation Conditional Diffusion Models Multimodal Diffusion Models

Apr 30, 2026

Information-geometric adaptive sampling for graph diffusion

Standard diffusion models for graph generation typically rely on uniform time-stepping, an approach that overlooks the non-homogeneous dynamics of distributional evolution on complex manifolds. In this paper, we present an information-geometric framework that reinterprets the diffusion sampling trajectory as a parametric curve on a Riemannian manifold. Our key observation is that the Fisher-Rao metric provides a principled measure of the intrinsic distance. By analyzing this metric, we derive the Drift Variation Score (DVS), a geometry-aware indicator that quantifies the instantaneous rate of distributional change. Unlike prior heuristic-based adaptive samplers, our DVS solver enforces a constant informational speed on the statistical manifold, automatically maintaining a uniform rate of distributional change along the sampling trajectory. This equal arc-length strategy ensures that each discretization step contributes equally to the information speed. Theoretical analysis verifies that DVS characterizes the local stiffness of the sampling dynamics in the Fisher-Rao sense. Experimental results on molecule and social network generation show that DVS significantly improves structural fidelity and sampling efficiency. Code is at https://github.com/kunzhan/DVS

Yuhui Lu, Wenjing Liu, Kun Zhan

School of Information Science and Engineering, Lanzhou University, Gansu, China.

Information Geometry Graph Generation Diffusion Sampling Adaptive Sampling Diffusion Timestep Scheduling Graph Diffusion

Apr 30, 2026

A unified perspective on fine-tuning and sampling with diffusion and flow models

We study the problem of training diffusion and flow generative models to sample from target distributions defined by an exponential tilting of a base density; a formulation that subsumes both sampling from unnormalized densities and reward fine-tuning of pre-trained models. This problem can be approached from a stochastic optimal control (SOC) perspective, using adjoint-based or score matching methods, or from a non-equilibrium thermodynamics perspective. We provide a unified framework encompassing these approaches and make three main contributions: (i) bias-variance decompositions revealing that Adjoint Matching/Sampling and Novel Score Matching have finite gradient variance, while Target and Conditional Score Matching do not; (ii) norm bounds on the lean adjoint ODE that theoretically support the effectiveness of adjoint-based methods; and (iii) adaptations of the CMCD and NETS loss functions, along with novel Crooks and Jarzynski identities, to the exponential tilting setting. We validate our analysis with reward fine-tuning experiments on Stable Diffusion 1.5 and 3.

Carles Domingo-Enrich, Yuanqi Du, Michael S. Albergo

Microsoft Research New England · Harvard University, Kempner Institute

Stochastic Optimal Control Diffusion Sampling Diffusion Model Fine-Tuning Flow-Based Generative Modeling Score-Based Generative Modeling

Apr 27, 2026

BiMol-Diff: A Unified Diffusion Framework for Molecular Generation and Captioning

Bridging molecular structures and natural language is essential for controllable design. Autoregressive models struggle with long-range dependencies, while standard diffusion processes apply uniform corruption across positions, which can distort structurally informative tokens. We present BiMol-Diff, a unified diffusion framework for the paired tasks of text-conditioned molecule generation and molecule captioning. Our key component is a token-aware noise schedule that assigns position-dependent corruption based on token recovery difficulty, preserving harder-to-recover substructures during the forward process. On ChEBI-20 and M3-20M, BiMol-Diff improves molecule reconstruction with a 15.4% relative gain in Exact Match and achieves strong captioning results, attaining best BLEU and BERTScore among compared baselines. These results indicate token-aware noising improves fidelity in molecular structure-language modelling.

Aditya Hemant Shahane, Anuj Kumar Sirohi, Devansh Arora +3

Indian Institute of Technology Delhi · Indian Institute of Technology Ropar · Indian Institute of Science Bengaluru

Diffusion Sampling Molecular Generation Diffusion Language Models Conditional Diffusion Models

Apr 26, 2026

$Z^2$ -Sampling: Zero-Cost Zigzag Trajectories for Semantic Alignment in Diffusion Models

Diffusion models have achieved unprecedented success in text-aligned generation, largely driven by Classifier-Free Guidance (CFG). However, standard CFG operates strictly on instantaneous gradients, omitting the intrinsic curvature of the data manifold. Recent methods like Zigzag-sampling (Z-Sampling) explicitly traverse multi-step forward-backward trajectories to probe this curvature, significantly improving semantic alignment. Yet, these explicit traversals triple the Neural Function Evaluation (NFE) cost and introduce unconstrained truncation errors from off-manifold evaluations, causing cumulative drift from the true marginal distribution. In this paper, we theoretically demonstrate that the explicit zigzag sequence is topologically reducible. We propose Implicit Z-Sampling, rigorously proving that intermediate states can be algebraically annihilated via operator dualities, physically eliminating off-manifold approximation errors. To push sampling efficiency to its theoretical lower bound, we introduce

Z^2

-Sampling (Zero-cost Zigzag Sampling). Exploiting the Probability Flow ODE's temporal coherence,

Z^2

-Sampling couples implicit algebraic collapse with a dynamically cached Temporal Semantic Surrogate. This restores the standard 2-NFE baseline without sacrificing semantic exploration. We formally prove via Backward Error Analysis that this discrete collapse inherently synthesizes a directional derivative curvature penalty. Finally, extensive evaluations demonstrate that

Z^2

-Sampling structurally shatters the performance-efficiency Pareto frontier. We validate its universal applicability across diverse architectures (U-Nets, DiTs) and modalities (image/video), establishing seamless orthogonality with advanced alignment frameworks (AYS, Diffusion-DPO).

Haosen Li, Wenshuo Chen, Shaofeng Liang +3

1The Hong Kong University of Science and Technology (Guangzhou) · 2Griffith University & Data61/CSIRO

Diffusion Model Alignment Diffusion Sampling Diffusion Model Inference Acceleration

Apr 21, 2026

FASTER: Value-Guided Sampling for Fast RL

Some of the most performant reinforcement learning algorithms today can be prohibitively expensive as they use test-time scaling methods such as sampling multiple action candidates and selecting the best one. In this work, we propose FASTER, a method for getting the benefits of sampling-based test-time scaling of diffusion-based policies without the computational cost by tracing the performance gain of action samples back to earlier in the denoising process. Our key insight is that we can model the denoising of multiple action candidates and selecting the best one as a Markov Decision Process (MDP) where the goal is to progressively filter action candidates before denoising is complete. With this MDP, we can learn a policy and value function in the denoising space that predicts the downstream value of action candidates in the denoising process and filters them while maximizing returns. The result is a method that is lightweight and can be plugged into existing generative RL algorithms. Across challenging long-horizon manipulation tasks in online and batch-online RL, FASTER consistently improves the underlying policies and achieves the best overall performance among the compared methods. Applied to a pretrained VLA, FASTER achieves the same performance while substantially reducing training and inference compute requirements. Code is available at https://github.com/alexanderswerdlow/faster .

Perry Dong, Alexander Swerdlow, Dorsa Sadigh +1

Stanford University

Diffusion Sampling Diffusion Policy RL for Diffusion Models Diffusion Model Inference Acceleration

Apr 21, 2026

Denoising, Fast and Slow: Difficulty-Aware Adaptive Sampling for Image Generation

Diffusion- and flow-based models usually allocate compute uniformly across space, updating all patches with the same timestep and number of function evaluations. While convenient, this ignores the heterogeneity of natural images: some regions are easy to denoise, whereas others benefit from more refinement or additional context. Motivated by this, we explore patch-level noise scales for image synthesis. We find that naively varying timesteps across image tokens performs poorly, as it exposes the model to overly informative training states that do not occur at inference. We therefore introduce a timestep sampler that explicitly controls the maximum patch-level information available during training, and show that moving from global to patch-level timesteps already improves image generation over standard baselines. By further augmenting the model with a lightweight per-patch difficulty head, we enable adaptive samplers that allocate compute dynamically where it is most needed. Combined with noise levels varying over both space and diffusion time, this yields Patch Forcing (PF), a framework that advances easier regions earlier so they can provide context for harder ones. PF achieves superior results on class-conditional ImageNet, remains orthogonal to representation alignment and guidance methods, and scales to text-to-image synthesis. Our results suggest that patch-level denoising schedules provide a promising foundation for adaptive image generation.

Johannes Schusterbauer, Ming Gui, Yusong Li +3

Diffusion Model Sampling Diffusion Sampling Diffusion-Based Image Generation Diffusion Timestep Scheduling Diffusion Model Inference Acceleration Diffusion-Based Denoising

Apr 18, 2026

Noise-Adaptive Diffusion Sampling for Inverse Problems Without Task-Specific Tuning

Diffusion models (DMs) have recently shown remarkable performance on inverse problems (IPs). Optimization-based methods can fast solve IPs using DMs as powerful regularizers, but they are susceptible to local minima and noise overfitting. Although DMs can provide strong priors for Bayesian approaches, enforcing measurement consistency during the denoising process leads to manifold infeasibility issues. We propose Noise-space Hamiltonian Monte Carlo (N-HMC), a posterior sampling method that treats reverse diffusion as a deterministic mapping from initial noise to clean images. N-HMC enables comprehensive exploration of the solution space, avoiding local optima. By moving inference entirely into the initial-noise space, N-HMC keeps proposals on the learned data manifold. We provide a comprehensive theoretical analysis of our approach and extend the framework to a noise-adaptive variant (NA-NHMC) that effectively handles IPs with unknown noise type and level. Extensive experiments across four linear and three nonlinear inverse problems demonstrate that NA-NHMC achieves superior reconstruction quality with robust performance across different hyperparameters and initializations, significantly outperforming recent state-of-the-art methods. The code is available at https://github.com/NA-HMC/NA-HMC.

Yingzhi Xia, Setthakorn Tanomkiattikun, Liangli Zhen +1

Institute of High Performance Computing, Agency for Science, Technology and Research, Singapore · Agency for Science, Technology and Research, Singapore · Johns Hopkins University +1

Markov Chain Monte Carlo Bayesian Inverse Problems Diffusion Sampling Diffusion-Based Inverse Problems Diffusion Posterior Sampling Image Inverse Problems

Apr 17, 2026

Elucidating the SNR-t Bias of Diffusion Probabilistic Models

Diffusion Probabilistic Models have demonstrated remarkable performance across a wide range of generative tasks. However, we have observed that these models often suffer from a Signal-to-Noise Ratio-timestep (SNR-t) bias. This bias refers to the misalignment between the SNR of the denoising sample and its corresponding timestep during the inference phase. Specifically, during training, the SNR of a sample is strictly coupled with its timestep. However, this correspondence is disrupted during inference, leading to error accumulation and impairing the generation quality. We provide comprehensive empirical evidence and theoretical analysis to substantiate this phenomenon and propose a simple yet effective differential correction method to mitigate the SNR-t bias. Recognizing that diffusion models typically reconstruct low-frequency components before focusing on high-frequency details during the reverse denoising process, we decompose samples into various frequency components and apply differential correction to each component individually. Extensive experiments show that our approach significantly improves the generation quality of various diffusion models (IDDPM, ADM, DDIM, A-DPM, EA-DPM, EDM, PFGM++, and FLUX) on datasets of various resolutions with negligible computational overhead. The code is at https://github.com/AMAP-ML/DCW.

Meng Yu, Lei Sun, Jianhao Zeng +2

Lanzhou University · AMAP Alibaba Group

Diffusion Model Sampling Diffusion Sampling Diffusion-Based Image Generation Diffusion Models

Apr 12, 2026

Query Lower Bounds for Diffusion Sampling

Diffusion models generate samples by iteratively querying learned score estimates. A rapidly growing literature focuses on accelerating sampling by minimizing the number of score evaluations, yet the information-theoretic limits of such acceleration remain unclear. In this work, we establish the first score query lower bounds for diffusion sampling. We prove that for

d

-dimensional distributions, given access to score estimates with polynomial accuracy

\varepsilon=d^{-O(1)}

(in any

L^p

sense), any sampling algorithm requires

\widetildeΩ(\sqrt{d})

adaptive score queries. In particular, our proof shows that, within any polynomial total-query budget, successful sampling requires searching over

\widetildeΩ(\sqrt{d})

distinct noise levels, providing a formal explanation for why multiscale noise schedules are necessary in practice.

Zhiyang Xun, Eric Price

UT Austin

Query Complexity Diffusion Sampling Information-Theoretic Lower Bounds Diffusion Models

Apr 11, 2026

From AR to Diffusion: Efficiently Adapting Large Language Models with Strictly Causal and Elastic Horizons

Diffusion models promise efficient parallel text generation but rely on bidirectional attention, creating a structural mismatch with pre-trained Autoregressive (AR) models. This incompatibility precludes reusing robust AR priors, necessitating prohibitive pre-training from scratch. To bridge this gap, we propose FLUID, a framework that efficiently adapts AR backbones to the diffusion paradigm. By enforcing Strictly Causal Alignment, FLUID enables seamless initialization from standard GPT-style checkpoints, circumventing the need for massive pre-training. Furthermore, we introduce Elastic Horizons, an entropy-driven mechanism that dynamically modulates denoising strides based on local information density rather than fixed schedules. Experiments demonstrate that FLUID achieves state-of-the-art performance while reducing training costs by orders of magnitude, effectively reconciling established AR foundations with efficient parallel generation. Our code is available at https://github.com/Oli-lab-nun/FLUID/tree/main.

Xiangyu Ma, Teng Xiao, Zuchao Li +1

School of Artificial Intelligence, Wuhan University, Wuhan, China · School of Computer Science, Wuhan University, Wuhan, China

Diffusion Language Models Diffusion Language Model Inference Diffusion Sampling

Apr 10, 2026

EGLOCE: Training-Free Energy-Guided Latent Optimization for Concept Erasure

As text-to-image diffusion models grow increasingly prevalent, the ability to remove specific concepts-mostly explicit content and many copyrighted characters or styles-has become essential for safety and compliance. Existing unlearning approaches often require costly re-training, modify parameters at the cost of degradation of unrelated concept fidelity, or depend on indirect inference-time adjustment that compromise the effectiveness of concept erasure. Inspired by the success of energy-guided sampling for preservation of the condition of diffusion models, we introduce Energy-Guided Latent Optimization for Concept Erasure (EGLOCE), a training-free approach that removes unwanted concepts by re-directing noisy latent during inference. Our method employs a dual-objective framework: a repulsion energy that steers generation away from target concepts via gradient descent in latent space, and a retention energy that preserves semantic alignment to the original prompt. Combined with previous approaches that either require erroneous modified model weights or provide weak inference-time guidance, EGLOCE operates entirely at inference and enhances erasure performance, enabling plug-and-play integration. Extensive experiments demonstrate that EGLOCE improves concept removal while maintaining image quality and prompt alignment across baselines, even with adversarial attacks. To the best of our knowledge, our work is the first to establish a new paradigm for safe and controllable image generation through dual energy-based guidance during sampling.

Junyeong Ahn, Seojin Yoon, Sungyong Baik

KAIST AI · Hanyang University

Diffusion Model Unlearning Safe T2I Generation T2I Diffusion Models Diffusion Sampling

Apr 8, 2026

VASR: Variance-Aware Systematic Resampling for Reward-Guided Diffusion

Sequential Monte Carlo (SMC) samplers for reward-guided diffusion models often suffer from rapid lineage collapse: a few high-reward particles dominate the population within a handful of resampling steps, destroying diversity and degrading sample quality. We propose a variance-decomposition framework for reward-guided diffusion SMC that separates continuation variance

V_t^{\mathrm{cont}}

from residual variance

V_t^{\mathrm{res}}

, revealing that high offspring-count variance under the commonly used multinomial resampling drives this collapse. This motivates VASR (Variance-Aware Systematic Resampling), which addresses both variance terms via variance-optimal mass allocation

m_t \propto w_t e^{r_t}

(minimizing

V_t^{\mathrm{cont}}

) and systematic resampling (controlling

V_t^{\mathrm{res}}

). For latent diffusion models where intermediate rewards are noisy due to stochastic continuations, we propose VASR-Max, a deliberately biased high-selection variant for variance-sensitive reward optimization. Both methods are training-free, fully parallelizable, and add only linear overhead. On MNIST and CIFAR-10, VASR achieves as high as

26\%

better FID than prior SMC methods while remaining 66 times faster than MCTS-based value methods at matched compute. On text-to-image generation, VASR-Max consistently outperforms the strongest SMC baseline across compute budgets and matches MCTS-based methods within 2.5--3% reward at high budgets while being approximately times faster.

Shivanshu Shekhar, Sagnik Mukherjee, Jia Yi Zhang +1

Siebel School of Computing and Data Science University of Illinois Urbana-Champaign · Department of Statistics University of Illinois Urbana-Champaign

Diffusion Model Sampling Diffusion Sampling Diffusion Model Guidance

Apr 5, 2026

1.x-Distill: Breaking the Diversity, Quality, and Efficiency Barrier in Distribution Matching Distillation

Diffusion models produce high-quality text-to-image results, but their iterative denoising is computationally expensive.Distribution Matching Distillation (DMD) emerges as a promising path to few-step distillation, but suffers from diversity collapse and fidelity degradation when reduced to two steps or fewer. We present 1.x-Distill, the first fractional-step distillation framework that breaks the integer-step constraint of prior few-step methods and establishes 1.x-step generation as a practical regime for distilled diffusion models.Specifically, we first analyze the overlooked role of teacher CFG in DMD and introduce a simple yet effective modification to suppress mode collapse. Then, to improve performance under extreme steps, we introduce Stagewise Focused Distillation, a two-stage strategy that learns coarse structure through diversity-preserving distribution matching and refines details with inference-consistent adversarial distillation. Furthermore, we design a lightweight compensation module for Distill--Cache co-Training, which naturally incorporates block-level caching into our distillation pipeline.Experiments on SD3-Medium and SD3.5-Large show that 1.x-Distill surpasses prior few-step methods, achieving better quality and diversity at 1.67 and 1.74 effective NFEs, respectively, with up to 33x speedup over original 28x2 NFE sampling.

Haoyu Li, Tingyan Wen, Lin Qi +6

Tsinghua University · Central Media Technology Institute, Huawei

Diffusion Model Distillation Diffusion Sampling Few-Step Diffusion Sampling Distribution Matching Distillation T2I Diffusion Models

Apr 3, 2026

Generating DDPM-based Samples from Tilted Distributions

Given

n

independent samples from a

d

-dimensional probability distribution, our aim is to generate diffusion-based samples from a distribution obtained by tilting the original, where the degree of tilt is parametrized by

θ\in \mathbb{R}^d

. We define a plug-in estimator and show that it is minimax-optimal. We develop Wasserstein bounds between the distribution of the plug-in estimator and the true distribution as a function of

n

and

θ

, illustrating regimes where the output and the desired true distribution are close. Further, under some assumptions, we prove the TV-accuracy of running Diffusion on these tilted samples. Our theoretical results are supported by extensive simulations. Applications of our work include finance, weather and climate modelling, and many other domains, where the aim may be to generate samples from a tilted distribution that satisfies practically motivated moment constraints.

Himadri Mandal, Dhruman Gupta, Rushil Gupta +5

Indian Statistical Institute, Kolkata. · Ashoka University. · Tata Institute of Fundamental Research +2

Diffusion Sampling Constrained Generative Modeling

Apr 2, 2026

Why Gaussian Diffusion Models Fail on Discrete Data and How to Prevent It?

Diffusion models have become a standard approach for generative modeling in continuous domains, yet their application to discrete data remains challenging. We investigate why Gaussian diffusion models with the DDPM solver struggle to sample from discrete distributions that are represented as a mixture of delta-distributions in the continuous space. Using a toy Random Hierarchy Model, we identify a critical sampling interval in which the density of noisified data becomes multimodal. In this regime, DDPM occasionally enters low-density regions between modes producing out-of-distribution inputs for the model and degrading sample quality. We show that existing heuristics, including self-conditioning and a solver we term q-sampling, help alleviate this issue. Furthermore, we demonstrate that combining self-conditioning with switching from DDPM to q-sampling within the critical interval improves generation quality on real data. We validate these findings across conditional and unconditional tasks in multiple domains, including text, programming code, and proteins.

Alexander Shabalin, Simon Elistratov, Viacheslav Meshchaninov +2

Constructor University, Bremen, Germany · Lomonosov Moscow State University, Moscow, Russia

Diffusion Models Diffusion Sampling

Apr 1, 2026

Locally Confident, Globally Stuck: The Quality-Exploration Dilemma in Diffusion Language Models

Diffusion large language models (dLLMs) theoretically permit token decoding in arbitrary order, a flexibility that could enable richer exploration of reasoning paths than autoregressive (AR) LLMs. In practice, however, random-order decoding often hurts generation quality. To mitigate this, low-confidence remasking improves single-sample quality (e.g., Pass@

1

) by prioritizing confident tokens, but it also suppresses exploration and limits multi-sample gains (e.g., Pass@

k

), creating a fundamental quality--exploration dilemma. In this paper, we provide a unified explanation of this dilemma. We show that low-confidence remasking improves a myopic proxy for quality while provably constraining the entropy of the induced sequence distribution. To overcome this limitation, we characterize the optimal distribution that explicitly balances quality and exploration, and develop a simple Independent Metropolis--Hastings sampler that approximately targets this distribution during decoding. Experiments across a range of reasoning benchmarks including MATH500, AIME24/25, HumanEval, and MBPP show that our approach yields better exploration-quality tradeoff than both random and low-confidence remasking.

Liancheng Fang, Aiwei Liu, Henry Peng Zou +7

University of Illinois Chicago · Tsinghua University · MBZUAI +1

Diffusion Language Model Decoding Diffusion Sampling

Mar 31, 2026

Total Variation Guarantees for Sampling with Stochastic Localization

Motivated by the success of score-based generative models, a number of diffusion-based algorithms have recently been proposed for the problem of sampling from a probability measure whose unnormalized density can be accessed. Among them, Grenioux et al. introduced SLIPS, a sampling algorithm based on Stochastic Localization. While SLIPS exhibits strong empirical performance, no rigorous convergence analysis has previously been provided. In this work, we close this gap by establishing the first guarantee for SLIPS in total variation distance. Under minimal assumptions on the target, our bound implies that the number of steps required to achieve an

\varepsilon

-guarantee scales linearly with the dimension, up to logarithmic factors. The analysis leverages techniques from the theory of score-based generative models and further provides theoretical insights into the empirically observed optimal choice of discretization points.

Jakob Kellermann

Weierstrass Institute Berlin

Sampling from Unnormalized Distributions Diffusion Sampling Convergence Analysis

Mar 30, 2026

Stepwise Credit Assignment for GRPO on Flow-Matching Models

Flow-GRPO successfully applies reinforcement learning to flow models, but uses uniform credit assignment across all steps. This ignores the temporal structure of diffusion generation: early steps determine composition and content (low-frequency structure), while late steps resolve details and textures (high-frequency details). Moreover, assigning uniform credit based solely on the final image can inadvertently reward suboptimal intermediate steps, especially when errors are corrected later in the diffusion trajectory. We propose Stepwise-Flow-GRPO, which assigns credit based on each step's reward improvement. By leveraging Tweedie's formula to obtain intermediate reward estimates and introducing gain-based advantages, our method achieves superior sample efficiency and faster convergence. We also introduce a DDIM-inspired SDE that improves reward quality while preserving stochasticity for policy gradients.

Yash Savani, Branislav Kveton, Yuchen Liu +5

Carnegie Mellon University · Adobe Research

RL for Diffusion Models Step-Level Credit Assignment in RL Flow-Based Generative Modeling Flow Matching Diffusion Sampling

Mar 26, 2026

A Priori Sampling of Transition States with Guided Diffusion

Transition states, the first-order saddle points on the potential energy surfaces, govern the kinetics and mechanisms of chemical reactions and conformational changes. Locating them is challenging because transition pathways are topologically complex and can proceed via an ensemble of diverse routes. Existing methods address these challenges by introducing heuristic assumptions about the pathway or reaction coordinates, which limits their applicability when a good initial guess is unavailable or when the guess precludes alternative, potentially relevant pathways. We propose to bypass such heuristic limitations by introducing ASTRA, A Priori Sampling of TRAnsition States with Guided Diffusion, which reframes the transition state search as an inference-time scaling problem for generative models. ASTRA trains a score-based diffusion model on configurations from known metastable states. Then, ASTRA guides inference toward the isodensity surface separating the basins of metastable states via a principled composition of conditional scores. A Score-Aligned Ascent (SAA) process then approximates a reaction coordinate from the difference between conditioned scores and combines it with physical forces to drive convergence onto first-order transition states. Validated on benchmarks ranging from 2D potentials to biomolecular conformational changes and a chemical reaction, ASTRA locates transition states with high precision and discovers multiple reaction pathways, enabling mechanistic studies of complex molecular systems.

Hyukjun Lim, Soojung Yang, Lucas Pinède +3

Department of Materials Science and Engineering, Seoul National University, 1 Gwanak-ro, Gwanak-gu, Seoul, 08826, Republic of Korea. · Computational and Systems Biology, MIT, 77 Massachusetts Avenue, Cambridge, 02139, Massachusetts, USA. · Chimie ParisTech, PSL University, 11 rue Pierre et Marie Curie, Paris, 75005, France. +2

Diffusion Model Guidance Diffusion Sampling Inference-Time Search

Mar 26, 2026

DRiffusion: Draft-and-Refine Process Parallelizes Diffusion Models with Ease

Diffusion models have achieved remarkable success in generating high-fidelity content but suffer from slow, iterative sampling, resulting in high latency that limits their use in interactive applications. We introduce DRiffusion, a parallel sampling framework that parallelizes diffusion inference through a draft-and-refine process. DRiffusion employs skip transitions to generate multiple draft states for future timesteps and computes their corresponding noises in parallel, which are then used in the standard denoising process to produce refined results. Theoretically, our method achieves an acceleration rate of

\tfrac{1}{n}

or

\tfrac{2}{n+1}

, depending on whether the conservative or aggressive mode is used, where

n

denotes the number of devices. Empirically, DRiffusion attains 1.4

\times

-3.7

\times

speedup across multiple diffusion models while incur minimal degradation in generation quality: on MS-COCO dataset, both FID and CLIP remain largely on par with those of the original model, while PickScore and HPSv2.1 show only minor average drops of 0.17 and 0.43, respectively. These results verify that DRiffusion delivers substantial acceleration and preserves perceptual quality.

Runsheng Bai, Chengyu Zhang, Yangdong Deng

CSAIL, MIT · IST, Nanjing University · School of Software, Tsinghua University; FuturististAI Lab, Shanghai Tsinghua International Innovation Center

Diffusion Model Sampling Diffusion Model Inference Acceleration Diffusion Sampling Diffusion-Based Image Generation Diffusion-Based Denoising

Mar 25, 2026

Polynomial Speedup in Diffusion Models with the Multilevel Euler-Maruyama Method

We introduce the Multilevel Euler-Maruyama (ML-EM) method compute solutions of SDEs and ODEs using a range of approximators

f^1,\dots,f^k

to the drift

f

with increasing accuracy and computational cost, only requiring a few evaluations of the most accurate

f^k

and many evaluations of the less costly

f^1,\dots,f^{k-1}

. If the drift lies in the so-called Harder than Monte Carlo (HTMC) regime, i.e. it requires

ε^{-γ}

compute to be

ε

-approximated for some

γ>2

, then ML-EM

ε

-approximates the solution of the SDE with

ε^{-γ}

compute, improving over the traditional EM rate of

ε^{-γ-1}

. In other terms it allows us to solve the SDE at the same cost as a single evaluation of the drift. In the context of diffusion models, the different levels

f^{1},\dots,f^{k}

are obtained by training UNets of increasing sizes, and ML-EM allows us to perform sampling with the equivalent of a single evaluation of the largest UNet. Our numerical experiments confirm our theory: we obtain up to fourfold speedups for image generation on the CelebA dataset downscaled to 64x64, where we measure a

γ\approx2.5

. Given that this is a polynomial speedup, we expect even stronger speedups in practical applications which involve orders of magnitude larger networks.

Arthur Jacot

Diffusion Model Inference Acceleration Diffusion Sampling Stochastic Differential Equations Diffusion Models