stat.MLJan 10, 2026

Inference-Time Alignment for Diffusion Models via Variationally Stable Doob's Matching

Authors: Jinyuan Chang, Chenguang Duan, Yuling Jiao, Ruoxuan Li, Yi Xu, Jerry Zhijian Yang

Organizations: Joint Laboratory of Data Science and Business Intelligence, Southwestern University of Finance and Economics, Chengdu, Sichuan 611130, China. · State Key Laboratory of Mathematical Sciences, Academy of Mathematics and Systems Science, Chinese Academy of Sciences, Beijing 100190, China · Institut für Geometrie und Praktische Mathematik, RWTH Aachen University, Im Süsterfeld 2, 52072 Aachen,Germany. · School of Artificial Intelligence, Wuhan University, Wuhan, Hubei 430072, China. · 

Abstract

Inference-time alignment for diffusion models aims to adapt a pre-trained reference diffusion model toward a target distribution without retraining the reference score network, thereby preserving the generative capacity of the reference model while enforcing desired properties at inference time. A central mechanism for achieving such alignment is guidance, which modifies the sampling dynamics through an additional drift term. In this work, we introduce variationally stable Doob's matching, a novel framework for provable guidance estimation grounded in Doob's hh-transform. Our approach formulates guidance as the gradient of the logarithm of an underlying Doob's hh-function and employs gradient-regularized regression to simultaneously estimate both the hh-function and its gradient, resulting in a consistent estimator of the guidance. Theoretically, we establish non-asymptotic convergence rates for the estimated guidance. Moreover, we analyze the resulting controllable diffusion processes and prove non-asymptotic convergence guarantees for the generated distributions in the Wasserstein-2 distance. Finally, we show that variationally stable guidance estimators are adaptive to unknown low dimensionality, effectively mitigating the curse of dimensionality under low-dimensional linear subspace assumptions. Numerical experiments demonstrate the effectiveness of our proposed method and show the advantages compared to the baselines.

Figures & tables

Explore similar work

Sep 30, 2026

Steepest Guidance: A Practical and Principled Approach to Inference-Time Alignment of Flow and Diffusion-based Models

Inference-time alignment of flow and diffusion-based models is critical for achieving flexible generative modeling. Theoretically, Doob's hh-transform provides an elegant solution to this problem, and most existing methods are based on this principle. However, in practice, estimating the optimal guidance derived from Doob's hh-transform at inference time is challenging. To deal with this issue, we regard inference-time alignment as a sequential optimization problem in the space of probability measures and propose a novel framework called Steepest Guidance, based on the principle of maximizing local improvement in the objective. We provide a theoretical analysis of the proposed method and demonstrate its effectiveness through extensive experiments.
Sep 30, 2026

WEIRDO: WEak resIdual Regularized DOob's h-transform diffusion alignment

We study the problem of estimating the guidance that steers the distribution learned by a diffusion generative model toward a tilted target q0∝w p0q_0 \propto w\,p_0 at inference time. Relying on the stochastic optimal control approach, we observe that the exact drift correction is the gradient of the logarithm of Doob's hh-function, and we study the problem of estimating it from a sample. In the present paper, we assume that the score of the pretrained model is available, that the tilting weight is bounded and positive, and that the reference distribution has a bounded support, no smoothness of the weight is required. Introducing a penalized least-squares risk in which the penalty is the residual of the space-time harmonicity equation satisfied by the hh-function, measured in a dual Sobolev norm, we derive high-probability bounds on the squared error of the resulting guidance estimate. Since the penalty vanishes at the target, the estimator is free of regularization bias, and in favourable scenarios its rate of convergence is faster than the minimax rate of estimating first-order derivatives of a smooth regression function. Assuming that ww is bounded and positive with Ep0[w−s]<∞\mathbb{E}_{p_0}[w^{-\mathrm{s}}] < \infty for some s∈(0,∞]\mathrm{s} \in (0,\infty], and that the reference data are compactly supported, we prove that the guidance is estimable in squared L2L^2 at rate εns/(s+4)\varepsilon_n^{\mathrm{s}/(\mathrm{s}+4)}, where εn=n−2(β−1)/(2(β−1)+d).\varepsilon_n = n^{-2(β-1)/(2(β-1)+d)}. We also transfer the obtained bounds to the total variation distance between the marginals of the estimated and the exactly guided samplers, and illustrate the performance of the suggested approach with numerical experiments.
Feb 18, 2026

Training-Free Adaptation of Diffusion Models via Doob's hh-Transform

Adaptation methods have been a workhorse for unlocking the transformative power of pre-trained diffusion models in diverse applications. Existing approaches often abstract adaptation objectives as a reward function and steer diffusion models to generate high-reward samples. However, these approaches can incur high computational overhead due to additional training, or rely on stringent assumptions on the reward such as differentiability. Moreover, despite their empirical success, theoretical justification and guarantees are seldom established. In this paper, we propose DOIT (Doob-Oriented Inference-time Transformation), a training-free and computationally efficient adaptation method that applies to generic, non-differentiable rewards. The key framework underlying our method is a measure transport formulation that seeks to transport the pre-trained generative distribution to a high-reward target distribution. We leverage Doob's hh-transform to realize this transport, which induces a dynamic correction to the diffusion sampling process and enables efficient simulation-based computation without modifying the pre-trained model. Theoretically, we establish a high probability convergence guarantee to the target high-reward distribution via characterizing the approximation error in the dynamic Doob's correction. Empirically, on D4RL offline RL benchmarks, our method consistently outperforms state-of-the-art baselines while preserving sampling efficiency.