Inference-Time Alignment for Diffusion Models via Variationally Stable Doob's Matching
Organizations: Joint Laboratory of Data Science and Business Intelligence, Southwestern University of Finance and Economics, Chengdu, Sichuan 611130, China. · State Key Laboratory of Mathematical Sciences, Academy of Mathematics and Systems Science, Chinese Academy of Sciences, Beijing 100190, China · Institut für Geometrie und Praktische Mathematik, RWTH Aachen University, Im Süsterfeld 2, 52072 Aachen,Germany. · School of Artificial Intelligence, Wuhan University, Wuhan, Hubei 430072, China. ·
Abstract
Inference-time alignment for diffusion models aims to adapt a pre-trained reference diffusion model toward a target distribution without retraining the reference score network, thereby preserving the generative capacity of the reference model while enforcing desired properties at inference time. A central mechanism for achieving such alignment is guidance, which modifies the sampling dynamics through an additional drift term. In this work, we introduce variationally stable Doob's matching, a novel framework for provable guidance estimation grounded in Doob's -transform. Our approach formulates guidance as the gradient of the logarithm of an underlying Doob's -function and employs gradient-regularized regression to simultaneously estimate both the -function and its gradient, resulting in a consistent estimator of the guidance. Theoretically, we establish non-asymptotic convergence rates for the estimated guidance. Moreover, we analyze the resulting controllable diffusion processes and prove non-asymptotic convergence guarantees for the generated distributions in the Wasserstein-2 distance. Finally, we show that variationally stable guidance estimators are adaptive to unknown low dimensionality, effectively mitigating the curse of dimensionality under low-dimensional linear subspace assumptions. Numerical experiments demonstrate the effectiveness of our proposed method and show the advantages compared to the baselines.
Figures & tables
| Grid | Ring | |||
| Method | Mean reward | Mean reward | ||
| Pre-trained model | ||||
| RL ( Fan et al., 2023 ; Black et al., 2024 ) | ||||
| Direct backpropagation ( Clark et al., 2024 ) | ||||
| Approximate guidance ( Chung et al., 2023 ) | ||||
| DAS ( Kim et al., 2025 ) | ||||
| Task | Method | PSNR | SSIM | LPIPS |
| Gaussian Deblur | DPS | 25.54 | 0.757 | 0.155 |
| Ours | 27.35 | 0.833 | 0.112 | |
| Motion Deblur | DPS | 19.85 | 0.495 | 0.239 |
| Ours | 20.60 | 0.540 | 0.213 | |
| Inpainting ( ) | DPS | 21.90 | 0.761 | 0.161 |
| Ours | 22.50 | 0.835 | 0.104 |
Diffusion Model Alignment
Diffusion model alignment steers generated content and internal representations toward prompts, human preferences, safety goals, and task constraints.
See all 194 papers