Doob H-Transform

Follow this topic Ask about this topic

A Doob H-transform changes the dynamics of a stochastic process by weighting states according to how likely they are to lead to a desired outcome. In diffusion and flow models, it provides a way to guide generation toward rewards, constraints, or source-consistent edits, including when rewards are non-differentiable or the model cannot be retrained. Approaches include estimating the guiding function from samples, adjusting reverse transitions in discrete diffusion, and distilling guidance into a model; related bridge methods soften endpoint constraints for greater stability. A central challenge is that approximate guidance can distort sampling or miss high-reward modes, so work continues to balance fidelity, diversity, computational cost, and guarantees on approximation error.

Latest papers

13 papers3 papers in the last four weeks, against 1 the four weeks before. 0.0% of all new papers.

Doob-Based Generative Policy Optimization
Oct 20, 2026Maojiang Su, Qijie Zhu, Yu Xiao +4Policy Optimization RL for Diffusion Models
Department of Computer Science, Northwestern University · Center for Foundation Models and Generative AI, Northwestern University · Department of Statistics and Data Science, Northwestern University +1
D-DOIT: Training-free Adaptation of Discrete Diffusion via Doob's h-Transform
Oct 4, 2026Jieke Wu, Qijie Zhu, Weimin Wu +3Diffusion Model Adaptation Diffusion Model Guidance
Department of Computer Science, King Abdullah University of Science and Technology · Department of Statistics and Data Science, Northwestern University · Center for Foundation Models and Generative AI, Northwestern University Department of Computer Science, Northwestern University +1
WEIRDO: WEak resIdual Regularized DOob's h-transform diffusion alignment
Sep 30, 2026Denis SuchkovDiffusion Model Alignment Doob H-Transform
HSE University, Russian Federation
DOHF: Online Diffusion Fine-tuning with Doob's $h$ -transform Guidance
Sep 25, 2026Zhengyi Guo, Jiayuan Sheng, Wenpin Tang +1Diffusion Model Alignment Diffusion Model Guidance
SDDBMs: Soft Denoising Diffusion Bridge Models
Aug 9, 2026Shiyi Qi, Kun He, Mingmou LiuDiffusion Models for Image Restoration Doob H-Transform
Nanjing University · Renmin University of China
h-Flow: Flexible Flow-based Image Editing via Doob's h-Transform
Jul 12, 2026Zehui Guo, Zhen Wang, Junwei Shu +3Doob H-Transform Text-Guided Image Editing
East China Normal University, Shanghai, China · The Hong Kong University of Science and Technology, Hong Kong SAR, China
Intrinsic-Noise Consolidation: A Doob-Barrier-Conditioned Diffusion Turns Analog Device Noise into a Continual-Learning Resource
Jul 8, 2026Gunner Levi HoweContinual Learning Stability-Plasticity Dilemma
Variance-Tilted Diffusion Models for Diverse Sampling
Jun 20, 2026Iskander Azangulov, Leo Zhang, Kianoosh AshouritaklimiDoob H-Transform Diffusion Model Sampling
Department of Statistics, University of Oxford, Oxford, UK.
Are we really tilting? The mechanics of reward guidance in flow and diffusion models
Jun 1, 2026Sanjit Dandapanthula, Nicholas M. BoffiReward Hacking Diffusion Model Guidance
Carnegie Mellon University
Free energy Estimation on Any State Space
May 29, 2026Jiajun He, Zijing Ou, Francisco Vargas +4Free Energy Calculation Doob H-Transform
University of Cambridge · Imperial College London · Xaira Therapeutics +1
Future Validity is the Missing Statistic: From Impossibility to $Φ$ -Estimation for Grammar-Faithful Speculative Decoding
May 8, 2026Wenhua Nie, Zijie Meng, Kun Zou +5Doob H-Transform Speculative Decoding
National Taiwan University
DPRM: A Plug-in Doob h transform-induced Token-Ordering Module for Diffusion Language Models
Apr 27, 2026Dake Bu, Wei Huang, Andi Han +4Doob H-Transform Process Reward Models
City University of Hong Kong · CFAR and IHPC, A*STAR · RIKEN AIP +4
Training-Free Adaptation of Diffusion Models via Doob's $h$ -Transform
Feb 18, 2026Qijie Zhu, Zeqi Ye, Han Liu +2Diffusion Model Adaptation Diffusion Models
Department of Statistics and Data Science, Northwestern University. · Department of Industrial Engineering and Management Sciences, Northwestern University. · Department of Computer Science, Northwestern University.

Oct 20, 2026J/K move · Enter open · S save

Doob-Based Generative Policy Optimization

Maojiang Su, Qijie Zhu, Yu Xiao, Shuyang Yu+3

Department of Computer Science, Northwestern University · Center for Foundation Models and Generative AI, Northwestern University · Department of Statistics and Data Science, Northwestern University+1

Policy Optimization RL for Diffusion Models RL for Flow Matching Models Robotic Control Doob H-Transform

Open paper AskPDF arXiv

Diffusion and flow models provide expressive policy classes for robotic and sequential decision-making systems. However, their action likelihoods are generally intractable, limiting the direct use of widely adopted policy optimization methods such as PPO and GRPO. We propose Doob Policy Optimization (DoobPO), a framework that leverages Doob's

h

-transform for likelihood-free reinforcement learning fine-tuning of generative policies. DoobPO separates policy improvement from generative policy learning. It first learns a lightweight ratio model that represents the desired policy update, and then distills the resulting ratio-tilted target policy into the generative policy via conditional drift matching. This formulation applies uniformly to both diffusion and flow policies and supports a broad family of policy improvement rules, including PPO and GRPO. We prove that the practical conditional drift matching objective is gradient-equivalent to the ideal Doob drift matching objective. We further show that the induced diffusion process solves an endpoint-constrained stochastic optimal control problem. This establishes that it achieves the minimum-energy modification required to realize the desired policy tilt. Across MuJoCo continuous-control benchmarks, DoobPO achieves strong performance against competitive diffusion- and flow-policy baselines, showing a principled and practical route to RL fine-tuning for generative policies.