Training-Free Adaptation of Diffusion Models via Doob's $h$-Transform

Qijie Zhu; Zeqi Ye; Han Liu; Zhaoran Wang; Minshuo Chen

cs.LGFeb 18, 2026

Training-Free Adaptation of Diffusion Models via Doob's $h$ -Transform

Authors: Qijie Zhu, Zeqi Ye, Han Liu, Zhaoran Wang, Minshuo Chen

Organizations: Department of Statistics and Data Science, Northwestern University. · Department of Industrial Engineering and Management Sciences, Northwestern University. · Department of Computer Science, Northwestern University.

Abstract

Adaptation methods have been a workhorse for unlocking the transformative power of pre-trained diffusion models in diverse applications. Existing approaches often abstract adaptation objectives as a reward function and steer diffusion models to generate high-reward samples. However, these approaches can incur high computational overhead due to additional training, or rely on stringent assumptions on the reward such as differentiability. Moreover, despite their empirical success, theoretical justification and guarantees are seldom established. In this paper, we propose DOIT (Doob-Oriented Inference-time Transformation), a training-free and computationally efficient adaptation method that applies to generic, non-differentiable rewards. The key framework underlying our method is a measure transport formulation that seeks to transport the pre-trained generative distribution to a high-reward target distribution. We leverage Doob's $h$ -transform to realize this transport, which induces a dynamic correction to the diffusion sampling process and enables efficient simulation-based computation without modifying the pre-trained model. Theoretically, we establish a high probability convergence guarantee to the target high-reward distribution via characterizing the approximation error in the dynamic Doob's correction. Empirically, on D4RL offline RL benchmarks, our method consistently outperforms state-of-the-art baselines while preserving sampling efficiency.

Figures & tables

Figure 1: DOIT : At each $t_{l}$ , we simulate $M$ trajectories (here, $M=3$ ) starting from $x_{t_{l}}$ to approximate $\nabla\log h(x_{t_{l}},t_{l})$ via ( 9 ), then utilize it to modify the sampling dynamics.

Figure 2: Violin plots of aesthetic scores for the samples generated by Stable Diffusion v1.5 , comparing the vanilla generation result and applying DOIT across different

(\tau,\gamma)

settings. The blue bars indicate the minimum and maximum scores, the orange bars represent the first & third quantiles, and the red marker denotes the mean.

	Min	Mean	Max	Runtime
Pre-trained	5.737 $\pm$ 0.102	6.372 $\pm$ 0.031	7.052 $\pm$ 0.085	1.260 $\pm$ 0.046
Surrogate	5.987 $\pm$ 0.180	6.726 $\pm$ 0.072	7.501 $\pm$ 0.055	1.712 $\pm$ 0.036
Simulation	6.006 $\pm$ 0.099	6.714 $\pm$ 0.055	7.553 $\pm$ 0.113	39.584 $\pm$ 0.142

Table 1: Comparison of aesthetic score statistics between the surrogate and full trajectory simulation methods. Runtime (seconds) is reported as the generation time per image, excluding reward evaluation time.

$K$	BoK	BoK + DOIT (Ours)	FK-Steering	DAS	TreeG	SVDD	BFS	BFS + DOIT (Ours)
4	$0.702\pm 0.057$	$0.875\pm 0.017$	$0.743\pm 0.037$	$0.878\pm 0.028$	$0.860\pm 0.033$	$0.667\pm 0.076$	$0.882\pm 0.029$	$0.950\pm 0.016$
8	$0.896\pm 0.031$	$1.001\pm 0.015$	$0.926\pm 0.042$	$1.052\pm 0.033$	$1.023\pm 0.018$	$0.775\pm 0.087$	$1.087\pm 0.031$	$1.115\pm 0.009$

Table 2: Performance comparison on ImageReward.

K

represents the number of particles. Combining DOIT with both BoK and BFS provides a significant boost.

Dataset	Environment	IQL ( $\clubsuit$ )	Diffuser ( $\clubsuit$ )	D-QL ( $\clubsuit$ )	QGPO ( $\clubsuit$ )	TFG ( $\spadesuit$ )	DAS ( $\spadesuit$ )	TTS ( $\spadesuit$ )	DOIT ( $\spadesuit$ ) (Ours)
Medium-Expert	HalfCheetah	86.7	79.8	96.1	93.5	$90.2\pm 0.2$	$93.3\pm 0.3$	${\bf 93.9}\pm 0.3$	${\bf 93.9}\pm 0.5$
Medium-Expert	Hopper	91.5	107.2	110.7	108.0	$100.2\pm 3.5$	$105.4\pm 5.1$	$104.4\pm 3.1$	${\bf 107.7}\pm 5.6$
Medium-Expert	Walker2d	109.6	108.4	109.7	110.7	$108.1\pm 0.1$	$111.4\pm 0.1$	$111.4\pm 0.1$	${\bf 113.2}\pm 0.7$
Medium	HalfCheetah	47.4	44.2	50.6	54.1	$53.1\pm 0.1$	$53.4\pm 0.1$	$54.8\pm 0.1$	${\bf 55.3}\pm 0.3$
Medium	Hopper	66.3	58.5	82.4	98.0	$96.2\pm 0.5$	$71.3\pm 2.7$	$99.5\pm 1.7$	${\bf 101.0}\pm 0.3$
Medium	Walker2d	78.3	79.7	85.1	86.0	$83.2\pm 1.4$	$83.9\pm 0.9$	$86.5\pm 0.2$	${\bf 86.9}\pm 0.3$

Table 3: Performance comparison of different methods on D4RL locomotion tasks. Training-time methods are marked with

\clubsuit

and inference-time methods are marked with

\spadesuit

. For each task, the best-performing inference-time method has its mean highlighted in bold.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 3: Comparison of dog images generated by Stable Diffusion v1.5 . The upper row displays samples from vanilla generation process, while the bottom row shows samples guided by DOIT using the aesthetic score as the reward.

Doob $\gamma$ $\backslash$ $\tau$	0.2	0.3	0.4	0.5
0.0	5.688 ± 0.142	5.688 ± 0.142	5.688 ± 0.142	5.688 ± 0.142
1.0	5.728 ± 0.242	5.763 ± 0.116	5.733 ± 0.128	5.736 ± 0.137
2.0	5.646 ± 0.417	5.740 ± 0.238	5.752 ± 0.152	5.741 ± 0.140
3.0	5.782 ± 0.414	5.856 ± 0.227	5.760 ± 0.245	5.791 ± 0.167
4.0	5.665 ± 0.345	5.757 ± 0.256	5.804 ± 0.218	5.794 ± 0.185
5.0	5.004 ± 0.316	5.821 ± 0.226	5.877 ± 0.195	5.836 ± 0.263

Appendix

Table 4: Minimum aesthetic score.

Doob $\gamma$ $\backslash$ $\tau$	0.2	0.3	0.4	0.5
0.0	6.157 ± 0.041	6.157 ± 0.041	6.157 ± 0.041	6.157 ± 0.041
1.0	6.200 ± 0.050	6.199 ± 0.038	6.193 ± 0.038	6.188 ± 0.034
2.0	6.270 ± 0.042	6.216 ± 0.040	6.209 ± 0.031	6.208 ± 0.026
3.0	6.368 ± 0.019	6.282 ± 0.022	6.244 ± 0.032	6.235 ± 0.037
4.0	6.433 ± 0.028	6.323 ± 0.017	6.272 ± 0.024	6.265 ± 0.031
5.0	6.252 ± 0.101	6.393 ± 0.008	6.311 ± 0.011	6.282 ± 0.015

Appendix

Table 5: First quartile (Q1) of aesthetic score.

Doob $\gamma$ $\backslash$ $\tau$	0.2	0.3	0.4	0.5
0.0	6.357 ± 0.029	6.357 ± 0.029	6.357 ± 0.029	6.357 ± 0.029
1.0	6.452 ± 0.025	6.421 ± 0.025	6.403 ± 0.024	6.396 ± 0.026
2.0	6.525 ± 0.033	6.472 ± 0.023	6.453 ± 0.024	6.434 ± 0.024
3.0	6.600 ± 0.037	6.537 ± 0.028	6.490 ± 0.018	6.470 ± 0.023
4.0	6.645 ± 0.022	6.576 ± 0.033	6.520 ± 0.028	6.498 ± 0.018
5.0	6.476 ± 0.097	6.635 ± 0.036	6.569 ± 0.035	6.527 ± 0.023

Appendix

Table 6: Mean aesthetic score.

Doob $\gamma$ $\backslash$ $\tau$	0.2	0.3	0.4	0.5
0.0	6.560 ± 0.037	6.560 ± 0.037	6.560 ± 0.037	6.560 ± 0.037
1.0	6.674 ± 0.027	6.635 ± 0.031	6.617 ± 0.030	6.596 ± 0.047
2.0	6.787 ± 0.013	6.676 ± 0.023	6.655 ± 0.048	6.639 ± 0.051
3.0	6.798 ± 0.080	6.736 ± 0.039	6.691 ± 0.031	6.679 ± 0.057
4.0	6.852 ± 0.065	6.846 ± 0.070	6.728 ± 0.041	6.691 ± 0.035
5.0	6.810 ± 0.054	6.857 ± 0.028	6.831 ± 0.054	6.719 ± 0.038

Appendix

Table 7: Third quartile (Q3) of aesthetic score.

Doob $\gamma$ $\backslash$ $\tau$	0.2	0.3	0.4	0.5
0.0	7.028 ± 0.091	7.028 ± 0.091	7.028 ± 0.091	7.028 ± 0.091
1.0	7.161 ± 0.139	7.125 ± 0.148	7.085 ± 0.111	7.078 ± 0.115
2.0	7.291 ± 0.219	7.197 ± 0.185	7.164 ± 0.161	7.111 ± 0.165
3.0	7.363 ± 0.192	7.296 ± 0.147	7.185 ± 0.169	7.183 ± 0.179
4.0	7.292 ± 0.073	7.338 ± 0.187	7.251 ± 0.112	7.241 ± 0.156
5.0	7.299 ± 0.125	7.385 ± 0.170	7.323 ± 0.133	7.288 ± 0.162

Appendix

Table 8: Maximum aesthetic score.

Dataset	Environment	$\bm{\eta}$	$\bm{\gamma}$	$\bm{\tau}$	$\bm{t^{\star}}$	$\bm{K}$
Medium-Expert	HalfCheetah	0.7	0.25	0.5	8	4
Medium-Expert	Hopper	0.8	0.25	0.7	4	4
Medium-Expert	Walker2d	0.4	0.5	0.4	10	8
Medium	HalfCheetah	0.2	0.25	0.5	10	4
Medium	Hopper	0.6	0.75	0.4	8	8
Medium	Walker2d	0.3	0.5	0.3	10	4

Appendix

Table 9: Hyperparameter settings for our method across datasets and environments.

\eta

denotes the DDIM stochasticity parameter,

\gamma

the Doob correction scale,

\tau

the temperature,

t^{\star}

the correction start timestep, and

K

the number of candidate samples selected via best-of-

K

Explore similar work

D-DOIT: Training-free Adaptation of Discrete Diffusion via Doob's h-Transform
Oct 4, 2026Jieke Wu, Qijie Zhu, Weimin Wu +3Diffusion Model Guidance Masked Diffusion Models
Department of Computer Science, King Abdullah University of Science and Technology · Department of Statistics and Data Science, Northwestern University · Center for Foundation Models and Generative AI, Northwestern University +1
DOHF: Online Diffusion Fine-tuning with Doob's $h$ -transform Guidance
Sep 25, 2026Zhengyi Guo, Jiayuan Sheng, Wenpin Tang +1Diffusion Model Alignment Diffusion Model Guidance
Source-Prior-Driven Selective Adaptation for Efficient Diffusion Model Finetuning
Jul 23, 2026Yi Xiong, Yuan-Yuan Cheng, Xiao-Ming FuFine-Tuning Diffusion Model Fine-Tuning
University of Science and Technology of China, China

Oct 4, 2026J/K move · Enter open · S save

D-DOIT: Training-free Adaptation of Discrete Diffusion via Doob's h-Transform

Jieke Wu, Qijie Zhu, Weimin Wu, Zeqi Ye+2

Department of Computer Science, King Abdullah University of Science and Technology · Department of Statistics and Data Science, Northwestern University · Center for Foundation Models and Generative AI, Northwestern University+1

Diffusion Model Guidance Masked Diffusion Models Doob H-Transform

Open paper AskPDF arXiv

We propose D-DOIT (Discrete Doob-Oriented Inference-time Transformation), a training-free and efficient adaptation method for discrete diffusion models with generic rewards. D-DOIT formulates adaptation as sampling from a reward-tilted target distribution and realizes this transport through Doob's h-transform of the discrete diffusion reverse kernel, using only reward values rather than reward gradients. Unlike continuous diffusion, masked discrete diffusion samples categorical token-reveal transitions rather than continuous state updates. D-DOIT derives the corresponding discrete Doob's h-transform, which guides sampling by reweighting reverse transition probabilities instead of adding a drift correction. To make this transformation practical, D-DOIT avoids expensive future rollouts. At each guided step, D-DOIT samples candidate next states, uses the model prediction head to complete each candidate into a clean sequence, evaluates each completion with the reward oracle, and resamples the next state with probabilities proportional to the rewards. An optional late-stage best-of-K refinement further improves sample quality by branching trajectories only near the end of denoising, avoiding the

K

-fold cost over the full trajectory. Empirically, across regulatory DNA design and protein inverse folding benchmarks, D-DOIT outperforms training-free guidance baselines. It improves enhancer activity and cell-type specificity while preserving sequence naturalness, and achieves the highest success rate in protein inverse folding.

Abstract

Explore similar work

D-DOIT: Training-free Adaptation of Discrete Diffusion via Doob's h-Transform

DOHF: Online Diffusion Fine-tuning with Doob's hhh-transform Guidance

Source-Prior-Driven Selective Adaptation for Efficient Diffusion Model Finetuning

DOHF: Online Diffusion Fine-tuning with Doob's $h$ -transform Guidance