VaLoP: Value Gradient Lookahead and Pullback for Reward-Guided Diffusion

Kwanyoung Kim

cs.AIOct 26, 2026

VaLoP: Value Gradient Lookahead and Pullback for Reward-Guided Diffusion

Authors: Kwanyoung Kim

Abstract

Reward guidance steers a pretrained diffusion or flow model toward images or actions that a reward model prefers, at sampling time and without retraining the model for each reward. It follows the gradient of a value function, the expected reward given the noisy state. The two common estimates of this gradient, gradients through a one-step denoised estimate and learned value models, are least accurate at high noise, where the coarse structure of the sample is decided. Interestingly, we find that both errors are posterior covariances, which grow with the uncertainty about the clean sample, and hence at higher noise. To reduce them, we evaluate the value at a less noisy state, one sampler step ahead, where this uncertainty is smaller, and pull its gradient back to the current state. This lookahead places both estimators at the two ends of one family and reveals that an intermediate lookahead requires one additional term, a correction through the generator. We therefore propose Value Gradient Lookahead and Pullback (VaLoP), which computes this correction with one vector-Jacobian product, and VaLoP-Fast, which amortizes it with a small head that, for text-to-image models, is trained with a single reward and reused for the others. Across various models, rewards, and benchmark datasets, both robustly outperform existing value-gradient guidance on the target reward, VaLoP-Fast at about the cost of learned-value guidance. Both extend to critic-guided robot policies, where VaLoP-Fast attains the highest success rates on LIBERO and RoboCasa.

Figures & tables

Figure 2: Comparison and pipeline overview. (a) Where the value is read: DPS [ 6 ] at the denoised estimate (

s=0

), StitchVM [ 19 ] at

{\bm{z}}_{t}

(

s=t

), and VaLoP in between, at a lookahead state

{\bm{z}}_{s}

(❶), whose gradient (❷) it pulls back (❸) into the guidance

{\bm{g}}^{\mathrm{VaLoP}}_{t}

toward the rewarded mode (❹). AGF [ 25 ] learns the guidance with a head trained by adjoint rollouts. (b) Per-step computation and training. VaLoP needs no extra training, and VaLoP-Fast amortizes its correction with one head for all rewards, avoiding the generator backward pass.

Figure 3: Value-gradient error at high noise. (a, b) Mixture with a misspecified value model and known exact value: samples at the best strength (a) and cosine to the exact gradient (b, VaLoP: $\nabla V_{\psi}$ at ${\bm{z}}_{s}$ ). (c) HPSv2 gain per guided step on SD3.5-Medium (mean $\pm$ s.d.).

	SD3.5-Medium						SD3.5-Large						FLUX.1-dev
Method	HPSv2 $\scriptstyle\uparrow$	Aes. $\scriptstyle\uparrow$	CLIP $\scriptstyle\uparrow$	Pick $\scriptstyle\uparrow$	IR $\scriptstyle\uparrow$	GenE. $\scriptstyle\uparrow$	HPSv2 $\scriptstyle\uparrow$	Aes. $\scriptstyle\uparrow$	CLIP $\scriptstyle\uparrow$	Pick $\scriptstyle\uparrow$	IR $\scriptstyle\uparrow$	GenE. $\scriptstyle\uparrow$	HPSv2 $\scriptstyle\uparrow$	Aes. $\scriptstyle\uparrow$	CLIP $\scriptstyle\uparrow$	Pick $\scriptstyle\uparrow$	IR $\scriptstyle\uparrow$	GenE. $\scriptstyle\uparrow$
Unguided	0.286	5.347	28.80	22.65	0.987	0.694	0.294	5.461	28.61	22.87	1.065	0.721	0.298	5.729	27.71	22.79	0.995	0.660
HPSv2 reward	✓						✓						✓
DPS	0.361	5.426	28.99	23.54	1.130	0.711	0.349	5.519	28.82	23.55	1.111	0.715	0.320	5.440	27.09	22.78	0.922	0.633
DPS + StitchVM	0.366	5.541	28.77	23.28	1.302	0.647	0.360	5.544	28.65	23.20	1.248	0.662	0.314	5.690	27.82	22.79	1.080	0.652
VaLoP (Ours)	0.382	5.654	29.13	23.77	1.332	0.726	0.371	5.633	29.24	23.78	1.290	0.755	0.328	5.590	28.36	23.12	1.190	0.701

Table 1: Reward guidance on DrawBench and GenEval (

1024^{2}

, strengths in Sec. D.2 ). Rows are grouped by the guided reward, whose column is marked by ✓ , and the other columns are held-out metrics. DrawBench metrics average three seeds and GenE., the GenEval overall score, averages four (three for DPS, s.d. in Sec. E.1 ). Bold: best guided result per target and model.

Success (%) $\uparrow$	LIBERO				RoboCasa
Method	Goal	Spatial	LIBERO-10	Avg.	Atomic
Base	76.4	64.4	43.4	61.4	44.6
Q-BoN	78.2	69.6	41.2	63.0	46.7
QGF	79.0	69.4	48.6	65.7	47.1
QDPS	80.2	68.2	47.6	65.3	45.7
AGF	80.0	69.2	48.0	65.7	45.9

Table 2: Critic guidance on robot policies. Success rate (%) of SmolVLA on LIBERO and

\pi_{0.5}

on RoboCasa, averaged over the tasks of each suite. Bold: best. Rollouts in Sec. E.5 .

	SD3.5-Medium		SD3.5-Large		FLUX.1-dev
Method	Time $\downarrow$	Mem. $\downarrow$	Time $\downarrow$	Mem. $\downarrow$	Time $\downarrow$	Mem. $\downarrow$
DPS	54.5	43.9	83.5	66.6	43.6	41.7
DPS + StitchVM	17.8	18.9	36.8	34.9	9.9	32.2
VaLoP (Ours)	30.4	24.6	61.6	52.2	28.6	33.8
VaLoP-Fast (Ours)	15.4	18.6	31.1	34.7	8.6	31.0
	Time per robot action chunk (s) $\downarrow$

Table 3: Cost on one H100: time (s) and peak memory (GB) per image at

1024^{2}

, and time (s) per robot action chunk. Bold: lowest.

	HPSv2 $\uparrow$	Aes. $\uparrow$	CLIP $\uparrow$	Pick $\uparrow$	IR $\uparrow$
VaLoP (Ours)	0.382	5.665	29.06	23.79	1.339
Lookahead and correction
No lookahead ( $s=t$ )	0.358	5.505	28.63	23.28	1.322
No correction ( ${\bm{g}}_{s}$ only)	0.365	5.614	28.23	23.52	1.324
Form of the correction
$J$ instead of $J^{\!\top}$	0.368	5.598	28.25	23.59	1.318

Table 4: Lookahead and correction (SD3.5-Medium,

1024^{2}

, DrawBench, HPSv2 reward,

\eta=300

, one seed). Bold: best.

	Goal	Spatial	LIBERO-10
No lookahead ( $s=t$ )	79.0	69.0	47.4
No correction ( ${\bm{g}}_{s}$ only)	76.6	67.0	46.4
VaLoP (Ours)	80.6	69.4	49.0

Table 5: Lookahead and correction on SmolVLA (success %), all with the critic at the denoised action. Bold: best.

Figure 4: Lookahead ratio (SD3.5-Medium, HPSv2 reward, DrawBench at

1024^{2}

), relative to learned-value guidance at

s=t

Appendix figures & tables22 assets

Supplementary material from the paper’s appendix.

Appendix

	Value	Backprop		One for
Method	at	per step	Training	all rewards
Training-free
DPS, FreeDoM [ 6 , 73 ]	$\hat{\bm{z}}_{0\mid t}$	G, D, R	–	–
QGF, FlowChef [ 75 , 44 ]	$\hat{\bm{z}}_{0\mid t}$	D, R ( $D_{t}{=}I$ )	–	–
DOODL, D-Flow [ 64 , 2 ]	${\bm{z}}_{0}$	G (all steps), D, R	–	–
Learned

Appendix

Table 6: Evaluation state and cost of each estimator. G: generator, D: decoder, R: reward, V: value model, H: head, F: flow map.

Method	$\eta$	HPSv2	Aes.	CLIP	Pick	IR
Unguided	–	0.302	5.967	29.02	22.58	1.126
	200	0.402	6.102	28.97	23.69	1.449
	400	0.417	5.939	28.64	23.56	1.408
DPS + StitchVM	800	0.417	5.789	27.76	23.16	1.265
	150	0.428	6.127	29.40	24.27	1.526
	200	0.434	6.128	29.42	24.21	1.510

Appendix

Table 7: Guidance strength (SD3.5-Medium,

512^{2}

, HPSv2 reward, HPDv2 test prompts). Bold: best guided result per column.

	SD3.5-M			SD3.5-L			FLUX.1-dev
Method	H	A	C	H	A	C	H	A	C
DPS	100	0.3	0.6	50	0.3	0.3	200	0.3	0.6
StitchVM	400	12	75	600	8	45	400	16	100
VaLoP	300	6	37.5	300	4	22.5	800	16	100
VaLoP-Fast	450	6	37.5	450	8	45	800	16	100

Appendix

Table 8: Guidance strengths in Tab. 1 for the HPSv2 (H), aesthetic (A), and CLIP (C) rewards.

	SD3.5-Medium	SD3.5-Large	FLUX.1-dev
Parameters	0.41B	0.87B	1.39B
States at $512^{2}$	20,032 (200)	10,240 (200)	10,240 (400)
States at $1024^{2}$	7,680 (200)	1,280 (300)	2,560 (800)
Steps at $512^{2}$	30k (16)	15k (8)	20k (8)
Steps at $1024^{2}$	8k (8)	3k (4)	3k (4)

Appendix

Table 9: Correction head : parameters, distillation states (strength

\eta

), and training steps (batch).

	Goal					Spatial					LIBERO-10
$w$	0.25	0.5	1	2	4	0.25	0.5	1	2	4	0.25	0.5	1	2	4
QGF	76.4	76.2	78.6	79.0	78.6	67.6	65.0	68.2	68.8	69.4	46.4	44.4	44.2	45.8	48.6
QDPS	77.8	79.4	78.8	74.8	80.2	65.8	65.0	66.6	68.2	68.0	43.6	46.0	41.6	47.6	47.4
AGF	77.4	80.0	77.6	77.6	80.0	64.6	64.8	65.6	69.2	68.0	46.0	46.0	48.0	44.8	45.2
VaLoP	79.2	78.0	80.4	79.0	80.6	66.6	65.0	65.8	69.4	68.2	45.4	47.4	45.2	49.0	45.2
VaLoP-Fast	78.0	77.0	79.0	76.0	81.8	67.6	67.8	66.6	68.6	69.8	44.8	49.4	44.8	44.4	45.8

Appendix

Table 10: LIBERO over the strength grid. Success rate (%) per suite, with the base policy at 76.4, 64.4, and 43.4. Bold: highest, reported in Tab. 2 .

$w$	0.5	1	2	4
QGF	44.8	45.3	44.4	47.1
QDPS	44.0	45.0	45.2	45.7
AGF	44.3	44.9	45.9	42.7
VaLoP	44.4	43.4	46.3	47.4
VaLoP-Fast	44.7	46.8	45.8	47.4

Appendix

Table 11: RoboCasa over the strength grid. Success rate (%) over the 18 atomic tasks, with the base policy at 44.6. Bold: highest, reported in Tab. 2 .

Inference, time (ms) and memory (MB)

Training per task

SmolVLA

\pi_{0.5}

Param.

SmolVLA

\pi_{0.5}

Method

Time

\downarrow

Mem.

\downarrow

Time

\downarrow

Mem.

\downarrow

s/step

\downarrow

\downarrow

\downarrow

\downarrow

\downarrow

Base

448

1,338

228

9,169

–

Q-BoN (

N=8

)

463 (1.03)

1,435 (+97)

250 (1.09)

9,638 (+469)

5.3M / 6.0M

–

QGF

587 (1.31)

1,370 (+32)

303 (1.33)

9,249 (+80)

5.3M / 6.0M

–

QDPS

1,045 (2.33)

1,425 (+87)

633 (2.77)

9,249 (+80)

5.3M / 6.0M

–

Appendix

Table 12: Cost of critic guidance. Inference per action chunk, with the ratio of the time to Base and the added memory in parentheses, and parameters of the guidance module. Training is per task. The critic, trained once per task and shared by all methods (about 1 h on SmolVLA and 1.5 h on

\pi_{0.5}

), is not counted.

Method	HPSv2 $\uparrow$	Aes. $\uparrow$	CLIP $\uparrow$	Pick $\uparrow$	IR $\uparrow$	GenE. $\uparrow$
SD3.5-Medium
Unguided	0.286 $\pm$ 0.001	5.347 $\pm$ 0.024	28.80 $\pm$ 0.09	22.65 $\pm$ 0.06	0.987 $\pm$ 0.032	0.694 $\pm$ 0.017
HPSv2 reward
DPS	0.361 $\pm$ 0.000	5.426 $\pm$ 0.024	28.99 $\pm$ 0.18	23.54 $\pm$ 0.04	1.130 $\pm$ 0.028	0.711 $\pm$ 0.010
DPS + StitchVM	0.366 $\pm$ 0.001	5.541 $\pm$ 0.002	28.77 $\pm$ 0.16	23.28 $\pm$ 0.03	1.302 $\pm$ 0.048	0.647 $\pm$ 0.012
VaLoP (Ours)	0.382 $\pm$ 0.001	5.654 $\pm$ 0.014	29.13 $\pm$ 0.07	23.77 $\pm$ 0.03	1.332 $\pm$ 0.015	0.726 $\pm$ 0.011

Appendix

Table 13: Means and standard deviations of Tab. 1 over three DrawBench seeds and four GenEval seeds (three for DPS).

Setting	Value	HPSv2 $\uparrow$	Aes. $\uparrow$	CLIP $\uparrow$	Pick $\uparrow$	IR $\uparrow$
Unguided		0.287	5.370	28.71	22.69	0.995
VaLoP	$s/t=0.6$ , $\beta=1$ , $\gamma$ schedule	0.382	5.665	29.06	23.79	1.339
Ratio $s/t$	$1$ (StitchVM)	0.358 ( $\mathbf{-13.1}$ )	5.505 ( $-4.8$ )	28.63 ( $-1.9$ )	23.28 ( $-9.6$ )	1.322 ( $-0.5$ )
	$0.8$	0.385 ( $\mathbf{+2.1}$ )	5.623 ( $-1.7$ )	29.36 ( $+1.6$ )	23.79 ( $-0.1$ )	1.386 ( $+1.8$ )
	$0.4$	0.369 ( $\mathbf{-7.8}$ )	5.599 ( $-2.3$ )	29.23 ( $+0.8$ )	23.60 ( $-4.1$ )	1.267 ( $-2.2$ )
	$0.2$	0.359 ( $\mathbf{-11.9}$ )	5.535 ( $-3.7$ )	29.27 ( $+0.8$ )	23.41 ( $-6.4$ )	1.217 ( $-2.9$ )

Appendix

Table 14: Ablation at

1024^{2}

(SD3.5-Medium, DrawBench, HPSv2 reward,

\eta=300

, one seed). Each row changes one setting of VaLoP , and parentheses give the paired

t

-statistic against VaLoP over the 200 images. Bold:

|t|\geq 2

on the target reward.

Table 18

Method	Head	HPSv2 $\uparrow$	Aes. $\uparrow$	CLIP $\uparrow$	Pick $\uparrow$	IR $\uparrow$
CLIP reward
StitchVM, $\eta=75$	–	0.249	4.849	34.49	21.70	0.762
VaLoP-Fast, $\eta=37.5$	own	0.274	5.092	34.95	22.24	0.987
VaLoP-Fast, $\eta=37.5$	HPSv2	0.277	5.138	34.99	22.34	1.011
Aesthetic reward
StitchVM, $\eta=12$	–	0.279	5.970	27.09	22.26	0.799

Appendix

Table 17: One head for all rewards (SD3.5-Medium,

1024^{2}

). Head trained with the HPSv2 value or per reward.

Method	$\eta$	Bwd.	HPSv2 $\uparrow$	Aes. $\uparrow$	CLIP $\uparrow$	Pick $\uparrow$	IR $\uparrow$
DPS + StitchVM	400	0 / 1	0.417	5.939	28.64	23.56	1.408
VaLoP-Fast, computed	200	0 / 2	0.426	6.095	29.31	24.05	1.455
VaLoP-Fast, fitted	200	0 / 1	0.425	6.118	29.36	24.12	1.509
VaLoP	200	1 / 2	0.434	6.128	29.42	24.21	1.510

Appendix

Table 18: Guidance magnitude of VaLoP-Fast (SD3.5-Medium,

512^{2}

, HPDv2 test prompts, HPSv2 reward). Backward passes through the generator / value model.

	SD3.5-Medium					SD3.5-Large					FLUX.1-dev
Method	HPSv2 $\scriptstyle\uparrow$	Aes. $\scriptstyle\uparrow$	CLIP $\scriptstyle\uparrow$	Pick $\scriptstyle\uparrow$	IR $\scriptstyle\uparrow$	HPSv2 $\scriptstyle\uparrow$	Aes. $\scriptstyle\uparrow$	CLIP $\scriptstyle\uparrow$	Pick $\scriptstyle\uparrow$	IR $\scriptstyle\uparrow$	HPSv2 $\scriptstyle\uparrow$	Aes. $\scriptstyle\uparrow$	CLIP $\scriptstyle\uparrow$	Pick $\scriptstyle\uparrow$	IR $\scriptstyle\uparrow$
Unguided	0.302	5.840	27.59	22.17	1.087	0.308	5.940	27.77	22.31	1.130	0.304	6.158	26.70	22.23	1.031
HPSv2 reward	✓					✓					✓
DPS	0.384	5.955	28.25	23.13	1.284	0.368	6.006	28.14	23.10	1.266	0.334	5.930	26.63	22.17	0.953
DPS + StitchVM	0.387	5.967	27.86	22.96	1.393	0.380	5.940	27.82	22.77	1.347	0.325	6.119	26.89	22.21	1.127
VaLoP (Ours)	0.407	6.011	28.42	23.34	1.471	0.393	6.065	28.52	23.33	1.437	0.346	6.102	27.44	22.60	1.268

Appendix

Table 19: Reward guidance on Pick-a-Pic (

1024^{2}

, 499 prompts, one seed). Columns and bold as in Tab. 1 .

Value model	StitchVM	VaLoP
MLP, forward-noised actions	77.8	80.6
Stitched to the policy	77.4	79.6
MLP, policy sampling path	80.2	80.4

Appendix

Table 20: Learned value on SmolVLA (LIBERO-Goal, success %, base 76.4), read at

{\bm{z}}_{t}

(StitchVM) or with the lookahead (VaLoP).

$s/t$	$w=1$	$w=2$	$w=4$	Avg.
0.4	77.0	79.2	80.6	78.9
0.6	80.4	79.0	80.6	80.0
0.8	79.0	78.4	80.0	79.1

Appendix

Table 21: Lookahead ratio of VaLoP on LIBERO-Goal. Success rate (%). Bold: best.

Figure 5: Qualitative comparison on SD3.5-Medium under the target reward of each row. ✓ / ✗ : GenEval correctness.

Figure 6: Qualitative comparison on SD3.5-Large under the target reward of each row. ✓ / ✗ : GenEval correctness.

Figure 7: Qualitative comparison on FLUX.1-dev under the target reward of each row. ✓ / ✗ : GenEval correctness.

Figure 8: Rollouts on LIBERO (SmolVLA) from the same initial state, just before VaLoP-Fast succeeds and at the end of each episode, with bars for the steps to success or the time limit. Only VaLoP-Fast completes the first two tasks, and it is first on the third.

Figure 9: Rollouts on LIBERO , continued. QDPS also completes both tasks, one and five steps earlier than VaLoP-Fast.

Figure 10: Rollouts on RoboCasa ( $\pi_{0.5}$ ), shown as in Fig. 8 . Only VaLoP-Fast opens the drawer and closes the toaster oven door, and only our two variants slide out the rack.

Figure 11: Rollouts on RoboCasa , continued. Several baselines also complete these tasks, but later than VaLoP-Fast.

Diffusion Model Guidance

Diffusion model guidance steers denoising toward desired conditions, rewards, constraints, or away from unwanted concepts.

See all 364 papers

Aug 14, 2026

Designing Reinforcement Learning for Diffusion Models: A Unified Path-Space View

Reinforcement learning (RL) post-training provides a direct way to align diffusion models with human preferences and task-specific rewards. However, current RL algorithms for diffusion models remain fragmented: reverse-trajectory methods rely on discretized likelihood ratios, whereas forward-matching methods train on reward-labeled noising versions of the rollout samples. This paper shows that these seemingly different losses arise from a single path-space principle. Starting from the regularized diffusion-RL objective, we use importance sampling between sampling SDEs to obtain an explicit policy-gradient estimator on trajectory space. The estimator contains the stochastic Itô integral underlying Flow-GRPO-type updates; we derive an equivalent variance-reduced value-gradient form that recovers the forward-matching structure of AWM and DiffusionNFT. This identifies the empirical gap between these method families as a variance-reduction effect rather than a difference in RL principle. The derivation yields a unified design space organized by value-gradient estimation, weight functions, and sampling choices. Within this space, we propose a multi-sample KDE value-gradient estimator that reuses rollout groups, together with scale-bounded weight families that retain stable existing recipes while excluding singular ones. Experiments on SD3.5-M and Qwen-Image models validate the variance-reduction explanation and show that the resulting recipe improves over prior diffusion-RL baselines.

Yixian Xu, Yuanrui Zhang, Shengjie Luo +2

State Key Laboratory of General Artificial Intelligence, School of Intelligence Science and Technology, Peking University · ByteDance Seed

Diffusion Model Alignment Policy Gradient Methods RL for Diffusion Models Variance Reduction

Ask PDF

Oct 16, 2026

DiffusionRVGE: Reinforcement Learning via Regression-Based Value Gradient Estimation

Reinforcement learning improves diffusion models using rewards on generated images, but repeated denoising and gradient estimation make training expensive. We introduce \method, an efficient method that constructs useful update directions by regressing centered rewards on centered final denoised images. It requires neither reward derivatives nor adjoint backpropagation through the sampling trajectory. A unified first-variation formulation identifies the value-gradient target with or without a trajectory-KL penalty. A population analysis under Gaussian noising explains the regression geometry, while a subspace analysis motivates clean-image coordinates. The implementation adapts rollout bandwidth and query weights to reward continuity: local rollouts suit smooth rewards, whereas broader groups provide informative contrasts for discrete rewards. Trained jointly on five rewards, one SD3.5-Medium model matches or exceeds DiffusionNFT's joint model on seven of eight metrics, reaching GenEval

0.94

, OCR

0.92

, and PickScore

23.82

at about one fifth of its estimated training compute. In single-reward training, \method reaches matched reward thresholds

2.1

6.7\times

faster than reproduced DiffusionNFT on identical hardware. Ablations support the clean-image estimator, direct direction transfer, and reward-dependent rollout choices. Code is available at https://github.com/jy-wang-tsinghua/DIFFUSIONRVGE.

Jiayuan Wang, Tongyao Pang

Qiuzhen College, Tsinghua University, Beijing 100084, China · Yau Mathematical Sciences Center, Shuangqing Complex, Tsinghua University, Beijing 100084, China

RL for Image Generation RL for Diffusion Models Diffusion Model Fine-Tuning

Ask PDF

May 25, 2026

DRM: Diffusion-based Reward Model With Step-wise Guidance

Current mainstream methods of aligning diffusion models with human preferences typically employ VLM-based reward models. However, these reward models, pre-trained for semantic alignment, struggle to capture the essential perceptual qualities-such as aesthetics, composition, and visual harmony. In this work, we argue that a model capable of high-fidelity generation must possess a profound understanding of these visual attributes. Based on this insight, we introduce the Diffusion-based Reward Model (DRM), a novel paradigm that use the pre-trained diffusion model as a powerful evaluative backbone. A key advantage of the DRM is its unique ability to assess not only the final image but also the noisy intermediate latents at any stage of the generative process. We leverage this step-wise evaluative capacity in two ways. First, we propose Step-wise GRPO, a reinforcement learning algorithm that provides dense, per-step rewards to resolve the imprecise credit assignment problem in GRPO algorithm, leading to more stable and effective alignment. Second, we introduce Step-wise Sampling, a novel inference strategy that employs the DRM as a dynamic guide to evaluate multiple generation paths at each step, steering the process towards higher-quality outcomes. Extensive experiments confirm that our approach significantly enhances the final quality of generated images. Code: https://github.com/jjaxonx/DRM.

Jaxon Zhang, Binxin Yang, Hubery Yin +2

Peking University · Work done during an internship at WeChat Vision, Tencent Inc. · WeChat Vision, Tencent Inc.

Diffusion Model Alignment Diffusion Model Guidance RL for Diffusion Models T2I Diffusion Models

Ask PDF

Abstract

Diffusion Model Guidance

Explore similar work

Designing Reinforcement Learning for Diffusion Models: A Unified Path-Space View

DiffusionRVGE: Reinforcement Learning via Regression-Based Value Gradient Estimation

DRM: Diffusion-based Reward Model With Step-wise Guidance