Steepest Guidance: A Practical and Principled Approach to Inference-Time Alignment of Flow and Diffusion-based Models

Shokichi Takakura; Akifumi Wachi; Rei Higuchi; Kohei Miyaguchi; Taiji Suzuki

stat.MLSep 30, 2026

Steepest Guidance: A Practical and Principled Approach to Inference-Time Alignment of Flow and Diffusion-based Models

Authors: Shokichi Takakura, Akifumi Wachi, Rei Higuchi, Kohei Miyaguchi, Taiji Suzuki

Organizations: LY Corporation · The University of Tokyo · RIKEN AIP

Abstract

Inference-time alignment of flow and diffusion-based models is critical for achieving flexible generative modeling. Theoretically, Doob's $h$ -transform provides an elegant solution to this problem, and most existing methods are based on this principle. However, in practice, estimating the optimal guidance derived from Doob's $h$ -transform at inference time is challenging. To deal with this issue, we regard inference-time alignment as a sequential optimization problem in the space of probability measures and propose a novel framework called Steepest Guidance, based on the principle of maximizing local improvement in the objective. We provide a theoretical analysis of the proposed method and demonstrate its effectiveness through extensive experiments.

Figures & tables

Figure 1: Relative bias of the REINFORCE estimators for the toy Gaussian problem.

Figure 2: Toy experiment comparing Doob’s

h

-transform (left), Steepest Guidance (middle), and Regularized Steepest Guidance (right).

Model	Reward	Unguided	Steepest (Ours)	DOIT	SVDD
SD	Blueness	$-0.72\pm 0.06$	$\mathbf{-0.12}\pm 0.21$	$-0.69\pm 0.06$	$-0.67\pm 0.06$
	ImageReward	$-0.39\pm 0.49$	$\mathbf{0.86}\pm 0.53$	$0.10\pm 0.45$	$0.68\pm 0.44$
	PickScore	$20.97\pm 0.48$	$\mathbf{21.90}\pm 0.48$	$21.38\pm 0.45$	$21.84\pm 0.44$
	Compressibility	$-14.35\pm 1.80$	$\mathbf{-5.25}\pm 1.65$	$-14.17\pm 1.74$	$-13.54\pm 1.77$
FLUX	Blueness	$-0.33\pm 0.05$	$\mathbf{-0.13}\pm 0.06$	$-0.29\pm 0.04$	$-0.23\pm 0.04$
	ImageReward	$0.05\pm 0.28$	$\mathbf{1.66}\pm 0.24$	$0.45\pm 0.30$	$0.92\pm 0.29$

Table 1: Text-to-image reward summary for a prompt “A portrait photo of a golden-yellow lion”. Results are reported as mean

\pm

standard deviation over 32 generated images for each model, reward function, and method.

Appendix figures & tables13 assets

Supplementary material from the paper’s appendix.

Appendix

Method	(1)	(2)	(3)
Steepest Guidance (Ours)	$\checkmark$	$\checkmark$	$\checkmark$
DOIT ( Zhu et al., 2026 )	$\checkmark$	$\checkmark$	$\times$
FDC ( Santi et al., 2025 )	$\times$	$\times$	$\checkmark$
Adjoint Matching ( Domingo-Enrich et al., 2024 )	$\times$	$\times$	$\times$
Reinforce Adjoint Matching ( Bergmeister et al., 2026 )	$\times$	$\checkmark$	$\times$
SALD ( Nitanda et al., 2026 )	$\checkmark$	$\checkmark$	$\times$

Appendix

Table 2: Systematic comparison with previous works on reward-guided generation. (1) Training-free: The method does not require additional training of models. (2) Derivative-free: The method does not require gradients of the reward functional or the generative process. (3) Non-linear reward functional: The method can handle non-linear reward functionals.

Application	Functional $R[\mu]$
Expected reward	$\mathbb{E}_{x\sim\mu}\left[r(x)\right]$
KL divergence	$D_{\mathrm{KL}}(\mu\\|\pi_{1}):=\int\mu(x)\log\frac{\mu(x)}{\pi_{1}(x)}\,\d{x}$
Conditional value-at-risk	$\mathbb{E}_{x\sim\mu}\left[r(x)\mid r(x)\leq q_{\alpha}\right]$
Variance	$\mathbb{E}_{x\sim\mu}\left[(r(x)-\mathbb{E}_{x\sim\mu}\left[r(x)\right])^{2}\right]$
Entropy	$\mathrm{Ent}[\mu]:=-\mathbb{E}_{x\sim\mu}\left[\log\mu(x)\right]$
Optimal experimental design	$\mathsf{s}\!\left(\mathbb{E}_{x\sim\mu}\left[\Phi(x)\Phi(x)^{\top}-\lambda I\right]\right)$

Appendix

Table 3: Representative reward functionals for various applications.

\displaystyle\hat{g}_{t_{l}}^{\mathrm{steepest}}(y_{l}^{(j)})=\lambda\sigma_{t_{l}}^{2}\frac{1}{k}\sum_{i=1}^{k}\functionalderivative{R}{\mu}[\hat{\mu}_{1,l}](y_{l,i}^{(j)})\gradient_{y_{l}^{(j)}}\log\pi_{1}(y_{l,i}^{(j)}\mid Y_{t_{l}}=y_{l}^{(j)}).

Appendix

Algorithm 1 Steepest Guidance for Reward-Guided Generation

		SD		FLUX
Reward	Grid	Steepest	Doob	Steepest	Doob
Lion
Blueness	[50, 100, 200, 300, 400, 500]	500	300	500	400
ImageReward	[20, 30, 40, 50, 60, 70]	50	30	40	30
PickScore	[20, 30, 40, 50, 60, 70]	40	40	30	40
Compressibility	[10, 20, 40, 60, 80, 100]	100	20	100	100

Appendix

Table 4: A summary of the grid search for the guidance scale

\lambda

in the text-to-image experiments. The table shows the search grid and selected optimal

\lambda

for each prompt, reward function, model, and method. Lion, Tokyo, and Lake denote the prompts in Tables 1 , 5 , and 6 , respectively.

Model	Reward	Unguided	Steepest (Ours)	DOIT	SVDD
SD	Blueness	$-0.34\pm 0.07$	$\mathbf{0.15}\pm 0.21$	$-0.31\pm 0.07$	$-0.28\pm 0.07$
	ImageReward	$1.32\pm 0.29$	$\mathbf{1.78}\pm 0.14$	$1.55\pm 0.24$	$1.75\pm 0.15$
	PickScore	$22.12\pm 0.47$	$22.90\pm 0.48$	$22.46\pm 0.50$	$\mathbf{22.95}\pm 0.51$
	Compressibility	$-16.35\pm 1.73$	$\mathbf{-4.61}\pm 1.99$	$-15.98\pm 1.77$	$-15.60\pm 1.67$
FLUX	Blueness	$-0.22\pm 0.04$	$\mathbf{-0.03}\pm 0.07$	$-0.17\pm 0.04$	$-0.11\pm 0.03$
	ImageReward	$0.93\pm 0.52$	$\mathbf{1.84}\pm 0.07$	$1.41\pm 0.48$	$1.56\pm 0.34$

Appendix

Table 5: Text-to-image reward summary for the prompt “A cinematic night photograph of a rainy Tokyo street with neon reflections.” Results are reported as mean

\pm

standard deviation over

n=32

generated samples.

Model	Reward	Unguided	Steepest (Ours)	DOIT	SVDD
SD	Blueness	$-0.47\pm 0.10$	$\mathbf{0.32}\pm 0.20$	$-0.44\pm 0.10$	$-0.40\pm 0.09$
	ImageReward	$0.03\pm 0.43$	$\mathbf{0.64}\pm 0.16$	$0.33\pm 0.26$	$0.58\pm 0.18$
	PickScore	$21.84\pm 0.54$	$22.54\pm 0.52$	$22.12\pm 0.53$	$\mathbf{22.57}\pm 0.52$
	Compressibility	$-11.76\pm 1.42$	$\mathbf{-4.21}\pm 0.96$	$-11.41\pm 1.44$	$-10.98\pm 1.59$
FLUX	Blueness	$-0.58\pm 0.07$	$\mathbf{0.08}\pm 0.09$	$-0.48\pm 0.07$	$-0.38\pm 0.07$
	ImageReward	$0.97\pm 0.21$	$\mathbf{1.42}\pm 0.12$	$1.17\pm 0.18$	$1.34\pm 0.17$

Appendix

Table 6: Text-to-image reward summary for the prompt “A dreamy watercolor painting of a mountain lake at sunrise.” Results are reported as mean

\pm

standard deviation over

n=32

generated samples.

Panel of Figure 5 — Figure 5: Selected images generated with SD for the Blueness, ImageReward, PickScore, and Compressibility reward functions (top to bottom).

Panel of Figure 6 — Figure 6: Selected images generated with FLUX for the Blueness, ImageReward, PickScore, and Compressibility reward functions (top to bottom).

Method	DINO diversity $\uparrow$
Rao’s Quadratic Entropy	$0.555$
Entropy	$0.504$
Unguided	$0.428$

Appendix

Table 7: Within-batch diversity for the eight images in Fig. 7 , measured as mean pairwise cosine distance between normalized DINOv2 image embeddings. Higher is more diverse.

Figure 7: Comparison of diversity-seeking generation using SD. From top to bottom, the rows show Steepest Guidance with Rao’s quadratic entropy, entropy regularization, and the unguided baseline.

Figure 8: Ablation study on the guidance scaling factor

\gamma

for FLUX with ImageReward and Compressibility. Average performance is shown as horizontal lines.

Figure 9: Comparison of the proposed steepest guidance method with SALD ( Nitanda et al., 2026 ) for FLUX with ImageReward and Compressibility. Average performance is shown as horizontal lines.

Figure 10: Ablation study on the number of lookahead particles

k

for FLUX with ImageReward and Compressibility. Average performance is shown as horizontal lines.

Explore similar work

WEIRDO: WEak resIdual Regularized DOob's h-transform diffusion alignment
Sep 30, 2026Denis SuchkovDiffusion Alignment Wasserstein Gradient Flows
HSE University, Russian Federation
DiFA: Inference-Time Forward-Process Alignment for Diffusion Models
Jul 20, 2026Shigui Li, Delu ZengDiffusion Alignment Diffusion Models
School of Mathematics, South China University of Technology, Guangzhou, China · School of Electronic and Information Engineering, South China University of Technology, Guangzhou, China
PG-MAP: Joint MAP Optimization for Inference-Time Alignment of Diffusion and Flow-Matching Models
Jun 22, 2026Ruolan Sun, Pawel PolakDiffusion Alignment Contrastive Flow Matching
Stony Brook University

Sep 30, 2026, math.STJ/K move · Enter open · S save

WEIRDO: WEak resIdual Regularized DOob's h-transform diffusion alignment

Denis Suchkov

HSE University, Russian Federation

Diffusion Alignment Wasserstein Gradient Flows Regularization Guidance Stochastic Optimal Control Generative Models

Open paper AskPDF arXiv

We study the problem of estimating the guidance that steers the distribution learned by a diffusion generative model toward a tilted target

q_0 \propto w\,p_0

at inference time. Relying on the stochastic optimal control approach, we observe that the exact drift correction is the gradient of the logarithm of Doob's

h

-function, and we study the problem of estimating it from a sample. In the present paper, we assume that the score of the pretrained model is available, that the tilting weight is bounded and positive, and that the reference distribution has a bounded support, no smoothness of the weight is required. Introducing a penalized least-squares risk in which the penalty is the residual of the space-time harmonicity equation satisfied by the

h

-function, measured in a dual Sobolev norm, we derive high-probability bounds on the squared error of the resulting guidance estimate. Since the penalty vanishes at the target, the estimator is free of regularization bias, and in favourable scenarios its rate of convergence is faster than the minimax rate of estimating first-order derivatives of a smooth regression function. Assuming that

w

is bounded and positive with

\mathbb{E}_{p_0}[w^{-\mathrm{s}}] < \infty

for some

\mathrm{s} \in (0,\infty]

, and that the reference data are compactly supported, we prove that the guidance is estimable in squared

L^2

at rate

\varepsilon_n^{\mathrm{s}/(\mathrm{s}+4)}

, where

\varepsilon_n = n^{-2(β-1)/(2(β-1)+d)}.

We also transfer the obtained bounds to the total variation distance between the marginals of the estimated and the exactly guided samplers, and illustrate the performance of the suggested approach with numerical experiments.

Abstract

Explore similar work

WEIRDO: WEak resIdual Regularized DOob's h-transform diffusion alignment

DiFA: Inference-Time Forward-Process Alignment for Diffusion Models

PG-MAP: Joint MAP Optimization for Inference-Time Alignment of Diffusion and Flow-Matching Models