Imperfect Verifiers, Exact Targets: Inference-Time Scaling for Continuous Generation

Kijung Jeon; Molei Tao

cs.LGOct 11, 2026

Imperfect Verifiers, Exact Targets: Inference-Time Scaling for Continuous Generation

Authors: Kijung Jeon, Molei Tao

Organizations: Georgia Institute of Technology

Abstract

Verifier-guided inference-time scaling uses additional computation to score intermediate states and steer generation toward better outputs. However, imperfect intermediate verifiers can distort sampling, and repeated refinement alone does not ensure recovery of the desired reward-tilted distribution. We introduce Continuous-State Verifier-Guided Backtracking (CS-VGB), an inference-time scaling method for sampling from reward-tilted distributions with continuous generative models through repeated denoising and renoising. With correctly paired denoising and renoising transitions, the stationary distribution of CS-VGB at the clean checkpoint equals the desired reward-tilted target, even when intermediate verifiers are inaccurate. Under stated regularity conditions, CS-VGB converges exponentially to the target despite verifier error, and the expected number of updates needed to reach a clean sample grows only linearly with the number of checkpoints. In analytically tractable settings, we empirically demonstrate that CS-VGB recovers the desired reward-tilted distribution despite verifier error, whereas the compared verifier-based sampling methods fail to recover the target under the same verifier errors. We also instantiate the framework with diffusion models and one-step stochastic posterior maps. Across prompt-guided ImageNet generation, facial-attribute generation on CelebA, and multi-robot planning, recurrent CS-VGB search improves reward and task success as inference budgets grow, with image-generation rewards continuing beyond the observed plateaus of denoising-only baselines.

Figures & tables

Model	Denoising: $K_{j}^{\mathrm{F}}(dy\mid x)$	Renoising: $K_{j-1}^{\mathrm{B}}(dx\mid y)$
Diffusion models (VP SDE)	$K_{j}^{\mathrm{F}}=k_{\kappa_{j}}^{\mathrm{F}}\cdots k_{\kappa_{j-1}+1}^{\mathrm{F}}$ , $k_{k}^{\mathrm{F}}$ : reverse diffusion step ( Equation B.23 )	$\mathcal{N}(dx;\alpha_{t\mid s}y,\sigma_{t\mid s}^{2}I)$
One-step posterior map (MFM/PDM)	$K_{j}^{\mathrm{F}}(dy\mid x)=\operatorname{Law}(Y\in dy\mid x)$ , $Y$ : Gaussian bridge sample ( Figure B.6 )	$\mathcal{N}(dx;a_{j}y,\sigma_{j}^{2}I)$

Table 1: Stochastic checkpoint kernels for CS-VGB. Here

t=t_{\kappa_{j}}

and

s=t_{\kappa_{j-1}}

. For the VP SDE with noise-rate schedule

\beta

\alpha_{t\mid s}:=\exp(-\frac{1}{2}\int_{s}^{t}\beta(u)\,du)

and

\sigma_{t\mid s}^{2}:=1-\alpha_{t\mid s}^{2}

. For the linear-interpolant constructions,

a_{j}:=(1-t)/(1-s)

and

\sigma_{j}^{2}:=t^{2}-a_{j}^{2}s^{2}

. Full transitions and endpoint conventions are given in Sections A.1 and B.4 .

Panel of Figure 1 — Figure 1: Normalizing $\widehat{C}_{j}$ gives both sampling transitions, while its endpoint marginals determine the directional masses.

Panel of Figure 2 — Figure 2: Interior updates: move, stay, or flip. Here $Y\sim Q_{j}^{\mathrm{F}}$ and $Z\sim Q_{j}^{\mathrm{B},\rho}$ .

Figure 3: Target recovery of CS-VGB under process-verifier error. Left: reference and target samples. Center: CS-VGB at successive clean returns (top) and baseline outputs (bottom), all at

\lambda=20

, each showing 512 samples overlaid on target samples in gray. Right: empirical

W_{2}

across 10 random seeds for

\lambda\in\{0,3,10,15,20\}

(mean

\pm 1

SD). Each return count is per chain. The dashed line averages

W_{2}

over 100 independent 512-sample target sets against the fixed target reference.

Figure 4: Small-pool recovery with reverse insertion. Empirical

W_{2}

after

2^{11}

clean returns per chain at

\lambda=10

Figure 5: Reward-guided image generation with posterior maps. Optimized reward (ImageNet: HPSv2, CelebA: Attribute GM) and independent VQA scores versus actual NFE. Means over three seeds, each with 100 prompts or conditions; bands show

\pm 1

SD where available. F and F+B denote gradient guidance in denoising only and both directions, respectively.

Task / variant	Restart	Recurrent
ImageNet / -G	$39.20$	$\mathbf{45.57}$
ImageNet / Search	$32.23$	$\mathbf{32.69}$
CelebA / -G	$0.86$	$\mathbf{0.98}$
CelebA / Search	$0.80$	$\mathbf{0.82}$

Table 2: Restart versus recurrence. ImageNet: HPSv2; CelebA: attribute GM. Three-seed means over 100 conditions; target NFE 4,096.

Figure 6: Multi-robot planning. Joint success over four tasks with 64 instances each (mean

\pm 1

SD, five seeds) versus actual model NFE.

Appendix figures & tables21 assets

Supplementary material from the paper’s appendix.

Appendix

Symbol	Definition	Mathematical description
$t_{k}$	Time grid	$0=t_{0}<\cdots<t_{T}=1$
${\kappa_{j}}$	Checkpoint indices	${0=\kappa_{0}<\cdots<\kappa_{J}\leq T,\quad J\geq 2}$
${\pi_{j}}$	Reference marginal	${\pi_{j}=\pi_{t_{\kappa_{j}}},\quad\pi_{j-1}=\pi_{j}K_{j}^{\mathrm{F}}}$
${\pi_{j}^{\star}}$	Reward-tilted marginal	${\pi_{j}^{\star}(dx)=V_{j}(x)\pi_{j}(dx)/Z}$
$r$	Terminal reward	$r:\mathcal{X}\to[0,\infty)$
${V_{j}}$	Ideal process verifier	${V_{j}(x):=\mathbb{E}_{X_{0}\sim(K_{j}^{\mathrm{F}}\cdots K_{1}^{\mathrm{F}})(\cdot\mid x)}[r(X_{0})]}$

Appendix

Table A.1: Table of Key Notations

Panel of Figure 10 — Figure A.1: Roadmap of the theoretical analysis of CS-VGB.

Panel of Figure 11 — Figure B.1: Fine steps and checkpoint moves. Candidate paths connect neighboring checkpoints; the highlighted F move selects one of $L_{F}$ candidates. The B pool contains $L_{B}$ candidates in the opposite direction.

Panel of Figure 12 — Figure B.2: Reward-tilted edge-conductance measure between adjacent checkpoints ( ${j\geq 2}$ ). The joint measure ${\widehat{C}_{j}(dx,dy)}$ assigns mass to state pairs $(x,y)$ at levels ${j}$ and ${j-1}$ . Its marginals ${\widehat{C}_{j}(dx,\mathcal{X})}$ and ${\widehat{C}_{j}(\mathcal{X},dy)}$ give the incident masses at $x$ and $y$ , while conditioning the same joint measure in the two directions yields the transition kernels ${Q_{j}^{\mathrm{F}}(dy\mid x)}$ and ${Q_{j-1}^{\mathrm{B},\rho}(dx\mid y)}$ .

Current direction	Action	Probability	Next state
$\mathrm{F}$	Move	${p_{j}^{\mathrm{F}}(x)}$	${(j-1,Y,\mathrm{F})}$ , ${Y\sim Q_{j}^{\mathrm{F}}(\cdot\mid x)}$
$\mathrm{F}$	Flip	${p_{j}^{\mathrm{B}}(x)-\chi\ell_{j}(x)}$	${(j,x,\mathrm{B})}$
$\mathrm{F}$	Stay	${\chi\ell_{j}(x)}$	${(j,x,\mathrm{F})}$
$\mathrm{B}$	Move	${p_{j}^{\mathrm{B}}(x)}$	${(j+1,Y,\mathrm{B})}$ , ${Y\sim Q_{j}^{\mathrm{B},\rho}(\cdot\mid x)}$
$\mathrm{B}$	Flip	${p_{j}^{\mathrm{F}}(x)-\chi\ell_{j}(x)}$	${(j,x,\mathrm{F})}$
$\mathrm{B}$	Stay	${\chi\ell_{j}(x)}$	${(j,x,\mathrm{B})}$

Appendix

Table B.1: Momentum update at an interior checkpoint.

Panel of Figure 14 — Figure B.3: Algorithm comparison of CS-VGB. (a) One-pass candidate selection, following SVDD ( Li et al., 2025 ) , selects one candidate at each denoising step and stops at the first clean output. (b) CS-VGB may instead renoise and repeat the search, while the exact terminal reward anchors every visit to $t=0$ .

Panel of Figure 15 — Figure B.4: Reverse insertion for an F move. Select $Y_{I}$ and insert the previous $x$ into a uniform B slot. All remaining entries are freshly drawn.

Panel of Figure 16 — Figure B.5: Lazy pool evaluation from direction F, shown when only the B mark has been generated. First draw $I\sim\operatorname{Unif}(\{1,\ldots,L_{F}\})$ , then choose move, stay, or flip using the probabilities in Equation B.21 , conditioned on $I$ and the index $K$ of the existing B mark. (a) An F move discards that index and independently selects $Y_{I}$ by weight, creates the destination pools, and reverse-inserts $x$ as the new B mark. Panel (a) assumes an interior destination ( $j\geq 2$ ); for $j=1$ , arrival at checkpoint $0$ resets the direction to B and retains only the B pool, with no separate mark. (b) An F-to-B flip leaves $(j,x)$ unchanged, fills missing B entries, and marks the preselected $Y_{I}$ in the retained F pool. Already generated candidates and values are reused. Stay changes nothing and reuses the same pools and mark.

Panel of Figure 17 — Figure B.6: Posterior sampling followed by a Gaussian bridge conditioned on both endpoints. For an exact posterior sampler, ${Y\sim K_{j}^{\mathrm{F}}(\cdot\mid x)}$ ( Proposition B.1 ).

Panel of Figure 18 — Figure C.1: Edge-conductance measures define both the CS-VGB update and its stationary law. For ${1\leq j<J}$ , $\mu_{j}$ sums the marginals of the two adjacent edge-conductance measures; at the boundaries, $\mu_{0}=\widehat{C}_{1}(\mathcal{X},\cdot)$ and ${\mu_{J}=\widehat{C}_{J}(\cdot,\mathcal{X})}$ . The resulting law and clean target are established in Theorem C.1 .

Panel of Figure 19 — Figure C.2: Exact expected first hitting times from $j=J$ under balanced masses ( Equation C.97 ). Resample direction moves directly to $j-1$ or $j+1$ with probabilities $p_{j}^{\mathrm{F}}$ or $p_{j}^{\mathrm{B}}$ , respectively; here both equal $1/2$ at interior checkpoints, and the move from $J$ is forced inward. Its mean is $J^{2}$ , compared with $2J-1$ for CS-VGB at $\chi=1$ .

$B$	$J$	$L_{F}$	$L_{B}$	$m_{g}$	$g_{F}$	$g_{B}$
$2^{9}$	95	1	1	1	3.5	1.0
$2^{10}$	79	1	1	2	4.0	1.5
$2^{11}$	64	1	1	4	2.0	1.5
$2^{12}$	64	1	1	4	1.75	1.5
$2^{13}$	64	4	4	8	2	1.5
$10{,}000$	64	4	4	8	2	1.5

Appendix

Table F.1: ImageNet guided CS-VGB configurations.

B

is the requested NFE budget;

J

counts checkpoint intervals.

$B$	$J$	$N$	$m_{e}$	$B$	$J$	$N$	$m_{e}$
1,433	16	4	8	10,623	32	8	24
1,854	16	4	16	12,637	32	8	24
1,902	16	4	16	12,767	32	8	24
1,953	16	4	16	16,827	32	8	24
2,037	16	4	16	21,716	64	8	24
3,975	16	4	32	25,859	64	8	24

Appendix

Table F.2: ImageNet unguided CS-VGB configurations. Columns give requested NFE budget

B

, intervals

J

, particles

N

, and clean endpoint candidates per particle

m_{e}

Panel of Table 22 — Table F.3: ImageNet prompt-aligned outputs near 65k NFE. Generated images with HPSv2 scores (reported on the $\times 100$ scale) and actual NFE. Bold marks the largest HPSv2 score in each row.

	Guided ( $N=1$ , $L=2$ )		Unguided
Target NFE	$J$	$m_{g}$	$J$	$N$	$L$	$m_{e}$
$2^{6}$	4	1	2	2	2	4
$2^{7}$	4	1	4	2	4	4
$2^{8}$	4	1	4	4	8	8
$2^{9}$	8	1	8	4	4	8
$2^{10}$	16	2	16	8	4	4

Appendix

Table F.4: CS-VGB settings for CelebA. Guided columns apply to both F-only and F+B variants. Here,

m_{g}

denotes the number of posterior endpoint samples used for guidance,

J

the number of checkpoint intervals,

N

the number of particles,

L=L_{F}=L_{B}

the candidate-pool size, and

m_{e}

the number of posterior endpoint samples per particle used for unguided search.

Condition	BoN	PDM-SMC	CS-VGB Search	PDM-G	CS-VGB-G
ID 011 $-$ smiling, $+$ eyeglasses, $-$ bangs, $+$ hat, $-$ mouth open, $+$ black hair, Beard, $+$ bushy brows, $-$ lipstick, $+$ earrings.	0.2688 NFE 4,096	0.7476 NFE 4,068	0.7634 NFE 4,152	0.9453 NFE 4,068	0.9805 NFE 4,101
ID 013 $-$ smiling, $+$ eyeglasses, $+$ bangs, $-$ hat, $+$ mouth open, $-$ blond hair, No beard, $+$ bushy brows, $+$ lipstick, $-$ earrings.	0.4232 NFE 4,096	0.7046 NFE 4,068	0.9511 NFE 4,184	0.9668 NFE 4,068	0.9923 NFE 4,103
ID 001 $+$ smiling, $+$ eyeglasses, $-$ bangs, $+$ hat, $-$ mouth open, $-$ black hair, No beard, $-$ bushy brows, $-$ lipstick, $+$ earrings.	0.5476 NFE 4,096	0.9456 NFE 4,068	0.9775 NFE 4,104	0.9678 NFE 4,068	0.9900 NFE 4,101
ID 003 $+$ smiling, $-$ eyeglasses, $-$ bangs, $+$ hat, $+$ mouth open, $+$ black hair, No beard, $-$ bushy brows, $-$ lipstick, $-$ earrings.	0.6736 NFE 4,096	0.7743 NFE 4,068	0.9759 NFE 4,184	0.9670 NFE 4,068	0.9889 NFE 4,107
ID 006 $+$ smiling, $-$ eyeglasses, $+$ bangs, $-$ hat, $-$ mouth open, $-$ brown hair, Beard, $-$ bushy brows, $-$ lipstick, $-$ earrings.	0.7856 NFE 4,096	0.9525 NFE 4,068	0.9921 NFE 4,168	0.9782 NFE 4,068	0.9969 NFE 4,106

Appendix

Table F.5: CelebA attribute-conditioned outputs at a 4,096-NFE target budget.

+

-

denote required presence/absence; beard is expressed directly. Bold marks the largest GM in each row.

Figure F.3: Independent image-quality evaluation. CLIP-IQA of images selected using HPSv2 on ImageNet and attribute GM on CelebA. Curves show mean

\pm 1

SD across three seeds, each evaluated on 100 conditions, against actual mean NFE. CLIP-IQA is not used for reward optimization.

Panel of Figure 26 — Figure F.4: Successful multi-robot trajectories from CS-VGB-G. Examples in Circle, Conveyor Moderate, Conveyor Hard, and Highways. Red curves show trajectories, green dots mark starts, purple diamonds mark goals, and shaded regions indicate obstacles. The Highways example uses eight robots, whereas the quantitative Highways experiments use 16.

Method	Overall $\pm$ SD	Mean NFE	Highways	Circle	Conv. Mod.	Conv. Hard
BoN-G	$67.58\pm 1.56$	6,355.5	64.69	5.62	100.00	100.00
DFS-G	$60.08\pm 1.25$	8,029.7	40.00	0.31	100.00	100.00
ABCD-G	$71.17\pm 1.52$	5,765.2	73.12	11.56	100.00	100.00
CS-VGB-G	$84.84\pm 1.98$	5,044.8	80.00	60.00	99.69	99.69

Appendix

Table F.6: Planning results at the largest evaluated setting. All methods use NFE cap

2^{14}

. Success entries are percentages.

		Mean NFE			Attribute GM
Variant	Updates	ON	OFF	Saved	ON	OFF
Guided	313	4,103.89	5,955.02	31.1%	0.98138	0.98182
Unguided	45	4,174.24	5,738.72	27.3%	0.83989	0.83953

Appendix

Table F.7: Lazy pool evaluation at matched update counts on CelebA. NFE averages all 100 runs.

(a) Attribute GM
$\chi$	G	U
0	0.9787	0.8214
0.3	0.9803	0.8296
0.5	0.9816	0.8233
0.7	0.9804	0.8282
1	0.9804	0.8264

Appendix

Table F.8: Directional persistence on CelebA with lazy ON. G/U denote guided/unguided search. Panel (a) reports supplemented attribute GM; panel (b) reports mean counts per run at the initial 4,096-NFE target, comparing

\chi=0

with

\chi=1

Inference-Time Guidance

Inference-time guidance steers pretrained generative models, language models, and policies toward desired outputs without requiring full retraining.

See all 102 papers

VGB for Masked Diffusion Model: Efficient Test-time Scaling for Reward Satisfaction and Sample Editing
Jun 26, 2026Kijung Jeon, Thuy-Duong Vuong, Molei TaoConstrained Generative Modeling Masked Diffusion Models
Georgia Tech · UCSD
Tiny Inference-Time Scaling with Latent Verifiers
Mar 23, 2026Davide Bucciarelli, Evelyn Turri, Lorenzo Baraldi +3Inference-Time Scaling Diffusion Models
University of Modena and Reggio Emilia, Italy · University of Pisa, Italy
Inference-Time Scaling in Diffusion Models through Iterative Partial Refinement
May 19, 2026Taegu Kang, Jaesik Yoon, Sungjin AhnInference-Time Scaling Diffusion Models
KAIST · KAIST & SAP · KAIST & NYU

Jun 26, 2026J/K move · Enter open · S save

VGB for Masked Diffusion Model: Efficient Test-time Scaling for Reward Satisfaction and Sample Editing

Kijung Jeon, Thuy-Duong Vuong, Molei Tao

Georgia Tech · UCSD

Constrained Generative Modeling Masked Diffusion Models Controllable Text Generation Diffusion Sampling Inference-Time Scaling

Open paper AskPDF arXiv

Inference-time scaling is a promising paradigm to improve generative models, especially when outputs must satisfy structural constraints or optimize downstream rewards. We consider Masked Diffusion Model (MDM) and introduce MDM-VGB, a discrete diffusion sampler that augments unmasking generation with theoretically principled reward-guided remasking. Inspired by the recent success of the classical Jerrum-Sinclair backtracking Markov chain in reward-tilted generation, MDM-VGB extends the backtracking random walk from a fixed prefix tree to a masked-state graph, allowing tokens to be unmasked and remasked at arbitrary positions. The resulting sampler favors unmasking and remasking moves that lead to higher-value partial configurations, enabling both effective high-reward generation and efficient repair of low-reward samples. We prove that MDM-VGB is robust to process-verifier noise and achieves quadratic complexity, while popular test-time heuristics such as best-of-

N

can incur exponential complexity due to error accumulation. Our theoretical findings are corroborated by strong empirical performance, particularly on popular constraint-satisfaction and scientific benchmarks such as Sudoku and QM9.

Abstract

Inference-Time Guidance

Explore similar work

VGB for Masked Diffusion Model: Efficient Test-time Scaling for Reward Satisfaction and Sample Editing

Tiny Inference-Time Scaling with Latent Verifiers

Inference-Time Scaling in Diffusion Models through Iterative Partial Refinement