LatticeSMC: Where to Spend Inference-Time Compute in Chunked Sequence Generators

Xuanchen Wang; Heng Wang; Weidong Cai

cs.LGOct 2, 2026

LatticeSMC: Where to Spend Inference-Time Compute in Chunked Sequence Generators

Authors: Xuanchen Wang, Heng Wang, Weidong Cai

Organizations: School of Computer Science The University of Sydney Sydney, NSW, Australia

Abstract

Long-form generators for music, motion and video produce sequences chunk by chunk, with each chunk generated by iterative denoising while rewards are defined over the full sequence. Existing inference-time steering methods typically act on one axis at a time: best-of-N at the end, Feynman-Kac steering across denoising steps, or streaming pruning across chunks, and are often compared under unmatched compute or different return rules. We introduce budget-matched chunked steering and propose LatticeSMC, a sampler derived from a Feynman-Kac model on the two-dimensional lattice of chunk index and denoising step. Two telescoping results make its design exact: for chunk-additive rewards, the two axes induce identical weights, so resampling should occur where lookahead is cheapest; for terminal rewards, any prefix score defines an exact intermediate potential, making prefix-evaluable rewards twists with no estimation or extra denoiser calls. LatticeSMC resamples on these potentials at chunk boundaries and, when scoring is free, within chunks, returning either a weighted draw or the best particle. Under matched compute, on music-to-dance diffusion and 40-second text-to-music generation, it raises beat alignment from 0.234 to 0.441 (best-of-N: 0.354) and prompt adherence from 0.470 to 0.560 at 32 particles, while preserving held-out quality. It also retains its advantage on long-range rewards and is preferred by human raters in 60-77 percent of pairwise comparisons. Finally, we show that commitment strength should follow the information in the current potential, while the value of lookahead is predicted by the within-set predictability of future reward.

Figures & tables

Figure 1: The lattice Feynman-Kac model and LatticeSMC. (a) One pass:

N

particles extend their clean prefix by one chunk, each completed prefix is scored (chunk reward

r_{k}

or prefix score

s_{k}

), the set is resampled systematically at the boundary when the ESS falls below

N/2

, and after the last chunk the sampler returns a weighted draw (sampling) or the best particle (search). (b) Where each method places its potentials on the lattice of chunk index

k

and denoising step

\tau

. (c) The telescoping results: denoising-axis potentials multiply to the chunk potential (Proposition 1 ); prefix-score increments with a terminal correction multiply to

\exp(r/\alpha)

(Proposition 2 ).

Figure 2: Reward against denoiser evaluations at matched budget. Four rewards,

N\in\{1,\dots,32\}

, means over 40 conditions with 95% bootstrap intervals; solid: sampling rule (draw), dashed: search rule (argmax). LatticeSMC uses the chunk potential on dance (

\alpha=0.02

) and the prefix-score potential on music; FK steering is its dense schedule.

	Beat alignment		Repetition		Prompt adherence		Motif recurrence
	draw	argmax	draw	argmax	draw	argmax	draw	argmax
Base ( $N=1$ )	0.234 [0.223, 0.243]		0.718 [0.628, 0.804]		0.470 [0.449, 0.491]		0.814 [0.786, 0.839]
Best-of-N	0.347	0.354	0.895	0.901	0.527	0.530	0.904	0.918
Chunk pruning (top- $M$ )	–	0.415	–	0.887	–	0.551	–	0.918
Twisted SMC, learned twist	0.385	0.419 a	0.871	0.895	–	–	–	–
LatticeSMC, dense (FK steering placement)	0.387 ⋆	0.441 ⋆	0.921 ⋆	0.962 ⋆	0.546	0.559	0.908	0.932

Table 1: Reward of the returned sequence at

N=32

particles and identical denoiser evaluations, under the sampling rule (draw) and the search rule (argmax), with 95% bootstrap intervals for the base. A star marks the schedule of LatticeSMC its cost model prescribes, dense where the plug-in reward is free (dance) and boundary where scoring costs a decode (music); the dense schedule is the Feynman-Kac steering placement (Proposition 1 ), so that row is also the baseline. Chunk pruning is at

M=N/4

(swept in Section 4.4 ). Dash: return rule absent or not run; a : additive scale, not ranked. Bold: best per column; underlined: second best; paired differences with intervals in Table 11 .

LatticeSMC (argmax) $-$ pruning at $N=$	2	4	8	16	32	argmax - draw
Beat alignment	$-0.032^{*}$	$-0.029^{*}$	$-0.015^{*}$	$-0.002$	$+0.013^{*}$	$+0.042^{*}$
Repetition	$+0.007$	$+0.026^{*}$	$+0.063^{*}$	$+0.078^{*}$	$+0.059^{*}$	$+0.023^{*}$
Prompt adherence	$-0.012^{*}$	$-0.002$	$+0.002$	$+0.004^{*}$	$+0.009^{*}$	$+0.012^{*}$
Motif recurrence	$+0.001$	$+0.008$	$+0.007^{*}$	$+0.012^{*}$	$+0.011^{*}$	$+0.026^{*}$

Table 2: The return rule and the crossover for the boundary schedule of LatticeSMC, so that the crossover measures resampling on chunk potentials alone: paired difference of LatticeSMC (argmax) minus chunk pruning by particle count (asterisk: 95% interval excludes zero; dance values at

N\neq 8,32

on the additive logged scale) and the gain of argmax over draw at

N=32

; the 38M model reproduces every crossover (Figure 7 ).

	Beat alignment			Repetition
$K=$	4	6	8	4	6	8
Best-of-N	0.354	0.333	0.335	0.901	1.795	2.544
Chunk pruning ( $M=N/4$ )	0.415	0.410	0.434	0.887	1.830	2.555
LatticeSMC, boundary	0.428	0.393	0.382	0.947	1.930	2.676
LatticeSMC, dense	0.441	0.400	0.396	0.962	1.930	2.711
LatticeSMC, boundary / dense, $\alpha=0.01$	–	0.436 / 0.441	0.455 / 0.450	–	–	–

Table 3: Horizon and per-chunk tilt, 4M dance model,

N=32

, argmax return,

\alpha=0.02

unless stated.

K=4

and

6

use the 40 conditions,

K=8

the 16 with a long enough recording; repetition is the sum over the

K-2

chunk pairs, beat alignment a mean over beats;

\alpha=0.01

is the prespecified second point, equal to the per-boundary tilt

\alpha\propto 1/K

K=8

, run on beat alignment, where the fixed-

\alpha

ordering reversed. Best per column in bold, second best underlined; dash: not run. Paired differences, resampling rates, base and draw rows: Appendix C.6 .

Twist	cross-prompt $R^{2}$	within-set corr.	residual (nats)	effect at $N=32$
$\psi\equiv 1$ (exact potentials only)	–	–	–	$0$
Learned regressor	$0.90$	$-0.04$	$4.3$	$-0.053\ [-0.083,-0.028]$
Learned regressor, 38M model	$0.91$	–	$3.3$	$-0.018\ [-0.032,-0.006]$
Regressor trained on within-set weights	–	$0.01$	–	$-0.014\ [-0.033,-0.002]$
Oracle, 16 rollouts per particle	–	$1$	$\approx 0.6$	$+0.058\ [+0.034,+0.083]$

Table 4: Twists on the repetition reward at

\alpha=0.02

N=32

(4M unless stated): cross-prompt fit, within-set Pearson correlation with the oracle after chunk one, root-mean-square residual against the rollout target on 500 fresh prefixes in nats, and effect relative to

\psi\equiv 1

with paired intervals; the oracle is not budget-matched; dash: not measured.

Appendix figures & tables18 assets

Supplementary material from the paper’s appendix.

Appendix

Method	Lattice points with $G\neq 1$	Potential	Resampling	Return
Best-of-N	$(K,S)$	$\exp(r/\alpha)$	IS or none	draw or argmax
Feynman-Kac steering	$(k,\tau)$ , $\tau\in\mathcal{T}_{k}$	plug-in difference	SYS	draw
Chunk pruning	$(k,S)$	prefix reward, $\alpha\to 0$	TOP- $M$	argmax
LatticeSMC, boundary	$(k,S)$	exact chunk term or prefix increment	SYS	draw or argmax
LatticeSMC, dense	$(k,\tau)$ and $(k,S)$	as above, plus plug-in differences	SYS	draw or argmax

Appendix

Table 5: Inference-time steering methods as placements on the lattice. IS: importance resampling at the terminal corner; SYS: systematic resampling gated by

\mathrm{ESS}<N/2

; TOP-

M

: deterministic replication of the top

M

particles.

Figure 3: When lookahead pays (schematic). (a) Within-set predictability: prefixes drawn under one condition are each continued

M

times; the spread within a prefix is rollout noise, the spread between prefixes is the signal a twist can rank on ( Equation 3 ). (b) Twist accuracy is a within-set property: a regressor can fit realized future reward across prompts through prompt-level offsets while ranking the particles of one set at random (measured values in Table 4 and Appendices B.2 and B.3 ). (c) Three sources of a content-axis twist: a prefix-evaluable reward (exact, free), rollouts (unbiased, charged), a learned predictor (cheap, validate within sets).

Reward	Prefix pool	$k=1$	$k=2$	$k=3$
Beat alignment (chunk-additive)	pooled, $M=8$	$\leq 0.13$	$\leq 0.13$	$\leq 0.14$
Repetition (long range)	pooled, $M=16$	$0.92$	$0.94$	$0.93$
Prompt adherence (terminal)	within condition, $M=8$	$0.47$	$0.62$	$0.66$

Appendix

Table 6: Within-set predictability of future reward from base-model rollouts. Pooled values draw prefixes from many conditions; within-condition values hold the condition fixed. Beat alignment and repetition use the dance model; prompt adherence uses the music model. The repetition and prompt-adherence rows decompose the variance of

S_{k}

(the conditional-mean diagnostic of Equation 3 ); the beat-alignment row decomposes the variance of the exponential target

\exp(S_{k}/\alpha)

itself and reports the ratio of between- to within-prefix variance, an upper bound on the share. The within-condition repetition share is bounded through the rollout-target spread of Appendix B.3 .

Figure 4: Two chunked generators. Top: music-to-dance diffusion on AIST++. The denoiser sees a 300-frame window, the clean prefix chunk and the noisy chunk, and predicts the chunk’s root as a displacement integrated from the prefix root, so the prefix is fixed and the seam continuous by construction; steering rewards are beat alignment (chunk-additive) and repetition of chunk

k

against chunk

k-2

(long range). Bottom: text-to-music flow, 40 s as four 10 s chunks. Each chunk is generated by inpainting past the clip’s edge with the prefix latents replaced at every flow step; scoring a prefix requires one decode and one CLAP pass, which is charged; steering rewards are prompt adherence (terminal, evaluable on any prefix) and motif recurrence of the last chunk against the first.

Figure 5: Temperature and effective sample size on beat alignment. Left: reward at

N=32

for the proper methods and chunk pruning at

\alpha\in\{0.2,0.05,0.02\}

; the boundary and dense schedules of LatticeSMC coincide at every temperature, and both approach pruning as the tilt sharpens. Right: the fraction of chunk boundaries at which the ESS falls below

N/2

and resampling fires; at

\alpha=0.2

the maximum weight ratio across 32 particles is 3 and no method resamples, at

\alpha=0.02

it is of order

10^{4}

and resampling fires at 91 to 98 percent of boundaries.

Figure 6: Three cost axes on the music model (prompt adherence). The same runs plotted against denoiser evaluations, total model calls (denoiser, decoder and CLAP) and measured wall-clock per sequence. The two schedules of LatticeSMC carry identical weights (the plug-in CLAP differences telescope to the prefix-score increments of Proposition 2 ) and coincide on the first axis; on a latent generator each within-chunk event costs a decode and a reward pass, so the dense schedule moves right by

3.6\times

against the boundary schedule and its curve flattens from

N=8

, while the boundary schedule keeps rising. Best-of-N is cheapest per particle and lowest in reward at every budget.

Figure 7: Proper resampling overtakes greedy pruning as the budget grows. Paired difference between LatticeSMC with argmax return and chunk pruning as a function of particle count, with 95% bootstrap intervals over conditions, for the four rewards and, on the dance rewards, both model sizes. Pruning’s early commitment leads at the smallest budgets on beat alignment and prompt adherence; ESS-gated resampling crosses it between

N=16

and

32

on beat alignment, at

N=8

on prompt adherence and at

N=4

on motif recurrence, and leads from

N=4

on the long-range repetition reward, where pruning replicates blind for two chunks. The 38M model reproduces the 4M crossovers.

Reward	$M$	Reward [95% CI]	Held out	$-$ ( $M=N/4$ )	$-$ boundary argmax
Beat alignment	1	$\mathbf{0.473}\ [0.453,0.491]$	PFC 1.73, $W_{1}$ 0.0078	$+0.058^{*}$	$+0.045^{*}$
	8	$0.415\ [0.399,0.431]$	PFC 2.05, $W_{1}$ 0.0082		$-0.013^{*}$
	16	$0.386\ [0.372,0.400]$	PFC 1.80, $W_{1}$ 0.0076	$-0.029^{*}$	$-0.042^{*}$
Repetition	1	$0.850\ [0.755,0.941]$	PFC 1.62, $W_{1}$ 0.0081	$-0.037^{*}$	$-0.096^{*}$
	8	$0.887\ [0.796,0.976]$	PFC 1.36, $W_{1}$ 0.0077		$-0.059^{*}$
	16	$0.916\ [0.826,1.013]$	PFC 1.43, $W_{1}$ 0.0081	$+0.028^{*}$	$-0.031^{*}$

Appendix

Table 7: Chunk pruning at

N=32

as a function of its strength

M

(argmax by construction;

M=1

is the

\alpha\to 0

limit of the boundary schedule of LatticeSMC with argmax return when the maximum is unique), with 95% bootstrap intervals over conditions, held-out quantities, and paired differences against the

M=N/4

row and against the boundary schedule of LatticeSMC with argmax return; an asterisk marks an interval excluding zero. Dance: 4M model,

\alpha=0.02

, held-out PFC

\times 10^{4}

and

W_{1}

. Music: held-out tempo adherence, 8 kHz energy fraction and seed diversity.

Beat alignment , $\alpha=0.02$
Method	$N$	Reward [95% CI]	fired	$\mathrm{ESS}/N$	PFC	$W_{1}$	diversity
Base	1	$0.236\ [0.228,0.243]$			1.70	0.0078	0.178
Best-of-N (draw)	8	$0.292\ [0.283,0.301]$			1.58	0.0075	0.180
Best-of-N (argmax)	8	$0.296\ [0.288,0.304]$			1.75	0.0079	0.178
Chunk pruning	8	$0.376\ [0.365,0.387]$	1.00	0.25	1.90	0.0079	0.181
LatticeSMC, boundary (draw)	8	$0.310\ [0.298,0.323]$	0.35	0.53–0.62	1.69	0.0079	0.185

Appendix

Table 8: Horizon

K=6

(30 s, 40 conditions, 4M dance model): full reward of the returned sequence with 95% bootstrap intervals, fraction of boundaries at which resampling fired, range of the mean

\mathrm{ESS}/N

before the boundaries, and held-out quantities (PFC

\times 10^{4}

W_{1}

, root-relative pose diversity). Base intervals on these conditions: PFC

[1.05,2.51]

W_{1}

[0.0065,0.0094]

. Paired differences at

N=32

, beat alignment: boundary argmax

-

pruning

-0.017\ [-0.024,-0.010]

; dense argmax

-

pruning

-0.010\ [-0.016,-0.003]

; boundary argmax,

\alpha=0.01

-

pruning

+0.026\ [+0.019,+0.033]

; dense argmax,

\alpha=0.01

-

pruning

+0.031\ [+0.021,+0.042]

; boundary argmax,

\alpha=0.01

-

best-of-N argmax

+0.103\ [+0.092,+0.113]

. Repetition (sum over the four chunk pairs; rows give the mean of the returned sequence’s sum): boundary argmax

-

pruning

+0.100\ [+0.070,+0.135]

; boundary argmax

-

best-of-N argmax

+0.135\ [+0.099,+0.164]

; dense argmax

-

boundary argmax

-0.001\ [-0.046,+0.041]

Beat alignment , $\alpha=0.02$
Method	$N$	Reward [95% CI]	fired	$\mathrm{ESS}/N$	PFC	$W_{1}$	diversity
Base	1	$0.231\ [0.216,0.245]$			1.62	0.0067	0.191
Best-of-N (draw)	8	$0.290\ [0.272,0.305]$			2.27	0.0088	0.214
Best-of-N (argmax)	8	$0.296\ [0.277,0.312]$			1.68	0.0070	0.197
Chunk pruning	8	$0.392\ [0.370,0.412]$	1.00	0.25	2.14	0.0084	0.213
LatticeSMC, boundary (draw)	8	$0.316\ [0.291,0.336]$	0.33	0.50–0.65	2.54	0.0076	0.223

Appendix

Table 9: Horizon

K=8

(40 s, the 16 conditions with an 8-chunk recording, 4M dance model), as Table 8 . Base intervals on these conditions: PFC

[0.76,2.66]

W_{1}

[0.0046,0.0092]

. Paired differences at

N=32

, beat alignment: boundary argmax

-

pruning

-0.052\ [-0.057,-0.046]

; dense argmax

-

pruning

-0.038\ [-0.045,-0.029]

; boundary argmax,

\alpha=0.01

-

pruning

+0.021\ [+0.012,+0.031]

; dense argmax,

\alpha=0.01

-

pruning

+0.016\ [-0.002,+0.035]

; boundary argmax,

\alpha=0.01

-

best-of-N argmax

+0.120\ [+0.108,+0.134]

. Repetition (sum over the six chunk pairs; rows give the mean of the returned sequence’s sum): boundary argmax

-

pruning

+0.121\ [+0.060,+0.190]

; boundary argmax

-

best-of-N argmax

+0.133\ [-0.118,+0.317]

; dense argmax

-

boundary argmax

+0.035\ [-0.053,+0.106]

	Dance			Music
	PFC	$W_{1}$	diversity	tempo	8 kHz	PQ	seam	diversity
Base	1.97	0.0080	0.183	0.903	0.0069	8.17	1.27	0.061
Steered, all methods	1.84–2.24	0.0077–0.0083	0.174–0.184	0.894–0.944	0.0061–0.0067	8.12–8.16	1.25–1.34	0.048–0.060
LatticeSMC, argmax	2.24	0.0081	0.183	0.924	0.0062	8.13	1.34	0.050

Appendix

Table 10: Held-out quantities at

N=32

, never used by any method, as ranges over all steered methods against the base; dance values for beat-alignment steering, music values for prompt-adherence steering. PFC

\times 10^{4}

(lower is better; base 95% interval

[1.25,2.72]

);

W_{1}

realism statistic (base interval

[0.0066,0.0094]

); PQ is the production-quality aesthetics axis; diversity is across seeds.

Difference	Beat alignment	Repetition	Prompt adherence	Motif recurrence
boundary $-$ best-of-N, draw	$+0.038\ [+0.028,+0.048]$	$+0.028\ [+0.019,+0.038]$	$+0.020\ [+0.016,+0.024]$	$-0.002\ [-0.009,+0.006]$
boundary $-$ chunk pruning, argmax	$+0.013\ [+0.005,+0.021]$	$+0.059\ [+0.039,+0.083]$	$+0.009\ [+0.006,+0.012]$	$+0.011\ [+0.005,+0.017]$
boundary $-$ chunk pruning, draw	–	$+0.036\ [+0.014,+0.060]$	–	–
dense $-$ chunk pruning, argmax	$+0.026\ [+0.015,+0.039]$	$+0.075\ [+0.052,+0.101]$	–	–
dense $-$ boundary, argmax	$+0.013\ [+0.003,+0.025]$	$+0.016\ [-0.008,+0.034]$	$-0.000\ [-0.004,+0.005]$	$+0.003\ [-0.003,+0.008]$
dense $-$ boundary, draw	–	–	$-0.002\ [-0.007,+0.003]$	$+0.006\ [-0.003,+0.015]$

Appendix

Table 11: Paired 95% bootstrap intervals over the 40 conditions for the differences reported in Table 1 and Section 4.3 at

N=32

(4M dance model unless stated). A dash marks a pair for which no paired interval is stored.

$N$	1	2	4	8	16	32
Base	0.234	–	–	–	–	–
Best-of-N (argmax)	0.234	0.260	0.289	0.312	0.334	0.354
Best-of-N (draw)	0.234	0.260	0.288	0.309	0.329	0.347
Chunk pruning	0.234	0.289	0.342	0.371	0.398	0.415
LatticeSMC, dense (FK steering), draw	0.234	0.256	0.316	0.350	0.370	0.387
LatticeSMC, dense (FK steering), argmax	0.234	–	–	–	–	0.441

Appendix

Table 12: Beat alignment (4M dance model), full reward of the returned sequence by particle count, draw rule unless marked;

\alpha=0.02

. Argmax rows are regenerated at

N=8

and

32

(dense schedule:

N=32

); a dash marks a budget at which the argmax sequences were not regenerated, or the base model’s single sample.

$N$	1	2	4	8	16	32
Base	0.718	–	–	–	–	–
Best-of-N (argmax)	0.718	0.768	0.805	0.850	0.875	0.901
Best-of-N (draw)	0.718	0.768	0.804	0.849	0.871	0.895
Chunk pruning	0.718	0.761	0.792	0.811	0.835	0.887
LatticeSMC, dense (FK steering), draw	0.718	0.768	0.836	0.876	0.901	0.921
LatticeSMC, dense (FK steering), argmax	0.718	–	–	0.898	–	0.962

Appendix

Table 13: Repetition (4M dance model),

\alpha=0.02

; the additive and full forms coincide. A dash marks a budget at which the argmax sequences were not regenerated.

$N$	1	2	4	8	16	32
Base	0.470	–	–	–	–	–
Best-of-N (argmax)	0.470	0.488	0.502	0.513	0.520	0.530
Best-of-N (draw)	0.470	0.488	0.502	0.513	0.519	0.527
Chunk pruning	0.470	0.499	0.523	0.536	0.543	0.551
LatticeSMC, dense (FK steering), draw	0.470	0.488	0.520	0.538	0.542	0.546
LatticeSMC, dense (FK steering), argmax	0.470	–	–	0.545	–	0.559

Appendix

Table 14: Prompt adherence (terminal CLAP, music model),

\alpha=0.0068

. LatticeSMC uses the prefix-score potential;

\beta

tempers the increments (Proposition 2 ). A dash marks a budget at which the argmax sequences were not regenerated.

$N$	1	2	4	8	16	32
Base	0.814	–	–	–	–	–
Best-of-N (argmax)	0.814	0.849	0.876	0.897	0.910	0.918
Best-of-N (draw)	0.814	0.849	0.874	0.892	0.901	0.904
Chunk pruning	0.814	0.848	0.870	0.894	0.906	0.918
LatticeSMC, dense (FK steering), draw	0.814	0.849	0.875	0.895	0.898	0.908
LatticeSMC, dense (FK steering), argmax	0.814	0.849	0.885	0.910	0.922	0.932

Appendix

Table 15: Motif recurrence (music model),

\alpha=0.019

; the prefix score is available at the last boundary.

	Beat alignment				Repetition
	4M		38M		4M		38M
	draw	argmax	draw	argmax	draw	argmax	draw	argmax
Base ( $N=1$ )	0.234		0.225		0.718		0.654
Best-of-N	0.347	0.354	0.353	0.360	0.895	0.901	0.780	0.784
Chunk pruning	–	0.415	–	0.423	–	0.887	–	0.795
LatticeSMC, dense (FK steering)	0.387	0.441	0.406	0.455	0.921	0.962	0.822	0.874

Appendix

Table 16: Scale: reward at

N=32

on the 4M and 38M dance models,

\alpha=0.02

, both return rules, with the paired difference of LatticeSMC (argmax) against chunk pruning. Orderings at

N=32

on beat alignment are identical at both sizes.

	Dance (beat alignment)		Music (prompt adherence)
LatticeSMC preferred over	follows beat	overall	matches prompt	overall
Base model	73.5 [65.7, 81.2]	66.0 [57.8, 74.5]	77.0 [69.3, 84.2]	69.0 [60.0, 77.3]
Best-of-N (argmax)	66.0 [58.0, 74.0]	60.5 [53.0, 68.0]	69.0 [59.5, 77.5]	62.0 [54.0, 69.8]
Chunk pruning	69.0 [57.8, 78.2]	63.0 [53.5, 72.0]	71.5 [62.0, 79.8]	65.0 [56.2, 73.5]

Appendix

Table 17: Human evaluation at

N=32

: percentage of pair judgements preferring LatticeSMC (prescribed schedule, argmax) over each comparator, with 95% bootstrap intervals over the 40 conditions of each testbed; ties counted as one half. Each cell contains 40 distinct pairs and 200 judgements, five valid ratings per pair.

Explore similar work

May 29, 2026

Parallel Tempering Initial Sampling in Inference-Time Reward Alignment

Inference-time reward alignment steers pretrained diffusion and flow-based generative models to satisfy user-specified rewards without retraining. Recently, Sequential Monte Carlo (SMC) has emerged as a powerful framework for this task by iteratively filtering and propagating multiple particles. However, we show that standard SMC-based methods often suffer from poor performance because they initialize particles from a standard prior, whereas high-reward regions in complex reward landscapes are extremely rare. Further, we show that even recent reward-aware initial sampling approaches remain vulnerable to getting trapped in local modes, as complex reward landscapes are often multi-modal. To overcome these limitations, we propose PATHS (PArallel Tempering for High-complexity reward Sampling), a novel initialization method that couples multiple sampling chains through parallel tempering. PATHS maintains a ladder of reward-tempered chains and periodically performs Metropolis swaps, enabling efficient exploration across flattened reward landscapes, thereby mitigating the mode-trapping issues. Our analysis reveals that this mechanism substantially enhances the finite-budget exploration of rare, high-reward regions that are typically challenging to sample. Experiments on layout-to-image and quantity-aware generation show that PATHS achieves consistent gains in alignment quality, particularly on complex prompts.

Myeongjun Oh, Gwangho Kim, Sungyoon Lee

Department of Artificial Intelligence 2Department of Computer Science Hanyang University, Seoul, Korea

Diffusion Model Alignment Markov Chain Monte Carlo Diffusion Sampling Parallel Tempering

Ask PDF

Mar 9, 2026

Reject, Resample, Repeat: Understanding Parallel Reasoning in Language Model Inference

Inference-time methods that aggregate and prune multiple samples have emerged as a powerful paradigm for steering large language models, yet we lack any principled understanding of their accuracy-cost tradeoffs. In this paper, we introduce a route to rigorously study such approaches using the lens of particle filtering algorithms such as Sequential Monte Carlo (SMC). Given a base language model and a process reward model estimating expected terminal rewards, we ask: how accurately can we sample from a target distribution given some number of process reward evaluations? Theoretically, we identify (1) simple criteria enabling non-asymptotic guarantees for SMC; (2) algorithmic improvements to SMC; and (3) a fundamental limit faced by all particle filtering methods. Empirically, we demonstrate that our theoretical criteria effectively govern the sampling error of SMC, though not necessarily its final accuracy, suggesting that theoretical perspectives beyond sampling may be necessary.

Noah Golowich, Fan Chen, Dhruv Rohatgi +4

Microsoft Research · MIT · NYU

LLM Inference Inference-Time Guidance Efficient Language Model Reasoning

Ask PDF

Aug 20, 2026

Discrete Diffusion Inference-Time Control with Nested Sequential Monte Carlo

We study inference-time control for text generation in discrete diffusion language models, where the goal is to steer sampling toward sequence-level rewards without retraining. Prior work in this domain has focused on particle-based methods such as best-of-

n

sampling and bootstrap sequential Monte Carlo, which may suffer from overoptimism and weight degeneracy, respectively. We address these limitations using nested sequential Monte Carlo methods. We formulate nested SMC (NSMC) and fully-adapted nested SMC (FA-NSMC) for Feynman--Kac steering, identifying and correcting errors in prior formulations that lead to biased final estimates. We evaluate these methods on toxicity and fluency steering tasks, showing that NSMC and FA-NSMC consistently outperform best-of-

n

and bootstrap SMC.

Lohithsai Yadala Chanchu, Hany Abdulsamad, Christian A. Naesseth

University of Amsterdam

Inference-Time Guidance Diffusion Language Model Inference Diffusion Language Models

Ask PDF