Inference-time sequential Monte Carlo (SMC) decoding can improve the reasoning accuracy of masked diffusion language models (diffusion LLMs) without post-training by resampling a population of partial generations according to a reward. Its effectiveness therefore depends on the resampling step, which must concentrate computation on promising candidates while retaining enough diversity to explore alternative reasoning paths. The reward can be internal, the decoder's own confidence on the committed prefix, or external, a process reward model scoring that prefix. Under either reward, multinomial resampling can disrupt this balance by duplicating high-weight particles and discarding lineages that may still reach a correct solution. The effective sample size (ESS) does not register this loss: multinomial resampling resets it to its maximum even as distinct lineages disappear. To retain such lineages, we introduce lineage-preserving SMC (LP-SMC), which combines Chopthin resampling with an input weight floor. Instead of equalizing weights, Chopthin bounds the ratio between the largest and smallest weights it returns and carries the unequal weights forward. The floor limits how far below the maximum a weight may fall before resampling so that lineages with low early weight keep a chance to recover. Across mathematical-reasoning and code-generation benchmarks on two base models, LP-SMC raises oracle coverage, the fraction of problems with at least one correct particle, in every setting, by 1.8 to 25.6 points, and raises end-to-end accuracy under a common reward-model selector in eleven of twelve settings, by up to 7.9 points on LLaDA-1.5 and 18.3 on Dream-7B. The coverage gain persists under the external reward model, which suggests that diversity-preserving resampling and the guidance signal contribute separately to inference-time search.
Figures & tables
Figure 1: Multinomial resampling can remove a useful lineage before its completion is known. Illustrative example, N=4 , problem (1+2)×3 , answer 9 ; one token per box, two blocks, resampling after block 1 ( ESS=1.9<N/2 ). Colors are founding lineages, bars are normalized weights; the blue lineage has the largest weight but ends wrong, orange and teal end right. (a) Multinomial resampling: three blue copies and one orange, two lineages left, ESS reset to 4 . (b) Chopthin: blue, orange and teal survive with unequal weights, three lineages, ESS=3.3 . Resampling copies states and assigns weights; the token differences arise in later decoding.
Figure 2: Oracle coverage, the fraction of problems with at least one correct particle ( N=16 , full test sets), under the decoder’s confidence (conf) or the external PRM in the weight. LP-SMC is higher in all twelve settings. Bars: unpaired 95% binomial intervals; paired counts and bootstrap intervals: Appendix E . The axis is clipped at 40 .
Model
Guidance
Population management
MATH-500
GSM8K
HumanEval
MBPP
LLaDA-1.5
Internal
SR-SMC
49.0
88.2
48.2
49.8
LP-SMC
52.4
90.5
56.1
50.6
External PRM
Multinomial SMC
53.8
90.4
n.a.
n.a.
LP-SMC
55.0
90.8
n.a.
n.a.
Dream-7B
Internal
SR-SMC
41.8
78.2
48.8
59.8
LP-SMC
46.8
86.7
67.1
62.0
Table 1: End-to-end accuracy under reward-model selection (Qwen2.5-Math-PRM-7B ( Yang et al., 2024 ) on mathematics, AceCodeRM-7B ( Zeng et al., 2025 ) on code), N=16 , L=256 , paired problems, full test sets. Internal guidance: the decoder’s confidence in the weight; External PRM: the reward model in the weight. n.a.: no step-level code reward model meeting our criteria is available, and AceCodeRM-7B, an outcome model, can rank finished programs but not weight a prefix (Appendix C ). Paired tests: Appendix E .
Surviving lineages
Distinct outputs
Coverage (%)
Model
Benchmark
Mult.
Chop.
Chop.+floor
Mult.
Chop.
Chop.+floor
Mult.
Chop.
Chop.+floor
LLaDA-1.5
MATH-500
2.71
3.93
6.98
6.90
8.71
10.53
54.2
57.2
63.6
GSM8K
5.21
7.52
10.04
7.80
9.65
11.28
90.8
92.9
94.2
HumanEval
2.96
4.58
7.79
2.45
3.56
5.96
50.0
54.9
59.8
MBPP
10.36
11.76
13.45
5.88
6.89
8.37
55.6
56.2
58.7
Dream-7B
MATH-500
1.55
2.18
5.37
3.47
4.17
7.70
44.2
47.4
58.8
Table 2: Adding one component at a time under the decoder’s confidence: multinomial resampling (Mult.), Chopthin without the floor (Chop.), Chopthin with the floor (Chop.+floor), which is LP-SMC. Lineages: mean number of founding particles still represented at the end, out of 16; distinct outputs: mean number of distinct x0 per problem; coverage as in Figure 2 . Table 3 crosses the floor with the resampler. Benchmark sizes are as in Table 4 .
Configuration
Surviving lineages
Distinct outputs
Lost per event
Coverage (%)
Multinomial
5.21
7.80
9.14
90.8
Multinomial + floor
6.25
8.80
8.48
92.5
Systematic
6.21
8.43
7.98
90.8
Systematic + floor
7.62
9.48
6.97
93.6
Chopthin
7.52
9.65
5.41
92.9
Chopthin + floor (LP-SMC)
10.04
11.28
3.61
94.2
Table 3: Ancestor draw and floor separated: LLaDA-1.5 on GSM8K ( n=1319 ), decoder confidence, all configurations paired. Systematic resampling is the standard variance-reduced draw. The six rows are a full factorial of ancestor draw against weight floor. Lost per event: mean number of the 16 particles left without offspring at a resampling event (lower is better).
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 3: Floor depth on MATH-500 (LLaDA-1.5, N=16 , reward-model selector). From left to right: multinomial resampling, Chopthin without the floor, and Chopthin with the floor at 1/32 , 1/16 ( κ=lnN , used throughout) and 1/8 of the maximum weight. Among the Chopthin configurations, accuracy and surviving lineages rise monotonically as the floor is raised.
Benchmark
Guidance
Population management
Confidence
Majority
Distinct
Reward model
MATH-500 n=500
Internal
SR-SMC
41.8
41.4
41.2
49.0
LP-SMC
39.2
44.4
45.0
52.4
External PRM
Multinomial SMC
48.8
52.6
51.4
53.8
LP-SMC
43.6
52.6
50.2
55.0
GSM8K n=1319
Internal
SR-SMC
82.7
84.4
84.8
88.2
LP-SMC
83.5
86.4
86.8
90.5
Appendix
Table 4: Accuracy under four selection rules applied to the same final populations (LLaDA-1.5, N=16 , L=256 ). Guidance is where the reward model acts during generation; the Reward model column is a selection rule applied afterwards to the finished population (Qwen2.5-Math-PRM-7B on mathematics, AceCodeRM-7B on code) and is available under either guidance. Bold: better within a pair.
Benchmark
Guidance
Population management
Confidence
Majority
Distinct
Reward model
MATH-500
Internal
SR-SMC
37.8
42.2
42.6
41.8
n=500
LP-SMC
33.4
47.0
49.6
46.8
External PRM
Multinomial SMC
44.0
54.2
53.6
49.2
LP-SMC
40.8
55.0
54.6
48.2
GSM8K
Internal
SR-SMC
77.6
77.7
78.1
78.2
n=1319
LP-SMC
70.6
72.9
81.3
86.7
Appendix
Table 5: Dream-7B under every selection rule. As in Table 4 , Guidance is where the reward model acts during generation and the Reward model column is a selection rule applied afterwards to a finished population; the two are independent. Bold: better within a pair.
Model
Repository
License
Type
AceCodeRM-7B/32B
TIGER-Lab/AceCodeRM-*
MIT
O
CodeScaler-1.7B–8B
LARK-Lab/CodeScaler-*
MIT
O
Themis-RM-0.6B–32B
project-themis/Themis-RM-*
Apache-2.0
O
CodePRM
SIMONLQY/CodePRM
not declared
P
Appendix
Table 6: Open-weight code reward models considered for function-completion experiments (September 2026). Type O: outcome model, one score per finished program; P: step-level process model, the only kind we regard as suitable inside a resampling weight. SWE-Shepherd is not listed because it scores repository-agent actions on SWE-Bench rather than function-completion steps.
Benchmark
Population management
Confidence
Coverage
HumanEval ( n=164 )
Multinomial SMC
50.0
59.1
LP-SMC
42.7
63.4
MBPP ( n=482 )
Multinomial SMC
42.7
58.1
LP-SMC
41.5
60.4
Appendix
Table 7: Both configurations on code under External PRM guidance, the mathematics PRM in the weight and so out of distribution by construction. Guidance is identical across configurations, so the comparison isolates the LP-SMC population-management intervention.
Model
Population management
Correct
Median
>0.5
>0.8
>0.9
LLaDA-1.5
SR-SMC
1198
0.625
64.4%
24.6%
9.5%
LP-SMC
1243
0.780
87.4%
31.1%
0.0%
Dream-7B
SR-SMC
1036
0.118
20.5%
6.4%
2.7%
LP-SMC
1186
0.792
95.7%
42.1%
0.0%
Appendix
Table 8: Worst-case deletion probability (1−wρ)N faced by a lineage that produced a correct answer, GSM8K ( n=1319 ): median, and the fraction of correctly answered problems above each threshold. Per problem the maximum is taken jointly over every fired resampling event and every correct lineage.
Model
Guidance
Benchmark
Quantity
n
a – b
Δ
95% CI
p
LLaDA-1.5
Internal
MATH-500
Accuracy
500
44 – 27
+3.4
[+0.2,+6.8]
0.057
LLaDA-1.5
Internal
GSM8K
Accuracy
1319
76 – 45
+2.4
[+0.7,+4.0]
0.006
LLaDA-1.5
Internal
HumanEval
Accuracy
164
13 – 0
+7.9
[+4.3,+12.2]
0.00024
LLaDA-1.5
Internal
MBPP
Accuracy
482
14 – 10
+0.8
[−1.0,+2.9]
0.541
LLaDA-1.5
External PRM
MATH-500
Accuracy
500
26 – 20
+1.2
[−1.4,+3.8]
0.461
LLaDA-1.5
External PRM
GSM8K
Accuracy
1319
44 – 39
+0.4
[−1.0,+1.7]
0.661
Appendix
Table 9: Paired statistics for every cell of Table 1 and every pair of Figure 2 . a – b : discordant counts. CI: percentile bootstrap on the difference, 10,000 resamples over problems. Differences are computed from the underlying counts; displayed accuracy and coverage percentages elsewhere are rounded.
Inference-time power sampling via Sequential Monte Carlo (SMC) can substantially improve large language model (LLM) reasoning without requiring post-training. However, many existing SMC approaches rely on equal-weight resampling, which can aggressively prune low-weight trajectories, discarding potentially correct reasoning paths and degrading the genealogical diversity of the search space. To address this, we introduce Chopthin-Consensus Power Sampling (CCPS). Our method applies the Chopthin resampler to LLM decoding: rather than equalizing weights and forcing unnecessary particle duplication, it enforces an upper bound on the ratio between the largest and smallest weights and carries the unequal weights forward. This targeted intervention preserves a richer set of distinct reasoning paths, keeps the weighted SMC approximation unchanged in conditional expectation, and guarantees a lower bound on the post-resampling effective sample size (ESS). To fully exploit this enriched population, we employ a semantic-majority selection mechanism that merges token-identical final trajectories, clusters semantically equivalent answers, and returns the answer supported by the largest number of distinct trajectories. Evaluating across three open-weight models and five reasoning benchmarks, we show that Chopthin increases oracle coverage in 13 of 15 settings. Combined with semantic-majority selection, CCPS matches or exceeds the final-answer accuracy of the Power-SMC baseline in 14 of 15 settings, delivering absolute gains of up to 10.6 percentage points. These findings demonstrate that diversity-preserving resampling and diversity-aware selection are complementary mechanisms for training-free LLM reasoning. Code is available at github.com/MinooAhmadii/chopthin-consensus-power-sampling.
Speculative decoding (SD) accelerates language model inference by drafting tokens from a cheap proposal model and verifying them against an expensive target model via rejection sampling. Because rejection truncates the draft block at the first error, throughput degrades when draft and target diverge. Rather than rejecting draft tokens outright, we propose to reweight them. To this end, we introduce sequential Monte Carlo speculative decoding (SMC-SD), which replaces token-level rejection with importance-weighted resampling over a population of draft particles. SMC-SD is a principled approximate inference scheme that trades exactness for additional speed, while preserving theoretical bounds on its per-step approximation error. Because LLM inference is memory bandwidth-bound, the arithmetic needed to draft particles and to score them in parallel comes nearly for free -- SMC-SD uses idle compute to turn verification into a vectorized, fixed-size operation with no rollback. Empirically, SMC-SD achieves 2.36x speed-up over speculative decoding and a 5.2x speed-up over autoregressive decoding, while remaining within 3% of the target model's accuracy on reasoning, instruction-following, and coding benchmarks.
Yahya Emara, Mauricio Barba da Costa, Chi-Chih Chang +4
Inference-time methods that aggregate and prune multiple samples have emerged as a powerful paradigm for steering large language models, yet we lack any principled understanding of their accuracy-cost tradeoffs. In this paper, we introduce a route to rigorously study such approaches using the lens of particle filtering algorithms such as Sequential Monte Carlo (SMC). Given a base language model and a process reward model estimating expected terminal rewards, we ask: how accurately can we sample from a target distribution given some number of process reward evaluations? Theoretically, we identify (1) simple criteria enabling non-asymptotic guarantees for SMC; (2) algorithmic improvements to SMC; and (3) a fundamental limit faced by all particle filtering methods. Empirically, we demonstrate that our theoretical criteria effectively govern the sampling error of SMC, though not necessarily its final accuracy, suggesting that theoretical perspectives beyond sampling may be necessary.