Remasking allows diffusion language models to revisit generated text, but effective correction requires knowing when and what to revise. Our diagnosis on Sudoku reveals two gaps. Hidden states contain error information that token confidence does not reliably expose, and remasking an isolated error usually reproduces it when the surrounding context remains fixed. These findings motivate DiVR , a verifier that predicts whether a partially generated sequence will reach a correct answer. Trained on the final correctness of the model's own trajectories, DiVR decides when to regenerate a block and which alternative to retain, while the generator itself stays unchanged. On five reasoning benchmarks with LLaDA-8B-Base, DiVR raises the average accuracy of the fine-tuned generator from 35.3% to 43.4% at 1.9-3.5× the cost of greedy decoding, above search baselines that use 3.8-7.4×. It also complements post-training such as supervised fine-tuning and reinforcement learning, and generalizes to other generators, including Dream.
Figures & tables
Figure 1: Diagnosis on uniquely solvable 4×4 Sudoku. (a) Wrongness AUROC of hidden-state probes and confidence on matched observations. (b) After re-masking a wrong cell, the original digit stays highly probable and the gold digit rarely outranks it. (c) Gold-guided repair on 77 failures: reopening the wrong cells alone fixes few, and the per-puzzle best setting is an oracle upper bound.
Figure 2: DiVR-guided block re-decoding. DiVR scores the canvas, masked suffix included, after each block ( B=4 here, 32 in the experiments). A score below θ triggers four alternatives of the block from the same prefix, and the best-scoring state ( s1 , 0.91) replaces the original. Text and scores are illustrative.
Model / method
Math
Code
GSM8K
MATH-500
Countdown
HumanEval
MBPP
Avg.
Acc.
Cost
Acc.
Cost
Acc.
Cost
Acc.
Cost
Acc.
Cost
Acc.
LLaDA-8B-Base
Greedy
61.8
×1.0
24.2
×1.0
10.6
×1.0
19.5
×1.0
30.2
×1.0
29.3
+ DiVR-guided ∗
71.7
×1.9
26.8
×1.7
30.2
×2.6
23.8
×2.7
37.6
×2.8
38.0
+ SFT
68.7
×1.0
24.6
×1.0
35.6
×1.0
23.2
×1.0
24.2
×1.0
35.3
Table 1: Main results across three generators. Each panel is one model and each block one generator (untuned or SFT for LLaDA-8B-Base, released or diffu-GRPO for LLaDA-8B-Instruct); every method in a block runs on that generator, with one verifier for the math columns and one for the code columns. Cost is forwards relative to the generator’s paired greedy run, verification included. Bold marks the best decoding method of a generator; the best-of-4 rows use about four times the greedy budget and are not ranked. ∗ Verified by the verifiers of the SFT generator. ‡ After 3,800 steps of d1’s diffu-GRPO recipe on GSM8K ( Zhao et al., 2025 ) , with verifiers trained on its own decodes (Table 7 ).
Repair block
Flagged at block 0
Benchmark
Fixed
0
1
2
≥ 3
TPR
FPR
GSM8K
154
114
28
7
5
76%
22%
MATH-500
52
25
15
9
3
50%
14%
Countdown
59
58
1
0
0
99%
1%
Table 2: Where guided decoding repairs the greedy answer. SFT generator on the math benchmarks. Fixed counts the greedy errors that guided decoding corrects, out of 413 on GSM8K, 377 on MATH-500 and 322 on Countdown, and the repair block is the first block at which an alternative replaced the original. TPR and FPR treat a wrong greedy answer as positive and a first-block score below θ as a flag. Most repairs begin in the first block.
Evaluation signal
Success (%)
Ctotal
Commit confidence
72.2
2.45×
Prompted Yes/No
67.9
3.62×
Head-only verifier (V0)
74.2
3.07×
DiVR
78.8
2.10×
Table 3: Same block re-decoding, different scores. GSM8K test with the SFT generator (paired greedy 68.7). All signals share K , the temperature and the redo cap, and each threshold is the best on the development split. Ctotal is the verification-inclusive cost relative to greedy.
Selector
GSM8K
MATH-500
Countdown
Majority vote
75.7
28.8
38.2
Mean log-probability
68.5
23.6
34.6
DiVR best-of-4
82.2
34.2
49.2
Oracle pass@4
85.4
43.4
49.8
DiVR pair accuracy
After block 1
0.705
0.665
0.957
Table 4: Selection and ranking on a fixed candidate pool. Success (%) on the same four SFT-generator responses per problem. Oracle pass@4 is the pool’s ceiling, and a uniformly chosen candidate scores 68.7, 24.6 and 33.7. Pair accuracy is the share of correct–incorrect candidate pairs of a question that DiVR ranks correctly from their states after each block.
Figure 3: Sensitivity to the regeneration threshold. SFT generator, K=4 , at most three regenerations, on the full test sets. Solid lines give success (%) and dashed lines the verification-inclusive cost relative to greedy (square). The vertical line is θ=0.5 , chosen on the GSM8K development set beforehand, and ∗ marks a paired McNemar difference from it at p<0.05 .
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
LLaDA-8B-Base + SFT
LLaDA-8B-Instruct
Dream-v0-Instruct-7B
Benchmark
Greedy
Vote
Top
Cost
Greedy
Vote
Top
Cost
Greedy
Vote
Top
Cost
GSM8K
68.7
71.9
67.6
×5.9
79.2
83.5
82.8
×5.0
77.3
85.1
83.1
×5.5
MATH-500
24.6
32.4
29.0
×6.0
35.0
43.4
38.0
×4.8
40.8
48.2
49.6
×5.4
Countdown
35.6
39.8
40.8
×7.0
15.6
21.2
27.4
×4.9
18.0
29.2
46.2
×5.0
HumanEval
23.2
20.1
25.0
×5.2
34.1
28.0
36.0
×5.1
45.1
42.7
53.7
×5.9
MBPP
24.2
22.4
28.4
×7.4
33.2
31.6
36.6
×5.6
43.8
46.4
53.2
×7.5
Appendix
Table 5: Prism answer selection. Vote is the majority over the final trajectories, as in Bai et al. (2026) , and is the number in Table 1 . Top is the released code’s choice, the trajectory with the highest self-verification score. Cost is that of the vote, including every pruning judgment.
Generator / method
GSM8K
MATH-500
Countdown
HumanEval
MBPP
Acc.
Cost
Acc.
Cost
Acc.
Cost
Acc.
Cost
Acc.
Cost
LLaDA-8B-Instruct
Released, greedy
79.2
×1.0
35.0
×1.0
15.6
×1.0
34.1
×1.0
33.2
×1.0
Released + DiVR
83.0
×1.6
38.2
×1.8
23.0
×2.6
34.8
×2.6
38.0
×2.6
+ GSM8K SFT, greedy
74.7
×1.0
30.8
×1.0
11.4
×1.0
26.2
×1.0
28.8
×1.0
+ GSM8K SFT + DiVR
81.6
×1.9
37.0
×1.8
42.2
×2.7
26.8
×2.3
31.6
×2.2
Appendix
Table 6: GSM8K SFT of the instruction-tuned generators. Same SFT data and recipe as the LLaDA-8B-Base SFT generator, each with a verifier trained on its own decodes. All verifiers here are GSM8K-trained, code columns included.
Model / method
Math
Code
GSM8K
MATH-500
Countdown
HumanEval
MBPP
Avg.
Acc.
Cost
Acc.
Cost
Acc.
Cost
Acc.
Cost
Acc.
Cost
Acc.
LLaDA-8B-Instruct
Greedy (Table 1 )
79.2
×1.0
35.0
×1.0
15.6
×1.0
34.1
×1.0
33.2
×1.0
39.4
+ diffu-GRPO
77.4
×1.0
39.4
×1.0
17.0
×1.0
31.1
×1.0
35.4
×1.0
40.1
+ diffu-GRPO + SR-SMC (4) †
81.2
×4.1
37.2
×4.0
23.6
×3.9
32.3
×4.0
37.6
×4.1
42.4
Appendix
Table 7: diffu-GRPO generators under every method. LLaDA-8B-Instruct and the untuned LLaDA-8B-Base after 3,800 steps of d1’s diffu-GRPO recipe on GSM8K, each with DiVR verifiers trained on its own decodes. The greedy rows of the untrained generators repeat Table 1 . † Local reimplementations.
HumanEval
MBPP
Generator / verifier
Guided
Cost
Best-of-4
Pair
Guided
Cost
Best-of-4
Pair
LLaDA-8B-Base , untuned ∗ (greedy 19.5 / 30.2)
GSM8K verifier
22.0
×2.0
27.4
0.778
30.2
×2.0
30.2
0.659
MBPP verifier
23.8
×2.7
20.7
0.616
37.6
×2.8
33.4
0.757
LLaDA-8B-Base + SFT (greedy 23.2 / 24.2)
GSM8K verifier
24.4
×2.1
24.4
0.813
24.2
×2.0
26.8
0.759
Appendix
Table 8: Code benchmarks with the GSM8K-trained and the MBPP-trained verifier. The two verifiers of a generator differ only in their training decodes, and Table 1 reports the MBPP verifier in its code columns. Pair is the within-problem accuracy of ranking a passing finished sample above a failing one. ∗ Verified by the verifiers of the SFT generator.
State source / signal
Calibration
AUROC ↑
ECE ↓
Brier ↓
NLL ↓
Sampled policy (20,744 states, 1,319 problems)
DiVR
Raw
0.913
0.095
0.130
0.464
Platt
0.913
0.021
0.111
0.359
Greedy policy (1,209 states, 300 problems)
Commit confidence
Raw
0.625
0.186
0.253
0.736
Platt
0.625
0.033
0.219
0.628
Appendix
Table 9: Outcome prediction and post-hoc calibration. GSM8K test states at observed block boundaries. Each signal has its own Platt map fitted on development states. Calibration preserves the ordering, so raw and calibrated AUROC agree.
Figure 4: Verifier reliability on greedy GSM8K states. Mean score versus empirical success at the first non-final block and the final state (300 observations each), and markers are bins with at least ten observations. Platt parameters are fitted on development states.
θ
Regen.
Gain (pred. / real.)
Chosen-action gap
0.1
26%
11.0 / 12.2
+1.7 [ −3.4 , +7.0 ]
0.3
30%
10.9 / 10.6
+2.0 [ −2.6 , +7.0 ]
0.5
34%
10.3 / 0 9.5
+2.2 [ −2.4 , +7.8 ]
0.7
38%
0 9.6 / 0 8.4
+2.4 [ −2.4 , +7.3 ]
0.9
52%
0 7.8 / 0 7.6
+2.1 [ −2.5 , +6.8 ]
Appendix
Table 10: Decision calibration for regenerate-or-continue. 436 paired greedy states from 150 GSM8K test problems. A rule regenerates when the raw score is below θ , and predictions apply the development-fitted Platt map to the state each action leads to. The gap is predicted minus realized success of the chosen action, with a 95% bootstrap interval.
Candidate
Score
Decoded block
(a) Regeneration of the first block. Jillian’s handbag cost 20lessthan3timesasmuchashershoescost.Ifhershoescost80, how much did her bag cost? (Answer 220, greedy answer 200.)
Original
0.019
Three times the cost of her shoes is 3*80=220\hookleftarrow20 less than 220 is 22
Resample 1
0.395
Three times as much as her shoes cost is 3*80=240 ↩20lessthan80 is $84
Resample 2
0.990
Let x be the cost of the bag. ↩ x = 3 * 80 = 240. ↩ x = 240
Resample 3
0.018
Three times the cost of her shoes is 3*80=20 ↩20lessthan20 is 20−
Resample 4
0.016
Three times the cost of her shoes is 3 * 80=200 ↩ Jillian’s handbag cost 200−
Appendix
Table 11: Regenerations that fix greedy errors. GSM8K test problems on which the SFT generator’s greedy answer is wrong and one regeneration corrects it. Each panel shows the block that fell below θ (Original), the four alternatives decoded from the same state, and their scores. The highest-scoring candidate (bold) is kept. ↩ marks a line break.