Learning When Diffusion Language Models Should Regenerate
Organizations: Seoul National University · KAIST
Abstract
Remasking allows diffusion language models to revisit generated text, but effective correction requires knowing when and what to revise. Our diagnosis on Sudoku reveals two gaps. Hidden states contain error information that token confidence does not reliably expose, and remasking an isolated error usually reproduces it when the surrounding context remains fixed. These findings motivate DiVR , a verifier that predicts whether a partially generated sequence will reach a correct answer. Trained on the final correctness of the model's own trajectories, DiVR decides when to regenerate a block and which alternative to retain, while the generator itself stays unchanged. On five reasoning benchmarks with LLaDA-8B-Base, DiVR raises the average accuracy of the fine-tuned generator from 35.3% to 43.4% at - the cost of greedy decoding, above search baselines that use -. It also complements post-training such as supervised fine-tuning and reinforcement learning, and generalizes to other generators, including Dream.
Figures & tables
| Model / method | Math | Code | |||||||||
| GSM8K | MATH-500 | Countdown | HumanEval | MBPP | Avg. | ||||||
| Acc. | Cost | Acc. | Cost | Acc. | Cost | Acc. | Cost | Acc. | Cost | Acc. | |
| LLaDA-8B-Base | |||||||||||
| Greedy | 61.8 | 24.2 | 10.6 | 19.5 | 30.2 | 29.3 | |||||
| + DiVR-guided ∗ | 71.7 | 26.8 | 30.2 | 23.8 | 37.6 | 38.0 | |||||
| + SFT | 68.7 | 24.6 | 35.6 | 23.2 | 24.2 | 35.3 | |||||
| Repair block | Flagged at block 0 | ||||||
|---|---|---|---|---|---|---|---|
| Benchmark | Fixed | 0 | 1 | 2 | 3 | TPR | FPR |
| GSM8K | 154 | 114 | 28 | 7 | 5 | 76% | 22% |
| MATH-500 | 52 | 25 | 15 | 9 | 3 | 50% | 14% |
| Countdown | 59 | 58 | 1 | 0 | 0 | 99% | 1% |
| Evaluation signal | Success (%) | |
|---|---|---|
| Commit confidence | 72.2 | |
| Prompted Yes/No | 67.9 | |
| Head-only verifier (V0) | 74.2 | |
| DiVR | 78.8 |
| Selector | GSM8K | MATH-500 | Countdown |
|---|---|---|---|
| Majority vote | 75.7 | 28.8 | 38.2 |
| Mean log-probability | 68.5 | 23.6 | 34.6 |
| DiVR best-of-4 | 82.2 | 34.2 | 49.2 |
| Oracle pass@4 | 85.4 | 43.4 | 49.8 |
| DiVR pair accuracy | |||
| After block 1 | 0.705 | 0.665 | 0.957 |
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
| LLaDA-8B-Base + SFT | LLaDA-8B-Instruct | Dream-v0-Instruct-7B | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Benchmark | Greedy | Vote | Top | Cost | Greedy | Vote | Top | Cost | Greedy | Vote | Top | Cost |
| GSM8K | 68.7 | 71.9 | 67.6 | 79.2 | 83.5 | 82.8 | 77.3 | 85.1 | 83.1 | |||
| MATH-500 | 24.6 | 32.4 | 29.0 | 35.0 | 43.4 | 38.0 | 40.8 | 48.2 | 49.6 | |||
| Countdown | 35.6 | 39.8 | 40.8 | 15.6 | 21.2 | 27.4 | 18.0 | 29.2 | 46.2 | |||
| HumanEval | 23.2 | 20.1 | 25.0 | 34.1 | 28.0 | 36.0 | 45.1 | 42.7 | 53.7 | |||
| MBPP | 24.2 | 22.4 | 28.4 | 33.2 | 31.6 | 36.6 | 43.8 | 46.4 | 53.2 | |||
| Generator / method | GSM8K | MATH-500 | Countdown | HumanEval | MBPP | |||||
| Acc. | Cost | Acc. | Cost | Acc. | Cost | Acc. | Cost | Acc. | Cost | |
| LLaDA-8B-Instruct | ||||||||||
| Released, greedy | 79.2 | 35.0 | 15.6 | 34.1 | 33.2 | |||||
| Released + DiVR | 83.0 | 38.2 | 23.0 | 34.8 | 38.0 | |||||
| + GSM8K SFT, greedy | 74.7 | 30.8 | 11.4 | 26.2 | 28.8 | |||||
| + GSM8K SFT + DiVR | 81.6 | 37.0 | 42.2 | 26.8 | 31.6 | |||||
| Model / method | Math | Code | |||||||||
| GSM8K | MATH-500 | Countdown | HumanEval | MBPP | Avg. | ||||||
| Acc. | Cost | Acc. | Cost | Acc. | Cost | Acc. | Cost | Acc. | Cost | Acc. | |
| LLaDA-8B-Instruct | |||||||||||
| Greedy (Table 1 ) | 79.2 | 35.0 | 15.6 | 34.1 | 33.2 | 39.4 | |||||
| + diffu-GRPO | 77.4 | 39.4 | 17.0 | 31.1 | 35.4 | 40.1 | |||||
| + diffu-GRPO + SR-SMC (4) † | 81.2 | 37.2 | 23.6 | 32.3 | 37.6 | 42.4 | |||||
| HumanEval | MBPP | |||||||
| Generator / verifier | Guided | Cost | Best-of-4 | Pair | Guided | Cost | Best-of-4 | Pair |
| LLaDA-8B-Base , untuned ∗ (greedy 19.5 / 30.2) | ||||||||
| GSM8K verifier | 22.0 | 27.4 | 0.778 | 30.2 | 30.2 | 0.659 | ||
| MBPP verifier | 23.8 | 20.7 | 0.616 | 37.6 | 33.4 | 0.757 | ||
| LLaDA-8B-Base + SFT (greedy 23.2 / 24.2) | ||||||||
| GSM8K verifier | 24.4 | 24.4 | 0.813 | 24.2 | 26.8 | 0.759 | ||
| State source / signal | Calibration | AUROC | ECE | Brier | NLL |
|---|---|---|---|---|---|
| Sampled policy (20,744 states, 1,319 problems) | |||||
| DiVR | Raw | 0.913 | 0.095 | 0.130 | 0.464 |
| Platt | 0.913 | 0.021 | 0.111 | 0.359 | |
| Greedy policy (1,209 states, 300 problems) | |||||
| Commit confidence | Raw | 0.625 | 0.186 | 0.253 | 0.736 |
| Platt | 0.625 | 0.033 | 0.219 | 0.628 | |
| Regen. | Gain (pred. / real.) | Chosen-action gap | |
|---|---|---|---|
| 0.1 | 26% | 11.0 / 12.2 | [ , ] |
| 0.3 | 30% | 10.9 / 10.6 | [ , ] |
| 0.5 | 34% | 10.3 / 0 9.5 | [ , ] |
| 0.7 | 38% | 0 9.6 / 0 8.4 | [ , ] |
| 0.9 | 52% | 0 7.8 / 0 7.6 | [ , ] |
| Candidate | Score | Decoded block |
| (a) Regeneration of the first block. Jillian’s handbag cost 80, how much did her bag cost? (Answer 220, greedy answer 200.) | ||
| Original | 0.019 | Three times the cost of her shoes is 3*\hookleftarrow20 less than 220 is 22 |
| Resample 1 | 0.395 | Three times as much as her shoes cost is 3*240 80 is $84 |
| Resample 2 | 0.990 | Let x be the cost of the bag. x = 3 * 80 = 240. x = 240 |
| Resample 3 | 0.018 | Three times the cost of her shoes is 3*20 20 is |
| Resample 4 | 0.016 | Three times the cost of her shoes is 3 * 200 Jillian’s handbag cost |
Diffusion Language Models
Diffusion language models generate and refine text through iterative denoising, enabling bidirectional context and parallel token updates.
See all 494 papers