cs.CLNov 12, 2026

Learning When Diffusion Language Models Should Regenerate

Authors: Minhae Oh, Jeonghye Kim, Jinwoo Koo, Yeongoon Kim, Jimin Seo, Minsik Seol, Jungwoo Lee

Organizations: Seoul National University · KAIST

Abstract

Remasking allows diffusion language models to revisit generated text, but effective correction requires knowing when and what to revise. Our diagnosis on Sudoku reveals two gaps. Hidden states contain error information that token confidence does not reliably expose, and remasking an isolated error usually reproduces it when the surrounding context remains fixed. These findings motivate DiVR , a verifier that predicts whether a partially generated sequence will reach a correct answer. Trained on the final correctness of the model's own trajectories, DiVR decides when to regenerate a block and which alternative to retain, while the generator itself stays unchanged. On five reasoning benchmarks with LLaDA-8B-Base, DiVR raises the average accuracy of the fine-tuned generator from 35.3% to 43.4% at 1.91.9-3.5×3.5\times the cost of greedy decoding, above search baselines that use 3.83.8-7.4×7.4\times. It also complements post-training such as supervised fine-tuning and reinforcement learning, and generalizes to other generators, including Dream.

Figures & tables

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Random Remasking Scales Masked Diffusion Inference

    Oct 22, 2026Zhaoyi Zhou, Frank Wu, Andrea ZanetteMasked Diffusion ModelsDiffusion Language Model Inference

  2. Corrective Diffusion Language Models

    Dec 17, 2025Shuibai Zhang, Fred Zhangzhi Peng, Yiheng Zhang +2Diffusion Language Model TrainingDiffusion Language Models

  3. Breaking Diffusion: Parallel Reasoning with Draft Refinement

    Oct 19, 2026Jaesang Yu, Seoyeon Byun, Yun Wang +2Diffusion Language ModelsParallel Reasoning in Language Models