cs.CLNov 7, 2026

Test-Time Scaling for Diffusion Language Models with Frequency-Aware Remasking

Authors: Bowen Zuo, Yue Yu, Dongruo Zhou, Yinglun Zhu

Organizations: University of California, Riverside · Indiana University

Abstract

Diffusion language models (DLMs) generate text through iterative denoising, where decisions about which tokens to commit or remask strongly affect output quality. Existing certainty-aware strategies rely on instantaneous confidence or entropy, but these local signals can be misleading on difficult reasoning problems. We formulate DLM test-time scaling as adaptive allocation of both inference compute and decoding strategies. Our two-stage approach uses a fast base sampler to solve easy questions, then applies frequency-aware remasking to the remaining hard questions. This strategy tracks how often each token's prediction changes across denoising steps and preferentially commits frequently changing tokens, encouraging samples to explore alternative reasoning paths rather than collapse onto the model's preferred trajectory. Experiments with LLaDA-8B-Instruct and Dream-7B-Instruct show consistent improvements over a strong adaptive allocation baseline, with absolute accuracy gains of up to 3.73 percentage points (LLaDA) and 7.33 percentage points (Dream) on MATH-500, and absolute coverage gains of up to 11.39 percentage points (LLaDA) and 10.98 percentage points (Dream) on HumanEval. On questions unsolved by the base certainty-aware strategy after 100 samples, frequency-aware remasking recovers solutions for 3.75% to 18.75% of examples.

Figures & tables

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. S3S^3: Stratified Scaling Search for Test-Time in Diffusion Language Models

    Apr 7, 2026Ahsan Bilal, Muhammad Ahmed Mohsin, Muhammad Umer +6Test-Time ScalingDiffusion Language Model Decoding

  2. Random Remasking Scales Masked Diffusion Inference

    Oct 22, 2026Zhaoyi Zhou, Frank Wu, Andrea ZanetteMasked Diffusion ModelsDiffusion Language Model Inference

  3. Locally Confident, Globally Stuck: The Quality-Exploration Dilemma in Diffusion Language Models

    Apr 1, 2026Liancheng Fang, Aiwei Liu, Henry Peng Zou +7Diffusion Language Model DecodingDiffusion Sampling