cs.LGNov 17, 2026

ReMaRL: Learning to Unmask and Remask in Frozen Diffusion Language Models

Authors: Egor Shibaev, Danila Biktimirov, Ivan Klimov, Artem Makoian, Viacheslav Meshchaninov, Dmitry Vetrov

Organizations: JetBrains Research · Constructor University · JetBrains

Abstract

Masked diffusion language models generate tokens in parallel, and the order in which tokens are unmasked affects final-answer accuracy. Several methods learn unmasking policies from task rewards, but these policies cannot remask earlier predictions. Remasking lets the model correct those predictions using newly generated context. Heuristic rules guide both unmasking and remasking, while supervised remaskers learn to detect token errors. Neither approach directly optimizes these decisions for final-answer correctness. To address this gap, we introduce ReMaRL (Remasking with Reinforcement Learning), a lightweight policy that learns which tokens to unmask, retain, or remask. We train this roughly 0.3M-parameter policy with Group Relative Policy Optimization (GRPO) from final-answer rewards while keeping the diffusion model frozen. With LLaDA-8B-Instruct, ReMaRL achieves the highest scores among the compared unmasking and remasking policies within the tested inference budgets of 4-32 model calls on math and 8-16 calls on code.

Figures & tables

Appendix figures & tables24 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Remask, Don't Replace: Token-to-Mask Refinement in Diffusion Large Language Models

    Apr 20, 2026Lin YaoLLM Self-CorrectionMulti-Token Prediction

  2. Mask-Aware Policy Gradients for Diffusion Language Models

    Jul 16, 2026Haran Raajesh, Kulin Shah, Adam Klivans +1Masked Diffusion ModelsRL for Language Model Reasoning

  3. Targeted Remasking: Replacing Token Editing with Token-to-Mask Refinement in Discrete Diffusion Language Models

    Apr 6, 2026Lin YaoDiffusion Language ModelsDiffusion Language Model Decoding