Masked diffusion language models have emerged as an alternative to autoregressive models, enabling parallel token generation. However, standard masked diffusion language models use a shared noise schedule across positions without explicitly encoding which positions should be resolved first. Many tasks exhibit a natural priority among positions: document titles and section headings can guide the generation of body text, while Sudoku cells determined directly from the initial clues can help resolve more difficult cells. To incorporate this notion of priority (or hierarchy) among token positions, we introduce the Hierarchical Masked Diffusion Language Model (HMDLM), which augments each token position with a hierarchy level available as part of the training data, and requires zero or minimal additional annotations. It factors generation into two passes: the first predicts a hierarchy distribution over masked positions, the second predicts token distributions conditioned on it. Giving each hierarchy level its own noise schedule induces a soft, controllable preference for revealing higher-priority positions earlier while keeping decoding fully parallel. Our formulation is mathematically grounded, and we re-derive the forward and reverse processes and the NELBO for this factorization. Our evaluation shows that for the task of language generation on LM1B, HMDLM reduces perplexity by more than 1.5 points over Anchored Diffusion Language Model (ADLM), a recently proposed strong diffusion baseline; on WikiHow, we see a drop in perplexity of up to 4.4 pts on ADLM and up to 7 pts against MDLM. An LLM judge on WikiHow prefers HMDLM generations (the two variants combined) more than 2× as often as the baseline. On the task of solving Sudoku puzzles requiring multi-step reasoning, it improves solution accuracy by up to 8.6% absolute points over existing diffusion baselines.
Figures & tables
Figure 1: HMDLM inference example. Positions i=0,3,5 are masked in (xt,ht) . Step t : Stage 1 reveals ht−15=1 , then Stage 2 predicts xt−15=ball . Step t−1 : Stage 1 reveals ht−20=0 and ht−23=0 , then Stage 2 predicts xt−20=The and xt−23=the .
Model
PPL ( ↓ )
33B tokens
Diffusion BERT ( He et al., 2023 )
≤63.78
SEDD ( Lou et al., 2024 )
≤32.79
MDLM ( Sahoo et al., 2024 )
≤27.04
ADLM ( Rout et al., 2025 )
≤26.40
HMDLM
≤24.30
Table 1: Test perplexities (PPL ↓ ) on LM1B. Best value is bolded.
Model
Gen. PPL ( ↓ )
MAUVE ⋆ ( ↑ )
Entropy
MDLM
177.94±0.82
0.66±0.03
6.022±0.003
HMDLM
83.35±0.83
0.76±0.06
5.876±0.014
Table 2: LM1B generative evaluation on 65 B checkpoints. Mean ± std over 3 seeds.
L
MDLM
HMDLM-P †
ADLM
HMDLM-F ‡
256
24.60±0.22
23.89±0.11
22.05±0.21
17.62±0.12
512
21.04±0.06
21.76±0.15
20.15±0.10
16.64±0.06
1024
22.65±0.04
22.58±0.04
20.92±0.05
16.90±0.09
Table 3: Model PPL ( ↓ ) across sequence lengths on WikiHow. Best value is in bold.
Gen. PPL
MAUVE ( ↑ )
Entropy
Model
256
512
1024
256
512
1024
256
512
1024
Gold
25.2
21.0
20.8
1.00
1.00
1.00
5.86
6.42
6.70
MDLM
76.0
67.4
62.7
0.33
0.11
0.12
6.47
7.10
7.60
ADLM
84.0
82.1
93.81
0.40
0.21
0.08
6.49
7.17
7.77
HMDLM-P
23.1
17.6
14.6
0.38
0.13
0.07
5.91
6.26
6.45
HMDLM-F
29.2
25.4
19.92
0.52
0.41
0.20
5.98
6.60
6.83
Table 4: Generative evaluation on WikiHow across sequence lengths L∈{256,512,1024} . HMDLM-P/F: position-/frequency-based hierarchy. Gold is the reference data.
Table 6: Sudoku-hard test set ( 18 k puzzles). Results are from 6 M-parameter models trained for 100 k steps with T=81 and confidence-based sampling. Mean ± std over 3 sampling seeds.
Appendix figures & tables10 assets
Supplementary material from the paper’s appendix.
Appendix
Model
g1
ρ=1−g1/3
Steps
Extra steps (epochs)
L=512
MDLM
–
1.000
28,650
–
HMDLM (summary)
22.34%
0.926
30,955
+2,305 ( +2.40 )
HMDLM ( τ=1 )
30.82%
0.897
31,930
+3,280 ( +3.41 )
L=1024
MDLM
–
1.000
41,137
–
Appendix
Table 7: Step-matched training budgets on WikiHow at context lengths 512 and 1024. g1 is the fraction of tokens at level 1. Under the summary rule, level 1 is the article summary. Under the count rule ( τ=1 ), level 1 is any token occurring at most once in its own sample.
Discrete diffusion language models offer a compelling alternative to autoregressive generation for tasks demanding bidirectional reasoning and global constraint satisfaction. Yet they share a structural bottleneck: when decoding in parallel, each token is sampled independently from its marginal, severing the statistical dependencies among the tokens decoded together. Continuous diffusion language models avoid this by denoising a shared continuous state, but their denoiser sees only that state, so nothing ties it to a valid token configuration until it is finally decoded. To address this, we propose Hierarchical Continuous Diffusion Language Models (HC-DLM), which couple discrete token generation with a continuous latent trajectory in a single, principled denoising process, whose training objective is derived from a variational bound on the token likelihood. In contrast to recent methods that attach continuous context to a self-contained discrete chain, HC-DLM makes the latent the only persistent generative state: tokens are read out from it at every step and feed back as a scaffold for the next latent update. On structured reasoning (Sudoku), mathematical planning (Countdown) and language modeling (LM1B), HC-DLM improves over discrete and continuous diffusion baselines at matched model size, in puzzle accuracy on Sudoku and Countdown and in generative perplexity on LM1B. Project page: https://hc-dlm.github.io/.
Hui Ren, Zihan Li, Chang Liu +2
University of Illinois Urbana-Champaign · Amazon.com, Inc.
Reinforcement learning has proven effective for improving reasoning in large language models, but extending it to Masked Diffusion Language Models (MDLMs) remains challenging due to the intractability of the log-likelihood estimation. Existing approaches approximate this log-likelihood by modeling only the token predictions, ignoring the order in which positions are unmasked during generation. We observe that MDLM generation involves two decisions at each step: what tokens to place at each masked position and which positions to remask. We formalize this as a two-stage action MDP, showing that the policy gradient naturally decomposes into a token term and a masking term. Combining optimization of both terms leads to state-of-the-art outcomes on mathematical reasoning and coding benchmarks, with scores of 87.1% on GSM8K and 53.4% on MBPP.
Haran Raajesh, Kulin Shah, Adam Klivans +1
Department of Computer Science The University of Texas at Austin
Masked diffusion language models enable parallel token generation and offer improved decoding efficiency over autoregressive models. However, their performance degrades significantly when generating multiple tokens simultaneously, due to a mismatch between token-level training objectives and joint sequence consistency. In this paper, we propose ME-DLM, an edit-based refinement framework that augments diffusion generation with lightweight post-editing steps. After producing an initial complete response, the model refines it through minimal edit operations, including replacement, deletion, and insertion, conditioned on the full sequence. Training supervision is derived from edit distance, providing a deterministic signal under a fixed canonicalization scheme for learning minimal corrections. This approach encourages sequence-level consistency through globally conditioned edits while preserving the efficiency benefits of parallel diffusion decoding. Extensive experiments demonstrate that ME-DLM improves the quality and robustness of multi-token parallel generation. In particular, when built upon LLaDA, our method achieves consistent gains of 11.6 points on HumanEval and 33.6 points on GSM8K while using one-eighth of the total diffusion steps. Code is available at https://github.com/renhouxing/ME-DLM.
Houxing Ren, Mingjie Zhan, Zimu Lu +5
CUHK MMLab · SenseTime Research · Shenzhen Loop Area Institute +1