dLoop: Looped Diffusion Language Models via Recurrent Latent Refinement
Authors: Handa Li, Pengxiang Li, Mingyu Cao, Xilu Wang, Li Shen, Da Li, Lu Yin
Organizations: University of Surrey · The Hong Kong Polytechnic University · Sun Yat-sen University · University of York · Shenzhen University of Advanced Technology
Diffusion language models (DLMs) support parallel generation, yet they often require many denoising steps because only a fraction of token predictions are sufficiently reliable to be committed at each step. Existing acceleration methods primarily optimize confidence learning or commitment policies. However, these approaches operate on the outputs of a single denoising forward pass, in which information from the prompt, previously committed tokens, and uncommitted positions may not yet be fully integrated. Consequently, some predictions remain under-refined when the commitment decision is made, and modifying the commitment policy alone cannot improve their underlying representations. We introduce dLoop, which restructures a pretrained diffusion language model into a Prelude-Loop-Coda architecture and recurrently applies a parameter-shared intermediate block to the same partially masked state. Rather than advancing the denoising process, each additional loop provides further latent computation, allowing information distributed across uncommitted positions to continue interacting and become more fully integrated before commitment. Experiments on GSM8K, MATH, MBPP, and HumanEval show that dLoop achieves the highest or tied-highest accuracy on every benchmark. With tokens per forward pass (TPF) adjusted for loop computation, it outperforms Fast-dLLM in both accuracy and TPF on all four benchmarks and dParallel on three, and is more accurate than d3LLM on all four, while d3LLM attains higher TPF; no compared method surpasses dLoop in both. Mechanistic analyses show that extra loops mainly help on slow, unstable trajectories and that updates at uncommitted positions drive changes in token decisions and later generation. Pre-commitment latent computation thus makes predictions more reliable, offering a route to DLM acceleration distinct from forcing confidence through the training loss.
Figures & tables
Figure 1: Overview of dLoop. Here, Xt denotes the partially masked input sequence at denoising step t , and Et is the corresponding Prelude output, which remains fixed as an input anchor throughout the extra passes. Ht(0) denotes the hidden state produced by the initial recurrent pass, whereas Ht(k) denotes the refined hidden state after k extra passes through the parameter-shared recurrent block. The Coda maps Ht(k) to output logits.
Benchmark
Method
TPF ↑
Acc. (%) ↑
LLaDA
1.00
72.6
Fast-dLLM
2.77
75.1
dParallel
5.11
72.9
d3LLM
9.06
72.9
GSM8K
dLoop
6.76
76.3
LLaDA
1.00
32.2
Table 1: Accuracy and tokens per forward pass (TPF) for LLaDA-based models. dLoop uses adjusted TPF. Blue marks the best TPF and red the best accuracy on each benchmark; rows shaded in light blue are our method.
Figure 2: Single-step source rollback. H0=[P0,C0,U0] and H1=[P1,C1,U1] denote the hidden states produced before and after one extra loop, respectively. P , C , and U denote the hidden states at prompt-token positions, previously committed-token positions, and currently uncommitted positions. Subscripts 0 and 1 indicate states before and after the extra loop. The intervention constructs [P1,C1,U0] by rolling back only the uncommitted component from U1 to U0 , while retaining P1 , C1 , and the same Coda and language-model head.
Reverted update
Margin decrease
Top-1 changed
Prompt ΔHP
0.043
19.4%
Committed ΔHC
−0.004
10.4%
Uncommitted ΔHU
0.130
47.8%
Table 2: Single-step rollback of different components of ΔHt .
Condition
Accuracy
Mean steps
Full extra loop [P1,C1,U1]
74.43%
24.62
Updated U only [P0,C0,U1]
73.14%
24.96
Updated P/C only [P1,C1,U0]
71.43%
26.10
Full rollback [P0,C0,U0]
70.86%
26.81
Table 3: Persistent source control during complete decoding on the first 700 GSM8K test examples. Full rollback is identical to decoding without an extra loop.
Position group
Relative displacement r
Uncommitted U
0.275
Committed C
0.233
Table 4: Hidden-state changes produced by one extra loop.
Attention scope
Top- k retention
Top-1
70.7%
Top-5
73.4%
Top-10
74.8%
Top-20
76.5%
Top-40
78.6%
Table 5: Overlap of the most-attended key positions before and after one extra loop.
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
Loop block
Layers
Avg. ( k=1 –3)
Best ( k )
6–9
4
74.83
76.25 (3)
10–13
4
74.00
75.50 (3)
14–17
4
75.42
77.75 (3)
18–21
4
76.67
77.75 (2)
22–25
4
76.17
77.00 (2)
10–15
6
74.00
74.75 (3)
Appendix
Table 6: GSM8K accuracy (%) of different loop blocks on the first 400 test problems: the average over 1–3 extra loops and the best result among them.
k
0
1
2
3
4†
5†
GSM8K Acc.
74.98
78.01
79.15
78.17
78.09
78.39
GSM8K Adj. TPF
8.77
7.71
6.77
6.04
5.41
4.93
MATH Acc.
25.37
27.17
27.37
26.17
26.47
26.77
MATH Adj. TPF
5.45
4.75
4.14
3.64
3.29
2.98
Appendix
Table 7: Accuracy (%) and adjusted TPF for different numbers of extra loops. † Not used during training.
τ
0.55
0.65
0.725
0.80
0.875
0.95
GSM8K Acc.
73.9
78.3
78.0
77.8
78.5
78.5
GSM8K steps
18.9
22.1
26.3
32.3
40.7
56.2
MATH Acc.
22.4
27.0
27.2
28.2
30.0
29.3
MATH steps
26.2
31.1
37.1
44.6
54.8
72.3
Appendix
Table 8: Accuracy (%) and mean decoding steps for different confidence thresholds.
Rolled back
GSM8K
MATH
MBPP
HumanEval
Prompt
6.8
8.0
7.0
5.7
Committed
8.0
8.5
7.3
8.6
Target itself
4.8
5.3
4.2
4.7
Other uncommitted
22.0
20.1
11.9
16.0
Appendix
Table 9: Top-1 changes (%) after single-step rollback over all masked targets in the commitment window.