Latent Refinement Decoding: Enhancing Diffusion-Based Language Models by Refining Belief States
Authors: Qinglin Zhu, Yizhen Yao, Runcong Zhao, Yanzheng Xiang, Siya Qi, Amrutha Saseendran, Chen Jin, Philip Teare, Bin Liang, Yulan He, Lin Gui
Organizations: King’s College London, UK · Centre for AI, Data Science & Artificial Intelligence, BioPharmaceuticals R&D, AstraZeneca, UK · The Chinese University of Hong Kong · MoE Lab, CUHK · The Alan Turing Institute, UK
Autoregressive (AR) models remain the dominant paradigm for natural language generation, but their strictly sequential decoding process leads to high inference latency. Recent diffusion-inspired language models, such as LLaDA and Dream, alleviate this issue through parallel generation, yet they still face two key limitations: information loss, since predictive distributions over non-finalised tokens are discarded at each step, and unstable commitment dynamics, where local token decisions are not sufficiently coordinated at the global level. We propose Latent Refinement Decoding (LRD), a two-stage decoding framework consisting of Latent Refinement and a Predictive Feedback Loop. In the first stage, LRD keeps masked positions as distributional mixtures of predicted tokens and the mask embedding, enabling the model to form more globally consistent beliefs before committing tokens. In the second stage, LRD progressively finalises confident tokens while preserving uncertain ones for further iterative refinement, with KL-divergence dynamics serving as a stable criterion for convergence and early stopping. Experiments show that LRD improves performance on both coding tasks, including HumanEval by 6.3 and MBPP by 2.6, and reasoning tasks, including GSM8K by 2.9 and MATH500 by 3.8, while achieving up to 10.6x overall speedup. Moreover, LRD is compatible with single-token decoding, multi-token threshold commitment, and system-level accelerators; when combined with Fast-dLLM, it reaches up to 18.4x speedup over vanilla decoding while further improving accuracy.
Figures & tables
Figure 1: Comparison between the existing decoding strategy and the proposed method. Different c o l o u r s represent distinct tokens, while gradient colours indicate predicted token representations. Top: In the existing strategy, all [MASK] tokens share the same embedding and are repeatedly remasked if not selected. Bottom: In LRD, Phase 1 refines each [MASK] embedding, and Phase 2 progressively commits confident tokens while keeping uncertain ones soft for context-aware decoding.
Model
Len
Commit
Method
HumanEval
MBPP
GSM8K
MATH500
Acc
Speed
Acc
Speed
Acc
Speed
Acc
Speed
Dream Base-7B
256
Single
Entropy-min
50.6
1.0 ×
55.8
1.0 ×
75.3
1.0 ×
36.9
1.0 ×
Prophet
51.1
1.2 ×
55.1
2.1 ×
75.6
1.5 ×
36.5
1.4 ×
LRD
56.9 +6.3
1.2 ×
57.6 +1.8
2.3 ×
78.2 +2.9
1.8 ×
39.8 +2.9
1.4 ×
Multi
Conf-thresh
48.9
1.4 ×
55.5
1.9 ×
74.7
2.5 ×
36.7
2.4 ×
KLASS
51.4
1.5 ×
56.6
1.4 ×
75.2
1.8 ×
37.2
1.2 ×
Table 1: Main comparison at sequence lengths 256 and 512. Speed is normalized to vanilla decoding (1.0 × ; higher is faster). Entropy-min is the standard hard-assignment decoder that commits the minimum-entropy token each step. Green deltas compare LRD with Entropy-min and Fast-dLLM+LRD with Conf-thresh (confidence threshold 0.9 ). Additional results are in Appendix A.2 .
Method
HumanEval
MBPP
GSM8K
MATH500
Acc
Speed
Acc
Speed
Acc
Speed
Acc
Speed
Baseline
43.9
1.0 ×
38.2
1.0 ×
81.3
1.0 ×
37.7
1.0 ×
SOAR
46.6
1.7 ×
39.2
1.6 ×
82.1
1.4 ×
39.0
1.2 ×
CreditDecoding
45.6
1.3 ×
38.9
1.5 ×
80.7
1.9 ×
37.6
1.5 ×
LRD
48.4
1.3 ×
40.6
1.5 ×
84.5
2.0 ×
39.8
1.6 ×
Table 2: LLaDA-Instruct-8B ( L=512 ). Speed is relative to native decoding.
Figure 4
Method
HE
MBPP
GSM8K
MATH
Baseline
56.1
56.7
80.2
38.6
GN, rf=0.1
49.2
52.2
70.4
33.4
GN, rf=0.2
47.0
51.4
69.8
32.4
KL-stop only
55.8
56.9
80.4
38.3
LRD w/o ent. wt.
57.7
57.9
76.8
40.2
LRD (full)
60.9
58.8
82.7
41.8
Table 3: Random Gaussian noise (GN) vs. soft embeddings on Dream-Instruct-7B ( L=512 ).
Method
HumanEval
MBPP
GSM8K
MATH500
Acc
Speed
Acc
Speed
Acc
Speed
Acc
Speed
Baseline
56.1
1.0 ×
56.7
1.0 ×
80.2
1.0 ×
38.6
1.0 ×
LRD (full)
60.9
2.9 ×
58.8
4.6 ×
82.7
3.6 ×
41.8
1.2 ×
w/o latent refinement
59.9
3.1 ×
57.8
4.9 ×
82.2
3.8 ×
41.0
1.2 ×
w/o mixed embeddings
58.0
2.7 ×
57.8
4.3 ×
80.7
3.0 ×
40.8
1.0 ×
w/o early stopping
61.2
1.2 ×
58.8
1.1 ×
82.9
1.1 ×
41.9
1.0 ×
Table 4: Dream-Instruct-7B ablations ( L=512 ). Speed is relative to the baseline.
Appendix figures & tables31 assets
Supplementary material from the paper’s appendix.
Appendix
Model
Len
Method
HumanEval
MBPP
GSM8K
MATH500
Acc
Speed
Acc
Speed
Acc
Speed
Acc
Speed
Dream-Base-7B
256
baseline
50.6
1.0 ×
55.8
1.0 ×
75.3
1.0 ×
36.9
1.0 ×
LRD
56.9 +6.3
1.2 ×
57.6 +1.8
2.3 ×
78.2 +2.9
1.8 ×
39.8 +2.9
1.4 ×
512
baseline
54.4
1.0 ×
55.8
1.0 ×
76.2
1.0 ×
37.5
1.0 ×
LRD
58.8 +4.4
2.6 ×
58.4 +2.6
4.5 ×
77.4 +1.2
3.4 ×
40.8 +3.3
1.8 ×
1024
baseline
54.8
1.0 ×
58.0
1.0 ×
76.8
1.0 ×
39.1
1.0 ×
Appendix
Table 5: Full LRD-only results across additional models and sequence lengths. Speed denotes relative runtime (baseline = 1.0 × ), where larger values indicate faster inference. Accuracy deltas in green are relative to the corresponding baseline row.
HumanEval
MBPP
GSM8K
MATH-500
Method
Acc.
Speed
Acc.
Speed
Acc.
Speed
Acc.
Speed
Baseline
56.1
1.0 ×
56.7
1.0 ×
80.2
1.0 ×
38.6
1.0 ×
Top k=1 by confidence
55.1
1.0 ×
55.0
1.0 ×
80.7
1.0 ×
37.4
1.0 ×
Top k=2 by confidence
41.9
1.9 ×
43.0
1.9 ×
75.9
2.1 ×
29.0
1.9 ×
Top k=3 by confidence
26.7
2.9 ×
27.2
2.8 ×
66.1
3.1 ×
26.2
2.9 ×
Top k=4 by confidence
21.6
3.8 ×
22.2
3.9 ×
57.7
4.1 ×
14.0
3.9 ×
Appendix
Table 6: Sampling strategy sweep on Dream-Instruct-7B ( L=512 ). Speed is normalised to the baseline. “Top k=1 by confidence” and “Top k=1 by entropy” coincide with the standard baseline decoder.
Length
Method
HumanEval
MBPP
GSM8K
MATH500
256
Baseline
38.7 / 1.0 ×
36.9 / 1.0 ×
77.4 / 1.0 ×
33.8 / 1.0 ×
256
SOAR
40.4 / 1.3 ×
37.6 / 1.3 ×
78.2 / 1.2 ×
34.7 / 1.0 ×
256
CreditDecoding
40.3 / 1.2 ×
37.5 / 1.3 ×
76.9 / 1.4 ×
33.6 / 1.3 ×
256
LRD
43.3 / 1.2 ×
40.0 / 1.3 ×
78.8 / 1.5 ×
35.8 / 1.4 ×
512
Baseline
43.9 / 1.0 ×
38.2 / 1.0 ×
81.3 / 1.0 ×
37.7 / 1.0 ×
512
SOAR
46.6 / 1.7 ×
39.2 / 1.6 ×
82.1 / 1.4 ×
39.0 / 1.2 ×
Appendix
Table 7: Additional training-free decoders on LLaDA-Instruct-8B. Cells are accuracy / speedup.
Method
Benchmark
Method accuracy
LRD accuracy
Soft-Masked DLMs
HumanEval
57.8
60.9
Soft-Masked DLMs
MBPP
56.4
58.8
EvoToken-DLM
GSM8K
84.0
84.5
EvoToken-DLM
MATH500
40.0
39.8
RCD
GSM8K
79.1
84.5
RCD
MATH500
37.0
39.8
Appendix
Table 8: Training-based predictive feedback versus frozen-model LRD. Soft-Masked DLM results are quoted; the other comparisons use released weights.
HumanEval
MBPP
GSM8K
MATH500
Method
Acc.
Speed
Acc.
Speed
Acc.
Speed
Acc.
Speed
Baseline
56.1
1.0 ×
56.7
1.0 ×
80.2
1.0 ×
38.6
1.0 ×
DAEDAL [ Li et al., 2025a ]
54.3
3.2 ×
56.0
7.7 ×
79.1
8.4 ×
38.2
1.2 ×
DAEDAL+LRD
58.8
3.3 ×
58.4
7.6 ×
81.9
8.6 ×
41.0
1.2 ×
LRD (ours)
60.9
2.9 ×
58.8
4.6 ×
82.7
3.6 ×
41.8
1.2 ×
Appendix
Table 9: Comparison with acceleration baselines on Dream-Instruct-7B ( L=512 ). All speedups are normalised to the same baseline. Combinations such as DAEDAL+LRD and Fast-dLLM+LRD demonstrate composability.
Len
Method
RACE
TruthfulQA
Acc
Speed
bleu_acc
rouge1_acc
rouge2_acc
Speed
256
baseline
81.1
1.0 ×
40.3
45.3
32.8
1.0 ×
Ours
82.2 +1.1
1.3 ×
42.5 +2.2
47.6 +2.3
34.2 +1.4
2.1 ×
512
baseline
80.9
1.0 ×
41.9
45.4
33.1
1.0 ×
Ours
82.4 +1.5
1.4 ×
44.3 +2.4
48.3 +2.9
35.1 +2.0
2.6 ×
1024
baseline
81.2
1.0 ×
41.7
45.5
33.6
1.0 ×
Appendix
Table 10: Closed-selection benchmarks: reading comprehension (RACE) and TruthfulQA on Dream-Ins-7B. Baseline results are shown in grey , and improvements of our method are highlighted in green . Speed is reported per task.
Len
Method
CNN/DailyMail
NQ-Open
BERTScore
ROUGE-1
ROUGE-L
Speed
EM
Speed
256
baseline
85.6
21.7
15.0
1.0 ×
12.0
1.0 ×
Ours
85.7 +0.1
22.3 +0.6
15.5 +0.5
1.3 ×
15.0 +3.0
2.2 ×
512
baseline
85.5
22.6
15.6
1.0 ×
12.8
1.0 ×
Ours
85.6 +0.1
23.0 +0.4
15.9 +0.3
1.4 ×
14.4 +1.6
2.6 ×
1024
baseline
85.7
22.8
15.7
1.0 ×
12.8
1.0 ×
Appendix
Table 11: Open-generation benchmarks: summarisation (CNN/DailyMail) and open-domain QA (NQ-Open) on Dream-Ins-7B. Baseline results are shown in grey , and improvements of our method are highlighted in green . Speed is reported per task.
Benchmark
Length
Native accuracy
LRD accuracy
LRD speedup
GPQA
256
34.2
36.8
1.2 ×
GPQA
512
35.5
38.4
1.5 ×
BBH
256
59.1
61.6
1.5 ×
BBH
512
60.4
63.2
2.9 ×
Appendix
Table 12: Additional benchmarks on Dream-Instruct-7B.
Model
Len
Method
HumanEval
MBPP
GSM8K
MATH500
Acc
Speed
Acc
Speed
Acc
Speed
Acc
Speed
LLaDA2.0- mini-preview
256
baseline
5.5
1.0×
19.2
1.0×
63.8
1.0×
16.2
1.0×
Ours
7.3 +1.8
1.0×
22.2 +3.0
1.1×
65.6 +1.8
1.2×
17.5 +1.3
1.0×
512
baseline
54.3
1.0×
54.7
1.0×
86.5
1.0×
44.6
1.0×
Ours
56.9 +2.6
1.1×
58.2 +3.5
1.6×
88.1 +1.6
2.1×
46.3 +1.7
1.1×
1024
baseline
74.2
1.0×
63.2
1.0×
87.7
1.0×
61.2
1.0×
Appendix
Table 13: Performance of LLaDA2.0-mini-preview (16B) across benchmarks. Speed denotes relative runtime (baseline = 1.0×), where larger values indicate faster inference. Baseline results are shown in grey , and our improvements in green .
Method
HumanEval
MBPP
GSM8K
MATH500
Nemotron-Labs-Diffusion-8B-Instruct
Native diffusion
69.8 / 1.0 ×
70.9 / 1.0 ×
87.4 / 1.0 ×
65.6 / 1.0 ×
+LRD
72.4 / 1.3 ×
73.6 / 1.5 ×
89.0 / 1.2 ×
68.0 / 1.1 ×
DiffusionGemma-26B-A4B-it
Native EB + adaptive stopping
76.3 / 1.0 ×
73.8 / 1.0 ×
89.6 / 1.0 ×
68.2 / 1.0 ×
+LRD-style adaptation
77.5 / 1.2 ×
75.3 / 1.1 ×
90.3 / 1.2 ×
69.6 / 1.1 ×
Appendix
Table 14: Additional model families. Cells are accuracy / speedup over the corresponding native decoder. DiffusionGemma uses a model-specific LRD-style adaptation.
Per-Step Cost
FLOPs (G)
Latency
Remark / Δ
Baseline (Standard)
14.14
81.8 ms
N/A
Backbone
14.14
68.6 ms
MatMul dominated
System Overhead
N/A
13.2 ms
Data dispatch
LRD (Proposed)
∼ 14.14
94.3 ms
N/A
Backbone
14.14
68.6 ms
Identical to baseline
KL / Mixing / Aux
≈0
12.6 ms
Memory-bound overhead
Appendix
Table 15: Efficiency Analysis Breakdown (HumanEval, length=512). We compare the per-step theoretical cost and wall-clock time between the Baseline and our method (Dream-Instruct-7B). Although our method introduces a minor overhead per step, the significant reduction in generation steps results in a substantial total speedup.
Figure 4: Quality vs. time Pareto analysis comparing our method with the baseline on HumanEval using Dream-Instruct under different diffusion-step budgets (2×A100 GPUs).
LRD
Fast-dLLM+LRD
Benchmark
Baseline E
E (change)
Scontent (no stop)
E (change)
Scontent (no stop)
HumanEval
90.4
84.6 ( −6.4% )
1.08 ×
98.6 ( +9.1% )
1.11 ×
MBPP
60.5
57.2 ( −5.5% )
1.06 ×
49.4 ( −18.3% )
1.19 ×
GSM8K
135.5
123.7 ( −8.7% )
1.07 ×
115.3 ( −14.9% )
1.13 ×
MATH500
482.3
437.3 ( −9.3% )
1.09 ×
377.2 ( −21.8% )
1.21 ×
Appendix
Table 16: Content-length controls at L=1024 . E counts tokens before [EOS] ; Scontent is measured with Phase 2 early stopping disabled.
Model
Steps
Method
HumanEval
MBPP
GSM8K
MATH500
Dream-Base-7B
64
Baseline
43.2 / 1.0 ×
50.2 / 1.0 ×
33.2 / 1.0 ×
10.9 / 1.0 ×
64
LRD
44.4 / 0.9 ×
50.6 / 1.0 ×
33.7 / 1.1 ×
11.6 / 0.9 ×
128
Baseline
47.2 / 1.0 ×
54.4 / 1.0 ×
63.4 / 1.0 ×
28.9 / 1.0 ×
128
LRD
49.0 / 1.1 ×
55.4 / 1.1 ×
64.6 / 1.3 ×
30.4 / 1.1 ×
LLaDA-Ins-8B
64
Baseline
31.2 / 1.0 ×
36.0 / 1.0 ×
61.0 / 1.0 ×
24.3 / 1.0 ×
64
LRD
31.8 / 0.9 ×
36.7 / 0.9 ×
61.5 / 1.0 ×
24.8 / 0.9 ×
Appendix
Table 17: Small-budget decoding. Cells are accuracy / speed relative to the baseline at the same budget.
k
HumanEval
MBPP
GSM8K
MATH-500
Baseline
LRD
Baseline
LRD
Baseline
LRD
Baseline
LRD
1
37.8
42.7
36.4
39.7
77.2
78.6
33.9
35.9
2
45.7
50.1
44.1
47.1
86.5
87.7
41.5
43.4
5
52.8
56.5
52.6
54.7
92.4
93.0
50.3
52.0
10
56.3
58.7
58.1
60.2
94.2
95.2
54.9
56.7
Appendix
Table 18: Pass@ k under temperature sampling ( T=0.8 ) on LLaDA-Ins-8B (L=256). LRD consistently improves pass@ k across all benchmarks and all k , confirming compatibility with temperature sampling.
Figure 5: Premature Commitment vs. Latent Refinement: A Case Study on Error Propagation (GSM8K)
Figure 6: Logical Reasoning Error.
Figure 7: Procedural Step Errors.
Figure 8: Problem Misinterpretation.
Length
Method
Accuracy
Speed
Etoken
HumanEval
MBPP
GSM8K
MATH500
HumanEval
MBPP
GSM8K
MATH500
HumanEval
MBPP
GSM8K
MATH500
256
baseline
55.4
57.4
80.8
37.9
1.0 ×
1.0 ×
1.0 ×
1.0 ×
117.2
53.5
132.4
228.4
Ours
61.6 +6.2
59.4 +2.0
83.0 +2.2
40.6 +2.7
1.4 × +0.4×
2.4 × +1.4×
1.4 × +0.4×
1.1 × +0.1×
108.4 -8.8
49.2 -4.3
128.6 -5.8
226.0 -2.4
w/o latent refinement
60.1 -1.5
58.6 -0.8
82.3 -0.7
39.4 -1.2
1.5 × +0.1×
2.5 × +0.1×
1.5 × +0.1×
1.1 × +0.0×
108.7 +0.3
50.4 +1.2
129.9 +1.3
226.8 +0.8
w/o mix embed
59.5 -2.1
58.8 -0.6
82.7 -0.3
38.9 -1.7
1.3 × -0.1×
2.2 × -0.2×
1.5 × +0.1×
1.0 × -0.1×
117.2 +8.8
49.5 +0.3
129.4 +0.8
228.4 +2.4
w/o early stop
61.8 +0.2
59.4 +0.0
83.2 +0.2
40.6 +0.0
1.0 × -0.4×
1.1 × -1.3×
1.1 × -0.3×
1.0 × -0.1×
109.7 +1.3
51.4 +2.2
129.9 +1.3
228.0 +2.0
Appendix
Table 19: Component ablation results on decoding variants. We report Accuracy, Speed, and effective token number Etoken . Red and green numbers indicate changes compared to our full method.
Final outcome
Benchmark
Positions
Hard p(y∗)
Soft p(y∗)
Δp [95% CI]
Rank: Hard → Soft
Baseline wrong / LRD correct
HumanEval
73
0.24
0.37
+0.13 [0.08, 0.18]
6.9→4.2
Baseline wrong / LRD correct
GSM8K
186
0.11
0.23
+0.12 [0.08, 0.16]
46.2→12.8
Baseline wrong / LRD wrong
HumanEval
108
0.17
0.21
+0.04 [0.01, 0.07]
11.6→9.5
Baseline wrong / LRD wrong
GSM8K
327
0.09
0.15
+0.06 [0.03, 0.09]
51.3→43.7
Appendix
Table 20: Matched-state hard/soft probe. Counts are error positions, not independent examples. Intervals are 95% example-clustered bootstrap confidence intervals; lower rank is better.
Method
HumanEval
MBPP
GSM8K
MATH500
Baseline
56.1
56.7
80.2
38.6
Ours
60.9 +4.8
58.8 +2.1
82.7 +2.5
41.8 +3.2
Auto
59.6 +3.5
57.7 +1.0
81.5 +1.3
41.4 +2.8
LF × 1
60.4 +4.3
57.8 +1.1
81.6 +1.4
40.2 +1.6
LF × 2
58.3 +2.2
57.2 +0.5
81.2 +1.0
40.2 +1.6
LF × 3
57.9 +1.8
57.8 +1.1
81.8 +1.6
39.0 +0.4
Appendix
Table 21: Ablation study on decoding variants at length 512, where Auto uses adaptive latent refinement, and LF × k enforces k latent refinement steps prior to each token commitment.
Figure 9: Convergence ratios across latent refinement steps for LLaDA-1.5 and Dream-Ins on four benchmarks. Since computing the difference in KL divergence requires at least three consecutive steps, the curves are plotted starting from step 2.
Figure 10: Step-wise predictive distributions of different distance metrics on LLaDA-1.5 (HumanEval). The red vertical line indicates where decoding begins after a fixed 20-step latent refinement.
Figure 11: Effect of top- p mixing on Dream-Ins across four benchmarks. The purple curve shows the log fraction of tokens included in the mixture.
Hyperparam
Value
HumanEval
MBPP
GSM8K
MATH-500
Avg Speed
τrefine
0.05
60.8
58.8
82.4
41.9
2.6 ×
τrefine
0.10 (default)
60.9
58.8
82.7
41.8
2.8 ×
τrefine
0.15
60.8
58.6
82.6
41.6
2.9 ×
τdecode
0.05
61.0
58.8
82.6
41.5
2.3 ×
τdecode
0.10 (default)
60.9
58.8
82.7
41.8
2.8 ×
τdecode
0.15
60.7
58.7
82.6
41.6
3.0 ×
Appendix
Table 22: Single-axis hyperparameter sweep on Dream-Instruct-7B ( L=512 ). For each row only the listed hyperparameter changes; the others remain at their default. “Avg Speed” is the geometric mean of the per-benchmark speedups over the four benchmarks. Default: τrefine=0.1 , τdecode=0.1 , Trefine=20 .
Parameter
Value
HumanEval
MBPP
GSM8K
MATH500
Avg. speed
τrefine
0.05
61.5
59.3
82.8
40.7
1.4 ×
0.10
61.6
59.4
83.0
40.6
1.5 ×
0.15
61.4
59.2
82.9
40.4
1.6 ×
τdecode
0.05
61.7
59.4
83.1
40.7
1.3 ×
0.10
61.6
59.4
83.0
40.6
1.5 ×
0.15
61.3
59.2
82.8
40.3
1.7 ×
Appendix
Table 23: Cross-window hyperparameter sweep on Dream-Instruct-7B at length 256.
Parameter
Value
HumanEval
MBPP
GSM8K
MATH500
Avg. speed
τrefine
0.05
60.9
58.9
83.3
44.0
5.1 ×
0.10
61.0
59.0
83.5
43.9
5.5 ×
0.15
60.8
58.8
83.4
43.7
5.8 ×
τdecode
0.05
61.1
58.9
83.6
44.0
4.6 ×
0.10
61.0
59.0
83.5
43.9
5.5 ×
0.15
60.7
58.8
83.2
43.6
6.1 ×
Appendix
Table 24: Cross-window hyperparameter sweep on Dream-Instruct-7B at length 1024. The length-512 sweep is in Table 22 .
Figure 12: Distribution of L2 norms for token embeddings in Dream and LLaDA models.
Operator
ρ^ : mild → strongest
CR: mild → strongest
Accuracy at strongest
LRD
0.03→12.43
0.2→232.0
0.14
SM-style
0.29→11.78
0.7→213.4
0.19
Evo-style
0.21→10.62
1.5→233.1
0.19
RCD-style
0.40→15.79
2.2→331.8
0.00
Appendix
Table 25: Feedback strength, embedding perturbation, and confidence drops in a frozen-decoder stress test. Accuracy is a fraction; CR is in units of 10−3 .
Figure 13: Empirical quantities for TCR verification on LLaDA (on MBPP dataset, commitment threshold = 0.9). Left: the average signed gap to the commitment threshold, where lower/high values indicate positions moving closer to/away from commitment. Right: the corresponding TCR curve, where positive/negative values indicate progress toward/backward commitment. Both figures demonstrate our framework is able to speed-up the commitment in parallel decoding.
Diffusion language model decoding generates text through iterative parallel refinement, balancing fewer denoising steps with accuracy, coherence, and control.
Diffusion-based large language models (dLLMs) support parallel text generation via iterative denoising, yet inference remains latency-heavy because many steps are spent on redundant refinement and repeated remasking of tokens whose final values are already determined. Prior acceleration methods mainly depend on step-local confidence heuristics or fixed schedules, which are sensitive to prompt and task variation and ignore strong positional effects within a sequence. We cast diffusion decoding as a dynamic control problem and show that token-wise denoising trajectories provide the key signal for reliable control. We propose a trace-aware decoding framework with two components. First, Temporal-Spatial Parallel Decoding (TSPD) uses a lightweight temporalspatial controller that consumes per-token trajectory features, including confidence, entropy, and momentum, together with token position, to decide when a token has converged and can be safely fixed. Second, we introduce Confidence Extrapolation (CE), a training-free state-space module that forecasts future logit trends with uncertainty to support proactive decisions, including safe look-ahead and targeted stabilization when trajectories are oscillatory or underconfident. Together, TSPD and CE reduce unnecessary denoising iterations while preserving output quality, and they compose cleanly with system optimizations such as KV caching.
Discrete diffusion language models enable parallel token generation, offering a pathway to low-latency decoding. However, selecting tokens independently by marginal confidence limits effective parallelism: tokens that appear reliable in isolation can form incompatible configurations when several positions are updated at once. We introduce a training-free decoding framework that coordinates these parallel updates. At each forward pass, the method assigns a commit score to each masked position and refines these scores using pairwise interactions derived from the model's predictive distributions. A variational relaxation yields a simple fixed-point update that suppresses conflicting simultaneous commitments within a single forward pass. This mechanism allows the decoder to commit more tokens in parallel while maintaining competitive generation quality. The method is lightweight, requires no auxiliary model or retraining, and drops into existing diffusion decoding pipelines without modification. Experiments on reasoning and code-generation benchmarks show consistent improvements in the quality-latency trade-off.
Tamim Zoabi, Ameen Ali, Liran Ringel +1
School of Electrical & Computer Engineering Tel Aviv University, Israel · School of Computer Science and AI Tel Aviv University, Israel · Department of Computer Science Technion, Israel Institute of Technology, Israel
Block-wise semi-autoregressive decoding is the standard inference paradigm for diffusion large language models (DLMs), but it imposes a strict dependency between blocks: the next block cannot begin until the current block is fully decoded or its denoising budget is exhausted. We observe that once a block exposes a reliable delimiter boundary or stable semantic prefix, continuation generation need not wait for every residual token to be resolved. We propose AsyncLane, a training-free decoding scheduler that decouples refinement from advancement. AsyncLane forks a generate lane at observed delimiter boundaries into a refine lane and a continuation generate lane: the prefix remains editable, while the continuation advances before prefix refinement finishes. The resulting lane tree records decoding dependencies and output order, while execution proceeds over the active lane set. To make this asynchronous schedule efficient under bidirectional attention, AsyncLane combines shared-prefix lane batching, lookahead draft reuse, cascading termination, and compact cache refresh with refresh-logit reuse, preventing model-call cost from scaling directly with the number of lanes. AsyncLane is a drop-in replacement for block-wise DLM samplers and requires no retraining. Experiments on mathematical reasoning and code generation show that AsyncLane consistently improves throughput while maintaining competitive quality. Across LLaDA and Dream backbones, AsyncLane achieves the highest TPS in all evaluated benchmark-length settings; relative to the fastest competing baseline, it reaches peak speedups of 2.95x on LLaDA and 3.04x on Dream, with especially large gains under longer generation budgets.
Yingxuan Ren, Yuxuan Lou, Yong Liu +4
1National University of Singapore · University of Southampton