Discrete diffusion language models (dLLMs) offer an attractive alternative to autoregressive generation by updating multiple tokens in parallel, but their inference efficiency is still limited by the cost of safely committing tentative predictions. In particular, speculative decoding for dLLMs is challenging because speculative tokens within the same diffusion block may depend on one another, making aggressive acceptance prone to error propagation and strict verification computationally expensive. We present PV-Decoding, a confidence-driven speculative decoding framework with progressive verification for block diffusion inference. PV-Decoding partitions masked positions into directly decoded tokens, speculative tokens, and unresolved tokens using two confidence thresholds, and then verifies speculative tokens with progressively stronger policies, including single-batch, dual-batch, and multi-batch verification. This design exposes a practical accuracy--efficiency trade-off: weaker verification minimizes overhead, whereas stronger verification improves acceptance precision and robustness. Experiments on three diffusion language models across GSM8K, MBPP, and HumanEval show that PV-Decoding reduces sequential model forward passes (MFPs) by 8%--37% relative to confidence-based decoding, while keeping the largest observed accuracy drop to 1.3%. On Dream-7B-Instruct/GSM8K, PV-Decoding (1-batch) and PV-Decoding (2-batch) improve throughput by 7.0%--13.6% under the same hardware budget, whereas PV-Decoding (multi-batch) is slower under the current runtime. These results suggest that confidence-guided speculation with structured verification is a practical path toward efficient and reliable dLLM inference.
Figures & tables
Figure 1: Overview of PV-Decoding under block diffusion decoding. Given an input sequence with partially masked tokens, one forward pass produces token predictions and confidence scores. Two thresholds, τspec and τdec , partition masked positions by confidence ci into unresolved tokens ( ci<τspec ), speculative tokens ( τspec≤ci<τdec ), and directly decoded tokens ( ci≥τdec ). The speculative tokens are then processed by progressively stronger verification policies: single-batch, dual-batch, and multi-batch verification.
Figure 2: Model-averaged accuracy–efficiency trade-off across benchmarks. For each benchmark and method, relative decoding efficiency and Δ Acc are averaged over LLaDA-8B-Instruct, Dream-7B-Instruct, and openPangu-7B-Diffusion-Base. The x-axis shows the relative decoding efficiency, computed as the ratio between the number of sequential model forward passes (MFPs) required by greedy decoding and that required by each method. The y-axis reports the accuracy difference with respect to greedy decoding.
Model
Dataset
Greedy
Fast-dLLM
PV-Decoding (1-batch)
PV-Decoding (2-batch)
PV-Decoding (multi-batch)
Acc (%)
MFP
Acc (%)
MFP
Acc (%)
MFP
Acc (%)
MFP
Acc (%)
MFP
Dream-7B-Instruct
GSM8K
72.4
122.4
72.4
58.3
72.4 (+0.0)
46.2 (-12.1)
72.9 (+0.5)
52.3 (-6.0)
72.3 (-0.1)
46.5 (-11.8)
MBPP
55.4
256.0
55.4
62.9
54.8 (-0.6)
42.7 (-20.2)
55.4 (+0.0)
47.2 (-15.7)
55.4 (+0.0)
45.6 (-17.3)
HumanEval
56.1
173.6
54.8
86.4
54.3 (-0.5)
75.5 (-10.9)
54.3 (-0.5)
78.8 (-7.6)
54.9 (+0.1)
78.2 (-8.2)
LLaDA-8B-Instruct
GSM8K
78.5
210.0
78.2
77.1
78.2 (+0.0)
70.9 (-6.2)
78.5 (+0.3)
62.2 (-14.9)
78.8 (+0.6)
70.7 (-6.4)
MBPP
36.2
56.0
36.6
33.2
36.4 (-0.2)
27.6 (-5.6)
36.4 (-0.2)
28.1 (-5.1)
36.8 (+0.2)
28.4 (-4.8)
Table 1: Main accuracy and model-forward-pass results across diffusion language models and benchmarks. Accuracy is reported in percentage, and MFP denotes the average number of model forward passes. Parenthetical changes of the accuracy and MFP in the PV-Decoding columns are reported relative to Fast-dLLM.
Throughput (tok/s) ↑
Total evaluation time (s) ↓
Concurrency
Fast-dLLM
PV-Decoding (1-batch)
PV-Decoding (2-batch)
PV-Decoding (multi-batch)
Fast-dLLM
PV-Decoding (1-batch)
PV-Decoding (2-batch)
PV-Decoding (multi-batch)
1
183.8
208.7 (+13.6%)
204.9 (+11.5%)
134.8 ( − 26.6%)
338.29
294.73 ( − 12.9%)
300.85 ( − 11.1%)
458.51 (+35.5%)
4
280.6
306.3 (+9.1%)
303.0 (+8.0%)
155.0 ( − 44.8%)
220.22
199.41 ( − 9.5%)
201.86 ( − 8.3%)
395.36 (+79.5%)
8
311.5
341.4 (+9.6%)
333.3 (+7.0%)
116.5 ( − 62.6%)
197.26
179.01 ( − 9.3%)
182.91 ( − 7.3%)
524.30 (+165.8%)
Table 2: Fixed-hardware performance on Dream-7B-Instruct/GSM8K using one Ascend NPU. Parentheses show changes from Fast-dLLM under the same concurrency.
Figure 3: Accuracy–efficiency trade-off on Dream-7B-Instruct under different speculative confidence thresholds τspec∈{0.6,0.7,0.8} .
Dataset
Method
Acc
MFP
GSM8K
Fast-dLLM
72.4
58.3
PV-Decoding (1-batch)
72.4
46.2
PV-Decoding (1-batch) w/o decode gating
70.6
49.9
HumanEval
Fast-dLLM
54.8
86.4
PV-Decoding (1-batch)
54.3
75.5
PV-Decoding (1-batch) w/o decode gating
50.0
70.8
Table 3: Ablation of verification-gated decoding in PV-Decoding (1-batch). “w/o decode gating” allows new high-confidence positions to be committed even when some speculative tokens fail verification.
Diffusion large language models (dLLMs) generate text by iteratively denoising masked token sequences. Although dLLMs can predict all masked positions in parallel within each step, the large number of denoising iterations still makes inference expensive. This cost can be reduced spatially by unmasking multiple tokens per step, or temporally by collapsing multiple denoising steps into one verification call. We propose Parallel Speculative Decoding (PSD), a training-free framework that jointly improves inference along both axes. Using the confidence scores from a single forward pass, PSD selects positions to unmask via a configurable, adaptive unmasking policy and constructs multi-depth speculative drafts without extra model calls. A final batched verification pass then applies hierarchical acceptance, keeping the deepest draft that remains consistent with the updated predictions. Experiments on three dLLMs across reasoning and code generation tasks show that PSD achieves favorable trade-offs between inference efficiency and generation quality, reaching up to 5.5× tokens per forward pass with accuracy comparable to greedy decoding.
Shengyin Sun, Yiming Li, Renxi Liu +7
Department of Computer Science City University of Hong Kong · Huawei Technologies · City University of Hong Kong
Diffusion-based language models (dLLMs) enable parallel token generation through iterative denoising, but existing decoding strategies collapse to single-token generation under low confidence, severely limiting throughput. Unlike autoregressive models where speculative decoding operates on token sequences in a fixed left-to-right order, dLLMs require speculating over denoising trajectories-sequences of multi-token updates with explicit positions and unmasking orders. We develop a trajectory-level speculative framework that constructs draft denoising trajectories via confidence-stratified tree exploration and verifies them through blockwise parallel evaluation with bidirectional attention masking. Our method further introduces inter-block speculation, exploiting diffusion models' bidirectional structure to perform cross-block lookahead. We formally characterize when this approach is exact and identify trajectory drift as the fundamental cost of increased parallelism. Building on Fast-dLLM's dual-cache infrastructure, our framework reduces denoising iterations by 30-40% and increases tokens-per-step from 2.6 to 4.3, achieving 7-14x speedup over vanilla dLLMs and 1.3x over Fast-dLLM with less than 1% accuracy change across reasoning and code benchmarks.
Diffusion large language models (dLLMs) have recently emerged as a promising alternative to autoregressive (AR) LLMs, offering faster inference through parallel or blockwise decoding. However, their masked language modeling formulation remains incompatible with standard token-level speculative decoding, one of the most effective acceleration techniques for AR models. In AR decoding, the causal mask preserves temporally valid token-level contexts, enabling a target model to verify multiple drafted tokens in a single forward pass. In contrast, dLLMs rely on mask tokens and bidirectional attention, causing the effective context to change across denoising steps and preventing direct token-level speculative verification. To bridge this gap, we propose a simple but effective speculative decoding algorithm for diffusion language models, named SimSD, which mainly adopts a plug-and-play masking strategy that equips dLLMs with temporally valid token-level contexts for speculative decoding. Our method explicitly introduces reference tokens from draft-model predictions and designs an attention mask that regulates their interaction with current-step tokens, allowing dLLMs to compute valid logits for drafted tokens in a single forward pass. This restores the key verification ability provided by causal masking in AR models while preserving the parallel decoding advantages of dLLMs. The proposed method is training-free and can be flexibly integrated with other acceleration techniques such as KV cache and blockwise decoding. Experiments on SDAR-family dLLMs across four benchmarks show that our method achieves up to 7.46x higher decoding throughput while maintaining and even improving average generation quality.
Junxia Cui, Haotian Ye, Runchu Tian +9
University of California San Diego · University of Illinois Urbana-Champaign · 3Google +1