PV-Decoding: Progressive Verification Decoding with Decode Gating for Diffusion Language Models

Renxi Liu; Yiming Li; Shengyin Sun; Chen Chen; Xian Wang; Shuai Yang; Huiling Zhen; Xianzhi Yu

cs.CLOct 17, 2026

PV-Decoding: Progressive Verification Decoding with Decode Gating for Diffusion Language Models

Authors: Renxi Liu, Yiming Li, Shengyin Sun, Chen Chen, Xian Wang, Shuai Yang, Huiling Zhen, Xianzhi Yu,

Organizations: Huawei Technologies · City University of Hong Kong

Abstract

Discrete diffusion language models (dLLMs) offer an attractive alternative to autoregressive generation by updating multiple tokens in parallel, but their inference efficiency is still limited by the cost of safely committing tentative predictions. In particular, speculative decoding for dLLMs is challenging because speculative tokens within the same diffusion block may depend on one another, making aggressive acceptance prone to error propagation and strict verification computationally expensive. We present PV-Decoding, a confidence-driven speculative decoding framework with progressive verification for block diffusion inference. PV-Decoding partitions masked positions into directly decoded tokens, speculative tokens, and unresolved tokens using two confidence thresholds, and then verifies speculative tokens with progressively stronger policies, including single-batch, dual-batch, and multi-batch verification. This design exposes a practical accuracy--efficiency trade-off: weaker verification minimizes overhead, whereas stronger verification improves acceptance precision and robustness. Experiments on three diffusion language models across GSM8K, MBPP, and HumanEval show that PV-Decoding reduces sequential model forward passes (MFPs) by 8%--37% relative to confidence-based decoding, while keeping the largest observed accuracy drop to 1.3%. On Dream-7B-Instruct/GSM8K, PV-Decoding (1-batch) and PV-Decoding (2-batch) improve throughput by 7.0%--13.6% under the same hardware budget, whereas PV-Decoding (multi-batch) is slower under the current runtime. These results suggest that confidence-guided speculation with structured verification is a practical path toward efficient and reliable dLLM inference.

Figures & tables

Figure 1: Overview of PV-Decoding under block diffusion decoding. Given an input sequence with partially masked tokens, one forward pass produces token predictions and confidence scores. Two thresholds, $\tau_{\mathrm{spec}}$ and $\tau_{\mathrm{dec}}$ , partition masked positions by confidence $c_{i}$ into unresolved tokens ( $c_{i}<\tau_{\mathrm{spec}}$ ), speculative tokens ( $\tau_{\mathrm{spec}}\leq c_{i}<\tau_{\mathrm{dec}}$ ), and directly decoded tokens ( $c_{i}\geq\tau_{\mathrm{dec}}$ ). The speculative tokens are then processed by progressively stronger verification policies: single-batch, dual-batch, and multi-batch verification.

Figure 2: Model-averaged accuracy–efficiency trade-off across benchmarks. For each benchmark and method, relative decoding efficiency and $\Delta$ Acc are averaged over LLaDA-8B-Instruct, Dream-7B-Instruct, and openPangu-7B-Diffusion-Base. The x-axis shows the relative decoding efficiency, computed as the ratio between the number of sequential model forward passes (MFPs) required by greedy decoding and that required by each method. The y-axis reports the accuracy difference with respect to greedy decoding.

Model	Dataset	Greedy		Fast-dLLM		PV-Decoding (1-batch)		PV-Decoding (2-batch)		PV-Decoding (multi-batch)
Model	Dataset	Acc (%)	MFP	Acc (%)	MFP	Acc (%)	MFP	Acc (%)	MFP	Acc (%)	MFP
Dream-7B-Instruct	GSM8K	72.4	122.4	72.4	58.3	72.4 (+0.0)	46.2 (-12.1)	72.9 (+0.5)	52.3 (-6.0)	72.3 (-0.1)	46.5 (-11.8)
	MBPP	55.4	256.0	55.4	62.9	54.8 (-0.6)	42.7 (-20.2)	55.4 (+0.0)	47.2 (-15.7)	55.4 (+0.0)	45.6 (-17.3)
	HumanEval	56.1	173.6	54.8	86.4	54.3 (-0.5)	75.5 (-10.9)	54.3 (-0.5)	78.8 (-7.6)	54.9 (+0.1)	78.2 (-8.2)
LLaDA-8B-Instruct	GSM8K	78.5	210.0	78.2	77.1	78.2 (+0.0)	70.9 (-6.2)	78.5 (+0.3)	62.2 (-14.9)	78.8 (+0.6)	70.7 (-6.4)
	MBPP	36.2	56.0	36.6	33.2	36.4 (-0.2)	27.6 (-5.6)	36.4 (-0.2)	28.1 (-5.1)	36.8 (+0.2)	28.4 (-4.8)

Table 1: Main accuracy and model-forward-pass results across diffusion language models and benchmarks. Accuracy is reported in percentage, and MFP denotes the average number of model forward passes. Parenthetical changes of the accuracy and MFP in the PV-Decoding columns are reported relative to Fast-dLLM.

	Throughput (tok/s) $\uparrow$				Total evaluation time (s) $\downarrow$
Concurrency	Fast-dLLM	PV-Decoding (1-batch)	PV-Decoding (2-batch)	PV-Decoding (multi-batch)	Fast-dLLM	PV-Decoding (1-batch)	PV-Decoding (2-batch)	PV-Decoding (multi-batch)
1	183.8	208.7 (+13.6%)	204.9 (+11.5%)	134.8 ( $-$ 26.6%)	338.29	294.73 ( $-$ 12.9%)	300.85 ( $-$ 11.1%)	458.51 (+35.5%)
4	280.6	306.3 (+9.1%)	303.0 (+8.0%)	155.0 ( $-$ 44.8%)	220.22	199.41 ( $-$ 9.5%)	201.86 ( $-$ 8.3%)	395.36 (+79.5%)
8	311.5	341.4 (+9.6%)	333.3 (+7.0%)	116.5 ( $-$ 62.6%)	197.26	179.01 ( $-$ 9.3%)	182.91 ( $-$ 7.3%)	524.30 (+165.8%)

Table 2: Fixed-hardware performance on Dream-7B-Instruct/GSM8K using one Ascend NPU. Parentheses show changes from Fast-dLLM under the same concurrency.

Figure 3: Accuracy–efficiency trade-off on Dream-7B-Instruct under different speculative confidence thresholds $\tau_{\mathrm{spec}}\in\{0.6,0.7,0.8\}$ .

Dataset	Method	Acc	MFP
GSM8K	Fast-dLLM	72.4	58.3
	PV-Decoding (1-batch)	72.4	46.2
	PV-Decoding (1-batch) w/o decode gating	70.6	49.9
HumanEval	Fast-dLLM	54.8	86.4
	PV-Decoding (1-batch)	54.3	75.5
	PV-Decoding (1-batch) w/o decode gating	50.0	70.8

Table 3: Ablation of verification-gated decoding in PV-Decoding (1-batch). “w/o decode gating” allows new high-confidence positions to be committed even when some speculative tokens fail verification.

Speculative Decoding for Diffusion Language Models

Speculative decoding for diffusion language models speeds generation by drafting tokens in parallel and verifying them efficiently.

See all 30 papers

PSD: Pushing the Pareto Frontier of Diffusion LLMs via Parallel Speculative Decoding
May 15, 2026Shengyin Sun, Yiming Li, Renxi Liu +7Parallel Speculative Decoding Masked Diffusion Models
Department of Computer Science City University of Hong Kong · Huawei Technologies · City University of Hong Kong
Trajectory-Level Speculative Decoding for Diffusion Language Models
Aug 27, 2026Tianxiang Pan, Baitao Gong, Mo Guang +5Speculative Decoding for Diffusion Language Models Diffusion Language Model Decoding
Li Auto Inc., Shanghai, China
SimSD: Simple Speculative Decoding in Diffusion Language Models
Jun 1, 2026Junxia Cui, Haotian Ye, Runchu Tian +9Masked Diffusion Models Speculative Decoding for Diffusion Language Models
University of California San Diego · University of Illinois Urbana-Champaign · 3Google +1

May 15, 2026J/K move · Enter open · S save

PSD: Pushing the Pareto Frontier of Diffusion LLMs via Parallel Speculative Decoding

Shengyin Sun, Yiming Li, Renxi Liu, Xinqi Li+6

Department of Computer Science City University of Hong Kong · Huawei Technologies · City University of Hong Kong

Parallel Speculative Decoding Masked Diffusion Models Speculative Decoding for Diffusion Language Models Speculative Decoding Diffusion Language Model Inference Diffusion Language Model Decoding+3

Open paper AskPDF arXiv

Diffusion large language models (dLLMs) generate text by iteratively denoising masked token sequences. Although dLLMs can predict all masked positions in parallel within each step, the large number of denoising iterations still makes inference expensive. This cost can be reduced spatially by unmasking multiple tokens per step, or temporally by collapsing multiple denoising steps into one verification call. We propose Parallel Speculative Decoding (PSD), a training-free framework that jointly improves inference along both axes. Using the confidence scores from a single forward pass, PSD selects positions to unmask via a configurable, adaptive unmasking policy and constructs multi-depth speculative drafts without extra model calls. A final batched verification pass then applies hierarchical acceptance, keeping the deepest draft that remains consistent with the updated predictions. Experiments on three dLLMs across reasoning and code generation tasks show that PSD achieves favorable trade-offs between inference efficiency and generation quality, reaching up to

5.5\times

tokens per forward pass with accuracy comparable to greedy decoding.

Abstract

Speculative Decoding for Diffusion Language Models

Explore similar work

PSD: Pushing the Pareto Frontier of Diffusion LLMs via Parallel Speculative Decoding

Trajectory-Level Speculative Decoding for Diffusion Language Models

SimSD: Simple Speculative Decoding in Diffusion Language Models