S2D2: Fast Decoding for Diffusion LLMs via Training-Free Self-Speculation

Ligong Han; Hao Wang; Han Gao; Kai Xu; Akash Srivastava

cs.CLMar 26, 2026

S2D2: Fast Decoding for Diffusion LLMs via Training-Free Self-Speculation

Authors: Ligong Han, Hao Wang, Han Gao, Kai Xu, Akash Srivastava

Organizations: Red Hat AI Innovation · MIT-IBM Watson AI Lab · Iowa State University · Core AI, IBM

Abstract

Block-diffusion language models offer a promising path toward faster-than-autoregressive generation by combining block-wise autoregressive decoding with within-block parallel denoising. However, in the few-step regime needed for practical acceleration, standard confidence-thresholded decoding is often brittle: aggressive thresholds hurt quality, while conservative thresholds require unnecessary denoising steps. Existing approaches that address this issue either require additional training or incur extra test-time compute. We present S2D2, a training-free self-speculative decoding framework for block-diffusion language models. Our key observation is that a block-diffusion model becomes autoregressive when the block size is reduced to one, allowing the same pretrained model to act as both drafter and verifier. S2D2 inserts a speculative verification step into standard block-diffusion decoding and uses lightweight routing policies to decide when verification is worth its cost. This yields a hybrid decoding trajectory in which diffusion proposes tokens in parallel, while the autoregressive mode acts as a local sequence-level critic. Across three mainstream block-diffusion families, S2D2 consistently improves the accuracy-speed tradeoff over strong confidence-thresholding baselines. On SDAR, we observe up to $4.7\times$ speedup over autoregressive decoding, and up to $1.57\times$ over a tuned dynamic decoding baseline while improving accuracy by up to $4.5$ points. On LLaDA2.1-Mini, S2D2 remains complementary to built-in self-correction, including a conservative setting where it is $4.4\times$ faster than the static baseline with slightly higher accuracy.

Figures & tables

Figure 1: Overview of S2D2 . (a) Standard block-diffusion decoding accepts drafted tokens by confidence thresholding. (b) S2D2 inserts a self-speculative verification step: the same model under block-size- $1$ autoregressive masking verifies the first contiguous masked span, accepts tokens by rejection sampling, and falls back to standard diffusion decoding when verification is not invoked or terminates early. (c) Verification-mode attention masks for right-shifted and standard position-aligned diffusion LLMs; we draw the full-block mask for illustration, though in practice only $C_{t}$ is verified. (d) Lightweight routing policies decide when verification is worth its additional cost.

Panel of Figure 2 — Figure 2: AR-ness (@ $k$ , $k=2$ ) and decoding-confidence statistics on GSM8K and MBPP. Top row: local and global AR-ness for SDAR-8B-Chat and LLaDA 2.1. Bottom row: normalized decoded-token confidence under static and dynamic diffusion decoding; dashed curves in dynamic decoding indicate the number of decoded tokens per step. Reference accuracies (GSM8K, MBPP): SDAR-8B-Chat, AR $(89.3\%,64.4\%)$ and diffusion $(89.6\%,61.0\%)$ ; LLaDA 2.1, AR $(90.8\%,65.8\%)$ and diffusion $(90.8\%,67.8\%)$ .

SDAR-1.7B-Chat
Method	GSM8K	MBPP	HumanEval	IFEval	Avg
AR	76.3 (1.0 $\times$ )	48.4 (1.0 $\times$ )	62.2 (1.0 $\times$ )	38.6 (1.0 $\times$ )	56.4 (1.0 $\times$ )
Diffusion (static)	77.9 (1.7 $\times$ )	13.6 (1.3 $\times$ )	51.8 (1.9 $\times$ )	43.3 (2.1 $\times$ )	46.7 (1.9 $\times$ )
Diffusion (dynamic)	77.2 (2.7 $\times$ )	21.6 (1.3 $\times$ )	51.8 (3.4 $\times$ )	43.0 (3.8 $\times$ )	48.4 (3.1 $\times$ )
S2D2 (config-A)	77.4 (2.3 $\times$ )	45.4 (1.5 $\times$ )	48.8 (2.3 $\times$ )	46.2 (2.7 $\times$ )	54.4 (2.5 $\times$ )
S2D2 (config-B)	73.8 ( 4.3 $\times$ )	44.4 ( 2.9 $\times$ )	52.4 ( 4.1 $\times$ )	41.1 ( 5.2 $\times$ )	52.9 ( 4.7 $\times$ )

Table 1: Main accuracy and speedup results on the SDAR family. Speedups are measured relative to the autoregressive baseline. For each model, we report standard diffusion decoding at the strongest baseline block size and two representative S2D2 settings: config-A for higher accuracy and config-B for higher speed.

Method	GSM8K	MBPP	HumanEval	IFEval	Avg
AR	85.1 (1.0 $\times$ )	51.8 (1.0 $\times$ )	32.9 (1.0 $\times$ )	67.7 (1.0 $\times$ )	59.4 (1.0 $\times$ )
Diffusion (config-A)	84.8 (2.5 $\times$ )	50.6 (2.3 $\times$ )	14.6 (1.4 $\times$ )	63.1 (1.6 $\times$ )	53.3 (2.1 $\times$ )
Diffusion (config-B)	83.2 (3.3 $\times$ )	47.8 (2.9 $\times$ )	11.0 (1.5 $\times$ )	61.6 (1.9 $\times$ )	50.9 (2.7 $\times$ )
Diffusion (config-C, SB=32)	78.3 (3.5 $\times$ )	46.2 (3.6 $\times$ )	8.5 (1.7 $\times$ )	59.4 (1.9 $\times$ )	48.1 (2.9 $\times$ )
S2D2 (config-A)	83.8 (1.6 $\times$ )	48.8 (2.0 $\times$ )	21.3 (1.5 $\times$ )	62.7 (1.3 $\times$ )	54.2 (1.6 $\times$ )
S2D2 (config-B)	84.2 (3.0 $\times$ )	45.0 (3.7 $\times$ )	22.0 (2.0 $\times$ )	64.0 ( 1.9 $\times$ )	53.8 (2.8 $\times$ )

Table 2: Main accuracy and speedup results on Fast-dLLM v2. We fix block size

\mathrm{B}=32

and vary the sub-block size

\mathrm{SB}

. Speedups are measured relative to the autoregressive-style baseline

\mathrm{B}=4,\mathrm{SB}=1

. Config-C corresponds to the normal BD3 without sub-blocks.

	$\text{Block-Size}=1$ (AR)			$\tau_{\mathrm{mask}}=1,\ \tau_{\mathrm{edit}}=0.9$
Method	GSM8K	MBPP	Avg	GSM8K	MBPP	Avg
Diffusion	90.8 (1.0 $\times$ )	65.8 (1.0 $\times$ )	78.3 (1.0 $\times$ )	90.8 (0.6 $\times$ )	67.6 (0.5 $\times$ )	79.2 (0.5 $\times$ )
	$\tau_{\mathrm{mask}}=0.7,\ \tau_{\mathrm{edit}}=0.5$			$\tau_{\mathrm{mask}}=0.95,\ \tau_{\mathrm{edit}}=0.9$
Diffusion	89.6 ( 2.7 $\times$ )	57.8 ( 2.8 $\times$ )	73.7 ( 2.7 $\times$ )	91.0 (1.7 $\times$ )	66.4 (1.8 $\times$ )	78.7 (1.7 $\times$ )
S2D2	90.8 (2.2 $\times$ )	64.0 (2.1 $\times$ )	77.4 (2.1 $\times$ )	89.8 ( 2.1 $\times$ )	68.8 ( 2.2 $\times$ )	79.3 ( 2.2 $\times$ )

Table 3: Accuracy and speedup results on LLaDA2.1-Mini. Speedups are measured relative to the AR baseline. We compare the built-in editing-based diffusion decoder against S2D2 under two threshold settings.

Panel of Figure 3 — Figure 3: Accuracy versus wall-clock time for SDAR-8B-Chat on GSM8K and MBPP. ITS denotes inference-time scaling, the ITS trend curve is fitted on points with accuracy $>30\%$ . Across block sizes, denoising steps, and decoding schedules, S2D2 generally achieves a better accuracy–speed frontier than BD3.

Appendix figures & tables18 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 4: Drafting and caching with autoregressive attention masks.

Setting	Method	$T=0$ GSM8K	$T=0$ MBPP	$T=0.5$ GSM8K	$T=0.5$ MBPP
Conservative	AR	90.8 (1.0 $\times$ )	65.8 (1.0 $\times$ )	$89.2{\pm}0.4$ (1.0 $\times$ )	$64.5{\pm}0.8$ (1.0 $\times$ )
	Diffusion	91.0 (1.6 $\times$ )	66.4 (1.7 $\times$ )	$89.8{\pm}0.7$ (2.4 $\times$ )	$56.5{\pm}0.3$ (2.9 $\times$ )
	S2D2	$90.3{\pm}0.5$ (2.0 $\times$ )	$67.2{\pm}1.7$ (2.1 $\times$ )	$90.3{\pm}0.5$ (1.8 $\times$ )	$65.8{\pm}0.9$ (2.0 $\times$ )
Quality	AR	90.8 (1.0 $\times$ )	65.8 (1.0 $\times$ )	$89.2{\pm}0.4$ (1.0 $\times$ )	$64.5{\pm}0.8$ (1.0 $\times$ )
	Diffusion	89.6 (2.5 $\times$ )	57.8 (3.0 $\times$ )	$85.3{\pm}0.8$ (2.8 $\times$ )	$38.0{\pm}1.2$ (3.5 $\times$ )
	S2D2	$89.8{\pm}0.6$ (1.9 $\times$ )	$64.8{\pm}1.0$ (2.2 $\times$ )	$89.0{\pm}0.7$ (1.8 $\times$ )	$60.8{\pm}1.1$ (2.1 $\times$ )

Appendix

Table 4: Multi-seed LLaDA2.1-Mini results. Accuracy is reported in percent with wall-clock speedup over AR in parentheses.

BS	Method	GSM8K			MBPP
BS	Method	Acc	Tput@128	Tput full	Acc	Tput@128	Tput full
1	BD3 dynamic	0.87	38	125	0.43	33	275
1	S2D2	0.86	45	143	0.72	46	354
32	BD3 dynamic	0.85	1064	3458	0.41	902	7485
32	S2D2	0.85	1378	4555	0.69	1195	8914
64	BD3 dynamic	0.85	1800	5809	0.40	1491	12092

Appendix

Table 5: Batched throughput for SDAR-8B at

B=16,S=16

. Throughput is reported in tokens/s.

BS	Method	GSM8K			MBPP
BS	Method	Acc	Tput@128	Tput full	Acc	Tput@128	Tput full
1	Baseline	0.78	146	342	0.41	83	237
1	S2D2	0.89	143	310	0.47	113	327
32	Baseline	0.83	2021	4515	0.44	1120	3215
32	S2D2	0.89	2535	5160	0.51	1715	4757
64	Baseline	0.83	2137	4752	0.44	1181	3385

Appendix

Table 6: Batched throughput for Fast-dLLM v2 at

B=32,\mathrm{SB}=32

. Throughput is reported in tokens/s.

Quadrant	Description	SDAR	Fast-dLLM v2
Q1	High conf., high ratio	47.9%, 99.2% acc.	44.6%, 99.3% acc.
Q2	Low conf., high ratio	28.5%, 97.8% acc.	21.3%, 96.4% acc.
Q3	High conf., low ratio	2.4%, 5.7% acc.	3.8%, 4.6% acc.
Q4	Low conf., low ratio	21.2%, 5.4% acc.	30.3%, 4.8% acc.

Appendix

Table 7: Per-token draft-confidence versus verifier-ratio analysis on GSM8K. Fractions are over verified tokens.

Config	Accuracy	Avg. NFE
AR ( $B=1$ )	20.0%	225.5
Diffusion ( $B=32$ )	50.0%	72.6
S2D2 ( $B=32$ , resampling)	33.3%	58.5
S2D2 ( $B=32$ , reject only)	46.7%	67.7

Appendix

Table 8: Failure case on a D2F LLaDA variant evaluated on GSM8K.

Config	$B=8$	$B=16$	$B=32$	Slowdown
Baseline	85.0 / 61.5	90.5 / 63.5	90.0 / 56.0	1 $\times$
Residual, $w=0.2S$	84.0 / 60.5	88.5 / 61.0	92.5 / 61.0	$\sim$ 3 $\times$
Residual, $w=S$	84.5 / 63.5	91.0 / 59.0	88.5 / 57.5	$\sim$ 10 $\times$
NLL, $w=0.2S$	85.5 / 63.0	90.5 / 58.5	91.0 / 57.5	$\sim$ 3 $\times$
NLL, $w=S$	90.5 / 63.0	92.0 / 61.5	92.5 / 59.0	$\sim$ 10 $\times$

Appendix

Table 9: Self-normalized importance sampling on LLaDA2.1-Mini with

k=8

. Accuracy is GSM8K/MBPP. Slowdown is approximate relative to single-sample baseline generation.

Method	GSM8K			MBPP
Method	Acc	NFE	Wall	P@1	NFE	Wall
AR ( $B=1$ )	0.900	1.00 $\times$	1.00 $\times$	0.665	1.00 $\times$	1.00 $\times$
S2D2 config-B	0.895	4.65 $\times$	3.62 $\times$	0.595	3.77 $\times$	3.04 $\times$
BD3 dynamic $\tau=0.85$	0.835	3.63 $\times$	3.58 $\times$	0.465	2.92 $\times$	2.73 $\times$
Periodic step $p=2$	0.865	4.33 $\times$	3.28 $\times$	0.480	3.29 $\times$	2.58 $\times$
Periodic block $bp=2$	0.835	4.47 $\times$	3.71 $\times$	0.490	3.43 $\times$	2.74 $\times$

Appendix

Table 10: Inference-time alternatives on SDAR-8B with

B=16

. Speedups are relative to AR.

Random baseline $\alpha_{i}\sim\mathcal{U}[0,1]$
Config	Mean Error	Std Error	MAE
–	-8.014	10.293	8.129
Soft entropy estimator $\alpha_{i}=\exp(-\beta\tilde{H}_{i})$
$\beta=0.5$	-0.445	8.598	5.843
$\beta=0.75$	-1.956	8.872	5.279
$\beta=1.0$	-2.861	9.031	5.084

Appendix

Table 11: Token acceptance estimator ablation. We report signed error (Mean Error), error standard deviation (Std Error), and mean absolute error (MAE) for predicting accepted prefix length. Lower absolute error is better.

Config

B=1 (AR)

B=4

B=8

B=16

B=32

GSM8K

MBPP

Avg

GSM8K

MBPP

Avg

GSM8K

MBPP

Avg

GSM8K

MBPP

Avg

GSM8K

MBPP

Avg

Static

89.0 (1.0

\times

)

63.6 (1.0

\times

)

76.3 (1.0

\times

)

90.5 (1.5

\times

)

62.6 (1.5

\times

)

76.6 (1.5

\times

)

90.5 (1.8

\times

)

50.8 (1.7

\times

)

70.7 (1.7

\times

)

89.0 (1.6

\times

)

36.2 (1.7

\times

)

62.6 (1.6

\times

)

86.0 (1.4

\times

)

12.4 (1.5

\times

)

49.2 (1.4

\times

)

Dynamic

–

91.0 ( 2.9

\times

)

60.8 ( 2.0

\times

)

75.9 ( 2.4

\times

)

90.5 ( 3.3

\times

)

53.6 ( 2.6

\times

)

72.0 ( 3.0

\times

)

86.0 ( 3.4

\times

)

49.0 ( 2.8

\times

)

67.5 ( 3.1

\times

)

83.0 ( 3.2

\times

)

43.6 ( 2.8

\times

)

63.3 ( 3.0

\times

)

Appendix

Table 12: Baseline BD3 results with static and dynamic confidence-based unmasking.

$\tau_{\mathrm{span}}$	B=4			B=8			B=16			B=32
	GSM8K	MBPP	Avg	GSM8K	MBPP	Avg	GSM8K	MBPP	Avg	GSM8K	MBPP	Avg
1	90.0 (1.9 $\times$ )	61.8 (1.9 $\times$ )	75.9 (1.9 $\times$ )	90.0 (2.9 $\times$ )	61.2 (2.5 $\times$ )	75.6 (2.7 $\times$ )	90.0 (3.5 $\times$ )	61.4 ( 3.0 $\times$ )	75.7 (3.2 $\times$ )	89.5 (4.7 $\times$ )	59.8 (3.2 $\times$ )	74.6 (3.8 $\times$ )
2	91.5 ( 2.1 $\times$ )	59.6 ( 2.0 $\times$ )	75.6 ( 2.0 $\times$ )	92.0 (3.2 $\times$ )	60.6 (2.5 $\times$ )	76.3 (2.8 $\times$ )	90.0 (3.9 $\times$ )	61.0 (2.9 $\times$ )	75.5 (3.3 $\times$ )	93.0 ( 4.8 $\times$ )	58.6 ( 3.4 $\times$ )	75.8 ( 4.0 $\times$ )
3	93.0 (1.9 $\times$ )	61.2 (1.9 $\times$ )	77.1 (1.9 $\times$ )	–	–	–	–	–	–	–	–	–
4	–	–	–	91.5 ( 3.3 $\times$ )	60.4 ( 2.8 $\times$ )	76.0 ( 3.0 $\times$ )	91.5 (4.1 $\times$ )	59.4 ( 3.0 $\times$ )	75.4 ( 3.5 $\times$ )	91.5 ( 4.8 $\times$ )	58.8 (3.1 $\times$ )	75.2 (3.8 $\times$ )
7	–	–	–	91.5 (3.2 $\times$ )	58.8 (2.6 $\times$ )	75.2 (2.9 $\times$ )	–	–	–	–	–	–

Appendix

Table 13: Ablation of the minimum-span policy. We vary the minimum contiguous mask-span threshold

\tau_{\mathrm{span}}

\tau_{\mathrm{score}}

c

B=4

B=8

B=16

B=32

GSM8K

MBPP

Avg

GSM8K

MBPP

Avg

GSM8K

MBPP

Avg

GSM8K

MBPP

Avg

Static score

s=\hat{K}-c

-5

90.0 (2.0

\times

)

62.0 (1.9

\times

)

76.0 (2.0

\times

)

92.5 (3.3

\times

)

62.6 (2.6

\times

)

77.6 (2.9

\times

)

90.0 (3.6

\times

)

59.0 (2.9

\times

)

74.5 (3.2

\times

)

89.5 ( 4.7

\times

)

59.8 ( 3.4

\times

)

74.6 ( 4.0

\times

)

93.0 (2.1

\times

)

62.0 (2.0

\times

)

77.5 (2.1

\times

)

91.5 (2.8

\times

)

62.6 (2.4

\times

)

77.0 (2.6

\times

)

91.0 (4.0

\times

)

59.8 (3.2

\times

)

75.4 (3.6

\times

)

90.0 (4.2

\times

)

57.0 (3.2

\times

)

73.5 (3.7

\times

)

93.0 (2.3

\times

)

61.8 (2.0

\times

)

77.4 (2.1

\times

)

91.5 (3.0

\times

)

61.2 (2.5

\times

)

76.4 (2.8

\times

)

90.5 ( 4.2

\times

)

61.4 (3.0

\times

)

76.0 (3.5

\times

)

90.0 (3.9

\times

)

59.8 (3.0

\times

)

74.9 (3.4

\times

)

-1

90.0 (2.2

\times

)

62.0 (2.0

\times

)

76.0 (2.1

\times

)

90.0 (3.0

\times

)

61.2 (2.5

\times

)

75.6 (2.7

\times

)

90.5 ( 4.2

\times

)

61.4 (3.2

\times

)

76.0 ( 3.7

\times

)

90.0 (4.4

\times

)

59.8 (3.1

\times

)

74.9 (3.7

\times

)

Appendix

Table 14: Ablation of score-threshold policies with entropy -based token acceptance estimator. Static score uses

s=\hat{K}-c

; dynamic score uses

s=\hat{K}-c\cdot N_{\mathrm{hi}}

\tau_{\mathrm{score}}

c

B=4

B=8

B=16

B=32

GSM8K

MBPP

Avg

GSM8K

MBPP

Avg

GSM8K

MBPP

Avg

GSM8K

MBPP

Avg

Static score

s=\hat{K}-c

-5

93.0 (2.2

\times

)

59.6 (2.1

\times

)

76.3 (2.1

\times

)

90.5 (3.1

\times

)

56.2 (2.7

\times

)

73.4 (2.9

\times

)

89.5 (3.4

\times

)

57.0 (3.1

\times

)

73.2 (3.3

\times

)

89.5 (3.7

\times

)

57.2 ( 3.6

\times

)

73.4 (3.6

\times

)

93.0 (2.2

\times

)

59.6 (1.9

\times

)

76.3 (2.1

\times

)

90.5 (3.1

\times

)

56.2 (2.7

\times

)

73.4 (2.9

\times

)

90.5 (3.5

\times

)

57.0 (3.2

\times

)

73.8 (3.3

\times

)

89.0 (3.7

\times

)

55.6 (3.2

\times

)

72.3 (3.4

\times

)

93.0 (2.2

\times

)

62.0 (2.0

\times

)

77.5 (2.1

\times

)

89.0 (3.1

\times

)

56.6 (2.9

\times

)

72.8 (3.0

\times

)

89.5 (3.6

\times

)

55.2 (3.2

\times

)

72.4 (3.4

\times

)

89.5 (3.7

\times

)

55.6 (3.3

\times

)

72.5 (3.5

\times

)

-1

93.0 (2.2

\times

)

61.8 (2.0

\times

)

77.4 (2.1

\times

)

91.0 (3.1

\times

)

62.6 (2.4

\times

)

76.8 (2.8

\times

)

90.5 (4.1

\times

)

61.4 (3.1

\times

)

76.0 (3.6

\times

)

92.5 (4.3

\times

)

59.8 (3.1

\times

)

76.2 (3.6

\times

)

Appendix

Table 15: Ablation of score-threshold policies with margin -based token acceptance estimator. Static score uses

s=\hat{K}-c

; dynamic score uses

s=\hat{K}-c\cdot N_{\mathrm{hi}}

$\tau_{\mathrm{on}}$	$\tau_{\mathrm{off}}$	B=4			B=8			B=16			B=32
		GSM8K	MBPP	Avg	GSM8K	MBPP	Avg	GSM8K	MBPP	Avg	GSM8K	MBPP	Avg
0	-5	90.5 (1.8 $\times$ )	62.4 (1.9 $\times$ )	76.4 (1.8 $\times$ )	91.5 (3.2 $\times$ )	63.6 (2.7 $\times$ )	77.6 (3.0 $\times$ )	89.0 (4.0 $\times$ )	60.2 (2.6 $\times$ )	74.6 (3.1 $\times$ )	91.0 (4.3 $\times$ )	59.2 (3.0 $\times$ )	75.1 (3.5 $\times$ )
0	-1	93.0 (2.1 $\times$ )	62.4 (1.9 $\times$ )	77.7 (2.0 $\times$ )	89.5 (3.2 $\times$ )	63.6 (2.6 $\times$ )	76.6 (2.9 $\times$ )	91.5 ( 4.1 $\times$ )	60.2 (3.0 $\times$ )	75.8 (3.5 $\times$ )	91.0 (4.0 $\times$ )	59.4 (3.1 $\times$ )	75.2 (3.5 $\times$ )
1	-5	91.5 (2.1 $\times$ )	62.4 ( 2.0 $\times$ )	77.0 (2.0 $\times$ )	89.0 (3.3 $\times$ )	59.8 (2.7 $\times$ )	74.4 (3.0 $\times$ )	91.5 (3.8 $\times$ )	59.4 (2.9 $\times$ )	75.4 (3.3 $\times$ )	90.5 (4.2 $\times$ )	55.8 ( 3.2 $\times$ )	73.2 ( 3.7 $\times$ )
1	-1	91.5 (1.9 $\times$ )	62.4 (1.9 $\times$ )	77.0 (1.9 $\times$ )	88.5 (3.1 $\times$ )	59.8 (2.6 $\times$ )	74.2 (2.9 $\times$ )	91.0 ( 4.1 $\times$ )	61.4 ( 3.2 $\times$ )	76.2 ( 3.6 $\times$ )	91.5 ( 4.4 $\times$ )	57.4 (3.1 $\times$ )	74.4 (3.6 $\times$ )
5	-5	91.0 ( 2.5 $\times$ )	60.6 ( 2.0 $\times$ )	75.8 ( 2.3 $\times$ )	89.5 ( 3.4 $\times$ )	53.6 ( 2.8 $\times$ )	71.5 ( 3.1 $\times$ )	92.5 (3.7 $\times$ )	50.4 (2.9 $\times$ )	71.5 (3.3 $\times$ )	87.5 (3.7 $\times$ )	51.6 (3.1 $\times$ )	69.5 (3.4 $\times$ )

Appendix

Table 16: Ablation of hysteresis policies with entropy -based token acceptance estimator. We use dynamic score only, with fixed

c=1

Tuple $(1,2,2)$
$\beta$	B=4			B=8			B=16			B=32
	GSM8K	MBPP	Avg	GSM8K	MBPP	Avg	GSM8K	MBPP	Avg	GSM8K	MBPP	Avg
0.2	92.5 (2.2 $\times$ )	53.4 (1.9 $\times$ )	73.0 (2.1 $\times$ )	87.0 (3.4 $\times$ )	49.0 (2.5 $\times$ )	68.0 (2.9 $\times$ )	87.0 (3.4 $\times$ )	50.2 (2.7 $\times$ )	68.6 (3.0 $\times$ )	83.5 (4.2 $\times$ )	52.6 ( 3.2 $\times$ )	68.0 (3.6 $\times$ )
0.5	91.5 (2.2 $\times$ )	55.4 (2.0 $\times$ )	73.5 (2.1 $\times$ )	89.5 ( 3.5 $\times$ )	51.6 ( 2.7 $\times$ )	70.5 ( 3.1 $\times$ )	89.5 (3.6 $\times$ )	52.0 (2.8 $\times$ )	70.8 (3.2 $\times$ )	82.5 (3.8 $\times$ )	49.6 (2.8 $\times$ )	66.0 (3.2 $\times$ )
1	92.5 (2.3 $\times$ )	55.0 (2.0 $\times$ )	73.8 (2.2 $\times$ )	85.5 (3.4 $\times$ )	47.6 ( 2.7 $\times$ )	66.5 (3.0 $\times$ )	86.0 (3.9 $\times$ )	50.8 (2.9 $\times$ )	68.4 (3.3 $\times$ )	81.5 (3.5 $\times$ )	48.6 (2.9 $\times$ )	65.0 (3.2 $\times$ )
2	91.5 (2.3 $\times$ )	53.6 (1.8 $\times$ )	72.5 (2.0 $\times$ )	87.5 (2.7 $\times$ )	49.8 (2.4 $\times$ )	68.7 (2.5 $\times$ )	87.5 (3.8 $\times$ )	48.0 (2.9 $\times$ )	67.8 (3.3 $\times$ )	83.0 (3.8 $\times$ )	50.0 (3.0 $\times$ )	66.5 (3.3 $\times$ )

Appendix

Table 17: Ablation of contextual-bandit UCB policies. The context tuple

(a,b,c)

denotes the numbers of bins for mask span length, decoding progress, and entropy, respectively. We report results for different UCB exploration coefficients

\beta

Panel of Figure 5 — Figure 5: AR-ness and decoding confidence statistics for SDAR-8B-Chat. Top row: local/global AR-ness on GSM8K and MBPP. Bottom row: normalized confidence statistics under static and dynamic decoding on GSM8K and MBPP.

Panel of Figure 6 — Figure 6: AR-ness and decoding confidence statistics for LLaDA-2.1-Mini. Top row: local/global AR-ness on GSM8K and MBPP. Bottom row: normalized confidence statistics under static and dynamic decoding on GSM8K and MBPP.

$\gamma$	Config-A			Config-B
$\gamma$	GSM8K	MBPP	Avg	GSM8K	MBPP	Avg
0.5	89.5 (1.02 $\times$ )	60.0 (1.06 $\times$ )	74.8 (1.03 $\times$ )	88.5 (1.02 $\times$ )	58.0 (1.00 $\times$ )	73.2 (1.01 $\times$ )
0.75	89.5 (0.99 $\times$ )	62.0 (1.06 $\times$ )	75.8 (1.01 $\times$ )	90.5 (0.99 $\times$ )	58.5 (0.95 $\times$ )	74.5 (0.98 $\times$ )
1.0	89.5 (1.00 $\times$ )	64.0 (1.00 $\times$ )	76.8 (1.00 $\times$ )	89.5 (1.00 $\times$ )	60.5 (1.00 $\times$ )	75.0 (1.00 $\times$ )
1.25	89.5 (1.00 $\times$ )	60.5 (1.07 $\times$ )	75.0 (1.02 $\times$ )	90.5 (0.93 $\times$ )	60.5 (0.96 $\times$ )	75.5 (0.94 $\times$ )
1.5	90.5 (1.00 $\times$ )	60.5 (0.96 $\times$ )	75.5 (0.99 $\times$ )	90.5 (1.01 $\times$ )	60.0 (0.96 $\times$ )	75.2 (1.00 $\times$ )

Appendix

Table 18: Ablation of rejection-sampling ratio tempering for SSD on SDAR-8B-Chat. We vary the tempering factor

\gamma

in the acceptance ratio

(q_{i}/p_{i})^{\gamma}

and report accuracy (speedup relative to

\gamma{=}1

Speculative Decoding for Diffusion Language Models

Speculative decoding for diffusion language models speeds generation by drafting tokens in parallel and verifying them efficiently.

See all 33 papers

Jun 3, 2026

D^2SD: Accelerating Speculative Decoding with Dual Diffusion Draft Models

Speculative decoding accelerates autoregressive large language model inference by drafting multiple tokens and verifying them in a single target-model forward pass. Recent diffusion-based drafters generate an entire block of tokens in parallel but usually commit to a single draft sequence per verification: once the first mismatch occurs, all subsequent draft tokens are discarded, resulting in a limited acceptance rate. Naively batching more draft candidate sequences only introduces a marginal improvement, as redundant or poorly placed branches increase the cost of drafting and verification without proportionally increasing the number of accepted tokens. We propose D^2SD, a dual diffusion draft speculative decoding framework that organizes candidates into a confidence-guided prefix tree, where the first diffusion drafter generates a block along with per-position confidence scores that are used to identify the most likely rejection boundary and select the top-K prefix ranges for recovery; the second variable-prefix diffusion drafter re-anchors at each selected prefix and proposes alternative continuations in one batched pass; the resulting shared-prefix candidates are jointly verified via cascade attention. Empirically, D^2SD shows clear improvements over both the underlying diffusion approach and strong autoregressive speculative decoding baselines.

Liyuan Zhang, Jiarui Zhang, Jinwei Yao +6

Peking University · Tsinghua University · HKUST +2

Speculative Decoding for Diffusion Language Models Speculative Decoding Diffusion Language Model Inference LLM Inference Acceleration Diffusion Language Model Decoding Parallel Speculative Decoding

Ask PDF

Jun 11, 2026

Accelerating Speculative Diffusions via Block Verification

Speculative decoding speeds up LLM inference by using a draft model to generate tokens, with an acceptance-rejection scheme that ensures that the output matches the target distribution. Adapting this to continuous diffusions is difficult because speculative sampling requires drawing from a residual distribution. While straightforward in discrete spaces, efficiently sampling this residual in continuous space is non-trivial. Consequently, existing diffusion adaptations either use computationally inefficient sampling techniques or rely on an alternative scheme. In this work, we introduce a novel scheme that efficiently implements the original speculative sampling mechanism for diffusion models. Our approach offers a critical advantage over current methods: it enables us to adapt block verification from LLMs to diffusions -- which provably improves the acceptance rate of drafts. Furthermore, we formalize and analyze the Free Drafter, a heuristic self-speculative drafter for diffusions that requires no training. By enabling block verification, our Free Drafter yields up to a 6.3% speedup over existing speculative methods with no additional training and negligible overhead beyond the existing parallel verification pass.

Alexander Soen, Hisham Husain, Valentin De Bortoli +1

KTH · Google Research† · Google DeepMind‡

Speculative Decoding for Diffusion Language Models Speculative Decoding Diffusion Model Inference Acceleration Diffusion Models

Ask PDF

Jun 1, 2026

SimSD: Simple Speculative Decoding in Diffusion Language Models

Diffusion large language models (dLLMs) have recently emerged as a promising alternative to autoregressive (AR) LLMs, offering faster inference through parallel or blockwise decoding. However, their masked language modeling formulation remains incompatible with standard token-level speculative decoding, one of the most effective acceleration techniques for AR models. In AR decoding, the causal mask preserves temporally valid token-level contexts, enabling a target model to verify multiple drafted tokens in a single forward pass. In contrast, dLLMs rely on mask tokens and bidirectional attention, causing the effective context to change across denoising steps and preventing direct token-level speculative verification. To bridge this gap, we propose a simple but effective speculative decoding algorithm for diffusion language models, named SimSD, which mainly adopts a plug-and-play masking strategy that equips dLLMs with temporally valid token-level contexts for speculative decoding. Our method explicitly introduces reference tokens from draft-model predictions and designs an attention mask that regulates their interaction with current-step tokens, allowing dLLMs to compute valid logits for drafted tokens in a single forward pass. This restores the key verification ability provided by causal masking in AR models while preserving the parallel decoding advantages of dLLMs. The proposed method is training-free and can be flexibly integrated with other acceleration techniques such as KV cache and blockwise decoding. Experiments on SDAR-family dLLMs across four benchmarks show that our method achieves up to 7.46x higher decoding throughput while maintaining and even improving average generation quality.

Junxia Cui, Haotian Ye, Runchu Tian +9

University of California San Diego · University of Illinois Urbana-Champaign · 3Google +1

Masked Diffusion Models Speculative Decoding for Diffusion Language Models Speculative Decoding Diffusion Language Model Inference Diffusion Language Model Decoding Diffusion Language Models

Ask PDF

Abstract

Speculative Decoding for Diffusion Language Models

Explore similar work

D^2SD: Accelerating Speculative Decoding with Dual Diffusion Draft Models

Accelerating Speculative Diffusions via Block Verification

SimSD: Simple Speculative Decoding in Diffusion Language Models