Speculative Diffusion is a Tale of Two Tradeoffs

Aaron Zorel; Guy Van den Broeck; Aditya Grover; Daniel Israel

cs.LGOct 22, 2026

Speculative Diffusion is a Tale of Two Tradeoffs

Authors: Aaron Zorel, Guy Van den Broeck, Aditya Grover, Daniel Israel

Organizations: Independent Researcher · University of California, Los Angeles

Abstract

Speculative decoding is a technique that accelerates the inference speed of autoregressive large language models (LLMs) by sampling from a faster, lower quality draft model. Traditionally, a fundamental characteristic of speculative decoding is that it is lossless with respect to the verifier distribution, guaranteeing speed without sacrificing quality. This differs from recently developed diffusion LLMs (dLLMs), which are notably able to achieve substantial speedups by degrading the quality of their outputs with less diffusion effort. Because diffusion models are able to sample faster than autoregressive models by generating tokens in parallel, they are considered strong candidates for draft models in speculative diffusion. While it is widely understood that diffusion models can sacrifice quality for speed, speculative decoding itself can also be generalized to generate lossy samples faster using more lenient acceptance criteria with respect to the verifier distribution. Presented with the two parameters of diffusion effort and speculative decoding leniency, this paper explores the full achievable Pareto frontier via joint variations of these parameters. Our experiments show that jointly varying diffusion effort and acceptance leniency can extend the speed-quality Pareto frontier over single step diffusion drafting.

Figures & tables

Figure 1: A conceptual overview. Diffusion has a speed-quality tradeoff parameterized by diffusion effort

K

, while speculative decoding has a speed-quality tradeoff parameterized by acceptance leniency

\lambda

. This work argues that speculative diffusion exposes both tradeoffs and therefore reveals optimal configurations that could not be found by varying only a single parameter.

Panel of Figure 2 — Figure 2: Illustrative examples. Left: Visualization of the proof of Theorem 1 shows that the linear interpolated distribution $\tilde{p}^{\star}$ is the optimal solution to maximize drafter $q$ and verifier $p$ alignment while staying within a constraint on deviation from the verifier $\epsilon$ . The solution to this constrained optimization is $\tilde{p}$ . Right: A motivating example shows that the speed-quality frontiers independently induced by diffusion effort and acceptance lenience in speculative decoding can occupy separate regions of the Pareto frontier. We compare diffusion only with AR speculative decoding with leniency.

Panel of Figure 3 — Figure 3: A model of speculative diffusion. Theoretical and measured tradeoffs for Dream-7B drafting for Qwen2.5-32B on GSM8K. (a) The theoretical acceptance rate (speed) and verifier agreement (quality) under linear acceptance for each of $K$ diffusion steps per round. (b) Empirically measured acceptance rate for each $K$ . (c) A transformation from acceptance rate to throughput according to a simple model of throughput. (d) Empirically measured throughput.

Figure 4: Examples of the Pareto frontier. We provide examples of speculative diffusion for both masked diffusion (left) and uniform diffusion (right). Each point signifies a configuration of acceptance leniency in verification and diffusion effort (

K

steps) in drafting. We observe the same qualitative trend that the speed-quality Pareto frontier is occupied by many different settings of diffusion effort which require different levels of lenience.

Figure 5: Comparison of Pareto frontiers. For a single Qwen3 8B verifier, we benchmark several speculative diffusion methods. Each line corresponds to the achievable speed-quality frontier of a specific draft model. Dashed curves fix one diffusion step and vary leniency, while solid curves are varied over diffusion effort and acceptance lenience. For the SDAR models that support multi-step speculative diffusion, varying over the parameters jointly significantly extends the Pareto frontier.

Appendix figures & tables13 assets

Supplementary material from the paper’s appendix.

Appendix

Model pair	Benchmark	$n$	Drafter (%)	Verifier (%)	Gap (pp)
Dream 7B $\to$ Qwen2.5 32B	GSM8K	500	$79.8\pm 1.8$	$95.0\pm 1.0$	$+15.2$
	HumanEval	164	$57.3\pm 3.9$	$84.8\pm 2.8$	$+27.4$
	GPQA Diamond	198	$22.7\pm 3.0$	$45.5\pm 3.5$	$+22.7$
SDAR 8B $\to$ Qwen3 8B	GSM8K	500	$85.0\pm 1.6$	$92.2\pm 1.2$	$+7.2$
	HumanEval	164	$39.6\pm 3.8$	$85.4\pm 2.8$	$+45.7$
	GPQA Diamond	198	$25.3\pm 3.1$	$43.4\pm 3.5$	$+18.2$

Appendix

Table 1: Baseline task accuracy with one standard error. The gap is verifier minus drafter accuracy in percentage points.

Model / role	Hugging Face checkpoint	Reference
Dream drafter	Dream-org/Dream-v0-Instruct-7B	Ye et al. (2025)
Qwen2.5 verifier	Qwen/Qwen2.5-32B-Instruct	Qwen Team (2024b)
DiffusionGemma drafter	google/diffusiongemma-26B-A4B-it	DiffusionGemma Team (2026)
Gemma verifier	google/gemma-4-31B-it	Gemma Team (2026)
Qwen3 verifier / Uno	s-sahoo/uno-qwen3-8B	Sahoo et al. (2026)
SDAR 8B drafter	JetLM/SDAR-8B-Chat-b16	Cheng et al. (2026a)

Appendix

Table 2: Hugging Face checkpoints used in the experiments.

Sampler	Temperature	Top- $p$	Top- $k$
Dream draft / standalone	0.2	0.95	–
Qwen2.5-32B verifier	0.7	0.8	20
DiffusionGemma draft / standalone	$0.8\to 0.4$	0.95	–
Gemma-4-31B verifier	1.0	0.95	64
SDAR draft	0.6	0.95	20
Uno draft	1.0	0.95	50

Appendix

Table 3: Sampling settings for the reported runs. A dash indicates no additional truncation recorded for that setting. DiffusionGemma anneals temperature during denoising and uses 0.4 for a single denoising step.

Model / pair	Dataset	Diffusion effort	Acceptance	Block	Token limit
Dream 7B + Qwen2.5 32B	GSM8K	$K\in\{1,2,3,4\}$	Cactus	$\leq 512$	512
	HumanEval	$K\in\{1,2,3,4\}$	Cactus	$\leq 512$	512
DiffusionGemma 26B-A4B + Gemma-4 31B	GPQA	$K\in\{1,2,4,8\}$	Cactus	256	2048
	GPQA	$K\in\{1,2,4,8\}$	Linear	256	2048
SDAR 8B	GSM8K, HumanEval	$\tau=0.3,0.5,0.7,0.9$ or fixed $K=1$	Linear	16	512
SDAR 1.7B	GSM8K, HumanEval	$\tau=0.3,0.5,0.7,0.9$ or fixed $K=1$	Linear	16	512

Appendix

Table 4: Main experimental configurations. The final five rows share the Qwen3-8B verifier. Block lengths are the configured values, distinct from full-completion limits. DFlash’s 16-position block includes one anchor and 15 draft positions. SDAR’s confidence threshold

\tau

controls denoising effort adaptively.

Model pair / dataset	$K$	Cactus budgets $\delta$
Dream–Qwen / GSM8K	1	$0,0.25,0.5,1,\ldots,8$
	2	$0,0.25,0.5,1,\ldots,11$
	3	$0,0.25,0.5,1,\ldots,13$
	4	$0,0.25,0.5,1,\ldots,15$
DiffusionGemma–Gemma / GPQA	1	$0,0.25,0.5,1,\ldots,5$
	2	$0,0.25,0.5,1,\ldots,6$

Appendix

Table 5: Evaluated Cactus budgets at each diffusion effort.

Figure 6: Per-round latency measurements supporting Figure 3 . Left: drafting costs and their linear fit in

K

. Right: the distribution of verifier costs across configurations, with its mean and one-standard-deviation band.

Panel of Figure 7 — Figure 7: Latency and throughput on GSM8K with the Dream-7B drafter and Qwen2.5-32B-Instruct verifier

Panel of Figure 8 — Figure 8: Latency and throughput on GPQA Diamond with the DiffusionGemma 26B drafter and Gemma4 31B verifier.

Panel of Figure 9 — Figure 9: Block-size ablation on GSM8K with Dream–Qwen.

Panel of Figure 10 — Figure 10: Latency and throughput on HumanEval with the Dream 7B drafter and Qwen2.5 32B verifier.

Panel of Figure 11 — Figure 11: Latency and throughput on HumanEval with the DiffusionGemma 26B drafter and Gemma4 31B verifier.

Figure 12: Complete sweeps for Figure 5 . Colors distinguish SDAR’s effort settings within each model size. Hollow markers show measured configurations, including dominated points. Thick curves show the joint and fixed-

K=1

frontiers, using the same definitions as the main figure.

Panel of Figure 13 — Figure 13: Uno with one, two, and four drafting steps. Colors indicate $K$ , with leniency varied within each color. Thick lines connect nondominated measurements, and hollow markers show the remaining configurations. The GSM8K sweep lacks completed $K=4$ runs at $\lambda\in\{0.8,1\}$ .

LLM Inference

Large language model inference studies how models generate outputs efficiently, reliably, and securely across hardware and workloads.

See all 276 papers

D^2SD: Accelerating Speculative Decoding with Dual Diffusion Draft Models
Jun 3, 2026Liyuan Zhang, Jiarui Zhang, Jinwei Yao +6Speculative Decoding for Diffusion Language Models Speculative Decoding
Peking University · Tsinghua University · HKUST +2
LibraSpec: Dynamic Diffusion-Based Speculative Decoding via Marginal-Gain-Driven Optimization
Aug 9, 2026Zexun Lin, Yuan Feng, Junlin Lv +2Inference-Time Optimization Speculative Decoding for Diffusion Language Models
Suzhou Institute for Advanced Research, University of Science and Technology of China Suzhou, Jiangsu, China
PSD: Pushing the Pareto Frontier of Diffusion LLMs via Parallel Speculative Decoding
May 15, 2026Shengyin Sun, Yiming Li, Renxi Liu +7Parallel Speculative Decoding Masked Diffusion Models
Department of Computer Science City University of Hong Kong · Huawei Technologies · City University of Hong Kong

Jun 3, 2026J/K move · Enter open · S save

D^2SD: Accelerating Speculative Decoding with Dual Diffusion Draft Models

Liyuan Zhang, Jiarui Zhang, Jinwei Yao, Ran Yan+5

Peking University · Tsinghua University · HKUST+2

Speculative Decoding for Diffusion Language Models Speculative Decoding Diffusion Language Model Inference LLM Inference Acceleration Diffusion Language Model Decoding Parallel Speculative Decoding

Open paper AskPDF arXiv

Speculative decoding accelerates autoregressive large language model inference by drafting multiple tokens and verifying them in a single target-model forward pass. Recent diffusion-based drafters generate an entire block of tokens in parallel but usually commit to a single draft sequence per verification: once the first mismatch occurs, all subsequent draft tokens are discarded, resulting in a limited acceptance rate. Naively batching more draft candidate sequences only introduces a marginal improvement, as redundant or poorly placed branches increase the cost of drafting and verification without proportionally increasing the number of accepted tokens. We propose D^2SD, a dual diffusion draft speculative decoding framework that organizes candidates into a confidence-guided prefix tree, where the first diffusion drafter generates a block along with per-position confidence scores that are used to identify the most likely rejection boundary and select the top-K prefix ranges for recovery; the second variable-prefix diffusion drafter re-anchors at each selected prefix and proposes alternative continuations in one batched pass; the resulting shared-prefix candidates are jointly verified via cascade attention. Empirically, D^2SD shows clear improvements over both the underlying diffusion approach and strong autoregressive speculative decoding baselines.

Abstract

LLM Inference

Explore similar work

D^2SD: Accelerating Speculative Decoding with Dual Diffusion Draft Models

LibraSpec: Dynamic Diffusion-Based Speculative Decoding via Marginal-Gain-Driven Optimization

PSD: Pushing the Pareto Frontier of Diffusion LLMs via Parallel Speculative Decoding