Organizations: National Key Laboratory for Novel Software Technology, Nanjing University · Data Science & Artificial Intelligence Research Institute, China Unicom · Unicom Data Intelligence, China Unicom
Speculative decoding accelerates autoregressive generation by allowing a lightweight drafter to propose future tokens while a target model verifies them in parallel. Its lossless guarantee has motivated a line of work that pushes the drafter itself toward parallel generation. The most recent paradigm is block-parallel generative drafting, including diffusion-based methods such as DFlash and DSpark, achieving up to 3.6x speedup on common daily chatting tasks. While this transition is well studied in text-only LLMs, its applicability to multimodal models remains an open question. Existing multimodal speculative decoding efforts focus on input compression, adapter alignment, candidate coverage, or modality-specific verification; however, block-parallel generative drafting remains largely unexplored. To bridge this gap, this paper combines a modality-centered survey with a cross-architecture empirical study to ask: Is multimodal speculative decoding ready for diffusion-based parallel drafting? In this survey, we systematically analyze a wide spectrum of multimodal models, spanning Vision-Language, Video-Language, Audio, and Vision-Language-Action (VLA) architectures, from the dual perspectives of drafting parallelism and cross-modal information interaction. We introduce a unified taxonomy that isolates drafter-side parallelism from orthogonal design choices such as tree construction and verification strategies. Furthermore, we provide a comprehensive empirical comparison of existing methods under varying degrees of parallelism across standardized multimodal benchmarks, including OCR, VQA, visual reasoning, and image captioning. Finally, we summarize the limitations of current approaches, discuss open challenges, and outline promising future directions for this rapidly evolving field.
Figures & tables
Method
Drafter
Mechanism
L
Condition
Candidate form
Acceptance
Adaptation
Vision–Language Models
SPD-MLLM ( 2024 )
L0
Text / direct
Chain
Exact
Static
MASSV ( 2025 )
L0
Target feat.
Chain
Exact
Static
DREAM ( 2025b )
L0
Target feat.
Chain
Exact
Static
HiViS ( 2026 )
L0
Target feat.
Chain
Exact
Static
Table 1: Taxonomy of representative multimodal speculative decoding methods across application domains, drafter designs, and speculative mechanisms.
Figure 1: Taxonomy and evolution of representative multimodal speculative decoding methods.
Application
Domain constraint
Draft profile
Acceptance semantics
Domain-specific cue
VLM
Grounding–cost trade-off
L0 / L1 / L2
Token exactness
Visual relevance
Video-LM
Long-context cost
L0
Token exactness
Temporal relevance
VLA
Closed-loop state
L0 / L2
Functional validity
Control phase
Speech/Audio
Temporal dependencies
L0 / L1
Sequence / perceptual
Prediction uncertainty
Visual Generation
Spatiotemporal coupling
L0 / L1 / L2 (∗)
Group / quality
Spatial locality
Table 2: Cross-domain synthesis of multimodal speculative decoding. L-level entries report representative candidate-generation mechanisms rather than exhaustive coverage within each domain.
Figure 2: Illustration of the L0–L2 taxonomy of drafter-side parallelism. L0 performs sequential drafting with one future position advanced per drafter forward; L1 predicts multiple predefined future positions in parallel from a shared prefix; and L2 treats the future block as a jointly generated state, enabling block-parallel drafting followed by parallel target verification.
Level
Drafting regime
Core definition
Representative methods
L0
Sequential drafting
One forward advances one draft depth; reaching depth K requires K causally dependent forwards
Small AR LM; EAGLE
L1
Multi-position parallel drafting
One forward predicts several predefined future positions in parallel from a shared prefix, but does not treat the future block as a jointly evolving generative state
Medusa; MTP; SpecFLASH
L2
Block-parallel generative drafting
The future block is the basic generative state; its positions are jointly generated or refined with bounded serial drafting depth
DFlash; DSpark
Table 3: A L0–L2 taxonomy of drafter-side parallelism. K denotes the number of future positions considered in one speculation round. The taxonomy describes how candidate trajectories are generated; candidate width, tree construction, and target-side verification are orthogonal.
Target
Method
GQA
F30K
TextVQA
CharXiv
MMMU
MT
Overall
MAT
Speedup
MAT
Speedup
MAT
Speedup
MAT
Speedup
MAT
Speedup
MAT
Speedup
MAT
Speedup
Q3VL-4B
AR
–
1.00 ×
–
1.00 ×
–
1.00 ×
–
1.00 ×
–
1.00 ×
–
1.00 ×
–
1.00 ×
EAGLE-3
1.95
0.71 ×
1.96
0.67 ×
1.98
0.71 ×
2.28
0.76 ×
2.41
0.81 ×
2.14
0.60 ×
2.12
0.71 ×
DFlash †
3.05
2.28 ×
2.84
2.36 ×
2.05
1.57 ×
2.63
2.06 ×
2.28
1.74 ×
2.13
1.70 ×
2.50
1.95 ×
Q3VL-8B
AR
–
1.00 ×
–
1.00 ×
–
1.00 ×
–
1.00 ×
–
1.00 ×
–
1.00 ×
–
1.00 ×
EAGLE-3
2.22
0.85 ×
2.20
0.85 ×
2.05
0.81 ×
2.66
0.99 ×
2.48
0.95 ×
2.17
0.85 ×
2.30
0.88 ×
Table 4: Speculative decoding performance with a maximum of 2,048 new tokens. MAT denotes the mean number of output tokens advanced per speculative step. Overall results are task-equal means. The best and second-best speculative results for each target are shown in bold and underlined, respectively. † Results measured on the HuggingFace Transformers backend; all other results use the SGLang backend. Each speedup is normalized to its own backend-matched autoregressive baseline. We abbreviate target models Q denotes Qwen.zheli
Figure 3: Stage-level latency decomposition of a representative decoding pass on Qwen3.6-27B. Vision conditioning includes vision encoding, multimodal prefill, and drafter-side target-feature-to-KV construction when applicable.
Resolution
Qwen3-VL-4B
Qwen3-VL-8B
Top1
Sampling
Top1
Sampling
MAT
Speedup
MAT
Speedup
MAT
Speedup
MAT
Speedup
HR-Bench 4K
2.90
1.03 ×
2.45
0.96 ×
3.22
1.06 ×
2.88
1.11 ×
HR-Bench 8K
2.86
0.90 ×
2.70
0.83 ×
3.09
0.99 ×
3.22
0.85 ×
Table 5: HR-Bench results under 4K and 8K resolutions on Qwen3-VL, with a 512-token generation cap on a common 30-sample manifest. MAT denotes the mean number of output tokens advanced per speculative step. Speedup is end-to-end latency speedup over the backend-matched autoregressive baseline. Top1 denotes top-1 acceptance; Sampling denotes speculate sampling.
Model
DFlash context setting
GQA
TextVQA
MMMU
Overall MAT ↑
Δ Overall vs. full
Qwen3-VL-8B
Full multimodal context
2.692
2.750
3.259
2.900
–
Text-only context
2.608
2.504
3.246
2.786
−0.114 ( −3.9% )
No-prefill context
2.845
1.953
2.946
2.581
−0.319 ( −11.0% )
Qwen3-VL-4B
Full multimodal context
2.587
2.305
2.583
2.491
–
Text-only context
2.487
2.221
2.453
2.387
−0.104 ( −4.2% )
No-prefill context
2.095
1.904
2.417
2.139
−0.353 ( −14.1% )
Table 6: Effect of multimodal target context on DFlash speculative acceptance. Results use top-1 verification with 100 examples per task and a maximum generation length of 2,048 tokens. Overall MAT is the equal-weight arithmetic mean over GQA, TextVQA, and MMMU. Text-only context masks visual-token K/V entries in the DFlash context, whereas no-prefill context masks the complete original prefill context. The target model always receives the full multimodal input.
Repository
Models
L2 Methods
Framework Support
DFlash
Qwen3/3.5/3.6, Gemma4
DFlash
MLX, SGLang, vLLM
DeepSpec
Qwen3, Gemma4
DFlash, DSpark
Train, Eval
AngelSpec
Qwen3, Hunyuan3
DFly, DFlash, DFlare, DSpark
Train, vLLM, SGLang
Speculators
Qwen3/3.6, Gemma4
DFlash, DSpark
Train, vLLM
SpecForge
Qwen3/3.6
DFlash, Domino, DSpark
Train, SGLang
Table 7: Representative framework support for L2 speculative decoding.
Speculative decoding accelerates autoregressive language models by using a lightweight drafter to propose multiple future tokens, which the target model then verifies in parallel. DFlash shows that a block diffusion drafter can generate an entire draft block in a single forward pass and achieve state-of-the-art speculative decoding performance, outperforming strong autoregressive drafters such as EAGLE-3. Vanilla DFlash, however, still verifies only a single drafted trajectory per round, potentially limiting its acceptance length. We introduce DDTree (Diffusion Draft Tree), a method that constructs a draft tree directly from the per-position distributions of a block diffusion drafter. Under a fixed node budget, DDTree uses a simple best-first heap algorithm to select the continuations that are most likely to match the target model according to a surrogate defined by the draft model's output. The resulting tree is verified efficiently in a single target model forward pass using an ancestor-only attention mask. Because DDTree builds on DFlash, a leading draft model for speculative decoding, these gains place DDTree among the leading approaches to speculative decoding.
Liran Ringel, Yaniv Romano
Department of Computer Science, Technion – Israel Institute of Technology. · Department of Electrical and Computer Engineering, Technion – Israel Institute of Technology.
Speculative decoding is a technique that accelerates the inference speed of autoregressive large language models (LLMs) by sampling from a faster, lower quality draft model. Traditionally, a fundamental characteristic of speculative decoding is that it is lossless with respect to the verifier distribution, guaranteeing speed without sacrificing quality. This differs from recently developed diffusion LLMs (dLLMs), which are notably able to achieve substantial speedups by degrading the quality of their outputs with less diffusion effort. Because diffusion models are able to sample faster than autoregressive models by generating tokens in parallel, they are considered strong candidates for draft models in speculative diffusion. While it is widely understood that diffusion models can sacrifice quality for speed, speculative decoding itself can also be generalized to generate lossy samples faster using more lenient acceptance criteria with respect to the verifier distribution. Presented with the two parameters of diffusion effort and speculative decoding leniency, this paper explores the full achievable Pareto frontier via joint variations of these parameters. Our experiments show that jointly varying diffusion effort and acceptance leniency can extend the speed-quality Pareto frontier over single step diffusion drafting.
Aaron Zorel, Guy Van den Broeck, Aditya Grover +1
Independent Researcher · University of California, Los Angeles
Speculative decoding (SD) accelerates large language model (LLM) inference by using a smaller model to draft future tokens, which are then verified by the target LLM. This preserves generation quality by accepting only aligned tokens. However, individual drafters, often trained for specific tasks or domains, exhibit limited effectiveness across diverse applications. To address this, we introduce MetaSD, a unified framework that integrates multiple drafters into the SD process. MetaSD dynamically allocates computational resources to heterogeneous drafters by leveraging alignment feedback and framing drafter selection as a multi-armed bandit problem. Extensive experiments show MetaSD consistently outperforms single-drafter approaches.