Organizations: National Key Laboratory for Novel Software Technology, Nanjing University · Data Science & Artificial Intelligence Research Institute, China Unicom · Unicom Data Intelligence, China Unicom
Speculative decoding accelerates autoregressive generation by allowing a lightweight drafter to propose future tokens while a target model verifies them in parallel. Its lossless guarantee has motivated a line of work that pushes the drafter itself toward parallel generation. The most recent paradigm is block-parallel generative drafting, including diffusion-based methods such as DFlash and DSpark, achieving up to 3.6x speedup on common daily chatting tasks. While this transition is well studied in text-only LLMs, its applicability to multimodal models remains an open question. Existing multimodal speculative decoding efforts focus on input compression, adapter alignment, candidate coverage, or modality-specific verification; however, block-parallel generative drafting remains largely unexplored. To bridge this gap, this paper combines a modality-centered survey with a cross-architecture empirical study to ask: Is multimodal speculative decoding ready for diffusion-based parallel drafting? In this survey, we systematically analyze a wide spectrum of multimodal models, spanning Vision-Language, Video-Language, Audio, and Vision-Language-Action (VLA) architectures, from the dual perspectives of drafting parallelism and cross-modal information interaction. We introduce a unified taxonomy that isolates drafter-side parallelism from orthogonal design choices such as tree construction and verification strategies. Furthermore, we provide a comprehensive empirical comparison of existing methods under varying degrees of parallelism across standardized multimodal benchmarks, including OCR, VQA, visual reasoning, and image captioning. Finally, we summarize the limitations of current approaches, discuss open challenges, and outline promising future directions for this rapidly evolving field.
Figures & tables
Method
Drafter
Mechanism
L
Condition
Candidate form
Acceptance
Adaptation
Vision–Language Models
SPD-MLLM ( 2024 )
L0
Text / direct
Chain
Exact
Static
MASSV ( 2025 )
L0
Target feat.
Chain
Exact
Static
DREAM ( 2025b )
L0
Target feat.
Chain
Exact
Static
HiViS ( 2026 )
L0
Target feat.
Chain
Exact
Static
Table 1: Taxonomy of representative multimodal speculative decoding methods across application domains, drafter designs, and speculative mechanisms.
Figure 1: Taxonomy and evolution of representative multimodal speculative decoding methods.
Application
Domain constraint
Draft profile
Acceptance semantics
Domain-specific cue
VLM
Grounding–cost trade-off
L0 / L1 / L2
Token exactness
Visual relevance
Video-LM
Long-context cost
L0
Token exactness
Temporal relevance
VLA
Closed-loop state
L0 / L2
Functional validity
Control phase
Speech/Audio
Temporal dependencies
L0 / L1
Sequence / perceptual
Prediction uncertainty
Visual Generation
Spatiotemporal coupling
L0 / L1 / L2 (∗)
Group / quality
Spatial locality
Table 2: Cross-domain synthesis of multimodal speculative decoding. L-level entries report representative candidate-generation mechanisms rather than exhaustive coverage within each domain.
Figure 2: Illustration of the L0–L2 taxonomy of drafter-side parallelism. L0 performs sequential drafting with one future position advanced per drafter forward; L1 predicts multiple predefined future positions in parallel from a shared prefix; and L2 treats the future block as a jointly generated state, enabling block-parallel drafting followed by parallel target verification.
Level
Drafting regime
Core definition
Representative methods
L0
Sequential drafting
One forward advances one draft depth; reaching depth K requires K causally dependent forwards
Small AR LM; EAGLE
L1
Multi-position parallel drafting
One forward predicts several predefined future positions in parallel from a shared prefix, but does not treat the future block as a jointly evolving generative state
Medusa; MTP; SpecFLASH
L2
Block-parallel generative drafting
The future block is the basic generative state; its positions are jointly generated or refined with bounded serial drafting depth
DFlash; DSpark
Table 3: A L0–L2 taxonomy of drafter-side parallelism. K denotes the number of future positions considered in one speculation round. The taxonomy describes how candidate trajectories are generated; candidate width, tree construction, and target-side verification are orthogonal.
Target
Method
GQA
F30K
TextVQA
CharXiv
MMMU
MT
Overall
MAT
Speedup
MAT
Speedup
MAT
Speedup
MAT
Speedup
MAT
Speedup
MAT
Speedup
MAT
Speedup
Q3VL-4B
AR
–
1.00 ×
–
1.00 ×
–
1.00 ×
–
1.00 ×
–
1.00 ×
–
1.00 ×
–
1.00 ×
EAGLE-3
1.95
0.71 ×
1.96
0.67 ×
1.98
0.71 ×
2.28
0.76 ×
2.41
0.81 ×
2.14
0.60 ×
2.12
0.71 ×
DFlash †
3.05
2.28 ×
2.84
2.36 ×
2.05
1.57 ×
2.63
2.06 ×
2.28
1.74 ×
2.13
1.70 ×
2.50
1.95 ×
Q3VL-8B
AR
–
1.00 ×
–
1.00 ×
–
1.00 ×
–
1.00 ×
–
1.00 ×
–
1.00 ×
–
1.00 ×
EAGLE-3
2.22
0.85 ×
2.20
0.85 ×
2.05
0.81 ×
2.66
0.99 ×
2.48
0.95 ×
2.17
0.85 ×
2.30
0.88 ×
Table 4: Speculative decoding performance with a maximum of 2,048 new tokens. MAT denotes the mean number of output tokens advanced per speculative step. Overall results are task-equal means. The best and second-best speculative results for each target are shown in bold and underlined, respectively. † Results measured on the HuggingFace Transformers backend; all other results use the SGLang backend. Each speedup is normalized to its own backend-matched autoregressive baseline. We abbreviate target models Q denotes Qwen.zheli
Figure 3: Stage-level latency decomposition of a representative decoding pass on Qwen3.6-27B. Vision conditioning includes vision encoding, multimodal prefill, and drafter-side target-feature-to-KV construction when applicable.
Resolution
Qwen3-VL-4B
Qwen3-VL-8B
Top1
Sampling
Top1
Sampling
MAT
Speedup
MAT
Speedup
MAT
Speedup
MAT
Speedup
HR-Bench 4K
2.90
1.03 ×
2.45
0.96 ×
3.22
1.06 ×
2.88
1.11 ×
HR-Bench 8K
2.86
0.90 ×
2.70
0.83 ×
3.09
0.99 ×
3.22
0.85 ×
Table 5: HR-Bench results under 4K and 8K resolutions on Qwen3-VL, with a 512-token generation cap on a common 30-sample manifest. MAT denotes the mean number of output tokens advanced per speculative step. Speedup is end-to-end latency speedup over the backend-matched autoregressive baseline. Top1 denotes top-1 acceptance; Sampling denotes speculate sampling.
Model
DFlash context setting
GQA
TextVQA
MMMU
Overall MAT ↑
Δ Overall vs. full
Qwen3-VL-8B
Full multimodal context
2.692
2.750
3.259
2.900
–
Text-only context
2.608
2.504
3.246
2.786
−0.114 ( −3.9% )
No-prefill context
2.845
1.953
2.946
2.581
−0.319 ( −11.0% )
Qwen3-VL-4B
Full multimodal context
2.587
2.305
2.583
2.491
–
Text-only context
2.487
2.221
2.453
2.387
−0.104 ( −4.2% )
No-prefill context
2.095
1.904
2.417
2.139
−0.353 ( −14.1% )
Table 6: Effect of multimodal target context on DFlash speculative acceptance. Results use top-1 verification with 100 examples per task and a maximum generation length of 2,048 tokens. Overall MAT is the equal-weight arithmetic mean over GQA, TextVQA, and MMMU. Text-only context masks visual-token K/V entries in the DFlash context, whereas no-prefill context masks the complete original prefill context. The target model always receives the full multimodal input.
Repository
Models
L2 Methods
Framework Support
DFlash
Qwen3/3.5/3.6, Gemma4
DFlash
MLX, SGLang, vLLM
DeepSpec
Qwen3, Gemma4
DFlash, DSpark
Train, Eval
AngelSpec
Qwen3, Hunyuan3
DFly, DFlash, DFlare, DSpark
Train, vLLM, SGLang
Speculators
Qwen3/3.6, Gemma4
DFlash, DSpark
Train, vLLM
SpecForge
Qwen3/3.6
DFlash, Domino, DSpark
Train, SGLang
Table 7: Representative framework support for L2 speculative decoding.
Department of Computer Science, Technion – Israel Institute of Technology. · Department of Electrical and Computer Engineering, Technion – Israel Institute of Technology.