Speculative decoding accelerates generation by letting a lightweight drafter propose tokens for parallel verification by a target model. Diffusion drafters such as DFlash propose an entire token block in a single forward pass, but mainly focus on text-only tasks, leaving multimodal drafting underexplored. To study this setting, we first introduce a metric of token-level visual dependence based on the KL divergence between next-token distributions with and without visual input. We find that acceptance rates decrease from 46% in the lowest visual-dependence quartile to 22% in the highest, with this disparity persisting after multimodal training. Motivated by this observation, we propose MMFlash, a simple and effective framework that adaptively upweights visually dependent tokens that the drafter predicts poorly. For long video inputs, MMFlash further employs sparse attention to reduce redundant attention computation and improve drafting quality. Across various multimodal benchmarks, MMFlash achieves lossless decoding speedups of up to 3.58×, outperforming EAGLE-3 and DFlash by up to 195.4% and 17.1%, respectively.
Figures & tables
Figure 1: Decoding speedup comparison with SOTA speculative decoding methods (EAGLE-3 & DFlash) on SGLang.
Figure 2: Overview of MMFlash. (a) The target VLM is run twice on the same prefix x<t , with and without the image I , and the visual dependence score dt is the KL divergence between the two next-token distributions. Each masked position of a block keeps its base weight bk and gains a visual bonus that depends on dt . (b) The MMFlash drafter adopts a simple sparse attention pattern for video inputs, shrinking a context of more than 40K tokens to about 1.6K.
Figure 3: Empirical studies on Qwen3.5-4B with three benchmarks. For each benchmark, we randomly select 200 samples and sample up to 128 anchor positions per sample. Top : acceptance rate of drafted tokens. Bottom : share of truncation points falling in each quartile. Visual dependence increases from Q1 to Q4.
Figure 4: Empirical studies on Qwen3.5-4B across video benchmarks (60 random samples per benchmark). (a) Per-token attention of the MMFlash drafter on one LongVideoBench question, relative to a uniform spread over the visible context; rows are decoding steps. (b) Cumulative share of draft attention within a given distance of the current position (solid), against the share of context tokens (dashed). (c) Acceptance length with the full draft context and with a window of the most recent 512 positions.
Model
Method
Text & Doc
General VQA
STEM
Avg.
ChartQA
CharXiv
MMStar
MMBench
DynaMath
MathVista
MathVerse
Temperature =0
τ
Speedup
τ
Speedup
τ
Speedup
τ
Speedup
τ
Speedup
τ
Speedup
τ
Speedup
τ
Speedup
Qwen3.5-4B
MTP (3)
3.18
1.69 ×
3.27
1.76 ×
2.95
1.61 ×
2.43
1.40 ×
3.27
1.69 ×
3.18
1.68 ×
3.17
1.69 ×
3.06
1.64 ×
MTP (7)
4.26
1.68 ×
4.64
1.87 ×
3.75
1.57 ×
2.58
1.24 ×
4.42
1.69 ×
4.29
1.67 ×
4.27
1.69 ×
4.03
1.63 ×
MTP (15)
4.67
1.26 ×
5.37
1.43 ×
3.99
1.14 ×
2.60
0.87 ×
4.76
1.23 ×
4.65
1.21 ×
4.61
1.22 ×
4.38
1.19 ×
EAGLE-3 (16)
4.31
1.74 ×
4.38
1.86 ×
3.49
1.49 ×
2.61
1.08 ×
4.29
1.67 ×
4.02
1.56 ×
3.95
1.68 ×
3.86
1.58 ×
Table 1: Decoding speedup over baseline and average acceptance length ( τ ) on Qwen3.5 models across various benchmarks. Parenthesized values indicate the number of draft steps for MTP, the draft tree size for EAGLE-3, and the diffusion block size for DFlash & MMFlash.
Method
LongVideoBench
MVBench
Video-MME
∼ 41.6 K tokens
∼ 42.3 K tokens
∼ 39.8 K tokens
τ
Speedup
τ
Speedup
τ
Speedup
Qwen3.5-4B
MTP (15)
3.99
1.31 ×
3.80
1.09 ×
3.50
1.00 ×
EAGLE-3 (4)
2.04
1.63 ×
1.91
1.45 ×
1.92
1.43 ×
DFlash (16)
3.14
2.11 ×
3.01
1.82 ×
2.81
1.75 ×
Table 2: Average acceptance length ( τ ) and decoding speedup over baseline on three video benchmarks at temperature =0 . The gray number under each benchmark gives its average context length.
Task
Method
Concurrency
Avg. τ
1
4
8
16
32
Qwen3.5-4B
CharXiv
Baseline
238
794
1306
2094
2733
–
MMFlash
826
2054
3102
3943
4395
6.23
3.46 ×
2.59 ×
2.37 ×
1.88 ×
1.61 ×
MMBench
Baseline
242
453
672
1141
1748
–
Table 3: Decode throughput (tok/s), speedup over the baseline, and average acceptance length ( τ ) on SGLang under increasing serving concurrency. For each MMFlash block the upper line reports throughput and the lower line the speedup at the same concurrency.
Model
Method
Text & Doc
General VQA
STEM
Avg.
ChartQA
CharXiv
MMStar
MMBench
DynaMath
MathVista
MathVerse
τ
Speedup
τ
Speedup
τ
Speedup
τ
Speedup
τ
Speedup
τ
Speedup
τ
Speedup
τ
Speedup
Qwen3.5-4B
MMFlash (8)
4.70
3.01 ×
4.77
3.13 ×
3.80
2.49 ×
4.52
2.89 ×
4.92
2.80 ×
4.68
2.74 ×
4.43
2.83 ×
4.55
2.84 ×
MMFlash (16)
5.97
3.28 ×
6.17
3.46 ×
4.49
2.55 ×
5.68
3.19 ×
6.26
2.87 ×
6.00
2.78 ×
5.35
2.88 ×
5.70
3.00 ×
Qwen3.5-9B
MMFlash (8)
5.07
3.22 ×
4.72
3.17 ×
4.01
2.65 ×
3.77
2.30 ×
5.18
2.94 ×
5.02
2.96 ×
4.68
2.95 ×
4.64
2.89 ×
MMFlash (16)
6.19
3.58 ×
5.67
3.42 ×
4.58
2.67 ×
4.20
2.39 ×
6.44
3.02 ×
6.49
3.17 ×
5.60
3.10 ×
5.60
3.05 ×
Table 4: Effect of the test-time block size on MMFlash. Parenthesized values indicate the test-time block size for MMFlash. All drafters are trained with block size 16.
Method
ChartQA
MMStar
DynaMath
τ
Speedup
τ
Speedup
τ
Speedup
Gemma4-E4B-it
MTP (5, official)
3.81
1.78 ×
3.19
1.48 ×
4.04
1.89 ×
MTP (15)
4.73
1.66 ×
3.74
1.33 ×
5.17
1.75 ×
EAGLE-3 (16)
4.87
2.27 ×
4.04
1.50 ×
6.23
1.90 ×
MMFlash (16)
6.39
2.87 ×
4.83
2.10 ×
7.15
3.02 ×
Table 5: Average acceptance length ( τ ) and decoding speedup on Gemma4-E4B-it and Qwen3.8-27B at temperature =0 .
Setting
CharXiv
MMBench
DynaMath
τ
Speedup
τ
Speedup
τ
Speedup
3-L
5.56
3.25 ×
4.69
2.86 ×
5.78
2.70 ×
5-L
6.17
3.46 ×
5.68
3.19 ×
6.26
2.87 ×
8-L
6.39
3.42 ×
6.21
3.21 ×
6.69
2.88 ×
Table 6: Effect of the number of drafter layers on MMFlash for Qwen3.5-4B at temperature =0 . n -L denotes a drafter with n layers; all drafters are trained with block size 16 and read hidden features from 5 layers of the target model.
Setting
CharXiv
MMBench
DynaMath
τ
Speedup
τ
Speedup
τ
Speedup
5-H
6.17
3.46 ×
5.68
3.19 ×
6.26
2.87 ×
8-H
6.18
3.47 ×
6.01
3.20 ×
6.37
2.88 ×
Table 7: Effect of the number of tapped target hidden states on MMFlash for Qwen3.5-4B at temperature =0 . n -H denotes a drafter that reads hidden features from n layers of the target model; all drafters have 5 layers and are trained with block size 16.
Figure 5: Block acceptance rate on Qwen3.5-4B with three benchmarks, under the same setup as Figure 3 . Each drafted block is grouped by the visual dependence quartile of its first predicted token. Visual dependence increases from Q1 to Q4.
Task
Method
Concurrency
Avg. τ
1
4
8
16
32
Qwen3.5-4B
CharXiv
Baseline
284
1022
1783
2690
3772
–
MMFlash
907
2555
3634
4590
5514
6.17
3.19 ×
2.50 ×
2.04 ×
1.71 ×
1.46 ×
MMBench
Baseline
274
479
705
1051
1768
–
Table 8: Decode throughput (tok/s), speedup over the baseline, and average acceptance length ( τ ) on vLLM under increasing serving concurrency, complementing Table 3 . For each MMFlash block the upper line reports throughput and the lower line the speedup at the same concurrency.
Setting
CharXiv
MMBench
DynaMath
τ
Speedup
τ
Speedup
τ
Speedup
3-L
4.91
2.59 ×
4.50
2.48 ×
5.34
2.43 ×
5-L
5.39
2.86 ×
5.25
2.58 ×
5.75
2.56 ×
8-L
5.59
2.83 ×
5.88
2.67 ×
5.94
2.54 ×
Table 9: Effect of the number of drafter layers on MMFlash for Qwen3.5-4B at temperature =1 (top- p=0.95 , top- k=20 ), complementing Table 6 . n -L denotes a drafter with n layers; all drafters are trained with block size 16 and read hidden features from 5 layers of the target model.
Setting
CharXiv
MMBench
DynaMath
τ
Speedup
τ
Speedup
τ
Speedup
5-H
5.39
2.86 ×
5.25
2.58 ×
5.75
2.56 ×
8-H
5.40
2.86 ×
5.40
2.59 ×
5.83
2.57 ×
Table 10: Effect of the number of tapped target hidden states on MMFlash for Qwen3.5-4B at temperature =1 , complementing Table 7 . n -H denotes a drafter that reads hidden features from n layers of the target model.
Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University, 361005, P.R. China · Shanghai Innovation Institute
National Key Laboratory for Novel Software Technology, Nanjing University · Data Science & Artificial Intelligence Research Institute, China Unicom · Unicom Data Intelligence, China Unicom