cs.CVMay 20, 2025

Speculative Decoding Reimagined for Multimodal Large Language Models

Authors: Luxi Lin, Zhihang Lin, Zhanpeng Zeng, Rongrong Ji

Organizations: Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University, 361005, P.R. China · Shanghai Innovation Institute

Abstract

Multimodal Large Language Models (MLLMs) deliver strong performance across vision-language tasks but suffer from high autoregressive decoding latency. Although speculative decoding can accelerate Large Language Models (LLMs) without changing their output distribution, directly applying existing approaches to MLLMs yields limited speedups because text and visual tokens exhibit distinct characteristics. We present Multimodal Speculative Decoding (MSD), a framework specifically designed for efficient MLLM inference. MSD is motivated by two key observations: text and visual tokens should be handled separately during drafting, and an effective draft model requires both strong language modeling and visual perception capabilities. Accordingly, MSD decouples text and visual token processing in the draft model and applies token-specific strategies to improve drafting efficiency. We further introduce a two-stage training strategy. The draft model is first trained on text-only instruction-tuning data to strengthen its language modeling ability and is then progressively exposed to multimodal data to develop visual perception while preserving its language competence. Extensive experiments across multimodal benchmarks demonstrate that MSD achieves up to 2.29×2.29\times inference speedup for LLaVA-1.5-7B and 2.46×2.46\times for LLaVA-1.5-13B without sacrificing generation quality. These results establish MSD as an effective and general approach for accelerating multimodal autoregressive generation. Our code is available at https://github.com/Lyn-Lucy/MSD

Figures & tables

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. MMFlash: Faster Speculative Decoding for Vision-Language Models

    Nov 13, 2026Sicheng Feng, Zigeng Chen, Song Wang +2Efficient VLM InferenceSpeculative Decoding

  2. MMSpec: Benchmarking Speculative Decoding for Vision-Language Models

    Mar 16, 2026Hui Shen, Xin Wang, Ping Zhang +11Speculative DecodingEfficient VLM Inference