cs.CVNov 13, 2026

MMFlash: Faster Speculative Decoding for Vision-Language Models

Authors: Sicheng Feng, Zigeng Chen, Song Wang, Gongfan Fang, Xinchao Wang

Organizations: National University of Singapore

Abstract

Speculative decoding accelerates generation by letting a lightweight drafter propose tokens for parallel verification by a target model. Diffusion drafters such as DFlash propose an entire token block in a single forward pass, but mainly focus on text-only tasks, leaving multimodal drafting underexplored. To study this setting, we first introduce a metric of token-level visual dependence based on the KL divergence between next-token distributions with and without visual input. We find that acceptance rates decrease from 46% in the lowest visual-dependence quartile to 22% in the highest, with this disparity persisting after multimodal training. Motivated by this observation, we propose MMFlash, a simple and effective framework that adaptively upweights visually dependent tokens that the drafter predicts poorly. For long video inputs, MMFlash further employs sparse attention to reduce redundant attention computation and improve drafting quality. Across various multimodal benchmarks, MMFlash achieves lossless decoding speedups of up to 3.58×3.58\times, outperforming EAGLE-3 and DFlash by up to 195.4% and 17.1%, respectively.

Figures & tables

Explore similar work

CardsList
  1. Speculative Decoding Reimagined for Multimodal Large Language Models

    May 20, 2025Luxi Lin, Zhihang Lin, Zhanpeng Zeng +1Efficient Multimodal InferenceSpeculative Decoding

  2. Is Multimodal Speculative Decoding Ready for Diffusion-Based Parallel Drafting? A Survey and Empirical Diagnosis

    Aug 21, 2026Yantao Li, Huanlin Gao, Fang Zhao +12Speculative DecodingSpeculative Decoding for Diffusion Language Models

  3. HAWK: Rethinking Multimodal Drafting for Speculative Decoding

    Sep 30, 2026Wenhan Yang, Anirudh Rao, Ashwin ChandraEfficient Multimodal InferenceVLM Distillation