cs.AINov 4, 2026

RECAP: Context-Aware Prediction Recycling with Request-Local and Persistent Memories for Multimodal Speculative Decoding

Authors: Hengjie Zhu, Dayan Wu, Xinze Liu, Peng Fu, Zheng Lin, Weiping Wang

Organizations: Institute of Information Engineering, Chinese Academy of Sciences · School of Cyber Security, University of Chinese Academy of Sciences

Abstract

Recycling a target model's own predictions offers a training-free source of proposals for speculative decoding. In vision-language models, however, existing recycling indexes predictions by the last token alone, conflating predictions computed under different images and prefixes; we find that such reuse fails most often at image-sensitive decoding states. Longer keys reduce this aliasing but match fewer states. We introduce \method, which stores target predictions under one-, two-, and three-token keys: contextual memories live only for the current request and are thus implicitly conditioned on its image, while token-level memory persists across requests to retain coverage. At each lookup, entropy-confidence weighting fuses the retrieved predictions according to their key specificity and concentration. Best-first search composes the fused transitions into a candidate tree, supplemented by suffix-matched continuations, and one target pass verifies the tree under greedy decoding. Without training or an auxiliary draft model, \method\ achieves mean speedups over autoregressive decoding of 1.66\speed1.66\speed and 1.56\speed1.56\speed across seven single-turn benchmarks on Qwen2.5-VL-7B and LLaVA-1.6-7B, exceeding the strongest training-free baseline, Token Recycling (1.51\speed1.51\speed and 1.37\speed1.37\speed). Our implementation is available in https://github.com/Senmo996/RECAP-official-code.

Figures & tables

Appendix figures & tables19 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. SP2^2ec: Adaptive Self-Speculative Decoding for Vision-Language Models

    Oct 13, 2026Yuqi Huang, Xingyao Li, Yunlong Hou +3Speculative DecodingEfficient VLM Inference

  2. RACER: Retrieval-Augmented Contextual Rapid Speculative Decoding

    Apr 16, 2026Zihong Zhang, Zuchao Li, Lefei Zhang +2Speculative DecodingLLM Inference Acceleration

  3. MMSpec: Benchmarking Speculative Decoding for Vision-Language Models

    Mar 16, 2026Hui Shen, Xin Wang, Ping Zhang +11Speculative DecodingEfficient VLM Inference