Overview

Speculative decoding aims to reduce the time and computation needed to generate text by letting a drafter propose future tokens for a target model to check. Approaches include small auxiliary drafters, early-exit or multi-token prediction within the target model, and tree-based or block-parallel drafting; retrieval and task-specific signals can also supply candidates. Exact verification can preserve the target model’s output distribution, while relaxed acceptance rules trade some fidelity for speed. A central challenge is making gains in accepted tokens translate into real latency or throughput improvements across different workloads, hardware, and serving loads.

Latest papers

CardsList

343 papers58 papers in the last four weeks, down 6% on the four weeks before. 0.4% of all new papers.

  1. MoE-Spec: Expert Budgeting for Efficient Speculative Decoding

    Feb 17, 2026Bradley McDanel, Steven Li, Sruthikesh Surineni +1Speculative DecodingLLM Inference Acceleration

  2. Sparrow: Text-Anchored Window Attention with Visual-Semantic Glimpsing for Speculative Decoding in Video LLMs

    Feb 17, 2026Libo Zhang, Zhaoning Zhang, Wangyang Hong +2Speculative DecodingLong-Context VLMs

  3. HSD: Training-Free Acceleration for Document Parsing Vision-Language Models with Hierarchical Speculative Decoding

    Feb 13, 2026Wenhui Liao, Hongliang Li, Pengyu Xie +15Efficient VLM InferenceSpeculative Decoding

  4. MoE-SpAc: Efficient MoE Inference Based on Speculative Activation Utility in Heterogeneous Edge Scenarios

    Feb 12, 2026Shuhuai Li, Jianghao Lin, Dongdong Ge +1Mixture-of-Experts InferenceEdge Inference

  5. Beyond Tokens: Semantic-Aware Speculative Decoding for Efficient Inference by Probing Internal States

    Feb 3, 2026Ximing Dong, Shaowei Wang, Dayi Lin +2Speculative DecodingLLM Inference Acceleration

  6. TriSpec: Ternary Speculative Decoding via Lightweight Proxy Verification

    Jan 30, 2026Haoyun Jiang, Junqi He, Feng Hong +9Speculative DecodingLLM Inference Acceleration

  7. JuDi: Revisiting Judge Decoding from First Principles via Training-Free Distributional Divergence

    Jan 8, 2026Shengyin Sun, Yiming Li, Renxi Liu +5Speculative DecodingLLM Inference Acceleration

  8. Entropy-Aware Token Rejection for Improving Speculative Decoding

    Dec 29, 2025Tiancheng Su, Meicong Zhang, Guoxiu HeSpeculative DecodingLLM Inference Acceleration

  9. Correctness Forensics for Batch Speculative Decoding: Diagnosing the Ragged Tensor Problem

    Oct 26, 2025Ranran Haoran Zhang, Soumik Dey, Ashirbad Mishra +3Speculative DecodingEfficient Language Model Inference

  10. Speculative Decoding Reimagined for Multimodal Large Language Models

    May 20, 2025Luxi Lin, Zhihang Lin, Zhanpeng Zeng +1Efficient Multimodal InferenceSpeculative Decoding

  11. Towards Optimal Multi-draft Speculative Decoding

    Feb 26, 2025Zhengmian Hu, Tong Zheng, Vignesh Viswanathan +5Speculative DecodingLLM Inference Acceleration

  12. Continuous Speculative Decoding for Autoregressive Image Generation

    Nov 18, 2024Zili Wang, Zheng Zhang, Kun Ding +3Visual Autoregressive ModelsSpeculative Decoding