Overview

Speculative decoding aims to reduce the time and computation needed to generate text by letting a drafter propose future tokens for a target model to check. Approaches include small auxiliary drafters, early-exit or multi-token prediction within the target model, and tree-based or block-parallel drafting; retrieval and task-specific signals can also supply candidates. Exact verification can preserve the target model’s output distribution, while relaxed acceptance rules trade some fidelity for speed. A central challenge is making gains in accepted tokens translate into real latency or throughput improvements across different workloads, hardware, and serving loads.

Latest papers

CardsList

343 papers58 papers in the last four weeks, down 6% on the four weeks before. 0.4% of all new papers.

  1. T2D: Target-to-Draft Model Pruning & Adaptation for Efficient Speculative Decoding

    Oct 25, 2026Shahriar Kabir Nahin, Xiao Ye, Muhao Chen +2Speculative DecodingLLM Inference Acceleration

  2. Distributed LLM Inference via Communication-Efficient Speculative Decoding

    Oct 25, 2026Joohyun Shin, Byeonghyeon Park, Keumseo Ryum +1Speculative DecodingLLM Inference Acceleration

  3. One Call Ahead: Packed Self-Speculative Decoding with Diffusion Drafting

    Oct 24, 2026Yuxiao Chen, Changxiao CaiSpeculative DecodingLLM Inference Acceleration

  4. NgramRL: Accelerating RL Post-Training for Structured Generation via Rollout-Aware N-gram Speculative Decoding

    Oct 23, 2026Nilesh Prasad Pandey, Jangseon Park, Jason Kong +2Speculative DecodingLLM Inference Acceleration

  5. SPECMEM: Accelerating Agentic Tool Calling via Live Memory Management

    Oct 22, 2026Zihe Song, Wenting Zhao, Ye Liu +4Tool-Using Language AgentsLLM Inference Efficiency

  6. Speculative Diffusion is a Tale of Two Tradeoffs

    Oct 22, 2026Aaron Zorel, Guy Van den Broeck, Aditya Grover +1LLM InferenceSpeculative Decoding

  7. Evaluating the Applicability of LLM Inference Optimizations on Apple Silicon

    Oct 22, 2026Kyrylo Yemets, Andrii Yaroshevych, Victor Muryn +5LLM InferenceOn-Device Language Model Inference

  8. ValuSpec: Plug-and-Play Candidate Valuation before Target Verification for Tree-Based Speculative Decoding

    Oct 22, 2026Liang He, SiYuan Ma, Qishi Zhan +5Speculative DecodingLLM Inference Acceleration

  9. Understanding Target Probability Reuse in Speculative Decoding

    Oct 21, 2026Jungseob Lee, Giyoon Jang, Chailin Nam +6Speculative DecodingParallel Speculative Decoding

  10. Target Regeneration Shrinks the Gains of Draft-Training Objectives in Speculative Decoding

    Oct 21, 2026Jungseob Lee, Hogun Seong, Euijun Choi +1Speculative DecodingLanguage Model Distillation

  11. Adaptive Multi-Token Prediction: How far should we go?

    Oct 20, 2026Emile Anand, Mohammed Rehan Parwani, Xinyuan CaoMulti-Token PredictionLLM Inference Acceleration

  12. BitMTP: When Multi-Token Prediction Meets Low-Bit Large Language Models

    Oct 20, 2026Ning Zhang, Shihao Wang, Jinrui Zhang +2LLM Inference AccelerationMulti-Token Prediction

  13. Beyond the Point Mass: Acceptance-Derived Weights and Distributional Losses for Speculative Decoding

    Oct 20, 2026Tianyu Wu, Zhuohan Wang, Jonathan Geuter +5Speculative DecodingLLM Inference Acceleration

  14. ProVe: Proposer-Verifier Decoding

    Oct 20, 2026Ziqi Jin, Weiwen Xu, Jiaxi Li +3LLM Inference AccelerationSpeculative Decoding

  15. Conjugate Compilation: Self-Improving Agentic Co-Compilation for Speculative Decoding

    Oct 18, 2026Annabelle Sujun Tang, Rohan Mahapatra, Christopher Priebe +4LLM Inference AccelerationSpeculative Decoding

  16. When Does Better Acceptance Pay? A Controlled Study of Speculative Decoding

    Oct 18, 2026Yongchao Long, Chunrui Zheng, Yuxuan Zhou +10Speculative DecodingLLM Inference Efficiency

  17. ECHO: Efficient Causal Self-Recursion Beyond Auxiliary Multi-Token Prediction

    Oct 17, 2026Yuxuan Zhou, Junyuan Shang, Tianyu Wang +5Multi-Token PredictionEfficient Language Model Inference

  18. FlashMap: Flow Map Speculative Decoding

    Oct 17, 2026Jeff Shen, Zhejian Zhou, Ryan Lee +2Speculative DecodingLLM Inference Acceleration

  19. DLite: History-free Speculative Decoding for Agentic AI

    Oct 16, 2026Jiaming Xu, Hanzhen Wang, Tianlang Zhao +7Speculative DecodingLLM Inference Acceleration

  20. Trajectory-Triggered Sparse Collaborative Decoding

    Oct 16, 2026Amit Ranjan Trivedi, Sruthikesh Surineni, Kyong Ho Lee +3Speculative DecodingLLM Reasoning

  21. Next Token or Vector? Learning How Far to Decode

    Oct 16, 2026Sai Sreenivas Chintha, Chidera Biringa, Rishab Sharma +6LLM Inference AccelerationMulti-Token Prediction

  22. Speculative Forcing for Efficient Autoregressive Video Generation

    Oct 15, 2026Hyunwook Choi, Seungryong Kim, Dongyoon Han +1Efficient Video GenerationAutoregressive Video Generation

  23. LoopAhead: Speculating Ahead for Faster Decoding in Looped Language Models

    Oct 15, 2026Zixuan Liu, Nan Xue, Rui Bao +4Speculative DecodingLLM Inference Acceleration

  24. HyCo: Hybrid Drafting for Cost-Efficient Mobile-Cloud Speculative Decoding

    Oct 15, 2026Hanbin Bae, Heejin Choi, Joohyung Lee +3Speculative DecodingLLM Inference Acceleration