Overview

Speculative decoding aims to reduce the time and computation needed to generate text by letting a drafter propose future tokens for a target model to check. Approaches include small auxiliary drafters, early-exit or multi-token prediction within the target model, and tree-based or block-parallel drafting; retrieval and task-specific signals can also supply candidates. Exact verification can preserve the target model’s output distribution, while relaxed acceptance rules trade some fidelity for speed. A central challenge is making gains in accepted tokens translate into real latency or throughput improvements across different workloads, hardware, and serving loads.

Latest papers

CardsList

343 papers58 papers in the last four weeks, down 6% on the four weeks before. 0.4% of all new papers.

  1. MMFlash: Faster Speculative Decoding for Vision-Language Models

    Nov 13, 2026Sicheng Feng, Zigeng Chen, Song Wang +2Efficient VLM InferenceSpeculative Decoding

  2. DForkTree: Accelerating Speculative Decoding with Fork-Aware Block Diffusion Draft Trees

    Nov 9, 2026Zhicheng Zheng, Cheng Zhang, Chenkai Tang +1Speculative DecodingLLM Inference Acceleration

  3. Multi-Future Drafting: Mixture Block Drafters for Tree-Verified Speculative Decoding

    Nov 9, 2026Haoyu Huang, Jialiang Kang, Linlin Yang +2Speculative DecodingLLM Inference Acceleration

  4. D3Tree: Learning to Prune Block-Diffusion Draft Trees for Efficient Verification

    Nov 8, 2026Hong Nie, Zhangchi Gao, Jun Yu +3Speculative DecodingLLM Inference Scheduling

  5. ChainFlow: Few-Step Latent Flow Matching for Speculative Decoding

    Nov 5, 2026Selim Aktas, Yusuf Erdem Erdem, Ahmed Zeer +3Speculative DecodingLLM Inference Acceleration

  6. Decode Before You Hear: Speculative Listening for Chunk-Interleaved Full-Duplex Speech Language Models

    Nov 4, 2026Yutong Bian, Yanglin Zhang, Yuxiang Wang +6Speech Language ModelsSpeculative Decoding

  7. D3D: Decoupled Diffusion Drafting for Asynchronous Speculative Decoding

    Nov 4, 2026Baoyou Chen, Ruiqiao Mei, Guancong Lin +4Speculative DecodingSpeculative Decoding for Diffusion Language Models

  8. RECAP: Context-Aware Prediction Recycling with Request-Local and Persistent Memories for Multimodal Speculative Decoding

    Nov 4, 2026Hengjie Zhu, Dayan Wu, Xinze Liu +3Efficient VLM InferenceSpeculative Decoding

  9. Give the Drafter a Second Chance to Guess: Parallel Reconsideration for Speculative Decoding

    Nov 3, 2026Hoang Anh Duy Le, Huibing Dong, Alireza Khadem +6Speculative DecodingLLM Inference Acceleration

  10. Adversarial Speculative Reasoning Improves Long Horizon Agents

    Nov 3, 2026Hongyin Luo, Wei Fang, Lijie Yang +5Speculative DecodingLong-Horizon LLM Reasoning

  11. HybridSpec: Throughput-Aware Speculative Decoding for Hybrid GDN-MoE Models

    Nov 2, 2026Haiyu Wang, Sai Qian Zhang, Zechun Liu +1Speculative DecodingLLM Inference Acceleration

  12. Testing Processor Contracts in Speculative Decoding Engines

    Nov 2, 2026Tianyu Ding, Yaocheng Tong, Muzhi Yang +2Speculative DecodingDifferential Testing

  13. SpecPilot: Autonomous Algorithm Discovery and Systems Optimization for Speculative Decoding

    Nov 2, 2026Kunjun Li, Wenhao Chai, Yihao Liang +7Speculative DecodingLLM Inference Acceleration

  14. FactorSpec: Serving Exact Relational Constraints as Speculative Drafts

    Nov 1, 2026Jianchang Su, Dawei Xiang, Wei ZhangLLM ServingSpeculative Decoding

  15. ReFrame: Silence-Aware Compute Reclamation for Full-Duplex Speech Models

    Nov 1, 2026Zheng Zhou, Guanyi Chen, Jing Wei +3Full-Duplex Spoken Dialogue SystemsSpeech Language Models

  16. Seeing What Matters, Verifying What Pays: Load-Adaptive Speculative Decoding for Vision-Language Models

    Nov 1, 2026Hong Nie, Zhangchi Gao, Yuqian Liang +4Efficient VLM InferenceSpeculative Decoding

  17. TraceSpec: Understanding and Improving Speculative Decoding for LLM Agents

    Oct 30, 2026Jihao Zhao, Shuaishuai Zu, Pengfei Li +4Speculative DecodingLLM Inference Efficiency

  18. Only Accepted Tokens Count: An Energy Accounting of Speculative Decoding Under Latency Targets

    Oct 29, 2026Bole Ma, Ayesha Afzal, Jan Eitzinger +2Speculative DecodingEnergy-Aware LLM Inference