Overview

Speculative decoding aims to reduce the time and computation needed to generate text by letting a drafter propose future tokens for a target model to check. Approaches include small auxiliary drafters, early-exit or multi-token prediction within the target model, and tree-based or block-parallel drafting; retrieval and task-specific signals can also supply candidates. Exact verification can preserve the target model’s output distribution, while relaxed acceptance rules trade some fidelity for speed. A central challenge is making gains in accepted tokens translate into real latency or throughput improvements across different workloads, hardware, and serving loads.

Latest papers

CardsList

343 papers58 papers in the last four weeks, down 6% on the four weeks before. 0.4% of all new papers.

  1. Long-Context Drafting Does Not Require Long-Context Drafters

    Oct 14, 2026Aref Jafari, Vansh Bhatia, Matt Qin +5Speculative DecodingLLM Inference Acceleration

  2. ReaDy: Recurrent States as Dynamic Context for Speculative Drafting in Hybrid Linear Attention Large Language Models

    Oct 14, 2026Bohan Sun, Zunhai Su, Xialie Zhuang +3LLM Inference AccelerationSpeculative Decoding

  3. FOLD: Position-Free Far Memory with Local Order for Long-Context Speculative Decoding

    Oct 14, 2026Shuxuan Li, Jialiang Kang, Yanghao Zhou +2Speculative DecodingLong-Context Language Model Inference

  4. TreeSpark: Learning to Draft the Trees You Verify for Speculative Decoding

    Oct 14, 2026Jihao Zhao, Qianyu Jiang, Shuaishuai Zu +6Speculative DecodingLLM Inference Acceleration

  5. JetSpec++: Foresight Parallel Drafting with Self Refinements

    Oct 14, 2026Zhaoxiang Feng, Lanxiang Hu, Haoran Yuan +6Parallel Speculative DecodingSpeculative Decoding

  6. SP2^2ec: Adaptive Self-Speculative Decoding for Vision-Language Models

    Oct 13, 2026Yuqi Huang, Xingyao Li, Yunlong Hou +3Speculative DecodingEfficient VLM Inference

  7. SpecFit: Not All Draft Tokens Cost the Same in Offloaded MoE Inference

    Oct 11, 2026Hongliang Cao, Xiuyu Li, Siqi Wang +3LLM Inference AccelerationSpeculative Decoding

  8. Prospective Forcing: Learning to Look Ahead for Autoregressive Long Video Diffusion

    Oct 10, 2026Yifan Shen, Yewei Xia, Gongxu Luo +6Autoregressive Video GenerationVideo Diffusion Models

  9. S3^3R: Fast and Faithful RL Rollouts via Self-Sparse Speculative Decoding

    Oct 9, 2026Yuhang He, Hongyu Ge, Zihan Lin +2Speculative DecodingLLM Inference Acceleration

  10. Contextual Correction of Parallel Draftsfor Speculative Decoding

    Oct 8, 2026Roy Eisenstadt, Harel Lidar, Tal Gonen +5Speculative DecodingLLM Inference Acceleration

  11. Where Draft Trees Lose Target Mass: Exit-Guided Speculative Decoding

    Oct 8, 2026Shijing Hu, Xuancheng Ren, Zhihui Lu +1Speculative DecodingLLM Inference Acceleration

  12. Speedbumps: Rejection Attacks on Speculative Decoding

    Oct 7, 2026Adam Y. J. Jones, Yu Yuan, Sergio MaffeisAdversarial AttacksSpeculative Decoding

  13. Training Parallel Speculative Draft Models by Directly Minimizing Expected Decoding Rounds

    Oct 7, 2026Yunxiao Zhao, Changxiao CaiSpeculative DecodingParallel Speculative Decoding

  14. Secure Speculative Decoding for Large Language Models

    Oct 6, 2026Yichi Zhang, Zhiqi Wang, Neil Gong +1Adversarial Attacks on LLMsSpeculative Decoding

  15. Nucleus Speculative Decoding: Plausibility-Aware Verification Beyond Exact Distribution

    Oct 6, 2026Shuhao Li, Fanghua Ye, Wanyu Lin +2Speculative DecodingLLM Inference Acceleration

  16. APEX: Speculate smarter, not deeper

    Oct 6, 2026Manvi Jha, Zach Zhang, Zhichao Xu +3Speculative DecodingLLM Inference Acceleration

  17. DLoop: Looped Speculative Decoding

    Oct 6, 2026Geonmo Gu, Byeongho Heo, HeeJae Jun +4Speculative DecodingLLM Inference Acceleration

  18. SchemaFill: Efficient LLM Tool Calling via Slot-Parallel Speculative Decoding

    Oct 5, 2026Zhi-Kai Chen, Song-Yan Li, De-Chuan Zhan +1Speculative DecodingLLM Inference Acceleration

  19. EdgeAgent: Orchestrating On-Device LLM inference for End-User Multi-Agent Systems on CPU-GPU Unified Memory Architectures

    Oct 2, 2026Yuhai Long, Yuanxin Wei, Kai Wu +3On-Device InferenceLLM Inference Scheduling

  20. BitNest: Bit-Nested Speculative Decoding for Memory-Efficient LLM Inference Acceleration

    Oct 2, 2026Chence Yang, Ningxi Cheng, Arash Akbari +9Speculative DecodingLLM Inference Acceleration

  21. DRelay: Global Draft Context for Prefix-Aware Parallel Speculative Decoding Repair

    Oct 1, 2026Zhuoyu Wang, Junnan Huang, Xinyu ChenParallel Speculative DecodingSpeculative Decoding

  22. UBTree: Parallel Tree Drafting via Unigram and Bigram Models for Speculative Decoding

    Sep 30, 2026Chumeng Liang, Linxuan Wang, Xinyu Peng +7Parallel Speculative DecodingSpeculative Decoding