Speculative Decoding

Ask about this topic

Momentum

26 papers in the last four weeks, up 86% on the four weeks before. 0.3% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 170

All topics
CardsList

  1. SharpDraft: Accelerating Long-Context Speculative Decoding with Cardinality-Aware Query Scaling

    Oct 4, 2026Jeonghoon Park, Seongwoon Jo, Jongwon Lee +1Speculative DecodingDecode Speedup

    UNIST

  2. DRelay: Global Draft Context for Prefix-Aware Parallel Speculative Decoding Repair

    Oct 1, 2026Zhuoyu Wang, Junnan Huang, Xinyu ChenSpeculative DecodingRepair

    The Hong Kong University of Science and Technology (Guangzhou)

  3. AgSpec: Pushing the Limits of Retrieval-Based Speculative Decoding in Coding Agent Pipelines

    Oct 1, 2026Sumin Lee, Sukmin Cho, Suengjae Lim +1Coding AgentsSpeculative Decoding

    School of Computing, KAIST

  4. HAWK: Rethinking Multimodal Drafting for Speculative Decoding

    Sep 30, 2026Wenhan Yang, Anirudh Rao, Ashwin ChandraDraftsSpeculative Decoding

    University of California, Los Angeles · Samsung Research America

  5. Recovering Off-Policy Supervision for Speculative Decoding

    Sep 30, 2026Jungseob Lee, Chanjun Park, Sugyeong Eo +1Speculative DecodingDrafts

    Korea University · Soongsil University · Yonsei University Mirae Campus +1

  6. AS2^2D: Accelerating On-Demand Audio Understanding on Mobile Devices

    Sep 29, 2026Yunzhe Li, Kyoungjun Park, Hongzi Zhu +1Audio UnderstandingSpeculative Decoding

    The University of Texas at Austin · Shanghai Jiao Tong University

  7. DScale: Scaling Block-Diffusion Speculative Decoding with Adaptive Verification

    Sep 29, 2026Rongjian Chen, Minxian Xu, Zhengxin Fang +2Speculative DecodingDiffusion Language Models

    Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen 518052, China · University of Chinese Academy of Sciences, Beijing 100049, China · Victoria University of Wellington, Wellington, New Zealand +1

  8. LongSpark: Efficient speculative decoding with a fixed-cost parallel drafter

    Sep 29, 2026Hao-Yuan He, Peng-Fei Liu, Si Shen +1Speculative DecodingBlock-Diffusion Drafter

  9. Draft in Parallel, Condition Through Depth: Adjacent Causal Injection for Speculative Decoding

    Sep 28, 2026Haohui Zhang, Keyu Chen, Haocheng Sun +4Speculative DecodingDrafter

    Shanghai Jiao Tong University · Tencent YouTu Lab · Xiamen University

  10. Shallow Queries, Mature Values: Depth-Asynchronous Self-Speculation for Looped Transformers

    Sep 28, 2026Guanghao Li, Zihan Su, Hao Yu +6Speculative DecodingSelf-Speculative

    Tsinghua University · Southern University of Science and Technology · The University of Hong Kong +2

  11. Reciprocal Guidance: Orchestrating Draft and Verify Budgets for Advancing the Diffusion-AR Self-Speculation Frontier

    Sep 28, 2026Linye Wei, Shutian Zheng, Haoyu Zeng +1Block-Diffusion DrafterSpeculative Decoding

    Institute for Artificial Intelligence, Peking University · School of Integrated Circuits, Peking University · College of Engineering, Peking University +2

  12. Faster Block-Diffusion Serving with Distribution-Free Risk Guarantees

    Sep 27, 2026Jungseob Lee, Dongyub Jude Lee, Chanjun Park +2Decode SpeedupSpeculative Decoding

    Korea University · Zoom Communications · Soongsil University +1

  13. Tsubame: Tree Replay for Diffusion-Based Speculative Decoding

    Sep 27, 2026Yepeng Weng, Qiao Hu, Takehisa YairiBlock-Diffusion DrafterSpeculative Decoding

    The University of Tokyo · National Center for Mathematics and Interdisciplinary Sciences (NCMIS), AMSS, CAS

  14. When Parallel Drafter Meets Parallel Speculative Decoding

    Sep 23, 2026Fuliang Liu, Xue Li, Kun Qian +4Speculative DecodingDrafter

    State Key Laboratory of Novel Software Technology, Nanjing University · Alibaba Group

  15. Diffusion Drafts, AR Verifies: Accelerating Document OCR with Self-Speculative Decoding

    Sep 22, 2026Dohyun Kim, Sungjun Han, Hyungguk Kim +4Lingdt-Vl-OcrBlock-Diffusion Drafter

    Trillion Labs · Korea University · Seoul National University

  16. Adapting Tree-Structured Speculative Decoding to DeepSeek-V4 for Efficient Inference

    Sep 21, 2026Changxu Liu, Zhaogeng LiSpeculative DecodingLLM Inference Optimization

    Baige AI Team, Baidu Inc. · Fudan University

  17. H-Spec: Parallel Speculative Decoding Without a Drafter-Side KV Cache

    Sep 21, 2026Weifan Jiang, Krishna Teja Chitty-Venkata, Megan Flynn +6Speculative DecodingBlock-Diffusion Drafter

    Harvard University · Capital One · Red Hat +2

  18. Acceptance-Aware Draft Model Training for Speculative Decoding

    Sep 21, 2026Tianhua Xia, Mugilan Ganesan, Yifei Feng +3Speculative DecodingModel Training

    New York University · University of Waterloo · Independent Researcher

  19. To Copy or Not to Copy: Controlling Speculative Decoding via Intrinsic Model Signals

    Sep 17, 2026Roy Eisenstadt, Ido Cohen, Edo Cohen-Karlik +2Speculative DecodingRepetition

    Tel Aviv University · Stealth Startup, Tel Aviv

  20. Block Parallelism For Efficient Distributed Long-Context Diffusion Language Model Training

    Sep 16, 2026Tarun Suresh, Pranshu Chaturvedi, Hangoo Kang +4Diffusion Language ModelsEfficient Long-Context Inference

    Stanford University

  21. ASPIRE: Asynchronous Batched Self-Speculative Decoding for Long-Context LLM Inference

    Sep 16, 2026Amir Ziashahabi, Hossein Entezari Zarch, Lei Gao +2Speculative DecodingLLM Inference Optimization

    University of Southern California, Los Angeles, CA, USA

  22. ECHO: Early-layer Collaborative Hierarchical Orchestration with Bonus Logits in Speculative Decoding

    Sep 15, 2026Ziyang Ma, Zihong Zhang, Zuchao Li +4Speculative DecodingLLM Inference Optimization

    School of Computer Science, Wuhan University, Wuhan, China · School of Artificial Intelligence, Wuhan University, Wuhan, China · 3Xiaomi, Beijing, China

  23. LoopSpec: Pipelined Self-Speculative Decoding for Looped Transformers

    Sep 15, 2026SangLyul Cho, Langqing Cui, Sehoon Kim +2Speculative DecodingTransformer Architectures

    1Seoul National University · 2KAIST

  24. Evaluating Losslessness in Speculative Decoding Under Finite-Precision Inference

    Sep 14, 2026Ilya Koziev, Leonid Sinev, Ivan OseledetsSpeculative DecodingFull-Precision Performance

  25. PELM: Power Efficient On-Device LLM Inference with Speculative Decoding and Dynamic Voltage Frequency Scaling

    Sep 9, 2026Weisi Yang, Stephen XiaLLM Inference OptimizationResource-Constrained Edge Devices

    Northwestern University Evanston, Illinois, USA

  26. Osprey: Target-agnostic Pre-training Makes Stronger Drafters in Speculative Decoding

    Sep 8, 2026Fengxiang Bie, Yuqing Jian, Yifan Yu +7Speculative DecodingDrafts

    Together AI · The University of Sydney, Australia · University of Illinois Urbana-Champaign, USA +1

  27. Unlocking Lossless Speedups in LLMs via Discrete Diffusion

    Sep 3, 2026Subham Sekhar Sahoo, Lingjie Chen, Khiem Pham +14Diffusion Language ModelsSpeculative Decoding

    Institue of Foundation Models · University of Illinois Urbana-Champaign · Cornell Tech +2

  28. Jina-OCR-v1: Efficient Document Parsing with Speculative Decoding and Dense Verifiable Rewards

    Sep 2, 2026Alejandro Barón García, Feng Wang, Emilia Garcia Casademont +1Lingdt-Vl-OcrDocument Parsing

    Jina AI by Elastic · New Montgomery Street, San Francisco, CA 94105, USA

  29. Vision Is Not Overhead: One-Pass Block Drafting for Lossless Speculative Decoding in Vision-Language Models

    Aug 31, 2026Jungseob Lee, Seongtae Hong, Dongyub Jude Lee +4Recent Vision-Language ModelsSpeculative Decoding

    Korea University · Zoom Communications · Soongsil University +2

  30. Ceiling-Clipped Acceptance Histograms Indicate Stranded Speed-up in Block-Diffusion Speculative Decoding

    Aug 31, 2026Ephrem WuBlock-Diffusion DrafterDrafter

    Advanced Micro Devices, Inc., USA

  31. Strong Drafts Need Compact Memories: Long-Context Speculative Decoding with Compressed KV Cache

    Aug 31, 2026Tong Yuan, Chengxi Liao, Zeyi WenSpeculative DecodingMemory-Augmented Framework

    Data Science and Analytics Thrust, Information Hub, The Hong Kong University of Science and Technology (Guangzhou)

  32. Verification-Aware Training for Speculative Decoding

    Aug 31, 2026Geonmo Gu, Byeongho Heo, HeeJae Jun +4Speculative DecodingDrafts

    NAVER AI Lab · NAVER AI Search Platform · Korea University

  33. ReTrace: Rejected-Trajectory Conditioning for Speculative Decoding

    Aug 30, 2026Luxi Lin, Zhanpeng Zeng, Shuang Peng +2Speculative DecodingDecoding

    1Xiamen University, 2ByteDance

  34. LiLiCorr: Lightweight Likelihood Correlation of Parallel Drafts for Speculative Decoding

    Aug 20, 2026Matan Rusanovsky, Yoav Miron, Roy Uziel +5Speculative DecodingDrafter

    NVIDIA

  35. DARTree: Speculative Diffusion Decoding with Autoregressive Draft Trees

    Aug 13, 2026Tianyi Li, Yaxin Luo, Xinyi Shang +1Speculative DecodingDrafts

    VILA Lab, MBZUAI

  36. SPADE: Speculative Decoding for Precise and Low Cost Distributed Edge Cloud Inference

    Aug 13, 2026Divya Jyoti Bajpai, Kishan Kumar Upadhyay, Manjesh Kumar HanawalLLM Inference OptimizationSpeculative Decoding

    MLiONS, Department of IEOR, IIT Bombay Mumbai, Maharashtra-400076, India

  37. Decoupled Contrastive Decoding via Expert-Aligned Drafting

    Aug 13, 2026Zhixuan Liu, Zhichen Dong, Yuanfu Wang +1Contrastive DecodingDrafter

    Shanghai Jiao Tong University · Shanghai Artificial Intelligence Laboratory

  38. Alignment Drift in Single-Model Speculative Decoding for ASR: Mechanism, Correction, and Cost

    Aug 13, 2026Xinyu Wang, Huapeng Zhou, Ziyu Zhao +5Speculative DecodingAutomatic Speech Recognition

    1Boson AI · 2McGill University

  39. MemSpec: Memory-Aware Runtime for Adaptive Draft Scheduling in Speculative Decoding on Edge Devices

    Aug 11, 2026Eunjeong Kim, Yeong Jun Jeon, Myeonggyun HanSpeculative DecodingLLM Inference Optimization

    Kyungpook National University Daegu, Republic of Korea

  40. Matryoshka Language Model Suites

    Aug 10, 2026Nathan Godey, Yoav ArtziLanguage ModelingMatryoshka Representation Learning

    Cornell University

  41. WA-SpecDec: World-Aware Speculative Decoding for Vision-Language-Action Models

    Aug 9, 2026Zikang Wen, Yuning Zhang, Dong YuanSpeculative DecodingForward Propagation

    The University of Sydney

  42. LibraSpec: Dynamic Diffusion-Based Speculative Decoding via Marginal-Gain-Driven Optimization

    Aug 9, 2026Zexun Lin, Yuan Feng, Junlin Lv +2Speculative DecodingDrafts

    Suzhou Institute for Advanced Research, University of Science and Technology of China Suzhou, Jiangsu, China

  43. DBLAST: Dependent Block Drafting for Stochastic Speculative Decoding

    Aug 5, 2026Amirmohammad Karimi, Chao Gao, Negar HassanpourSpeculative DecodingBlock-Diffusion Drafter

    Huawei Technologies Canada Co., Ltd.

  44. EdgeXpert: An Edge Device for Memory-Efficient LLM Inference with Mixture-of-Experts and Speculative Decoding

    Aug 5, 2026Sangwoo Ha, Hyunwoo Seo, Yurim Jo +2LLM Inference OptimizationMixture-Of-Experts

    KAIST Daejeon, South Korea

  45. SpecRoll: Fast-Slow Verifier-Feedback Adaptation for Speculative Reinforcement Learning Rollouts

    Aug 5, 2026Nhat Minh Pham, Duy Tung Doan, Thi Duyen Ngo +2Autoregressive RolloutLarge Language Model Reinforcement Learning

    VNU University of Engineering and Technology, Vietnam National University, Hanoi, Vietnam · Viettel AI, Viettel Group, Hanoi, Vietnam

  46. Oilbird: Training-Free Speculative Decoding with Keys the Verifier Already Computes

    Aug 4, 2026Tao Jin, Phuong Minh Nguyen, Zhenzhu Yan +2Speculative DecodingDrafter

    1Japan Advanced Institute of Science and Technology (JAIST) · 2Tohoku University

  47. Approximate Speculative Decoding

    Aug 4, 2026Yuannuo Feng, Zegang Peng, Yuxin Xie +5Speculative DecodingTruncation

    School of Integrated Circuit Science and Engineering, Beihang University, Beijing, China · Department of Precision Instrument, Tsinghua University, Beijing, China · Faculty of Engineering, The University of Hong Kong, Hong Kong SAR, China +2

  48. AcceptMoE: Commitment-Weighted Self-Sizing Verifier Expert Sets for Efficient MoE Speculative Decoding

    Aug 4, 2026Shuang Liang, Hao Mark Chen, Zhiwen Mo +4Mixture-Of-Expert InferenceLocalized Lora-Moe

    Imperial College London · Tile-AI

  49. xPress: Parallel Refinement for Diffusion Drafters in Speculative Decoding

    Aug 3, 2026Zheng Wang, Davis Wertheimer, Yu Chin Fabian Lim +4Block-Diffusion DrafterDrafts

    University of Illinois Urbana-Champaign · IBM

  50. From Chains to Trees: Parent-Conditioned Drafting for Semi-Autoregressive Speculative Decoding

    Aug 3, 2026Zixian Li, Tong Li, Chi Xie +2Speculative DecodingDrafts

    OPPO AI Center

  51. Bole: Efficient Tree Speculation for Hybrid-Attention Language Models

    Aug 3, 2026Li Wang, Yi Su, Xiabao Wu +9Speculative DecodingSelf-Speculative

    State Key Laboratory for Novel Software Technology, Nanjing University · Ant Group · Shanghai Jiao Tong University +1

  52. CURE: Local Uncertainty Repair for Block-Parallel Speculative Decoding

    Aug 1, 2026Aofan Liu, Jingxiang Meng, Fangxin Liu +1Speculative DecodingRepair

  53. A Sparse Glimpse of the Whole: Train-Free Self-Speculative Decoding

    Jul 30, 2026Yuesong Liu, Yuan Zeng, Min Lyu +3Speculative DecodingSelf-Speculative

    University of Science and Technology of China

  54. Beyond KV Reconstruction: Functional Reconstruction for MLA Draft Models in Speculative Decoding

    Jul 29, 2026Weiye Shi, Fanxu Meng, Muhan ZhangSpeculative DecodingLLM Inference Optimization

    Institute for Artificial Intelligence, Peking University

  55. Revisiting Lossy Verification in Speculative Decoding: Mechanisms, Trade-offs, and Failure Modes

    Jul 29, 2026Tianyu Wang, Yuxuan Zhou, Wenbin Wang +3Speculative DecodingLLM Inference Optimization

    1Independent Researcher · 2Baidu Inc. · 3Zhejiang University

  56. AngelSpec: Towards Real-World High Performance Inference with Speculative Decoding

    Jul 28, 2026Hong Liu, Rui Cen, Junhan Shi +10Speculative DecodingBlock-Diffusion Drafter

    Tencent Inc.

  57. DraftExpert: Expansion-Aware Self-Speculative Decoding for End-Device MoE Inference

    Jul 27, 2026Dengke HanMixture-Of-Expert InferenceMixture-Of-Experts

  58. Adversarial Prompts for Acceptance Collapse in Speculative Decoding

    Jul 23, 2026Run Wang, Chaoyi Zhou, Xi Liu +7Speculative DecodingAdversarial Prompts

    Clemson University · Wayne State University · University of Washington

  59. Windowed-MTP: Removing the Full-Context Draft-KV Tax at Million-Token Context

    Jul 23, 2026Alagappan ValliappanSpeculative DecodingMulti-Token Prediction

    NVIDIA

  60. Leaky Language Models: Stealing Architecture and Inference Optimizations via Per-Token Timing

    Jul 22, 2026Sadegh Majidi, Niloofar Mireshghallah, Kazem TaramAttacker Large Language ModelLLM Inference Optimization

    Purdue University West Lafayette, IN, USA · Carnegie Mellon University Pittsburgh, PA, USA