cs.LGNov 10, 2026

REAL Loss: Optimizing Real Expected Acceptance Length for Speculative Decoding

Authors: Yu Tian

Organizations: The University of Hong Kong

Abstract

Speculative decoding accelerates LLM inference by having a draft model propose multiple tokens for the target model to verify in one pass. The speedup is governed by the expected acceptance length (EAL), yet standard training objectives such as cross-entropy, KL divergence, and total variation focus only on distribution matching or single-step overlap. Recent studies instead propose block-level training objectives, aiming to optimize EAL directly. However, these objectives remain surrogates rather than exact formulations of EAL. For example, some average later acceptance over target-distributed continuations rather than the verification branches selected by accepted tokens, introducing a covariance gap that can reverse the gradient direction. We analyze this gap and derive REAL, a real EAL training loss that uses path weights for unbiased estimation and Rao-Blackwellization for variance reduction. In a controlled loss-level comparison with a Qwen3.6-35B-A3B target and a Domino drafter, REAL outperforms WTV, VSD, DBLast, and the static LK baseline on probabilistic serving acceptance length across 9 task subsets and 5 evaluation seeds. These gains require only a change to the training loss, with no modification to the drafter architecture or inference procedure, demonstrating the practicality of real EAL training. A reference implementation of the REAL loss is available at https://github.com/King-ty/speculators/tree/real-loss.

Figures & tables

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Acceptance-Aware Draft Model Training for Speculative Decoding

    Sep 21, 2026Tianhua Xia, Mugilan Ganesan, Yifei Feng +3Speculative DecodingLLM Inference Acceleration

  2. LK Losses: Direct Acceptance Rate Optimization for Speculative Decoding

    Feb 27, 2026Alexander Samarin, Sergei Krutikov, Anton Shevtsov +3Speculative DecodingLLM Inference Acceleration