Speculative decoding accelerates LLM inference by having a draft model propose multiple tokens for the target model to verify in one pass. The speedup is governed by the expected acceptance length (EAL), yet standard training objectives such as cross-entropy, KL divergence, and total variation focus only on distribution matching or single-step overlap. Recent studies instead propose block-level training objectives, aiming to optimize EAL directly. However, these objectives remain surrogates rather than exact formulations of EAL. For example, some average later acceptance over target-distributed continuations rather than the verification branches selected by accepted tokens, introducing a covariance gap that can reverse the gradient direction. We analyze this gap and derive REAL, a real EAL training loss that uses path weights for unbiased estimation and Rao-Blackwellization for variance reduction. In a controlled loss-level comparison with a Qwen3.6-35B-A3B target and a Domino drafter, REAL outperforms WTV, VSD, DBLast, and the static LK baseline on probabilistic serving acceptance length across 9 task subsets and 5 evaluation seeds. These gains require only a change to the training loss, with no modification to the drafter architecture or inference procedure, demonstrating the practicality of real EAL training. A reference implementation of the REAL loss is available at https://github.com/King-ty/speculators/tree/real-loss.
Figures & tables
Figure 1: Two-step illustration of branch dependence. The surrogate averages before multiplying, while the exact objective multiplies before averaging.
Figure 2: The two-step counterexample. (a) The exact objective weights each branch by its propose-and-accept mass, while the WTV surrogate redistributes that mass by target proportions. (b) As a function of the shared draft parameter θ , the two objectives have opposite gradients, coinciding in value at θ=0.5 .
Method
Avg
HumanEval
GSM8K
NQ
RAG
CNN/DM
BFCL
WMT
MATH-500
MBPP
static LK
4.740
5.227
5.899
3.643
4.449
4.171
3.968
4.542
5.752
5.011
WTV
4.715
5.208
5.891
3.610
4.426
4.144
3.923
4.522
5.744
4.967
VSD
4.657
5.122
5.963
3.514
4.356
4.017
3.956
4.383
5.716
4.887
DBLast
4.749
5.222
5.998
3.615
4.437
4.155
4.022
4.511
5.789
4.992
REAL w/o RB
4.791
5.264
6.030
3.649
4.498
4.196
4.023
4.574
5.846
5.041
REAL
4.792
5.270
6.021
3.645
4.499
4.218
4.007
4.590
5.831
5.047
Table 1: Probabilistic acceptance length after 3 training epochs, averaged over 5 seeds. Avg is the equal-weight mean over 9 task subsets. Bold values indicate the top two results in the Avg column and the best result in each subset column.
k
static LK
WTV
VSD
DBLast
REAL
REAL w/o RB
1
0.847
0.844
0.847
0.844
0.847
0.847
2
0.704
0.699
0.697
0.702
0.706
0.705
3
0.588
0.584
0.575
0.589
0.594
0.593
4
0.496
0.493
0.480
0.500
0.505
0.504
5
0.421
0.419
0.403
0.425
0.432
0.432
6
0.360
0.357
0.340
0.362
0.370
0.371
Table 2: Acceptance retention Sk at the three-epoch checkpoint, defined as the probability of accepting at least k draft tokens in a speculative round. Bold values mark the highest unrounded retention at each position.
Figure 3: Conditional acceptance ck at the three-epoch checkpoint, defined as the probability of accepting the k th draft token given the first k−1 tokens are accepted.
Figure 4: Training EAL proxy and serving AL during training for WTV, REAL , and REAL w/o RB . REAL leads on probabilistic serving AL, while WTV leads on the training EAL proxy.
Figure 5: The branch EAL estimate J\textscBranch and the WTV surrogate during training for WTV, REAL , and REAL w/o RB . The WTV surrogate stays below the branch estimate in all three cases.
Figure 6: Total covariance summed over all 6 adjacent pairs during training for WTV, REAL , and REAL w/o RB . REAL and REAL w/o RB track each other closely and stay above WTV throughout training.
epoch
k=1
k=2
k=3
k=4
k=5
k=6
k=7
agg
checkpoints of REAL
ep1
0
0.17
0.44
0.45
0.61
0.56
0.73
0.47
ep2
0
0.17
0.38
0.49
0.59
0.65
0.66
0.48
ep3
0
0.20
0.40
0.48
0.47
0.55
0.67
0.43
checkpoints of REAL w/o RB
ep1
0
0.24
0.44
0.45
0.42
0.58
0.68
0.47
Table 3: Var\textscBranch/Var\textscPath for per-term gradient variance at each position and for aggregate gradient variance. Lower values indicate greater variance reduction. The zero at k=1 is structural: given the fixed prefix, a1 is deterministic while b1 is sampled.
Figure 7: Training EAL proxy trajectories of REAL and REAL w/o RB with and without the auxiliary CE term, and of pure CE.
greedy
probabilistic
REAL w/o CE&RB
4.369
4.588
REAL w/o CE
4.397
4.624
pure CE
4.508
4.668
REAL w/o RB
4.515
4.791
REAL
4.537
4.792
Table 4: Ablation of the auxiliary CE term at the three-epoch budget. Removing CE from REAL or REAL w/o RB , or using pure CE without the EAL term, degrades serving AL under both protocols.
method
T=1.0
T=0.7
T=0.5
static LK
4.740
4.925
5.002
WTV
4.715
4.867
4.981
VSD
4.657
4.842
4.928
DBLast
4.749
4.912
4.975
REAL w/o RB
4.791
4.919
4.964
REAL
4.792
4.935
4.993
Table 5: Probabilistic acceptance length across serving temperatures at the three-epoch budget, averaged over the 5 seeds and 9 subsets. Bold values mark the best method per temperature.
Figure 8: Probabilistic acceptance length at the end of each epoch, including a continuation to epoch 4, averaged over 9 subsets and 5 seeds. REAL attains the highest AL at every epoch.
Appendix figures & tables14 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 9: Per-pair covariance trajectories during training for WTV, REAL , and REAL w/o RB . Each panel shows one adjacent pair (bk,ak+1) , k=1,…,6 .
component
setting
target model
frozen Qwen3.6-35B-A3B (MoE)
drafter
Domino block-parallel drafter, 5 layers on target layers 2/20/37
Open-PerfectBlend continuations regenerated by the target at T=1.0
optimizer
Muon for hidden-layer weight matrices, LR 2×10−4
Appendix
Table 6: Shared training and evaluation setup for the main study.
method
loss configuration
static LK
{TV:0.9,CE:0.1} , the static LK objective used in the DSpark recipe
WTV
{WTV:0.9,CE:0.1}
VSD
{VSD:0.9,CE:0.1} , weighted-likelihood M-step with detached path-acceptance weights, self-normalized across valid blocks and divided by γ
DBLast
{DBLast:0.9,CE:0.1} , acceptance threshold τ=0.1
REAL w/o RB
{J\textscPath:0.9,CE:0.1}
REAL
{J\textscBranch:0.9,CE:0.1}
Appendix
Table 7: Loss configurations of the final experiments. Each J entry denotes the corresponding loss 1−J/γ .
method
global steps
median
IQR
WTV
1,000–10,999
847.4
[843.0, 855.4]
50,000–59,999
850.2
[845.2, 859.7]
100,000–109,999
849.7
[844.7, 859.1]
VSD
1,000–10,999
834.7
[830.0, 845.1]
50,000–59,999
840.8
[834.5, 854.2]
100,000–109,999
845.9
[838.5, 862.6]
Appendix
Table 8: Recorded training step time in milliseconds over three 10,000-step windows. The bracketed interquartile range (IQR) [Q1,Q3] spans the middle 50% of step times.
method
0
777
13
1234
999983
mean
static LK
4.739
4.735
4.739
4.746
4.743
4.740
WTV
4.714
4.726
4.706
4.715
4.714
4.715
VSD
4.652
4.669
4.649
4.659
4.657
4.657
DBLast
4.757
4.748
4.745
4.744
4.751
4.749
REAL w/o RB
4.787
4.787
4.786
4.799
4.796
4.791
REAL
4.804
4.788
4.787
4.797
4.783
4.792
Appendix
Table 9: Per-seed and seed-averaged probabilistic AL after three training epochs. All methods are combined with the auxiliary CE term. Bold marks the best value in each column.
method
Throughput (tokens/s)
static LK
4622.1
WTV
4553.0
VSD
4535.8
DBLast
4648.6
REAL w/o RB
4664.8
REAL
4662.0
Appendix
Table 10: End-to-end vLLM serving throughput at the three-epoch budget, averaged over 5 seeds and 9 subsets. Bold values indicate the top two results.
Method
Avg
HumanEval
GSM8K
NQ
RAG
CNN/DM
BFCL
WMT
MATH-500
MBPP
static LK
4.544
5.000
5.555
3.533
4.349
4.123
3.665
4.460
5.425
4.783
WTV
4.530
5.021
5.617
3.518
4.300
4.092
3.612
4.440
5.411
4.757
VSD
4.497
4.988
5.587
3.462
4.283
3.988
3.636
4.373
5.421
4.736
DBLast
4.513
5.000
5.600
3.471
4.266
4.039
3.618
4.428
5.469
4.731
REAL w/o RB
4.515
5.002
5.582
3.482
4.292
4.062
3.611
4.396
5.453
4.751
REAL
4.537
5.039
5.638
3.504
4.284
4.066
3.651
4.420
5.465
4.766
Appendix
Table 11: Greedy acceptance length after 3 training epochs, averaged over 3 repetitions with the default seed. Avg is the equal-weight mean over 9 task subsets. Bold values indicate the top two results in the Avg column and the best result in each subset column.
epoch 1
epoch 2
epoch 3
epoch 4
method
greedy
prob.
greedy
prob.
greedy
prob.
greedy
prob.
static LK
4.319
4.458
4.473
4.631
4.544
4.740
4.570
4.768
WTV
4.302
4.438
4.428
4.603
4.530
4.715
4.548
4.737
VSD
4.239
4.313
4.378
4.511
4.497
4.657
4.511
4.683
DBLast
4.270
4.421
4.403
4.610
4.513
4.749
4.545
4.777
REAL w/o RB
4.260
4.456
4.413
4.658
4.515
4.791
4.533
4.814
Appendix
Table 12: Acceptance length under greedy and probabilistic serving at the end of each epoch, including a continuation to epoch 4. Means over 9 subsets, using 5 seeds for probabilistic serving and the default seed for greedy serving.
greedy
probabilistic
D-PACE
4.535
4.760
REAL w/o RB
4.515
4.791
REAL
4.537
4.792
Appendix
Table 13: Acceptance length after the full 3-epoch budget with D-PACE included. Means are equal-weight averages over 9 subsets, using 5 seeds for probabilistic serving and the default seed for greedy serving. Bold values mark the best result for each protocol.
greedy
probabilistic
method
epoch 3
epoch 4
epoch 3
epoch 4
D-PACE
4.535
4.555
4.760
4.784
REAL w/o RB -CE
4.515
4.533
4.791
4.814
REAL w/o RB -D-PACE
4.516
4.528
4.797
4.838
REAL -CE
4.537
4.556
4.792
4.816
REAL -D-PACE
4.538
4.554
4.796
4.823
Appendix
Table 14: Combination of REAL and REAL w/o RB with D-PACE position weighting at the three- and four-epoch budgets. The suffixes -CE and -D-PACE denote standard CE and D-PACE-weighted CE, respectively. Bold values mark the best result in each column.
Objective
Acceptance length (T=1)
Base draft
2.5833
CE (KL)
3.1354
TV ( γ=1 )
2.3750
LK λ
3.6406
WTV
3.9375
J\textscBranch
3.5312
Appendix
Table 15: Pilot study: verifier acceptance length at T=1 on 64 held-out GSM8K prompts.
Metric
WTV
REAL
REAL-V
REAL-K
Validation J\textscBranch
4.3570
4.3795
4.3790
4.3669
Within var. (greedy)
6.499
6.554
6.548
6.526
Within var. (prob.)
6.252
6.340
6.343
6.312
Between var. (greedy)
0.804
0.825
0.830
0.818
Between var. (prob.)
0.878
0.890
0.890
0.839
Max streak (greedy)
50
26
31
23
Appendix
Table 16: Outcomes after one additional training epoch with hot continuation, with WTV and REAL as standard three-epoch references. All methods include a CE term.
Figure 10: Distributions of accepted draft-token counts per decoding round for REAL , REAL-V , and REAL-K . All three methods show similar distributions under both serving protocols.
K
REAL
REAL-K
1
0.81
0.82
2
1.46
1.47
3
1.99
2.01
4
2.43
2.45
5
2.80
2.82
6
3.11
3.13
Appendix
Table 17: Probabilistic JK after hot continuation, evaluated at each cutoff K=1,…,7 . REAL-K is trained with K=5 . All methods include a CE term.
Speculative decoding accelerates large language model (LLM) inference by using a lightweight draft model to generate multiple candidate tokens that are verified by the target model in a single forward pass. Its speedup is largely determined by the acceptance length, yet existing draft-model training methods mainly optimize cross-entropy or Kullback-Leibler (KL) divergence as proxies. These objectives encourage distribution matching but do not directly optimize acceptance length, and the acceptance mechanism also differs between greedy and sampling-based decoding. In this work, we propose acceptance-length-aware training losses that directly optimize the expected number of accepted tokens within a speculative window. For greedy verification, we derive an expected accepted length (EAL) loss that explicitly maximizes expected acceptance length. For sampling-based decoding, we introduce a window total variation (WTV) loss that optimizes the overlap between temperature-scaled draft and target distributions while accounting for sequential acceptance dependencies. Both objectives can be further combined with a group-relative reinforcement learning stage (GRPO) using simulated acceptance length as the reward. Experiments across different target and draft models, tasks, and decoding settings show that our losses consistently improve acceptance length over KL-based training. WTV provides particularly strong gains under sampling-based decoding, while EAL better matches greedy verification. These results show that directly optimizing the acceptance objective, with losses tailored to the decoding mode, is more effective than conventional distribution-matching objectives.
Tianhua Xia, Mugilan Ganesan, Yifei Feng +3
New York University · University of Waterloo · Independent Researcher
Speculative decoding efficiency depends on the number of consecutively accepted draft tokens, yet token-level training objectives do not directly optimize this quantity. We introduce D-REAL (Drafter Reinforcement of Expected Acceptance Length), a post-training method that directly optimizes expected acceptance length for a fixed decoding context. D-REAL weights position-wise draft-target distributional overlap by prefix survival probabilities along a draft-sampled prefix, faithfully capturing the sequential dependence of verification on earlier proposals. By accounting for the parameter dependence of the proposal distribution, we derive an unbiased gradient estimator for this single-round objective under stochastic verification. We evaluate D-REAL on DSpark and DFlash2 with Qwen3-8B as the target model across ten benchmarks. Starting from baselines whose mean acceptance length has plateaued, one epoch of D-REAL post-training consistently improves the benchmark-averaged mean acceptance length by up to 0.23 tokens across both architectures under greedy (T = 0.0) and stochastic (T = 1.0) sampling, translating into up to a 9.4% end-to-end serving throughput improvement.
Speculative decoding accelerates autoregressive large language model (LLM) inference by using a lightweight draft model to propose candidate tokens that are then verified in parallel by the target model. The speedup is significantly determined by the acceptance rate, yet standard training minimizes Kullback-Leibler (KL) divergence as a proxy objective. While KL divergence and acceptance rate share the same global optimum, small draft models, having limited capacity, typically converge to suboptimal solutions where minimizing KL does not guarantee maximizing acceptance rate. To address this issue, we propose LK losses, special training objectives that directly target acceptance rate. Comprehensive experiments across four draft architectures and six target models, ranging from 8B to 685B parameters, demonstrate consistent improvements in acceptance metrics across all configurations compared to the standard KL-based training. We evaluate our approach on general, coding and math domains and report gains of up to 8-10% in average acceptance length. LK losses are easy to implement, introduce no computational overhead and can be directly integrated into any existing speculator training framework, making them a compelling alternative to the existing draft training objectives.
Alexander Samarin, Sergei Krutikov, Anton Shevtsov +3