Speculative decoding efficiency depends on the number of consecutively accepted draft tokens, yet token-level training objectives do not directly optimize this quantity. We introduce D-REAL (Drafter Reinforcement of Expected Acceptance Length), a post-training method that directly optimizes expected acceptance length for a fixed decoding context. D-REAL weights position-wise draft-target distributional overlap by prefix survival probabilities along a draft-sampled prefix, faithfully capturing the sequential dependence of verification on earlier proposals. By accounting for the parameter dependence of the proposal distribution, we derive an unbiased gradient estimator for this single-round objective under stochastic verification. We evaluate D-REAL on DSpark and DFlash2 with Qwen3-8B as the target model across ten benchmarks. Starting from baselines whose mean acceptance length has plateaued, one epoch of D-REAL post-training consistently improves the benchmark-averaged mean acceptance length by up to 0.23 tokens across both architectures under greedy (T = 0.0) and stochastic (T = 1.0) sampling, translating into up to a 9.4% end-to-end serving throughput improvement.
Figures & tables
Figure 1: Overview of draft-training objectives. Unlike (a) standard token-level losses and (b) D-PACE, which weight token likelihoods along fixed offline reference prefixes, (c) D-REAL directly optimizes expected acceptance length along online draft-sampled trajectories by combining exact local distributional overlap ( left inset ) with differentiable prefix survival factors ( right inset ) via stop-gradient proposal denominators.
Model
Method
MATH
CODE
Chat
Avg.
GSM8K
MATH-500
AIME25
HumanEval
MBPP
LCB
C-Eval
MT-Bench
Alpaca
Arena-Hard
T=0.0
Baseline
6.43
6.39
5.82
5.82
5.45
5.78
3.69
3.84
3.75
3.77
5.07
Baseline +1 epoch
6.44
6.41
5.80
5.87
5.47
5.79
3.68
3.85
3.78
3.77
5.09
D-PACE
6.47
6.41
5.82
5.88
5.47
5.80
3.69
3.85
3.78
3.77
5.09
DSpark
D-REAL
6.50
6.45
5.85
5.88
5.50
5.82
3.77
3.84
3.82
3.83
5.13
Table 1: Mean acceptance length ( τ ) across ten benchmarks, including one additional verifier-generated token per round. Bold indicates the best results. D-REAL (Ours) consistently outperforms the baseline and D-PACE, particularly under stochastic sampling ( T=1.0 ).
Model
Method
Output Tput ↑
Mean E2E Latency ↓
(tok/s)
(ms)
T=0.0
Baseline
17,634
458
DFlash2
D-REAL
19,288 ( +9.4% )
424 ( -7.4% )
Baseline
17,949
449
DSpark
D-REAL
18,972 ( +5.7% )
433 ( -3.6% )
Table 2: End-to-end serving performance of DFlash2 and DSpark on GSM8K in SGLang, measured on an 8-GPU Hopper node (concurrency =32 ). D-REAL translates higher acceptance lengths into substantial throughput gains and latency reductions without modifying the inference architecture.
Temp.
Post-Training Objective
GSM8K
MBPP
C-Eval
Avg.
T=0.0
Baseline (6 epochs)
6.43
5.45
3.69
5.19
+1 Ep. Baseline Loss
6.44
5.47
3.68
5.20
+1 Ep. TV + Static Decay
6.48
5.49
3.72
5.23
+1 Ep. D-REAL (Ours)
6.50
5.50
3.77
5.26
T=1.0
Baseline (6 epochs)
6.18
5.18
3.21
4.86
+1 Ep. Baseline Loss
6.20
5.22
3.24
4.89
Table 3: Ablation study on DSpark across representative benchmarks from each domain: GSM8K (Math), MBPP (Code), and C-Eval (Chat). Replacing the static decay with D-REAL ’s unbiased dynamic prefix weighting yields consistent gains across all domains and temperatures.
Figure 2: Per-task cumulative acceptance rate Pr(N≥t) across speculative positions t∈{1,…,7} for DFlash2 under (a) greedy ( T=0.0 ) and (b) stochastic ( T=1.0 ) decoding. Across both regimes and all ten benchmarks, D-REAL consistently slows down prefix survival decay compared to the baseline, yielding compounding gains with draft depth. Y-axis ranges vary by workload and are shared across temperatures; shading highlights the gap, and labels report the position-7 gain in percentage points.
Temp.
Method
1
2
3
4
5
6
7
Baseline
83.6
70.7
60.5
52.6
46.2
40.8
35.9
D-PACE
84.2
71.8
62.1
54.4
48.1
42.7
37.9
D-REAL
85.0
73.2
63.7
56.2
50.0
44.8
39.9
T=0.0
Δ vs. Base
+1.4pp
+2.5pp
+3.2pp
+3.6pp
+3.8pp
+4.0pp
+4.0pp
Baseline
82.1
68.6
58.1
49.8
43.1
37.5
32.6
D-PACE
80.8
67.0
56.3
48.3
41.7
36.3
31.7
Table 4: Macro-averaged cumulative acceptance rate Pr(N≥t) (%) across speculative positions t∈{1,…,7} for DFlash2 over all ten benchmarks ( τ≈1+∑t=17Pr(N≥t) , where the small systematic positive bias stems from early-EOS block truncation causing position- t rates to be computed conditional on the block reaching t ). While D-PACE drops below the baseline at T=1.0 , D-REAL achieves consistent, substantial gains across the entire draft depth.