Speculative decoding accelerates large language models by verifying drafted tokens in parallel. New draft-training objectives are usually credited with the acceptance they add over cross-entropy on externally written responses, yet many pipelines let the target rewrite them. We ask how much of that gain survives. With a vision-language target and a block drafter, we train three objectives on four versions of one corpus: cross-entropy on corpus tokens, token-level distillation from the target, and distillation gated to positions that would survive verification. Distillation and the gate add large gains on external responses, but after greedy regeneration, distillation adds a smaller gain and the gate little or none. The shrinkage persists across data orders, position weights, verifiers, decoding temperatures and, for distillation, initializations and learning rates. Most of it remains at matched supervised length and at each objective's best measured training length. A second vision corpus shows the shrinkage, as does a text-only replication, for distillation also with an autoregressive drafter. Objective gains should therefore be measured against a target-regenerated baseline and across training lengths. Our code is available at https://github.com/js-lee-AI/TargetRegeneration.
Figures & tables
Figure 1: One training example with external (a) and target-regenerated (b) responses. Below each corpus token is the target’s greedy token, and red marks disagreement. Cross-entropy (CE) trains the outlined slot on its corpus token, distillation (KD) on the target’s distribution.
Figure 2: Response treatments for one training example with image and prompt x . Sand and lavender blocks are external and target tokens, pale blocks carry no loss, the black bar closes the turn, and the dashed line marks the external length.
Table 1: Vision factorial and three ALLaVA controls after three epochs. Agreement is the share of slots whose corpus token is the target’s greedy token, and kept the share the gate keeps. Subscripts give standard deviations over runs, brackets 95% bootstrap intervals.