ReAnchor: Reusing Rejected Verifier States for Diffusion Drafting

Sanghyun Lee; Wonjin Nam; Seungho Jang; Jaeho Lee; Jiho Park; Jaewoon Lee; Seungryong Kim

cs.CLOct 19, 2026

ReAnchor: Reusing Rejected Verifier States for Diffusion Drafting

Authors: Sanghyun Lee, Wonjin Nam, Seungho Jang, Jaeho Lee, Jiho Park, Jaewoon Lee, Seungryong Kim

Organizations: KAIST · Seoul National University

Abstract

Block diffusion drafters accelerate speculative decoding by generating multiple draft tokens in a single forward pass. However, each pass produces only a fixed-length block from a single target-generated anchor, limiting how far the draft can extend. Meanwhile, target verification computes hidden states even at positions beyond the first rejection, but these states are typically discarded rather than reused. We propose ReAnchor, a training-free method that reanchors a pretrained drafter at shifted positions using discarded verifier states to generate parallel draft blocks. These blocks improve predictions within the original block and extend coverage beyond its endpoint. At sampling temperature of T=0, ReAnchor achieves its largest average gains on Gemma 4-12B, improving acceptance length by 19.4% and decoding speed by 17.9% across eight benchmarks. Averaged equally over four target models and eight benchmarks, it improves acceptance length by 15.2% and decoding speed by 13.4% with pretrained DFlash drafters.

Figures & tables

Figure 1: Comparison of Drafting Strategies. While (a) autoregressive drafting requires multiple sequential next-token predictions and (b) block diffusion drafting generates a single draft block in one forward pass, (c) ReAnchor generates multiple draft blocks through parallel, low-cost drafter forward passes, reusing verifier hidden states discarded after rejection in the previous round to draft from later positions.

Figure 2: Distribution of per-round acceptance length on (a) MATH-500 and (b) HumanEval. The spike at position 16 indicates frequent saturation at the draft-block boundary.

	GSM8K	HumanEval	MBPP	MATH-500
Agreement (%)	65.4	60.2	60.6	66.0

Table 1: Post-rejection agreement (%). Fraction of draft tokens after the first rejection that match the target model’s top-1 predictions.

Figure 3: Overview of ReAnchor’s Draft Construction. (top) ReAnchor is a training-free speculative decoding method using pretrained block-diffusion drafter $q$ . At each decoding round, ReAnchor generates draft and target model $p$ verifies it repeatedly. (bottom) Rejected draft tokens of previous round from target model are typically discarded at prior methods. However, ReAnchor leverages them by directly conditioning them into pretrained draft model $q$ . Specifically, ReAnchor generates $(B-1)(1+|\mathcal{D}|)$ draft tokens with parallel inference of draft model $q$ . Each group of $B-1$ draft tokens is generated from a different anchor position. We can construct the final proposal structure by matching and attaching generated draft tokens.

Temperature = 0
		Dataset
Target model	Drafting method	GSM8K	MATH	AIME25	HumanEval	MBPP	LiveCode Bench	MT-Bench	Alpaca
Qwen3-4B	DFlash	6.24 / 4.19 $\times$	7.37 / 5.10 $\times$	6.36 / 4.35 $\times$	6.51 / 4.39 $\times$	5.81 / 4.13 $\times$	5.87 / 4.19 $\times$	3.46 / 2.44 $\times$	2.89 / 2.01 $\times$
Qwen3-4B	+ ReAnchor	7.51 / 5.03 $\times$	8.45 / 5.78 $\times$	7.20 / 4.90 $\times$	7.50 / 5.00 $\times$	6.78 / 4.78 $\times$	6.84 / 4.76 $\times$	3.89 / 2.68 $\times$	3.17 / 2.17 $\times$
Qwen3-8B	DFlash	6.34 / 4.39 $\times$	7.28 / 5.05 $\times$	6.38 / 4.41 $\times$	6.49 / 4.43 $\times$	5.71 / 4.06 $\times$	5.94 / 4.10 $\times$	3.42 / 2.40 $\times$	2.87 / 1.99 $\times$
Qwen3-8B	+ ReAnchor	7.45 / 5.03 $\times$	8.31 / 5.67 $\times$	7.11 / 4.72 $\times$	7.52 / 5.05 $\times$	6.68 / 4.69 $\times$	7.24 / 4.83 $\times$	3.82 / 2.57 $\times$	3.16 / 2.14 $\times$

Table 2: DFlash decoding performance at sampling temperatures

T=0

and

T=1

. Each entry reports average acceptance length

\tau

/ end-to-end speedup. Temperature

T

is used for both drafter and target sampling in speculative decoding. Speedup is the ratio of target-only latency measured at the same temperature to speculative decoding latency;

\tau

includes the target bonus token produced at each verification round. Bold marks the best value per dataset and metric.

Temperature = 0
		Dataset
Target model	Drafting method	GSM8K	MATH	AIME25	HumanEval	MBPP	LiveCode Bench	MT-Bench	Alpaca
Qwen3-4B	Domino	9.32 / 5.82 $\times$	8.20 / 5.34 $\times$	6.49 / 4.19 $\times$	6.84 / 4.50 $\times$	6.57 / 4.42 $\times$	5.82 / 3.83 $\times$	4.06 / 2.71 $\times$	3.56 / 2.33 $\times$
	+ ReAnchor	11.01 / 6.84 $\times$	9.31 / 5.94 $\times$	7.46 / 4.72 $\times$	7.73 / 4.99 $\times$	7.75 / 5.09 $\times$	6.75 / 4.29 $\times$	4.48 / 2.93 $\times$	3.90 / 2.50 $\times$
	DSpark	6.42 / 4.38 $\times$	6.20 / 4.27 $\times$	5.52 / 3.85 $\times$	5.70 / 4.10 $\times$	5.49 / 4.07 $\times$	5.18 / 3.67 $\times$	3.98 / 2.90 $\times$	3.61 / 2.46 $\times$
	+ ReAnchor	6.81 / 4.72 $\times$	6.47 / 4.41 $\times$	5.87 / 4.05 $\times$	6.11 / 4.35 $\times$	5.93 / 4.36 $\times$	5.62 / 4.00 $\times$	4.30 / 3.11 $\times$	3.96 / 2.65 $\times$

Table 3: Decoding performance of DFlash-based drafter variants at

T=0

and

T=1

. Each entry reports average acceptance length

\tau

/ end-to-end speedup relative to target-only decoding at the same temperature. The accepted length includes the target bonus token. Bold marks the best value per dataset and metric.

Method	In-block	Beyond	$\Delta\tau$
DFlash	$\times$	$\times$	–
In-block only	$\checkmark$	$\times$	$+0.54$
Extension only	$\times$	$\checkmark$	$+0.31$

Table 4: In-block refinement vs. block extension.

\Delta\tau

over DFlash. In-block only uses shifted predictions within the native block; Extension only uses them only beyond its endpoint.

Task	Shifted only	Multiple
GSM8K	50.7	35.8
MATH-500	51.2	40.6
HumanEval	49.0	34.7
MBPP	45.5	38.0

Table 5: Start-token alignment (%). Shifted only : the anchor matches only at baseline positions other than its original offset. Multiple : it matches at more than one position. Categories may overlap.

Attachment rule	Acceptance $\tau$	Speedup
Original offset	7.24	$4.96\times$
Random match	7.51	$5.16\times$
Earliest match	7.56	5.16 $\times$
Longest-prefix match	7.51	$5.14\times$

Table 6: Ablation of attachment-site matching on Qwen3-4B with DFlash. Results are averaged over 100 examples each from GSM8K, HumanEval, MATH, and MBPP.

Configuration	MATH	GSM8K	MBPP	HumanEval	MT-Bench	Alpaca
DFlash	7.37	6.24	5.81	6.51	3.46	2.89
+ Fixed-offset ReAnchor	8.03	6.89	6.49	7.04	3.75	3.04
+ ReAnchor	8.45	7.51	6.78	7.50	3.89	3.17

Table 7: Cumulative ablation of ReAnchor components. Average acceptance length

\tau

on Qwen3-4B with the DFlash drafter at

T=0

. Fixed-offset attaches each shifted block at its original offset

d

; ReAnchor instead attaches it at the earliest matching position in the baseline draft. All configurations use the same shifted anchors and drafter evaluations.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

# Tested draft groups	# Earliest wins	Win rate
25,304	23,875	94.35%

Appendix

Table 8: Winning rate of earliest attachment selection. Each tested draft group is one verifier round and shifted offset with at least one attachment path tying the maximum accepted draft length among all verified paths, including the baseline. “Earliest wins” counts groups where the earliest positive-overlap site also ties this maximum; “Win rate” is this count divided by the number of tested groups.

Speculative Decoding

Speculative decoding accelerates generation by drafting multiple tokens and verifying them in parallel with a larger target model.

See all 291 papers

DFlow: Enabling Verifier Information Flow in Block Diffusion Speculative Decoding
Sep 6, 2026Yaojie Zhang, Linfeng Zhang, Bin Cui +1Self-Conditioning Speculative Decoding for Diffusion Language Models
School of Computer Science & Beijing Key Laboratory of Software and Hardware Cooperative Artificial Intelligence Systems, Peking University · University of Electronic Science and Technology of China · Shanghai Jiao Tong University +1
xPress: Parallel Refinement for Diffusion Drafters in Speculative Decoding
Aug 3, 2026Zheng Wang, Davis Wertheimer, Yu Chin Fabian Lim +4Speculative Decoding for Diffusion Language Models Speculative Decoding
University of Illinois Urbana-Champaign · IBM
Carryover Drafting: Recycling Rejected States for Speculative Decoding
Sep 13, 2026Jahyun Koo, Sunghyeon Woo, Jaeeun Kil +4Speculative Decoding LLM Inference Acceleration
Seoul National University · NAVER Cloud

Sep 6, 2026J/K move · Enter open · S save

DFlow: Enabling Verifier Information Flow in Block Diffusion Speculative Decoding

Yaojie Zhang, Linfeng Zhang, Bin Cui, Xupeng Miao

School of Computer Science & Beijing Key Laboratory of Software and Hardware Cooperative Artificial Intelligence Systems, Peking University · University of Electronic Science and Technology of China · Shanghai Jiao Tong University+1

Self-Conditioning Speculative Decoding for Diffusion Language Models Diffusion Language Model Inference Blockwise Diffusion Language Models Diffusion Model Inference Acceleration

Open paper AskPDF arXiv

Block diffusion speculative decoding improves LLM inference efficiency by proposing a block of future tokens in parallel and verifying them with a single forward pass through the target model. However, existing methods retain only the accepted prefix and discard the rejected suffix, preventing the computation spent on these positions from benefiting subsequent drafting rounds and forcing the drafter to repeatedly reconstruct representations for future tokens from scratch. We observe that rejection only determines whether a proposed token can be committed, while the verifier representations at rejected positions can still provide useful information for subsequent predictions. Based on this observation, we propose DFlow, a simple yet effective framework that enables verifier information to flow across drafting rounds. DFlow reuses the hidden states produced by the target verifier for the rejected suffix to guide subsequent drafting without additional target computation. To effectively learn this information flow across drafting rounds, we introduce a self-condition train strategy that feeds verifier representations from earlier predictions back into subsequent predictions. Experiments on Qwen3 models across diverse benchmarks demonstrate that DFlow consistently improves draft quality and acceptance length over DFlash.

Abstract

Speculative Decoding

Explore similar work

DFlow: Enabling Verifier Information Flow in Block Diffusion Speculative Decoding

xPress: Parallel Refinement for Diffusion Drafters in Speculative Decoding

Carryover Drafting: Recycling Rejected States for Speculative Decoding