Speculative decoding accelerates autoregressive LLMs by having a lightweight drafter propose tokens that the target model verifies in parallel. Diffusion-based drafters further reduce drafting latency by proposing multiple tokens at once. However, these tokens are predicted independently, so a single early error causes prefix verification to discard the rest of the draft, even when it contains useful downstream predictions. We introduce DEdit, a diffusion-based drafter that can not only draft by conventional parallel unmasking but also iteratively edit its draft through token-to-token predictions. Through editing, later predictions can serve as bidirectional context for repairing earlier errors and extending the accepted prefix. To teach the model to repair errors while preserving correct predictions, we propose ProposalMix, a training scheme that mixes draft predictions with ground-truth tokens based on first-pass confidence during training. Across seven benchmarks on Qwen3-4B and Qwen3-8B, DEdit achieves the highest macro-average token acceptance and speedup among the evaluated drafters, reaching macro-average speedups of 5.72× and 5.97× over autoregressive generation under greedy decoding, respectively. Further analysis shows that acceptance improves with more editing passes and wider drafting windows, and that ProposalMix halves harmful edits that shorten the accepted prefix. Moreover, restricting the editor to causal attention lowers acceptance, especially on highly predictable outputs, indicating that future context is a key source of these gains.
Figures & tables
Figure 1: Bidirectional editing before verification extends the accepted prefix.
Figure 2: DEdit inference and training. (a) At inference time, drafter generates and edits its proposal tokens through K passes, sending only the final proposal to the target model for verification. (b) ProposalMix constructs P2 inputs for joint proposal-and-edit training.
Model
Method
Math
Code
Chat
Overall
GSM8K
MATH-500
AIME25
HumanEval
MBPP
LCB
MT-Bench
Avg.
Temperature: 0
Speedup
τ
Speedup
τ
Speedup
τ
Speedup
τ
Speedup
τ
Speedup
τ
Speedup
τ
Speedup
τ
Qwen3-4B
DFlash
4.99 ×
6.04
6.52 ×
7.76
5.82 ×
7.07
4.45 ×
5.32
4.08 ×
5.06
4.81 ×
5.83
2.98 ×
4.17
4.81 ×
5.89
DSpark
5.37 ×
7.13
6.52 ×
9.02
6.06 ×
8.23
4.61 ×
6.23
4.37 ×
6.11
5.32 ×
7.14
2.67 ×
4.53
4.99 ×
6.91
Domino
5.53 ×
6.76
7.21 ×
8.87
6.65 ×
8.12
4.76 ×
5.84
4.56 ×
5.70
5.30 ×
6.39
3.13 ×
4.53
5.30 ×
6.60
DEdit P3
6.51 ×
7.75
8.18 ×
10.67
6.84 ×
9.28
4.95 ×
6.48
4.71 ×
6.13
5.71 ×
7.61
3.17 ×
5.11
5.72 ×
7.58
Table 1: End-to-end speedup and mean token acceptance ( τ ) under the evaluation protocol in Section 4.1 ; Avg. is the seven-suite macro average. Baselines operate at W=16 (Domino and DSpark draft 16 candidates; DFlash drafts 15), while DEdit uses the W=32 P3 model after a 3K-step continuation from W=16 . Bold marks the best value in each model/temperature block.
Method
Pass
W=16
W=24
W=32
DFlash
P1
5.89
6.01
6.01
Domino
P1
6.60
6.91
6.93
DEdit
P1
5.82
5.93
5.93
P2
6.63
7.02
7.12
P3
6.87
7.39
7.58
Table 2: Window scaling and training ablations on Qwen3-4B. Both panels report macro-average τ across seven benchmarks. (a) Proposal width and editing passes at Tp=0 ; Domino drafts W candidates and the other methods W−1 . (b) Training components in a controlled 100K setting: a frozen DFlash drafter supplies P1, and a separately trained editor is evaluated at P2 with W=16 ; Δτ is relative to Full.
Figure 3: Editing outcomes and future context on Qwen3-4B. (a) Edit-gain distribution AP2−AP1 for the Full and w/o ProposalMix editors of Table 2 (b). Percentages use all rounds; zero-gain rounds are omitted (60.0% without ProposalMix , 66.5% with it). (b) W=16 DEdit P2-minus-Domino gain versus correct future-token count. Points and bands show response-cluster means and 95% cluster-bootstrap CIs; gray bars show the number of rounds per bin on a log scale, from 56k to 127. Fully accepted rounds (7.7%) are excluded.
DEdit P2
DEdit P3
Task slice
Domino
Causal
Bidir.
Δ
Causal
Bidir.
Δ
General (7-suite avg.)
5.168
4.997
5.219
+0.222
5.192
5.482
+0.290
Reasoning
3.260
3.224
3.422
+0.199
3.371
3.624
+0.253
Summary
3.756
3.718
3.913
+0.195
3.881
4.141
+0.260
Copy-heavy
8.560
9.374
10.098
+0.724
9.819
10.717
+0.897
Table 3: Fixed-weight visibility intervention. Values are A over the first 16 candidate positions, averaged over rounds within each slice; General averages the seven suite-level means. Causal and bidirectional DEdit share a W=32 checkpoint and P1 proposal. Domino ( W=16 ) takes the first token of this proposal and drafts the remaining 15 with its own backbone and corrector. Δ : bidirectional minus causal; underlined: below Domino. Details in Appendix A.3 .
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 4: A full W=32 editing trace. On an MBPP prompt, successive passes extend the accepted prefix from 8 to 18 to 30 of 31 proposal tokens. Blue cells already match the target but remain blocked by an earlier error.
Figure 5: Fixed-weight intervention on future proposal visibility. Lines report response-cluster mean bidirectional-minus-causal accepted tokens with 95% response-bootstrap intervals; gray bars report each bin’s share of 181,205 eligible rounds. P2 compares both masks on the same P1 proposal, while P3 applies the next edit to each configuration. The hatched bar denotes the 10,382 rounds in which all 16 shared proposal tokens are accepted.
Source / partition
800K (Qwen3-4B)
800K (Qwen3-8B)
100K (Qwen3-4B)
Nemotron / chat
365,375 (45.68%)
365,511 (45.69%)
28,791 (28.79%)
Nemotron / code
175,000 (21.88%)
175,000 (21.88%)
2,195 (2.20%)
Nemotron / math
239,467 (29.94%)
239,467 (29.93%)
6,698 (6.70%)
Nemotron / STEM
–
–
23,197 (23.20%)
CodeAlpaca-20k
20,022 (2.50%)
20,022 (2.50%)
–
OpenR1 Math
–
–
12,429 (12.43%)
Appendix
Table 4: Training-corpus composition: records (percentage of each corpus). Counts are computed from the released source field and, for 100K, the joint source / split fields. Nemotron rows denote v2 partitions for 800K and the mirror’s nemotron labels for 100K. A dash denotes no records with that source label.
Setting
Controlled
Main
Role
Training-design ablations
Main system comparison
Target(s)
Qwen3-4B
Qwen3-4B and Qwen3-8B
Corpus
100K target-aligned
800K Nemotron-based mixture
Global batch
8
16
Epochs
6
Optimizer
AdamW, LR 6×10−4 , warmup 0.04, clip 1.0
Appendix
Table 5: Training configurations for the controlled ablations and main results. Shared entries span both columns.
Step
P1 accuracy
GT selection
Overwrite
12,221
33.19
28.10
5.79
36,663
42.80
37.35
5.84
73,326
49.87
44.91
5.84
Appendix
Table 6: ProposalMix input rates (%; η=0.5 ) when each checkpoint generates its own P1 on a fixed probe set. GT selection marks positions chosen for ground-truth conditioning; overwrite counts those whose tokens change.
Figure 6: Trade-off across editing passes for the same W=32 checkpoint at Tp=0 . Each configuration is evaluated twice with a shared autoregressive baseline. Drafting uses CUDA Graphs and target verification remains eager. Training supervises P1 and P2; P3 and P4 reuse the trained editor at inference.
Figure 7: Host enqueue-ahead during CUDA Graph replay. (a) The CPU submits eager target operations while the GPU executes the captured drafter; draft and target GPU operations remain sequential. A forced host barrier delays target submission. (b) Nsight traces decompose each round into the pre-target draft interval, target compute, and gaps between target operations.
Speculative decoding accelerates autoregressive large language model inference by drafting multiple tokens and verifying them in a single target-model forward pass. Recent diffusion-based drafters generate an entire block of tokens in parallel but usually commit to a single draft sequence per verification: once the first mismatch occurs, all subsequent draft tokens are discarded, resulting in a limited acceptance rate. Naively batching more draft candidate sequences only introduces a marginal improvement, as redundant or poorly placed branches increase the cost of drafting and verification without proportionally increasing the number of accepted tokens. We propose D^2SD, a dual diffusion draft speculative decoding framework that organizes candidates into a confidence-guided prefix tree, where the first diffusion drafter generates a block along with per-position confidence scores that are used to identify the most likely rejection boundary and select the top-K prefix ranges for recovery; the second variable-prefix diffusion drafter re-anchors at each selected prefix and proposes alternative continuations in one batched pass; the resulting shared-prefix candidates are jointly verified via cascade attention. Empirically, D^2SD shows clear improvements over both the underlying diffusion approach and strong autoregressive speculative decoding baselines.
Liyuan Zhang, Jiarui Zhang, Jinwei Yao +6
Peking University · Tsinghua University · HKUST +2
Speculative decoding accelerates autoregressive generation in large language models. In each drafting stage, a lightweight draft model proposes tokens that the target model subsequently verifies. With increasingly capable draft models, we find that the target model frequently accepts all tokens produced in a drafting stage. A verification nevertheless follows each drafting stage, resulting in unnecessary target-model forward passes even when drafting could have continued. Adaptive draft length methods decide during decoding how many draft tokens precede a verification, but they raise the speedup only for autoregressive draft models. For a parallel draft model, drafting further requires target-model hidden states for draft tokens that have not been verified. We propose DLoop, a looped form of speculative decoding that adaptively performs multiple drafting stages before verification. DLoop continues drafting while the draft model remains confident and verifies all accumulated draft tokens together. Loop-aware training keeps the draft model reliable in the additional drafting stages by exposing it to its own hidden states for unverified draft tokens. By spending additional draft-model forward passes, DLoop reduces the number of target-model forward passes required for verification. Across diverse speculative decoding methods including EAGLE-3, DFlash, Domino, DSpark, and multi-token prediction modules, DLoop improves the wall-clock speedup by 5 to 41 percent while preserving lossless decoding. Code will be available at https://github.com/naver-ai/DLoop.
Geonmo Gu, Byeongho Heo, HeeJae Jun +4
NAVER AI Lab · Korea University · NAVER AI Search Platform
Speculative decoding accelerates autoregressive language models by using a lightweight drafter to propose multiple future tokens, which the target model then verifies in parallel. DFlash shows that a block diffusion drafter can generate an entire draft block in a single forward pass and achieve state-of-the-art speculative decoding performance, outperforming strong autoregressive drafters such as EAGLE-3. Vanilla DFlash, however, still verifies only a single drafted trajectory per round, potentially limiting its acceptance length. We introduce DDTree (Diffusion Draft Tree), a method that constructs a draft tree directly from the per-position distributions of a block diffusion drafter. Under a fixed node budget, DDTree uses a simple best-first heap algorithm to select the continuations that are most likely to match the target model according to a surrogate defined by the draft model's output. The resulting tree is verified efficiently in a single target model forward pass using an ancestor-only attention mask. Because DDTree builds on DFlash, a leading draft model for speculative decoding, these gains place DDTree among the leading approaches to speculative decoding.
Liran Ringel, Yaniv Romano
Department of Computer Science, Technion – Israel Institute of Technology. · Department of Electrical and Computer Engineering, Technion – Israel Institute of Technology.