Organizations: National College for Excellent Engineers, Beihang University · Canva Research · Communication University of China · School of Artificial Intelligence, Beihang University
Block drafters such as DFlash draft a block of future tokens in one forward pass, and draft trees such as DDTree verify the prefixes ranked highest by the product of the drafter's per-position distributions. Under tree verification, the expected number of accepted tokens is the probability mass the tree covers. We show that a product misallocates it: when the continuation can follow several futures, it splices them into chimeras, prefixes stitched together from different futures that the target never produces, and gives them as much budget as the futures themselves. Barring ties, no product-ranked tree can hold two futures that disagree in m positions without 2m−1−1 chimeras, however well the drafter is trained, and in practice chimeras make up 27-33% of a 64-node tree. We propose Multi-Future Drafting (MFD), which restores the coupling with a latent variable instead of the chain rule: K learned mode embeddings at the drafter's input draft K coherent blocks in the same pass, and a mixture DDTree lets the verifier pick the future the target takes, losslessly and with K+1 extra vectors. On four Qwen targets from 4B to 27B parameters, at batch size 1, MFD reaches up to 7.8× speedup, 5-8% faster than the best single-mode tree trained identically and, on Qwen3-4B and Qwen3-8B, 9-16% faster than the released DARTree trained on the same prompts.
Figures & tables
Figure 1: Why a one-pass drafter needs several futures. (a, b) A real GSM8K context for Qwen3-4B: after Let’s break , the target says future A ( p=0.61 ) or future B ( p=0.37 ). (a) A one-pass drafter (DFlash fine-tuned with D-PACE) gives one distribution per position, each mixing A’s token and B’s; DDTree’s best path jumps between the futures, a chimera. 9 of the tree’s 15 nodes lie on prefixes that splice A and B, and it accepts 2 tokens. (b) MFD’s 8 modes run in the same pass; six follow A or B, two neither. Every prefix is scored by its best mode and the 15 best form the tree: 6 nodes on A, 9 on B, none a chimera; the verifier keeps A, 6 tokens. (c) Speedup ( T=0 , 5-task mean) against the tree budget B : MFD leads the strongest D-PACE tree under the same recipe at every budget, by about 2% at B=16 and 6–7% from B=64 on. DDTree uses the released drafter; dashed: chain drafters.
Figure 2: Multi-Future Drafting on the example of Figure 1 . (a) The block, the last verified token y0 and L−1 mask tokens, is copied K times, the mode embedding ek is added to every position of copy k , and all copies are drafted in one forward pass; each mode gives a distribution per position. (b) The mode that best covers the target’s continuation ( Equation 5 ) takes most of the update, so the modes specialize. (c) Every prefix is scored by its best mode and the B best form one tree, which keeps both futures and no chimera.
Qwen3-4B
Qwen3-8B
Qwen3.5-4B (thinking)
Qwen3.6-27B (thinking)
Method
Training
T=0
T=1
T=0
T=1
T=0
T=1
T=0
T=1
EAGLE-3
released
4.21 × (6.46)
3.87 × (6.01)
3.75 × (6.07)
3.42 × (5.58)
–
–
–
–
DFlash chain
released
4.38 × (5.98)
3.73 × (5.08)
4.26 × (5.99)
3.48 × (4.88)
4.85 × (6.25)
3.29 × (4.23)
4.66 × (5.81)
4.04 × (5.04)
DFlash 2 selector chain
FT 1 ep
4.73 × (6.63)
4.12 × (5.79)
4.73 × (6.74)
4.00 × (5.75)
5.38 × (7.01)
3.97 × (5.18)
4.89 × (6.09)
4.35 × (5.43)
DDTree
released
6.08 × (8.59)
5.44 × (7.70)
5.73 × (8.18)
5.00 × (7.13)
6.53 × (8.78)
4.77 × (6.51)
5.89 × (7.62)
5.27 × (6.86)
DFlash 2 + DDTree
FT 1 ep
6.13 × (8.84)
5.39 × (7.78)
5.80 × (8.60)
4.88 × (7.28)
6.80 × (9.29)
4.62 × (6.40)
5.73 × (7.40)
5.07 × (6.57)
Table 1: Speedup over autoregressive decoding and acceptance length τ (5-task means). Each tree method except DARTree is at its best budget, B∈{16,64,128,256} on Qwen3-4B and Qwen3-8B and {16,64,256} on the thinking-mode targets ( Appendix H ). DFlash 2 + DDTree: DFlash 2’s distributions in the same tree; its selector chain at T=1 is the faster of its greedy and sampled chains. FT: fine-tuned under our recipe for 1 or 3 epochs (on Qwen3.6-27B, the 1-epoch rows are the first-epoch checkpoints of the 3-epoch runs). Bottom block: fine-tuned on the 1.42M target-regenerated prompts of Domino ( Huang et al., 2026 ) , the data of DARTree’s drafter; DARTree verifies 64 nodes with its released drafter and code. Bold: best within each block.
Qwen3-4B
Qwen3-8B
T=0
T=1
T=0
T=1
Released drafters on SGLang’s own engine paths
EAGLE-3
2.75 × (5.21)
2.58 × (4.91)
3.12 × (5.69)
2.88 × (4.91)
DFlash chain
4.06 × (5.97)
3.46 × (5.10)
4.39 × (5.90)
3.62 × (4.88)
DFlash 2 selector chain
3.37 × (5.20)
3.07 × (4.72)
3.57 × (4.94)
3.26 × (4.52)
Tree verification in our SGLang engine
Table 2: Batch size 1 in SGLang 0.5.22 on one RTX PRO 6000 GPU: decode speedup over SGLang’s autoregressive decoding (prefill excluded), with τ in parentheses, 5-task means. Top: released drafters as distributed, each at its best setting on SGLang’s own engine paths ( Appendix I ). Bottom: our tree engine: DDTree with the released DFlash drafter, and D-PACE, MFD and MFD-D (with a single-mode default path) fine-tuned from it on the full data of Table 1 , at B=63 . Bold: fastest per column.
Figure 3: Where the budget goes (Qwen3-4B, 24 prompts per task, T=0 contexts, T=1 probabilities from the verify pass; the three-epoch MFD, the strongest D-PACE (one epoch) of Table 1 and the released DDTree). Left: share of the nodes at each depth whose last step the target takes with probability below 1% (chimeras and dead steps, Appendix G ). Right: probability mass kept at each depth. From depth 2 on, where chimeras first form, the product trees waste more nodes and keep less mass.
Chain
Tree
Gate
Oracle
Gate
Mixture
Oracle
K
B=64
B=16
B=64
B=512
B=64
4
−3.1
+20.2
−4.9
+2.5
+5.4
+6.2
+13.0
8
−4.6
+27.4
−7.3
+2.3
+6.4
+7.8
+17.7
16
−7.2
+34.7
−10.4
+1.0
+6.6
+8.4
+21.5
32
−8.8
+40.2
−12.1
+0.0
+6.1
+8.7
+24.4
Table 3: One drafter, several ways to decode it ( T=0 ; change in τ relative to single-mode D-PACE decoded the same way, as a chain or as a tree with the same B ). Gate: the mode the gate picks before verification. Oracle: each block’s best mode, known only after verification. Mixture: the tree of Equation 2 , with the gate’s prior. The multi-mode drafters train on 1024 draft blocks per sequence against 512 for D-PACE; at a matched 512, K=4 gains 5.0% / 5.6% at B=64 ( T=0 / T=1 ). Bold: the best K at each budget.
Injection
Params
B=64
B=512
Oracle chain
Input (layer 0)
13K
+5.0
+5.9
+19.1
Layer 2
13K
+4.1
+5.0
+18.0
Layer 4 (last)
13K
+1.5
+1.9
+14.8
Output, rank 128
2.6M
−0.6
−0.2
+12.9
Input embeddings
13K
+7.0
+7.4
–
Output, full rank
26M
+2.5
+2.6
–
Table 4: Where the modes enter ( K=4 , T=0 ; change in τ relative to D-PACE). Top: mode embeddings added at different depths of the five-layer drafter, or a low-rank per-mode map of the output. Bottom: input embeddings against a full-rank per-mode output expansion with a learned prior ( Karimi et al., 2026 ) , both trained with the mixture NLL, relative to a single mode trained with plain NLL. Params: parameters added, including the gate or prior head. Oracle chain: each block’s best mode.
Appendix figures & tables16 assets
Supplementary material from the paper’s appendix.
Appendix
B=16
B=64
B=256
B=512
K
Max
LSE
Max
LSE
Max
LSE
Max
LSE
T=0
4
+2.5
+3.0
+5.4
+5.7
+6.2
+6.4
+6.2
+6.4
8
+2.3
+3.7
+6.4
+7.1
+7.7
+8.1
+7.8
+8.1
16
+1.0
+3.3
+6.6
+7.7
+8.2
+8.8
+8.4
+9.0
32
+0.0
+3.3
+6.1
+7.5
+8.4
+9.0
+8.7
+9.2
Appendix
Table 5: Max and log-sum-exp scoring of the mixture tree: change in τ relative to single-mode D-PACE (5-task means; the quarter-epoch drafters of Table 3 , scored with the gate’s prior as there). Max: the rule of Equation 2 , whose T=0 columns at B∈{16,64,512} are those of Table 3 ; LSE: log-sum-exp ( Equation 6 ).
Method
Training
GSM8K
MATH-500
HumanEval
MBPP
MT-Bench
T=0
EAGLE-3
released
4.67 × (7.11)
3.88 × (6.00)
4.25 × (6.51)
4.31 × (6.59)
3.91 × (6.10)
DFlash chain
released
4.62 × (6.34)
5.58 × (7.59)
4.81 × (6.56)
4.35 × (5.97)
2.54 × (3.44)
DFlash 2 selector chain
FT 1 ep
5.07 × (7.11)
5.97 × (8.35)
5.17 × (7.26)
4.73 × (6.65)
2.70 × (3.77)
DDTree
released
6.47 × (9.01)
7.19 × (10.29)
6.58 × (9.28)
6.30 × (8.82)
3.85 × (5.54)
DFlash 2 + DDTree
FT 1 ep
6.59 × (9.38)
7.33 × (10.68)
6.66 × (9.60)
6.19 × (8.86)
3.88 × (5.67)
Appendix
Table 6: Per-task speedup over autoregressive decoding and τ (in parentheses) on Qwen3-4B: the cells of Table 1 by task, from the same runs (every tree at B=256 except EAGLE-3’s, tuned as in Appendix H ; DARTree verifies 64 nodes). Bold: fastest in each column within each block.
Method
Training
GSM8K
MATH-500
HumanEval
MBPP
MT-Bench
T=0
EAGLE-3
released
4.22 × (6.77)
3.56 × (5.86)
3.84 × (6.09)
3.82 × (6.09)
3.29 × (5.55)
DFlash chain
released
4.55 × (6.38)
5.57 × (7.85)
4.57 × (6.40)
4.17 × (5.89)
2.42 × (3.42)
DFlash 2 selector chain
FT 1 ep
5.09 × (7.24)
6.18 × (8.82)
5.06 × (7.21)
4.71 × (6.69)
2.62 × (3.73)
DDTree
released
6.21 × (8.69)
6.97 × (10.14)
6.28 × (8.88)
5.78 × (8.10)
3.43 × (5.10)
DFlash 2 + DDTree
FT 1 ep
6.35 × (9.18)
6.99 × (10.63)
6.28 × (9.34)
5.88 × (8.49)
3.51 × (5.36)
Appendix
Table 7: Per-task speedup over autoregressive decoding and τ (in parentheses) on Qwen3-8B: the cells of Table 1 by task, from the same runs (every tree at B=128 except EAGLE-3’s, tuned as in Appendix H ; DARTree verifies 64 nodes). Bold: fastest in each column within each block.
Method
Training
GSM8K
MATH-500
HumanEval
MBPP
MT-Bench
T=0
EAGLE-3
released
–
–
–
–
–
DFlash chain
released
4.52 × (5.82)
5.41 × (6.97)
5.43 × (7.01)
4.98 × (6.43)
3.88 × (5.00)
DFlash 2 selector chain
FT 1 ep
5.05 × (6.59)
5.93 × (7.72)
6.03 × (7.86)
5.54 × (7.21)
4.34 × (5.66)
DDTree
released
6.14 × (8.25)
7.10 × (9.57)
7.22 × (9.74)
6.76 × (9.07)
5.40 × (7.28)
DFlash 2 + DDTree
FT 1 ep
6.45 × (8.85)
7.34 × (10.04)
7.61 × (10.41)
6.93 × (9.39)
5.67 × (7.76)
Appendix
Table 8: Per-task speedup over autoregressive decoding and τ (in parentheses) on Qwen3.5-4B (thinking): the cells of Table 1 by task, from the same runs (every tree at B=256 ). Bold: fastest in each column within each block.
Method
Training
GSM8K
MATH-500
HumanEval
MBPP
MT-Bench
T=0
EAGLE-3
released
–
–
–
–
–
DFlash chain
released
4.86 × (6.07)
5.63 × (6.99)
4.94 × (6.18)
4.51 × (5.61)
3.37 × (4.21)
DFlash 2 selector chain
FT 1 ep
5.11 × (6.38)
6.12 × (7.62)
5.07 × (6.32)
4.58 × (5.70)
3.55 × (4.44)
DDTree
released
6.13 × (7.88)
6.82 × (8.83)
6.20 × (8.06)
5.79 × (7.44)
4.52 × (5.90)
DFlash 2 + DDTree
FT 1 ep
6.03 × (7.78)
6.93 × (8.94)
5.86 × (7.55)
5.42 × (6.94)
4.44 × (5.76)
Appendix
Table 9: Per-task speedup over autoregressive decoding and τ (in parentheses) on Qwen3.6-27B (thinking): the cells of Table 1 by task, from the same runs (every tree at B=64 ). Bold: fastest in each column within each block.
Qwen3.5-4B (thinking)
Qwen3.6-27B (thinking)
B
Method
T=0
T=1
T=0
T=1
16
DDTree
5.38 × (7.04)
3.88 × (5.06)
5.28 × (6.61)
4.72 × (5.92)
D-PACE+DDTree
5.91 × (7.74)
3.90 × (5.08)
5.88 × (7.35)
5.00 × (6.27)
MFD
5.69 × (7.63)
3.97 × (5.30)
5.78 × (7.36)
5.06 × (6.47)
64
DDTree
6.18 × (8.14)
4.47 × (5.86)
5.89 × (7.62)
5.27 × (6.86)
D-PACE+DDTree
6.65 × (8.75)
4.42 × (5.79)
6.46 × (8.35)
5.50 × (7.17)
Appendix
Table 10: Hybrid targets by tree budget: speedup over autoregressive decoding and τ (in parentheses), 5-task means, from the main runs of Table 1 (DDTree with the released drafter; D-PACE and MFD fine-tuned for three epochs).
Decoding
Nodes
Qwen3-4B
Qwen3-8B
One mode: the D-PACE drafter
Chain
15
6.18
6.26
8 sampled chains
≤ 120
7.05
7.14
8 chains, best-first
≤ 120
7.63
7.70
Tree
64 / 128
8.06 / 8.42
8.14 / 8.49
Eight modes: the MFD drafter
Appendix
Table 11: Ways to decode the same drafters: simulated τ ( T=0 , 5-task means over 128 prompts per task, 80 for MT-Bench) of the three-epoch drafters of Table 1 (uniform prior), with the nodes verified per round. Chains are verified side by side; best-first: the argmax chain and the seven single-token deviations with the largest expected gain. Per-mode trees: one tree of B/8 nodes per mode. Product of mean marginals: the DDTree tree of the average of the eight modes’ per-position distributions, the same drafter without coupling. Product of per-position maxima: the DDTree tree of ∏imaxkqk,i(vi) , each position scored by its best mode. Modes shuffled: the mixture tree after permuting the eight modes’ distributions independently at each position, which keeps each position’s distributions but not their coupling (mean of two seeds, which differ by at most 0.02). Mean / random table: every mode embedding replaced by the table’s mean, or by the mean plus a random direction with that mode’s norm about it. T=1 in Table 12 .
Decoding
Nodes
Qwen3-4B
Qwen3-8B
One mode: the D-PACE drafter
Chain
15
5.23
5.10
8 sampled chains
≤ 120
6.02
5.87
8 chains, best-first
≤ 120
6.55
6.38
Tree
64 / 128
6.97 / 7.30
6.77 / 7.10
Eight modes: the MFD drafter
Appendix
Table 12: Table 11 at T=1 .
Figure 4: Coupling the latent explains on the target’s continuations, and its cost against the best mode, by depth in the block (Qwen3-4B, three-epoch MFD drafter, K=8 , 5 tasks). The explained coupling grows with depth and passes logK , the largest possible cost, at depth 9–10.
Mode
Distinctive fork tokens
1
Our (3.9), But (3.7), implementation (3.2), To (2.6), However (2.5)
This (3.4), These (3.1), which (3.1), Key (2.6), Since (2.6)
4
Therefore (5.0), also (3.0), Test (2.7), correctly (2.6), Then (2.6)
5
without (2.8), We (2.8), may (2.7), They (2.6), we (2.5)
6
Analy (2.5), --- (2.2), in (2.1), The (2.1), denote (2.1)
Appendix
Table 13: Tokens each mode drafts at the first position where the modes’ argmax chains disagree, most often relative to the average mode (lift in parentheses; Qwen3-4B, T=0 ; tokens with at least 300 forks over all modes, variants that differ only in surrounding whitespace merged, byte fragments omitted).
Injection
Identical
First split
After the split
Gain
positions
share
Input (layer 0)
0.23
5.5
6.2
0.61
+5.0
Layer 2
0.23
5.6
5.8
0.59
+4.1
Layer 4 (last)
0.25
5.8
4.3
0.46
+1.5
Output, rank 128
0.28
5.7
4.1
0.43
−0.6
Appendix
Table 14: Where the modes’ argmax chains part (Qwen3-4B; the K=4 quarter-epoch drafters of Table 4 ; 8 held-out conversations per task, every fourth anchor, 4330 anchors, all six pairs of modes, 15-token argmax chains). Identical: share of pairs whose chains agree at every position. First split: mean position of the first differing token among the other pairs (1 is the first drafted token). After the split: mean number of differing positions from there on, and their share of the remaining positions. Gain: the mixture tree’s change in τ at B=64 relative to D-PACE (%, Table 4 ).
Qwen3-4B
Qwen3-8B
Drafter and offsets
T=0
T=1
T=0
T=1
D-PACE, identical copies (its DDTree)
8.06 / 8.71
6.96 / 7.60
8.14 / 8.78
6.77 / 7.38
D-PACE + MFD’s trained table
8.30 / 9.03
7.18 / 7.87
8.37 / 9.05
6.95 / 7.62
D-PACE + MFD’s random table
8.21 / 8.91
7.08 / 7.75
8.24 / 8.90
6.85 / 7.49
D-PACE + random directions
8.24 / 8.93
7.12 / 7.79
8.22 / 8.89
6.84 / 7.47
D-PACE + random directions ×2
8.06 / 8.80
6.91 / 7.65
8.19 / 8.89
6.80 / 7.44
Appendix
Table 15: Mixture trees over eight offset copies of the D-PACE drafter, which was not trained with modes, against the MFD drafter (simulated τ at B=64 / 256, 5-task means, three-epoch drafters of Table 1 , uniform prior). Random table: the trained table’s mean plus random directions with each mode’s norm about it, as in Table 11 ; random directions: the same directions without the mean, at 1, 2 and 4 times that norm.
K
B=16
B=64
B=256
B=512
Draft
Tree
T=0
1
5.08 × (7.10)
5.80 × (8.08)
6.23 × (8.77)
4.63 × (9.01)
3.70
0.37
2
5.04 × (7.19)
5.91 × (8.38)
6.33 × (9.08)
4.76 × (9.39)
4.22
0.34
4
5.11 × (7.29)
6.05 × (8.58)
6.49 × (9.31)
4.90 × (9.66)
4.24
0.36
8
5.10 × (7.29)
6.11 × (8.70)
6.60 × (9.49)
4.96 × (9.81)
4.28
0.44
16
5.02 × (7.22)
6.07 × (8.72)
6.60 × (9.55)
5.00 × (9.97)
4.38
0.58
Appendix
Table 16: Speedup and τ (in parentheses) of the quarter-epoch drafters with K modes on Qwen3-4B (5-task means, the harness of Table 1 ); K=1 is the single-mode D-PACE drafter (512 draft blocks per sequence). Draft and tree: drafting and tree-building time per round at B=256 (ms; drafting time hardly changes with B ). Bold: fastest at each temperature.
Drafter
B
Chimera
Dead
Live
Chimeras
Best path
(%)
(%)
(%)
per tree
(%)
Qwen3-4B
DDTree (released)
64
30.7
22.1
47.2
19.6
64.6
D-PACE FT 1 ep
64
29.0
20.9
50.0
18.6
63.0
D-PACE FT 3 ep
64
27.6
22.2
50.2
17.7
64.6
MFD FT 1 ep
64
23.8
19.4
56.7
15.2
44.6
Appendix
Table 17: Chimeras in product and mixture trees (628 prompts, T=0 contexts, T=1 target probabilities): share of a tree’s nodes that are chimeras, dead or live, chimeras per tree, and share of rounds whose best path passes through a chimera. FT: fine-tuned under our recipe.
T=0
T=1
Epoch
D-PACE
MFD
D-PACE
MFD
Fine-tuned from the released drafter
1
8.76
9.43 ( +7.7 )
7.67
8.35 ( +8.9 )
2
8.77
9.55 ( +8.8 )
7.65
8.43 ( +10.2 )
3
8.70
9.54 ( +9.7 )
7.60
8.42 ( +10.8 )
Trained from scratch
Appendix
Table 18: Training longer (Qwen3-4B; simulated τ at B=256 with a uniform prior, 5-task means; in parentheses, the change of MFD relative to D-PACE in %). Top: the three-epoch fine-tuning runs of Table 1 after each epoch. Bottom: both drafters trained from scratch with DFlash’s schedule and the D-PACE loss.
Figure 5: Speedup over SGLang’s autoregressive decoding at the same batch size (top: T=0 , bottom: T=1 ). Closed loop: 320 requests from the five tasks with as many in flight as the batch size, at most 2048 new tokens, steady-state output throughput (prefills of new requests included, so batch-1 values lie below those of Table 2 ), one RTX PRO 6000 GPU. MFD and MFD-D run their drafter’s linear layers in FP8 (SGLang’s online option: per-channel weights, per-token activations, no new checkpoint; the acceptance length is unchanged). Every released option is tuned for each batch size and shown at the better of its bf16 and FP8 timings: trees over two or three budgets in {7,15,31,63} , the DFlash chain over 16-token and, from batch 4, 8-token blocks, EAGLE-3 over 6–10 tree shapes in bf16 and the best of them in FP8. Dashed: the fastest released option on SGLang’s own engine paths (DFlash chain, DFlash 2, EAGLE-3) at each batch size, which is the DFlash chain except where DFlash 2 is faster (open squares). DDTree: the released DFlash drafter in our tree engine. MFD-D is one engine with a fixed schedule: its eight-mode mixture for one or two requests, its single-mode default path (open circles) from three, with the budget falling from 63 to 7 as the batch grows. Every configuration is timed against the autoregressive reference of its own session; the references of different sessions agree within 0.8%, and a configuration re-timed in bf16 within 2.8%.
Department of Computer Science, Technion – Israel Institute of Technology. · Department of Electrical and Computer Engineering, Technion – Israel Institute of Technology.