cs.AINov 9, 2026

Multi-Future Drafting: Mixture Block Drafters for Tree-Verified Speculative Decoding

Authors: Haoyu Huang, Jialiang Kang, Linlin Yang, Xiao Yang, Baochang Zhang

Organizations: National College for Excellent Engineers, Beihang University · Canva Research · Communication University of China · School of Artificial Intelligence, Beihang University

Abstract

Block drafters such as DFlash draft a block of future tokens in one forward pass, and draft trees such as DDTree verify the prefixes ranked highest by the product of the drafter's per-position distributions. Under tree verification, the expected number of accepted tokens is the probability mass the tree covers. We show that a product misallocates it: when the continuation can follow several futures, it splices them into chimeras, prefixes stitched together from different futures that the target never produces, and gives them as much budget as the futures themselves. Barring ties, no product-ranked tree can hold two futures that disagree in mm positions without 2m−1−12^{m-1}-1 chimeras, however well the drafter is trained, and in practice chimeras make up 27-33% of a 64-node tree. We propose Multi-Future Drafting (MFD), which restores the coupling with a latent variable instead of the chain rule: KK learned mode embeddings at the drafter's input draft KK coherent blocks in the same pass, and a mixture DDTree lets the verifier pick the future the target takes, losslessly and with K+1K+1 extra vectors. On four Qwen targets from 4B to 27B parameters, at batch size 1, MFD reaches up to 7.8×\times speedup, 5-8% faster than the best single-mode tree trained identically and, on Qwen3-4B and Qwen3-8B, 9-16% faster than the released DARTree trained on the same prompts.

Figures & tables

Appendix figures & tables16 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. DForkTree: Accelerating Speculative Decoding with Fork-Aware Block Diffusion Draft Trees

    Nov 9, 2026Zhicheng Zheng, Cheng Zhang, Chenkai Tang +1Speculative DecodingLLM Inference Acceleration

  2. Accelerating Speculative Decoding with Block Diffusion Draft Trees

    Apr 14, 2026Liran Ringel, Yaniv RomanoSpeculative DecodingParallel Speculative Decoding

  3. TreeFlash: Parallel AR-Approximation for Faster Speculative Decoding

    Jun 2, 2026Peer Rheinboldt, Frédéric Berdoz, Roger WattenhoferSpeculative DecodingLLM Inference Acceleration