cs.AINov 5, 2026

ChainFlow: Few-Step Latent Flow Matching for Speculative Decoding

Authors: Selim Aktas, Yusuf Erdem Erdem, Ahmed Zeer, Himmet Toprak Kesgin, Muzaffer Kaan Yuce, Mehmet Fatih Amasyali

Organizations: Department of Electronics Engineering, Koç University, Istanbul, Türkiye · Department of Computer Engineering, Yıldız Technical University, Istanbul, Türkiye

Abstract

Speculative decoding has become a standard way to accelerate LLM inference: a lightweight drafter proposes tokens that the target verifies in one parallel pass. Existing drafters face trade-offs that limits their speedups: autoregressive drafters generate tokens one at a time, so drafting cost grows with draft length, while diffusion drafters generate tokens in parallel but predict each position independently, ignoring inter-token dependencies. We introduce ChainFlow, a new speculative decoding paradigm and the first flow-matching drafter conditioned on the target's hidden states. ChainFlow treats drafting as a short transport over hidden states, needing one or two flow steps instead of the many denoising steps of generation from noise. A variational autoencoder compresses these states so drafting cost barely grows with target size, lightweight feedback heads restore inter-token dependencies, and tree drafting raises acceptance. On Qwen3.5 models, at matched drafter size and training data, ChainFlow outperforms EAGLE-3 and DFlash in vLLM by 39% and 11%, respectively. These gains further widen as drafters are adapted to new languages sequentially. Our ablations cover drafter design and concurrent serving, and we open-source our code and drafter models.

Figures & tables

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. DFlow: Enabling Verifier Information Flow in Block Diffusion Speculative Decoding

    Sep 6, 2026Yaojie Zhang, Linfeng Zhang, Bin Cui +1Self-ConditioningSpeculative Decoding for Diffusion Language Models

  2. FlashMap: Flow Map Speculative Decoding

    Oct 17, 2026Jeff Shen, Zhejian Zhou, Ryan Lee +2Speculative DecodingLLM Inference Acceleration

  3. SpecBlock: Block-Iterative Speculative Decoding with Dynamic Tree Drafting

    May 8, 2026Weijie Shi, Qiang Xu, Fan Deng +9Speculative DecodingLLM Inference Acceleration