cs.LGSep 30, 2026

Gumbel Straight Flow: Distilling Autoregressive Models into One-step Flow Maps

Authors: Yeongmin Kim, Arnaud Doucet, Andrew Campbell, Valentin De Bortoli, Thomas Mensink, David Ruhe

Organizations: Google DeepMind · Work done as a Student Researcher at Google DeepMind Amsterdam

Abstract

We present Gumbel Straight Flow (GSF), a continuous flow map language model that leverages the noise-data coupling of a pretrained autoregressive language (AR) model. We theoretically demonstrate that the coupling between Gumbel noise and one-hot token sequences induced by an autoregressive model yields non-intersecting linear paths connecting the noise to the sequence representations. To further enhance high-quality few-step path sampling, we use a flow map semigroup objective where the tangent (velocity) condition is guided directly by the AR teacher. Across various benchmarks, including pretraining and downstream tasks, GSF can outperform current few-step language generation baselines.

Figures & tables

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Self-conditioned Flow Map Language Models via Fixed-point Flows

    Jul 1, 2026Jaehoon Yoo, Wonjung Kim, Floor Eijkelboom +4Latent FlowText-Conditioned Diffusion Model

  2. A Few-Step Generative Model on Cumulative Flow Maps

    May 5, 2026Zhiqi Li, Duowen Chen, Yuchen Sun +1Flow MapFlow Models

  3. Flow-Map GRPO: Reinforcement Learning for Few-Step Flow-Map Generators via Anchored Stochastic Composition

    Jul 1, 2026Zhiqi Li, Wen Zhang, Bo ZhuFlow MapInvertible Flows