The order in which tokens are predicted in language modeling has a profound impact on the quality of generated sequences. Autoregressive and masked diffusion models iteratively fill in tokens and can exploit such orderings across generation steps, but how to leverage an ordering for one-step parallel language modeling is unclear. We introduce asynchronous flow map language models (aFMLMs), which incorporate prediction order into one-step modeling by learning the flow maps of asynchronous flows. These flows specify an ordering through a tokenwise time schedule that controls the relative denoising progress of tokens. We formulate a general class of asynchronous flows that admit both prescribed and learned, state-adaptive schedules, show that they define valid flow maps, and develop new methods for their flow map distillation. The resulting models can generate all tokens in a single forward pass by internalizing the ordering encoded in the flow into their hidden computations. On Sudoku solving and code generation for GSM8K math word problems, aFMLM establishes new state-of-the-art one-step performance.
Figures & tables
Figure 1: Overview. (Left) We formulate asynchronous flows that denoise tokens at different rates. To rhyme with “mitten,” “kitten” emerges early and guides the preceding words. (Right) We distill these flows into an asynchronous flow map that internalizes this ordering and generates the full sequence in one forward pass.
Figure 2: Accuracy of a FLM under different inference-time schedulers on Sudoku-Hard (left) and GSM8K (right). A well-chosen scheduler improves performance over the synchronous one (Sync.). Alg. order † denotes the a FLM whose denoiser is trained only along the algorithmic order.
Sudoku
GSM8K
Model
Easy
Medium
Hard
Accuracy
MDLM ( Sahoo et al., 2024 )
63.6
16.2
1.7
0.0
Duo ( Sahoo et al., 2025 )
69.8
24.6
2.4
0.0
CANDI ( Pynadath et al., 2025 )
69.0
36.1
4.0
0.0
FLM ( Lee et al., 2026 )
69.1
21.5
2.8
0.0
S-FLM ( Deschenaux and Gulcehre, 2026 )
77.2
36.0
5.2
0.0
Table 1: One-step exact solving accuracy (%) on Sudoku and zero-shot accuracy (%) on GSM8K (see Section C.2 for baseline details). For a FLM, the schedule does not matter in one-step generation.
Figure 4
Schedule
Teacher
Distillation
Total
Learned
1.7h
3.3h
5.0h
Prescribed
0h
3.2h
3.2h
Table 2: Training cost (GPU hours) on Sudoku-Hard.
Figure 5: A visualization of code solutions to a GSM8K problem generated by one-step a FMLM (EB) (middle) and a FMLM (Sync.) (bottom) in their layer-wise order, compared with the generation order of a FLM (EB) (top). Columns show when 10%, 50%, 85%, and 100% of the tokens are decided.
Schedule
Sudoku-Hard
GSM8K
Synchronous
0.500
0.091
Asynchronous
0.516
0.271
Table 3: Spearman correlation with the schedule-induced ordering.
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 6: Power schedules τj(t)=tαj with β=log3 (left) and their speeds ktj (right). Darker curves denote higher-priority positions, with exponents ranging from αj=1/3 (darkest) to αj=3 (lightest), and the dashed line denotes the synchronous schedule.
TinyGSM / GSM8K
Sudoku
Architecture (DiT)
Hidden size
768
512
adaLN cond. dim
128
128
Layers
12
8
Attention heads
12
8
FFN multiplier
4
4
Appendix
Table 4: Per-task hyperparameters. All models on the same task share the architecture and optimizer, and differ only in the training and inference schedules.
Figure 7: Algorithmic ordering for a hard Sudoku puzzle. Plain digits are the given clues, and labels r1 – r3 indicate the round in which each empty cell is resolved. Cells resolved in the current round are shaded in darker blue, while previously assigned rounds remain visible. Cells still blank after round 3 are resolved in later rounds (not shown).
Model
10%
50%
85%
100%
a FMLM (EB)
6
8
11
12
a FMLM (Sync.)
2
7
10
12
Appendix
Table 5: Transformer block (out of 12) at which each column of Figure 5 is taken.
Non-autoregressive generation offers a powerful paradigm for iterative refinement, allowing models to recursively critique, erase and regenerate arbitrary subsets of tokens. However, existing non-autoregressive models fail to realize this potential. Masked Diffusion Models (MDMs) suffer from factorization error, causing sample quality to collapse when generating multiple tokens simultaneously. Flow Map Language Models (FMLMs) circumvent this bottleneck via joint sequence transport for excellent few-step generation, but sacrifice the inference-time flexibility of MDMs. We introduce FMLM+, a framework that bridges this gap by equipping FMLM with masking-style noise schedules. While generating the full sequence in a single step, FMLM+ simultaneously scores the global consistency of each token a posteriori. We leverage this to introduce Posterior Refinement, a novel inference-time refinement strategy that enables the model to adaptively self-correct its outputs, matching the performance of discrete baselines with 32x fewer NFEs. Across diverse benchmarks, we demonstrate that FMLM+ with Posterior Refinement improves the speed--quality tradeoff over both MDM and FMLM families, providing a scalable foundation for high-fidelity language modeling.
Language models based on discrete diffusion have attracted widespread interest for their potential to provide faster generation than autoregressive models. Despite their promise, these models typically produce samples whose quality sharply degrades in the few-step regime, preventing a dramatic speedup in practice. Here, we show that language models based on continuous flows over one-hot token embeddings can outperform discrete diffusion in both quality and speed. Importantly, our continuous formulation defines a unique flow map that can be learned directly for efficient few-step inference, a structure we show is unavailable to discrete methods. In this setting, we show that both the flow and its associated flow map can be learned with simple cross-entropy objectives that respect the simplex geometry of the data, and we identify three distinct choices for flow map distillation whose performance we compare in practice. Using these insights, we build a flow language model (FLM), a continuous flow that matches state-of-the-art discrete diffusion baselines on the One Billion Words (LM1B) and OpenWebText (OWT) datasets. We then distill FLM into a flow map language model (FMLM), whose one-step generation exceeds the 8-step quality of recent few-step discrete diffusion language models. Our work challenges the widely-held hypothesis that discrete noising processes are necessary for generative modeling over discrete modalities and paves the way toward accelerated language modeling at scale. Code is available at https://github.com/david3684/flm.
Masked Diffusion Models (MDMs) promise fast, parallel language generation, but their reverse transition factorises across token positions -- an approximation that breaks down in the few-step sampling regime where parallel generation ought to provide the greatest efficiency gains. Flow Language Models (FLMs) sidestep this limitation by learning a continuous flow that transports noise toward clean sequences represented in Euclidean space, inducing a flow map that can be distilled for single-step generation. However, this makes complex tasks requiring multi-step reasoning problematic for FLMs, as FLMs are forced to decode every token during generation. To address this, we introduce Masked Language Flow Models (MLFMs), which incorporate masking into FLMs using a continuous stochastic interpolant to bridge partially masked and clean sequences. This design enables conditional generation via continuous flows and allows pretrained MDMs to be converted into MLFMs through a simple, lightweight adaptation. Leveraging this flexibility, we propose a novel sampler that alternates continuous denoising with the discrete unmasking of confident tokens to better support multi-step reasoning. We evaluate our approach on GSM8K and MT-Bench and find, for the first time, that flow-based language models can be scaled to solve downstream reasoning and instruction-following tasks.
Iskander Azangulov, Kianoosh Ashouritaklimi, Leo Zhang +2