Authors: Jiayu Liu, Ruizhi Zhao, Tianyun Ji, Yee Whye Teh, Zhenya Huang, Ning Miao
Organizations: Hong Kong Institute of AI for Science, City University of Hong Kong · Shenzhen University · State Key Laboratory of Cognitive Intelligence, University of Science and Technology of China · Department of Statistics, University of Oxford · Department of Data Science, City University of Hong Kong
Diffusion language models generate tokens in parallel, but still require multiple sampling steps to produce high-quality text. In this paper, we investigate whether a model can learn to generate an entire sequence in a single forward pass. We propose Seq-Drifting, a framework that applies drifting to sequences of continuous token representations and directly learns a map from Gaussian noise to text. Unlike existing DLMs that derive the supervision at each position directly from the observed token in a training sequence, Seq-Drifting builds a drift field whose position-wise targets adapt to the current generated context. At each position, attraction guides each output toward a set of contextually plausible token representations, while repulsion discourages collapse across samples and repetition within sequences. The generator learns from this drift field during training and produces all tokens jointly in one pass at inference. Empirically, Seq-Drifting achieves a generative perplexity of 36.01 on LM1B in one step and performs competitively in both unconditional and conditional generation. Experiments on translation, summarization, and reasoning further demonstrate the applicability of the framework beyond open-ended generation. These results suggest that drifting over token sequences offers a promising path toward effective one-step text generation. Our code is available at https://github.com/jyliu-ai/Seq-Drifting.
Figures & tables
Figure 1: Seq-Drifting achieves lower generative perplexity than prior few-step distilled DLMs, and is on par with or better than multi-step DLMs, with a 4 – 11× speedup over the few-step baselines and a 900 – 2500× speedup over the multi-step ones in wall-clock time. (Gen-PPL under GPT-2 Large; dataset: LM1B, T=128 . Detailed comparison in Figure 3 and Table 6 .)
Figure 2: Seq-Drifting. In one forward pass the generator Gθ maps a noise vector ϵ∼N(0,IC) (optionally with a prefix p1,…,pM ) to token embeddings h1,…,hT , each projected onto the unit sphere as et=ht/∥ht∥∈SD−1 . Supervision is applied per position . A positive set St of plausible tokens attracts et , while the negative sets Ntout (other samples) and Ntin (other positions of the same sample) repel it, weighted by a similarity kernel φ and strengths λin/out . The two combine into the drift force Ft ; a single geodesic step along Ft gives the stop-gradient target goalt , onto which the generator is regressed, L=E∥et−sg(goalt)∥2 .
Figure 3: Unconditional generation. Left: Gen-PPL (under GPT-2 Large) and unigram entropy for the few-step distilled models at 1 / 2 / 4 steps and Seq-Drifting at a single pass, on LM1B ( T=128 ) and OpenWebText ( T=1024 ); multi-step baselines are in Table 6 . Right: Gen-PPL vs. sampling steps and wall-clock running time per sample, showing Seq-Drifting achieves lower Gen-PPL in a single forward pass at a fraction of the cost.
Method
Step
64→64
64→512
Gen-PPL ↓
Ent.
Gen-PPL ↓
Ent.
FMLM
1
162.20
3.78
418.37
3.32
2
143.62
3.82
441.09
3.55
4
124.74
3.82
320.22
3.52
Duo+DCD
1
972.09
3.77
7639.94
3.85
2
384.35
3.72
846.20
3.72
Table 1: Conditional generation. A 64 -token prefix followed by a continuation.
Unconditional ( 128 )
Conditional ( 64→64 )
Decision
Variant
Gen-PPL ↓
Entropy
Gen-PPL ↓
Entropy
Seq-Drifting P
36.01
4.42
76.16
4.40
Geometry
Euclidean
diverges (loss blow-up)
Positives
off-policy
> 1000
> 6
> 10000
> 4
Anti-repetition
w/o intra
10.01
1.82
12.18
1.31
Repulsion
per-context (block)
–
–
81.27
4.27
Table 2: Ablation study on unconditional ( 128 ) and conditional ( 64→64 ) generation.
Figure 4: Hyperparameter sweeps of the repulsion strength λout , the anti-repetition weight λin , the attraction temperature τ , and the support size k . Setting: unconditional 128 -token generation (solid), conditional 64→64 generation (dashed).
Table 7
Model
SVAMP
ASDiv
ProofWriter
ACC ↑
Time/sample (ms) ↓
ACC ↑
Time/sample (ms) ↓
ACC ↑
Time/sample (ms) ↓
AR
27.92 ±0.36
49.42 ±4.36
44.97 ±0.50
31.07 ±1.18
85.90 ±0.38
41.90 ±0.10
NAR
4.72 ±0.76
2.70 ±0.04
1.73 ±0.57
1.30 ±0.07
0.00 ±0.00
1.42 ±0.06
Coconut
17.76 ±0.01
25.80 ±0.52
22.59 ±0.01
26.63 ±0.13
–
–
Seq-Drifting
13.60 ±0.50
5.88 ±0.17
19.11 ±0.79
1.05 ±0.03
79.31 ±0.47
3.68 ±0.02
Table 4: Performance on SVAMP, ASDiv, and ProofWriter.
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
Method
Step
Self-BLEU
LM1B
OpenWebText
FMLM
1
0.201
0.249
2
0.198
0.251
4
0.204
0.260
Duo+DCD
1
0.042
0.016
2
0.065
0.064
Appendix
Table 5: Self-BLEU across the step budget. Per-dataset Self-BLEU for the few-step distilled models and Seq-Drifting; a fully mode-collapsed generator scores 1.0 (bottom row).
Figure 7: Sampling time across step budgets. Seconds per sample on LM1B ( T=128 ) and OpenWebText ( T=1024 ). The x-axis is logarithmic in the number of sampling steps and the y-axis is logarithmic in seconds per sample. The dashed line marks Seq-Drifting.
Method
Step
LM1B ( T=128 )
OpenWebText ( T=1024 )
Gen-PPL ↓
Entropy
Gen-PPL ↓
Entropy
FLM
1
6.27
1.15
27.54
1.53
8
215.84
4.24
453.49
5.20
1024
85.24
4.29
63.97
5.35
Duo
1
1027.77
4.34
4007.98
5.80
8
121.37
4.27
196.45
5.57
Appendix
Table 6: Unconditional generation: multi-step models across the step budget, with Seq-Drifting at a single pass. LM1B ( T=128 ) and OpenWebText ( T=1024 ). A dash indicates that no corresponding released checkpoint or published reference result was available for that dataset–model setting.
Method
Step
64→64
64→512
Gen-PPL ↓
Entropy
Gen-PPL ↓
Entropy
FLM
1
1.95
0.83
201.21
2.18
8
6.00
1.81
522.85
3.63
1024
9.47
2.19
304.01
3.44
Duo
1
1103.17
3.83
7616.55
3.86
8
99.98
3.78
181.91
3.81
Appendix
Table 7: Conditional generation: multi-step models across the step budget, with Seq-Drifting at a single pass. A 64 -token prefix, then a continuation ( 64→64 on LM1B, 64→512 on OWT). A dash indicates that no corresponding released checkpoint or published reference result was available for that dataset–model setting.
Method
Step
LM1B
OpenWebText
Pythia-1B
OPT-1.3B
Pythia-1B
OPT-1.3B
FMLM
1
94.75
97.29
170.10
152.67
2
87.55
88.58
134.89
122.45
4
79.86
80.37
109.44
100.41
Duo+DCD
1
829.59
849.31
4654.30
4401.95
2
435.40
453.47
905.38
792.78
Appendix
Table 8: Unconditional Gen-PPL under alternative evaluators. Gen-PPL computed with Pythia-1B and OPT-1.3B for LM1B ( T=128 ) and OpenWebText ( T=1024 ).
Figure 8: Training dynamics of the conditional runs. Gen-PPL (log scale, left axis) and unigram entropy (right axis) versus training step, for Seq-Drifting P 64→64 (top left), Seq-Drifting S 64→64 (top right), Seq-Drifting P 64→512 (bottom left), and Seq-Drifting S 64→512 (bottom right). All panels show training through 300 k steps for both output lengths. The dashed line marks the warm-up → conditional switch at 200 k steps.
Figure 9: Off-policy training dynamics (unconditional, length 128 ). Gen-PPL (log scale, left axis) and unigram entropy (right axis) versus training step for the off-policy positives ablation. The dashed line marks the end of the temperature warm-up at 10 k steps, where τ reaches 0.1 and the run begins to oscillate and degrade.
Discrete diffusion language models (DDLMs) generate text by iteratively denoising categorical token sequences, while recent drifting methods for continuous generators suggest that part of this sampling-time correction can instead be absorbed into training through an anti-symmetric fixed-point objective. We study how to transfer this principle to DDLMs, where the main challenge is the interface with discrete text: hard token samples are non-differentiable, and categorical predictions do not directly provide continuous samples to drift. We formulate TokenDrift, a drifting objective that lifts categorical predictions to soft-token features, applies anti-symmetric drifting in a frozen semantic space, and backpropagates the resulting stop-gradient feature target to DDLM logits. In controlled continual-training experiments with masked and uniform-state diffusion backbones, TokenDrift improves fixed-NFE generation quality over matched continuation baselines, reducing Gen.-PPL at 4 NFEs by 89% on MDLM and 86% on DUO. These results suggest that drifting can provide a practical refinement objective for DDLMs.
Drifting Models generate a sample with a single function evaluation, making them attractive for fast parallel language generation, but their one-step formulation does not support iterative refinement. We propose Recurrent Language Generation via Drifting (ReLaDrift), which uses a shared Drifting Model to generate an initial draft from noise and iteratively refine it in a continuous latent space. We train this recurrence by sampling the number of rounds and applying the drifting loss only at the final round, stopping gradients through preceding rounds so that only one round requires backpropagation. Unlike the one-step Drifting Model, whose accuracy does not improve when iterated, ReLaDrift enables a trade-off between test-time computation and generation quality, as diffusion and flow language models do. On Sudoku Hard and GSM8K, it achieves competitive or superior performance compared with these iterative-generation approaches across a range of inference budgets. Our code is available at https://github.com/Yuki-Ots/ReLaDrift
Diffusion Language Models (DLMs) generate text by iteratively denoising a masked sequence, independently predicting multiple tokens at each step. This conditional independence discards inter-token dependencies and degrades coherence-an issue that parallels the multi-modality problem in Non-Autoregressive Translation (NAT). Drawing on the Directed Acyclic Transformer (DAT), which tackles this problem in NAT via a Directed Acyclic Graph (DAG), we propose DA-DLM, a model that adapts DAG-based dependency modeling to DLMs' iterative setting through a position-oriented DAG design. The position-oriented DAG binds node groups to fixed output positions so that tokens fixed in earlier steps anchor neighboring predictions via learned transitions, and evolves with denoising to focus on remaining uncertainty as anchors accumulate. On language modeling, open-ended generation, and summarization, DA-DLM consistently outperforms Block Diffusion, especially under fewer denoising steps, and matches autoregressive models while preserving the parallel generation advantage. Our code is publicly available at https://github.com/jipy0222/DA-DLM.
Pengyu Ji, Zichen Zhang, Xiang Hu +1
‡School of Information Science and Technology, ShanghaiTech University · §Shanghai Engineering Research Center of Intelligent Vision and Imaging · ¶Tencent