TDLM: Adaptive Temporal Integration for Diffusion Language Models

Bohang Sun; Max Zhu; Francesco Caso; Christopher Irwin; Jialin Yu; Pietro Liò

cs.LGOct 27, 2026

TDLM: Adaptive Temporal Integration for Diffusion Language Models

Authors: Bohang Sun, Max Zhu, Francesco Caso, Christopher Irwin, Jialin Yu, Pietro Liò

Organizations: Department of Computer Science and Technology, University of Cambridge · Department of Engineering Science, University of Oxford

Abstract

Masked diffusion language models repeatedly compute representations for unresolved tokens, but conventional denoisers do not explicitly retain these representations for reuse across steps. We introduce Temporal Self-Conditioning Diffusion Language Model (TDLM), which lets each unresolved position attend to its own history of hidden representations within a frozen denoiser. This gives later predictions access to computation already performed. Training uses self-generated trajectories and a simple cross-entropy objective for tokens revealed between consecutively sampled decoding steps, while causal temporal attention permits parallel evaluation during training. On LLaDA, the temporal modules add only approximately 0.84% trainable parameters. Across LLaDA and Dream, the main comparison shows up to 7.15 $\times$ faster generation than the corresponding original diffusion language model, with competitive answer quality on the evaluated tasks. These results motivate temporal memory as a practical way to make more effective use of repeated denoising computation.

Figures & tables

Figure 1: Accuracy–latency trade-off on GSM8K. Threshold sweeps compare TDLM with the evaluated baselines.

Figure 2: Reusing computation before commitment. Each column is a denoising evaluation, read from bottom to top. The bottom shows the partially revealed input; the top shows token predictions before commitment. Decoding Steps 1–3 use inputs $S_{0},S_{1},S_{2}$ at $s_{0}=0,s_{1}=1,s_{2}=2$ , respectively. The green position remains masked while its predictions can change. Green arrows retrieve earlier hidden representations at that position and layer. Yellow modules are trainable, and the backbone is frozen.

Figure 3: Sampled inputs and their training targets. The top row gives the teacher’s reveal step for each token position. The three input rows are

S_{1},S_{2},S_{3}

at steps

s_{1}=2,s_{2}=4,s_{3}=6

. Gray positions are revealed context. Green positions are masked inputs whose teacher tokens become visible in the next selected sequence and are used as CE targets. Red positions are also masked, but receive no loss on that input. Arrows show reuse of earlier hidden representations at the same position and layer.

		LLaDA-8B-Instruct				Dream-v0-Instruct-7B
Dataset	Model	ACC (%) $\uparrow$	NFE	Sec. $\downarrow$	Speedup	ACC (%) $\uparrow$	NFE	Sec. $\downarrow$	Speedup
MATH-500	TDLM	37.20	54.79	0.829	4.26 $\times$	44.20	74.62	1.194	3.05 $\times$
	Fast-dLLM	38.60	107.20	1.535	2.30 $\times$	44.00	113.50	1.646	2.21 $\times$
	MetaState	36.80	107.56	1.682	2.10 $\times$	40.80 †	115.88	1.671	2.18 $\times$
	CDLM	30.40	87.89	2.851	1.24 $\times$	26.00	96.93	1.438	2.53 $\times$
	d3LLM	31.80	35.67	1.244	2.84 $\times$	42.40	53.76	0.820	4.45 $\times$

Table 1: Main results on two backbones. ACC is mathematical accuracy or code pass@1; NFE and seconds are per-problem means. Speedup is relative to the original model of the same backbone on the same task. Gold , silver , and bronze mark the three best distinct displayed values within each backbone–task group; ties share a rank.

\dagger

: scoring-only tag removal for the Dream MetaState results is described in Appendix C .

Temporal context	Accuracy (%)	NFE	Sec.	GPU (GiB)
Current only (gated)	76.48	42.64	0.623	15.184
Last 1	77.62	44.06	0.637	15.188
Last 3	78.00	42.36	0.631	15.196
Last 5	78.68	41.57	0.601	15.204
Full history	79.08	40.08	0.585	15.330

Table 2: Restricting temporal context at inference. All conditions reuse the same trained LLaDA checkpoint without retraining. Current-only retains the Q/K/sink gate. GPU memory includes weights and compiled workspaces. Implementation details are in Appendix E .

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Method	Added trainable parameters	Training data
d3LLM [ 15 ]	335.54M (LoRA)	$\approx$ 92k examples
CDLM [ 11 ]	176.13M (LoRA)	$\approx$ 15k prompts; trajectories collected at two temperatures per prompt
MetaState [ 18 ]	51.08M	15,543 instruction–response examples
TDLM	67.11M	3,500 self-generated trajectories

Appendix

Table 3: Added trainable parameters and training-data scale for LLaDA-based models. M denotes one million parameters.

Temporal layers	Accuracy (%)	NFE	Avg. sec
8, dispersed	77.86	40.15	0.571
16, middle	79.00	45.72	0.683
16, dispersed (main)	79.08	40.08	0.585
32, all	Training OOM

Appendix

Table 4: Different temporal-layer allocations on GSM8K. Every temporal module uses four heads of width 64; measured models share the 750-update training recipe.

Heads $\times$ head width	Total width	Updates	Accuracy (%)	NFE	Avg. sec
$1\times 64$	64	500	76.35	47.34	0.682
$1\times 128$	128	500	77.71	47.66	0.701
$2\times 128$	256	750	78.47	45.84	0.688
$4\times 64$	256	750	79.00	45.72	0.683

Appendix

Table 5: Temporal-width and head configurations on GSM8K. All models augment the middle sixteen backbone layers (zero-based indices 8–23). Total width is the product of head count and head width. Training-recipe differences are described in the text.

Dataset	Model	ACC (%) $\uparrow$	NFE	Sec. $\downarrow$	Speedup
MATH-500	TDLM	36.00	32.60	0.366	3.50 $\times$
	Fast-dLLM	36.40	64.28	0.670	1.91 $\times$
	MetaState	35.80	65.14	0.742	1.72 $\times$
	CDLM	30.80	46.36	1.806	0.71 $\times$
	d3LLM	31.40	42.96	0.947	1.35 $\times$
	Original DLM	36.00	128.00	1.281	1.00 $\times$

Appendix

Table 6: LLaDA with a 128-token generation budget and block size 32. ACC is mathematical accuracy or code pass@1. Speedup is relative to original DLM on the same task at this budget. Medal colors and ties follow Table 1 . Values and speedup ratios are reported at the supplied precision.

Dataset	Model	ACC (%) $\uparrow$	NFE	Sec. $\downarrow$	Speedup
MATH-500	TDLM	42.80	70.83	1.389	4.81 $\times$
	Fast-dLLM	41.60	134.01	2.443	2.73 $\times$
	MetaState	41.20	135.23	2.599	2.57 $\times$
	CDLM	31.00	91.48	3.569	1.87 $\times$
	d3LLM	35.80	42.06	1.618	4.12 $\times$
	Original DLM	43.20	384.00	6.675	1.00 $\times$

Appendix

Table 7: LLaDA with a 384-token generation budget and block size 32. ACC is mathematical accuracy or code pass@1. Speedup is relative to original DLM on the same task at this budget. Medal colors and ties follow Table 1 . Values and speedup ratios are reported at the supplied precision.

Diffusion Language Models

Diffusion language models generate and refine text through iterative denoising, enabling bidirectional context and parallel token updates.

See all 473 papers

DiLaDiff: Distilled Latent-Augmented Diffusion for Language Modeling
May 22, 2026Jean-Marie Lemercier, Tomas Geffner, Karsten Kreis +3Diffusion Language Models Diffusion Language Model Distillation
NVIDIA
Revise, Don't Freeze: Sampler-Matched Training for Self-Correcting Masked Diffusion Language Models
May 31, 2026Longxuan Yu, Shaorong Zhang, Yu Fu +3Masked Diffusion Models Diffusion Model Fine-Tuning
University of California, Riverside · Microsoft
ALoDLM: Adaptively Looped Diffusion Language Models
Oct 3, 2026Liancheng Fang, Zhuowei Li, Youngeun Kim +10Adaptive Computation for Diffusion Models Diffusion Language Models
University of Illinois Chicago · Amazon AGI · Korea University

May 22, 2026J/K move · Enter open · S save

DiLaDiff: Distilled Latent-Augmented Diffusion for Language Modeling

Jean-Marie Lemercier, Tomas Geffner, Karsten Kreis, Morteza Mardani+2

NVIDIA

Diffusion Language Models Diffusion Language Model Distillation Diffusion Language Model Inference Masked Diffusion Models Continuous Diffusion Language Models Diffusion Model Inference Acceleration+1

Open paper AskPDF arXiv

Diffusion language models intrinsically fail to capture correlations between decoded tokens, which leads to a harsh trade-off between sampling quality and throughput. To solve this issue, we propose DiLaDiff, a variant of masked diffusion language models with three components: (1) a continuous latent space with semantic capabilities, learned by an auto-encoder fine-tuned from an existing masked diffusion language model; (2) a latent diffusion model learning the prior over the encoder distribution; (3) a consistency model distilling the learned prior into a few-step latent generative model. We show that, even without distillation, our latent-guided diffusion model outperforms the masked diffusion baseline while significantly accelerating inference. Consistency distillation further lowers the computational overhead of continuous diffusion, such that the latent is generated in negligible time compared to discrete decoding.

Abstract

Diffusion Language Models

Explore similar work

DiLaDiff: Distilled Latent-Augmented Diffusion for Language Modeling

Revise, Don't Freeze: Sampler-Matched Training for Self-Correcting Masked Diffusion Language Models

ALoDLM: Adaptively Looped Diffusion Language Models