ES-dLLM: Efficient Inference for Diffusion Large Language Models by Early-Skipping

Zijian Zhu; Fei Ren; Zhanhong Tan; Kaisheng Ma

cs.LGMar 10, 2026

ES-dLLM: Efficient Inference for Diffusion Large Language Models by Early-Skipping

Authors: Zijian Zhu, Fei Ren, Zhanhong Tan, Kaisheng Ma

Organizations: IIIS, Tsinghua University · Polar Bear Tech.

Abstract

Diffusion large language models (dLLMs) are emerging as a promising alternative to autoregressive models (ARMs) due to their ability to capture bidirectional context and the potential for parallel generation. Despite the advantages, dLLM inference remains computationally expensive as the full input context is processed at every iteration. In this work, we analyze the generation dynamics of dLLMs and find that intermediate representations, including key, value, and hidden states, change only subtly across successive iterations. Leveraging this insight, we propose ES-dLLM, a training-free inference acceleration framework for dLLM that reduces computation by skipping tokens in early layers based on the estimated importance. Token importance is computed with intermediate tensor variation and confidence scores of previous iterations. Experiments on LLaDA-8B and Dream-7B demonstrate that ES-dLLM achieves throughput of up to 226.57 and 308.51 tokens per second (TPS), respectively, on an NVIDIA H200 GPU, delivering 5.6 $\times$ to 16.8 $\times$ speedup over the vanilla implementation and up to 1.85 $\times$ over the state-of-the-art caching method, while preserving generation quality.

Figures & tables

Panel of Figure 1 — Figure 1: Confidence variation statistics using LLaDA-8B-Instruct. (a) uses a sample from the BBH dataset, while (b) and (c) present results using 100 samples from multiple datasets.

Panel of Figure 2 — Figure 2: Hidden state variation in layer 10 using LLaDA-8B-Instruct. (a) uses a single sample from the BBH dataset, while (b) presents results using 100 samples from multiple datasets. The red vertical line in (a) separates prompt and output tokens, and the distribution in (b) includes only output tokens.

Figure 3: Illustration of ES-dLLM compared with the vanilla implementation and DualCache, assuming block-1 is under processing. The figure presents only 4 tokens per block, while the actual block length can be much larger (e.g., 32 or 64).

Benchmark	Method	TPS	Speedup	Performance Score
GSM8K(5)	LLaDA	8.56	1.0 $\times$	76.72
	DualCache	112.15	13.1 $\times$	76.35
	ES-dLLM	143.93	16.8 $\times$	76.95
MATH(4)	LLaDA	14.04	1.0 $\times$	28.14
	DualCache	56.02	4.0 $\times$	26.94
	ES-dLLM	103.63	7.4 $\times$	27.24

Table 1: Performance comparison using LLaDA-8B-Instruct on five benchmark datasets. TPS (tokens per second) measures throughput, calculated as the generated token count divided by total time. Speedup reports relative throughput improvement in terms of TPS compared to the vanilla implementation (i.e., LLaDA). Performance score in percentage indicates the accuracy or pass rate. The number in parentheses of each benchmark indicates the number of shots for evaluation. Bold numbers highlight the best performance on each benchmark.

Benchmark	Method	TPS	Speedup	Performance Score
GSM8K(5)	Dream	19.80	1.0 $\times$	79.00
	DualCache	209.88	10.6 $\times$	77.94
	ES-dLLM	267.13	13.5 $\times$	77.94
MATH(4)	Dream	26.38	1.0 $\times$	33.94
	DualCache	86.38	3.3 $\times$	33.60
	ES-dLLM	147.44	5.6 $\times$	33.44

Table 2: Performance comparison using Dream-7B-Instruct on five benchmark datasets. Metrics and notations are the same as in Table 1 .

Panel of Figure 4 — Figure 4: Ablation studies on importance estimation configurations using LLaDA-8B-Instruct.

Appendix figures & tables17 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 5: Variation statistics of key, value, and query tensors in layer 10 using LLaDA-8B-Instruct. Left: single-sample heatmap from BBH; red line separates prompt and output tokens. Right: log-scale distribution for output tokens using 100 samples from multiple datasets.

Figure 6: Log-scale distributions of hidden state variation in layers 10, 20, and 30 using LLaDA-8B-Instruct.

Panel of Figure 7 — Figure 7: Confidence variation statistics using Dream-7B-Instruct. (a) uses a sample from the BBH dataset, while (b) and (c) present results using 100 samples from multiple datasets.

Figure 8: Variation statistics of hidden states and key, query, and value tensors in layer 10 using Dream-7B-Instruct. Left: single-sample heatmap from BBH; red line separates prompt and output tokens. Right: log-scale distribution for output tokens using 100 samples from multiple datasets.

Layer	0	4	8	16	24	31
Hidden	0.15	0.22	0.27	0.38	0.48	0.51
Query	N/A	0.18	0.24	0.39	0.45	0.66
Key	N/A	0.18	0.24	0.38	0.42	0.66
Value	N/A	0.16	0.25	0.43	0.45	0.65

Appendix

Table 3: Pearson correlation between intermediate tensor variation and probability change (confidence is the maximum probability) for all mask tokens in selected layers using LLaDA-8B-Instruct.

Benchmark	Generation Length	Block Length
GSM8K	256	64
MATH	256	256
BBH	256	64
HumanEval	512	64
MBPP	512	64

Appendix

Table 4: Generation length and block length for each benchmark.

Table 5: Cache refresh periods in ES-dLLM for each benchmark and model.

Model	Benchmark	Prompt	Block
LLaDA	BBH	32	4
Dream	BBH	16	4
Dream	MBPP	8	2

Appendix

Table 6: Cache refresh periods in ES-dLLM* for each benchmark and model.

Benchmark	Method	TPS	Speedup	Performance Score
GSM8K(5)	LLaDA	9.20	1.0 $\times$	71.11
	DualCache	116.46	12.7 $\times$	66.19
	ES-dLLM	140.45	15.3 $\times$	67.63
MATH(4)	LLaDA	15.12	1.0 $\times$	32.38
	DualCache	57.89	3.8 $\times$	30.48
	ES-dLLM	94.46	6.2 $\times$	30.60

Appendix

Table 7: Performance comparison using LLaDA-8B-Base on five benchmark datasets. Metrics and notations follow Table 1 .

Benchmark	Method	TPS	Speedup	Performance Score
GSM8K(5)	Dream	21.12	1.0 $\times$	74.15
	DualCache	211.67	10.2 $\times$	73.16
	ES-dLLM	282.67	13.4 $\times$	73.92
MATH(4)	Dream	27.98	1.0 $\times$	40.90
	DualCache	86.58	3.1 $\times$	40.10
	ES-dLLM	167.03	6.0 $\times$	39.60

Appendix

Table 8: Performance comparison of Dream-7B-Base on five benchmark datasets.

Skip Ratio & Position	FLOPs Prop.	TPS	Speedup	Performance Score
No skipping	100%	56.02	1.0 $\times$	26.94
$r_{4}=r_{8}=0.5$	40%	103.63	1.85 $\times$	27.24
$r_{8}=0.75$	46%	95.26	1.70 $\times$	27.26
$r_{8}=0.5$	64%	77.66	1.39 $\times$	27.36
$r_{8}=0.25$	82%	63.49	1.13 $\times$	27.58
$r_{0}=0.5$	52%	89.40	1.60 $\times$	26.76

Appendix

Table 9: Ablation study of skip ratio and position configurations on MATH using LLaDA-8B-Instruct. No skipping represents the DualCache baseline. Speedup is calculated against the TPS of DualCache, and the FLOPs proportion is normalized to the no-skipping baseline.

Skip Ratio& Position	FLOPs Prop.	GSM8K	MATH	BBH	HumanEval	MBPP
$r_{4}=0.7$	40%	76.27	27.44	53.95	35.98	39.4
$r_{4}=r_{8}=0.5$	40%	76.95	27.24	54.51	37.8	39.4
$r_{4}=r_{8}=r_{12}=0.405$	40%	76.35	27.48	53.86	36.59	39.8

Appendix

Table 10: Ablation study on skipping times across five datasets using LLaDA-8B-Instruct.

Benchmark	Method	TPS	Speedup	Performance Score
GSM8K(5)	DualCache+PD	172.02	1.53 $\times$	76.57
GSM8K(5)	ES-dLLM +PD	201.60	1.80 $\times$	75.74
MATH(4)	DualCache+PD	85.05	1.52 $\times$	26.94
MATH(4)	ES-dLLM +PD	152.19	2.72 $\times$	27.48
BBH(3)	DualCache+PD	269.48	2.07 $\times$	52.31
BBH(3)	ES-dLLM +PD	302.95	2.33 $\times$	53.26

Appendix

Table 11: Performance comparison for parallel decoding using LLaDA-8B-Instruct. Speedup is compared with the DualCache baseline without parallel decoding.

Benchmark	Method	TPS	Speedup	Performance Score
GSM8K(5)	DualCache+PD	364.48	1.74 $\times$	77.94
GSM8K(5)	ES-dLLM +PD	425.27	2.03 $\times$	78.01
MATH(4)	DualCache+PD	128.02	1.48 $\times$	33.58
MATH(4)	ES-dLLM +PD	215.61	2.50 $\times$	33.48
BBH(3)	DualCache+PD	546.51	2.41 $\times$	57.18
BBH(3)	ES-dLLM +PD	615.51	2.71 $\times$	57.07

Appendix

Table 12: Performance comparison for parallel decoding using Dream-7B-Instruct.

Benchmark	Method	TPS	Speedup	Performance Score
GSM8K(5)	Sparse-dLLM	140.26	1.25 $\times$	76.80
GSM8K(5)	ES-dLLM +Sparse	172.55	1.54 $\times$	75.97
MATH(4)	Sparse-dLLM	66.80	1.19 $\times$	28.10
MATH(4)	ES-dLLM +Sparse	121.91	2.18 $\times$	28.06
BBH(3)	Sparse-dLLM	160.35	1.23 $\times$	49.13
BBH(3)	ES-dLLM +Sparse	191.56	1.47 $\times$	51.13

Appendix

Table 13: Performance comparison for sparse attention using LLaDA-8B-Instruct. Speedup is compared to DualCache without sparse attention technique.

Benchmark	Method	TPS	Speedup	Performance Score
GSM8K(5)	Sparse-dLLM	227.68	1.08 $\times$	77.94
GSM8K(5)	ES-dLLM +Sparse	287.68	1.37 $\times$	77.94
MATH(4)	Sparse-dLLM	89.56	1.04 $\times$	35.60
MATH(4)	ES-dLLM +Sparse	157.19	1.82 $\times$	36.10
BBH(3)	Sparse-dLLM	243.35	1.07 $\times$	57.52
BBH(3)	ES-dLLM +Sparse	305.35	1.35 $\times$	57.09

Appendix

Table 14: Performance comparison for sparse attention using Dream-7B-Instruct.

LLaDA-8B-Instruct
Benchmark	GSM8K	MATH	BBH	HumanEval	MBPP
TPS	241.03	175.53	331.30	407.86	483.35
(vs. DualCache)	2.15 $\times$	3.13 $\times$	2.55 $\times$	2.31 $\times$	4.10 $\times$
Performance	74.30	28.46	52.17	34.76	38.6
(vs. DualCache)	-2.05	+1.52	-1.12	-0.61	+0.2
Dream-7B-Instruct

Appendix

Table 15: Performance of ES-dLLM when combined with both parallel decoding and sparse attention.

Explore similar work

Dynamic-dLLM: Dynamic Cache-Budget and Adaptive Parallel Decoding for Training-Free Acceleration of Diffusion LLM
May 27, 2026Tianyi Wu, Xiaoxi Sun, Yanhua Jiao +5Diffusion Language Model Inference Diffusion Language Model Decoding
*Equal contribution. The work is done during Tianyi’s internship at CBG Celia DeviceAI Team, Huawei. · 1Harbin Institute of Technology, Shenzhen · 2Huawei +1
Streaming-dLLM: Accelerating Diffusion LLMs via Suffix Pruning and Dynamic Decoding
Jan 25, 2026Zhongyu Xiao, Zhiwei Hao, Jianyuan Guo +4Adaptive Inference Blockwise Diffusion Language Models
School of information and Electronics, Beijing Institute of Technology, Beijing · Department of Computer Science, City University of Hong Kong, Hong Kong · School of Computer Science, Wuhan University, Wuhan +2
CAI-DLLM: Convergence Aware Inference for Diffusion Language Models
Aug 23, 2026Farhana Amin, Sabiha Afroz, Dimitrios S. NikolopoulosDiffusion Language Model Inference Diffusion Language Model Decoding
Virginia Tech

May 27, 2026J/K move · Enter open · S save

Dynamic-dLLM: Dynamic Cache-Budget and Adaptive Parallel Decoding for Training-Free Acceleration of Diffusion LLM

Tianyi Wu, Xiaoxi Sun, Yanhua Jiao, Yulin Li+4

*Equal contribution. The work is done during Tianyi’s internship at CBG Celia DeviceAI Team, Huawei. · 1Harbin Institute of Technology, Shenzhen · 2Huawei+1

Diffusion Language Model Inference Diffusion Language Model Decoding Diffusion Model Inference Acceleration

Open paper AskPDF arXiv

Diffusion Large Language Models (dLLMs) offer a promising alternative to autoregressive models, excelling in text generation tasks due to their bidirectional attention mechanisms. However, their computational complexity scales on the order of L cubed with the sequence length L. This poses significant challenges for long-sequence and real-time applications, primarily due to the lack of compatibility with key-value caching and the non-autoregressive nature of denoising steps. Existing acceleration methods rely on static caching or parallel decoding strategies, which fail to account for the dynamic behavior of token properties across layers and decoding steps. We propose Dynamic-dLLM, a training-free framework that enhances dLLM inference efficiency through two components: Dynamic Cache Updating (DCU), which adaptively allocates cache-update budgets based on layer-wise token dynamics, and Adaptive Parallel Decoding (APD), which dynamically calibrates decoding thresholds to balance generation quality and efficiency. Extensive experiments on models like LLaDA-8B-Instruct, LLaDA-1.5, and Dream-v0-7B-Instruct across benchmarks such as MMLU, GSM8K, and HumanEval demonstrate that Dynamic-dLLM significantly improves inference speed. It attains an average speedup exceeding 3 times while maintaining performance. Dynamic-dLLM outperforms state-of-the-art acceleration methods and provides a plug-and-play solution for efficient dLLM deployment without compromising performance. The code is available at https://github.com/TianyiWu233/DYNAMIC-DLLM.

Abstract

Explore similar work

Dynamic-dLLM: Dynamic Cache-Budget and Adaptive Parallel Decoding for Training-Free Acceleration of Diffusion LLM

Streaming-dLLM: Accelerating Diffusion LLMs via Suffix Pruning and Dynamic Decoding

CAI-DLLM: Convergence Aware Inference for Diffusion Language Models