Breaking Block Boundaries: Anchor-based History-stable Decoding for Diffusion Large Language Models

Shun Zou; Yong Wang; Zehui Chen; Lin Chen; Chongyang Tao; Feng Zhao; Xiangxiang Chu

cs.CLApr 10, 2026

Breaking Block Boundaries: Anchor-based History-stable Decoding for Diffusion Large Language Models

Authors: Shun Zou, Yong Wang, Zehui Chen, Lin Chen, Chongyang Tao, Feng Zhao, Xiangxiang Chu

Organizations: MoE Key Lab of BIPC, University of Science and Technology of China · AMAP, Alibaba Group

Abstract

Diffusion Large Language Models (dLLMs) have recently become a promising alternative to autoregressive large language models (ARMs). Semi-autoregressive (Semi-AR) decoding is widely employed in base dLLMs and advanced decoding strategies due to its superior performance. However, our observations reveal that Semi-AR decoding suffers from inherent block constraints, which cause the decoding of many cross-block stable tokens to be unnecessarily delayed. To address this challenge, we systematically investigate the identification of stable tokens and present three key findings: (1) naive lookahead decoding is unreliable, (2) token stability closely correlates with convergence trend, and (3) historical information is isolated. Building on these insights, we propose Anchor-based History-stable Decoding (AHD), a training-free, plug-and-play dynamic decoding strategy. Specifically, AHD monitors the stability trend of tokens in real time through dynamic anchors. Once a token reaches stability, it initiates early cross-block decoding to enhance efficiency and performance. Extensive experiments across language, vision-language, and audio-language domains demonstrate that AHD simultaneously improves both performance and inference efficiency. Notably, AHD effectively reverses the performance degradation typically observed in existing advanced decoding acceleration strategies. For instance, on the BBH benchmark, our approach reduces decoding steps by 80% while improving performance by 3.67%.

Figures & tables

Figure 1: Systematic analysis of Semi-AR decoding on dLLMs. (a) Heatmap visualization on a sample. Stars indicate the earliest step at which cross-block tokens reach stability, while black squares denote the finally unmasked step. For instance, position 65 is expected to be decoded no earlier than step 64; however, it stabilizes as early as step 53. (b) Qualitative analysis of cross - block delayed decoding. (c) Distribution of the first stable step.

Figure 2: Dynamics of confidence during the decoding. (a) Local fluctuations persist even after the token has become stable. (b) Prior to stabilization, the confidence remains flat or oscillatory; however, after crossing the first stable step, it exhibits a marked upward trend.

Figure 3: Illustration of Anchor - based History - stable Decoding (AHD). AHD retrospectively tracks historical trajectories from anchor points to dynamically monitor the absolute stability trend. Cross - block stable tokens are unlocked early in the absolute stability trend, reducing decoding steps while further unleashing the potential of parallel diffusion decoding and achieving significant performance gains.

Task	Metric	Vanilla	PC-sampler	Fast-dLLM	KLASS	Saber	Ours
HumanEval	Score ↑	40.85	40.24 (-0.61)	41.46 (+0.61)	40.85 (0.00)	36.59 (-4.26)	43.29 (+2.44)
HumanEval	Step ↓	256	256	78.36 ( $-69\%$ )	131.48 ( $-49\%$ )	82.83 ( $-68\%$ )	77.24 ( $-70\%$ )
MBPP	Score ↑	29.20	31.20 (+2.00)	29.40 (+0.20)	29.20 (0.00)	26.00 (-3.20)	31.20 (+2.00)
MBPP	Step ↓	256	256	70.05 ( $-73\%$ )	113.26 ( $-56\%$ )	71.79 ( $-72\%$ )	66.03 ( $-74\%$ )
BBH	Score ↑	53.11	48.55 (-4.56)	53.17 (+0.06)	53.03 (-0.08)	52.88 (-0.23)	56.78 (+3.67)
BBH	Step ↓	256	256	55.85 ( $-78\%$ )	98.04 ( $-62\%$ )	87.31 ( $-66\%$ )	51.48 ( $-80\%$ )

Table 1: Evaluation results of LLaDA-8B-Instruct on seven benchmarks. Values in parentheses denote the performance change relative to the baseline naive decoding and the reduction in decoding steps. The best performance and lowest decoding steps are highlighted in bold .

Task	Metric	Vanilla	PC-sampler	Fast-dLLM	KLASS	Saber	Ours
HumanEval	Score ↑	42.68	31.88 (-10.80)	43.29 (+0.61)	43.29 (+0.61)	42.07 (-0.61)	43.90 (+1.22)
HumanEval	Step ↓	256	256	75.88 ( $-70\%$ )	131.01 ( $-49\%$ )	82.07 ( $-68\%$ )	75.85 ( $-70\%$ )
MBPP	Score ↑	27.20	25.60 (-1.60)	28.00 (+0.80)	27.20 (0.00)	25.40 (-1.80)	28.60 (+1.40)
MBPP	Step ↓	256	256	73.01 ( $-71\%$ )	119.34 ( $-53\%$ )	77.70 ( $-70\%$ )	72.88 ( $-72\%$ )
BBH	Score ↑	50.35	47.03 (-3.32)	49.85 (-0.50)	50.28 (-0.07)	50.02 (-0.33)	51.90 (+1.55)
BBH	Step ↓	256	256	57.89 ( $-77\%$ )	101.11 ( $-61\%$ )	88.90 ( $-65\%$ )	56.00 ( $-78\%$ )

Table 2: Evaluation results of LLaDA-1.5 on seven benchmarks.

Task	Method	Score ↑	Speed-up ↑
MathVista-mini	MMaDA	32.90	1.00 $\times$
MathVista-mini	Ours	36.00 (+3.10)	2.37 $\times$
MATH-Vision	MMaDA	9.44	1.00 $\times$
MATH-Vision	Ours	10.00 (+0.56)	2.16 $\times$
ScienceQA	MMaDA	48.88	1.00 $\times$
ScienceQA	Ours	49.93 (+1.15)	2.70 $\times$

Table 3: Comparison of performance and inference speedup across different multi-modal benchmarks.

Task	Method	Score ↑	Step ↓
AlpacaEval	DIFFA	3.75	128
AlpacaEval	Ours	3.78 (+0.03)	80.19 ( $-37\%$ )
OpenBookQA	DIFFA	36.50	128
OpenBookQA	Ours	38.50 (+2.00)	28.67 ( $-78\%$ )
CommonEval	DIFFA	2.90	128
CommonEval	Ours	2.94 (+0.04)	72.45 ( $-44\%$ )

Table 4: Evaluation results of DIFFA on VoiceBench.

Figure 4: (a) Computational complexity of related operations; (b) Ablation study on

\varepsilon

; (c) Ablation study on

\mathcal{H}

Method	BBH		HumanEval
Method	Score $\uparrow$	Step $\downarrow$	Score $\uparrow$	Step $\downarrow$
LLaDA	53.28	1024	43.90	1024
PC-sampler	54.03	1024	42.07	1024
Fast-dLLM	53.68	65.54	42.68	184.79
KLASS	53.42	122.66	43.90	309.97
Saber	52.80	213.02	43.90	210.43

Table 5: Long-sequence results of LLaDA-8B-Instruct on two benchmarks (generation length 1024).

Task	Method	128		256		512
Task	Method	Score $\uparrow$	Step $\downarrow$	Score $\uparrow$	Step $\downarrow$	Score $\uparrow$	Step $\downarrow$
MBPP	LLaDA	39.00	128	29.20	256	16.20	512
MBPP	Ours	40.40	47.65	31.20	66.03	16.40	116.75
MMLU-Pro	LLaDA	37.42	128	35.57	256	37.46	512
MMLU-Pro	Ours	38.12	65.51	37.42	133.06	39.00	197.20
BBH	LLaDA	45.71	128	53.11	256	53.14	512

Table 6: Performance and inference speed on three benchmarks at different generation lengths.

Block	Method	MBPP		MMLU-Pro		BBH
length	Method	Score $\uparrow$	Step $\downarrow$	Score $\uparrow$	Step $\downarrow$	Score $\uparrow$	Step $\downarrow$
16	LLaDA	28.60	256	33.74	256	52.51	256
16	Ours	30.80	72.73	37.36	137.02	56.12	59.88
32	LLaDA	29.20	256	35.57	256	53.11	256
32	Ours	31.20	66.03	37.42	133.06	55.69	52.62
64	LLaDA	27.40	256	36.67	256	54.32	256

Table 7: Performance and inference speed on three benchmarks at different block lengths.

Method	16		32		64		128
Method	Score $\uparrow$	Speed Up $\uparrow$	Score $\uparrow$	Speed Up $\uparrow$	Score $\uparrow$	Speed Up $\uparrow$	Score $\uparrow$	Speed Up $\uparrow$
MMaDA	31.3	1.00 $\times$	32.1	1.00 $\times$	32.9	1.00 $\times$	32.5	1.00 $\times$
Ours	33.9	2.01 $\times$	34.0	2.08 $\times$	36.0	2.37 $\times$	32.9	2.04 $\times$

Table 8: Performance and inference speed on Mathvista-mini benchmark at different block lengths.

Method	temperature=0		temperature=0.1		temperature=0.4		temperature=0.7		temperature=1
Method	Score $\uparrow$	Step $\downarrow$	Score $\uparrow$	Step $\downarrow$	Score $\uparrow$	Step $\downarrow$	Score $\uparrow$	Step $\downarrow$	Score $\uparrow$	Step $\downarrow$
LLaDA	35.57	256	35.86	256	35.80	256	35.73	256	35.20	256
Ours	37.42	133.06	37.99	132.77	37.46	132.08	37.22	131.06	36.93	131.62

Table 9: Performance and decoding steps on MMLU-Pro under different temperature coefficients.

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Task	ICL
HumanEval	0-shot
MBPP	3-shot
BBH	3-shot
MMLU-Pro	0-shot
TruthfulQA	0-shot
Math	3-shot

Appendix

Table 10: In-context learning settings for different tasks.

Task	LLaDA-8B-Instruct		LLaDA-1.5
Task	$\epsilon$	$\mathcal{H}$	$\epsilon$	$\mathcal{H}$
HumanEval	0.02	6	0.01	6
MBPP	0.01	6	0.01	4
BBH	0.05	6	0.01	6
MMLU-Pro	0.01	6	0.01	6
TruthfulQA	0.01	6	0.01	6

Appendix

Table 11: Hyperparameters

\epsilon

and

\mathcal{H}

for different tasks.

Method	1024			2048
Method	Score $\uparrow$	TPF $\uparrow$	Step $\downarrow$	Score $\uparrow$	TPF $\uparrow$	Step $\downarrow$
LLaDA	43.90	1.00	1024	44.51	1.00	2048
PC-sampler	42.07	1.00	1024	41.46	1.00	2048
Fast-dLLM	42.68	5.54	184.79	44.51	10.50	195.13
KLASS	43.90	3.30	309.97	44.51	6.39	320.63
Saber	43.90	4.87	210.43	48.17	9.34	219.25

Appendix

Table 12: Long-sequence results of LLaDA-8B-Instruct on HumanEval benchmark.

Method	Score ↑	TPF ↑	Step ↓
LLaDA	79.53	1.00	256
PC-sampler	79.98 (+0.45)	1.00	256
Fast_dLLM	80.29 (+0.76)	3.41 ( $\times$ 3.41)	75.10 ( $-71\%$ )
KLASS	80.06 (+0.53)	1.99 ( $\times$ 1.99)	128.64 ( $-50\%$ )
Saber	79.61 (+0.08)	3.35 ( $\times$ 3.35)	76.49 ( $-70\%$ )
Ours	80.51 (+0.98)	3.41 ( $\times$ 3.41)	75.06 ( $-71\%$ )

Appendix

Table 13: Results of LLaDA - 8B - Instruct on the GSM8K - CoT benchmark.

Task	Method	128		256		512
Task	Method	Score $\uparrow$	Step $\downarrow$	Score $\uparrow$	Step $\downarrow$	Score $\uparrow$	Step $\downarrow$
MMLU-Pro	LLaDA	37.42	128	35.57	256	37.46	512
	PC-sampler	35.59 (-1.83)	128	33.90 (-1.67)	256	35.31 (-2.15)	512
	Fast_dllm	37.36 (-0.06)	67.72 ( $-47\%$ )	35.70 (+0.13)	134.76 ( $-47\%$ )	37.32 (-0.14)	199.31 ( $-61\%$ )
	KLASS	37.43 (+0.01)	88.96 ( $-31\%$ )	35.67 (+0.10)	187.17 ( $-27\%$ )	37.43 (-0.03)	293.96 ( $-43\%$ )
	Saber	36.48 (-0.94)	55.65 ( $-57\%$ )	36.10 (+0.53)	128.83 ( $-50\%$ )	37.16 (-0.30)	195.01 ( $-62\%$ )

Appendix

Table 14: Performance and steps on the MMLU-Pro and BBH benchmarks at various generation lengths.

Block	Method	MMLU-Pro		BBH
length	Method	Score $\uparrow$	Step $\downarrow$	Score $\uparrow$	Step $\downarrow$
16	LLaDA	33.74	256	52.51	256
	PC-sampler	33.98 (+0.24)	256	48.44 (-4.07)	256
	Fast_dllm	33.65 (-0.09)	138.76 ( $-46\%$ )	52.47 (-0.04)	63.37 ( $-75\%$ )
	KLASS	33.73 (-0.01)	192.61 ( $-25\%$ )	52.57 (+0.06)	112.90 ( $-56\%$ )
	Saber	34.55 (+0.81)	125.00 ( $-51\%$ )	51.64 (-0.87)	91.47 ( $-64\%$ )

Appendix

Table 15: Performance and inference speed on the MMLU-Pro and BBH benchmarks at different block lengths.

Diffusion Language Model Decoding

Diffusion language model decoding generates text through iterative parallel refinement, balancing fewer denoising steps with accuracy, coherence, and control.

See all 249 papers

STDec: Spatio-Temporal Stability Guided Decoding for dLLMs
Apr 7, 2026Yuzhe Chen, Jiale Cao, Xuyang Liu +3Diffusion Language Model Decoding Diffusion Language Models
1Tianjin University · 2Sichuan University · 3Chongqing University
Follow the Latent Roadmap: Navigating Revocable Decoding for Diffusion LLMs with Anchor Tokens
Jun 15, 2026Yizhen Yao, Qinglin Zhu, Runcong Zhao +4Diffusion Language Model Inference Diffusion Language Model Decoding
King’s College London · The Chinese University of Hong Kong · The Alan Turing Institute, UK
Stability-Weighted Decoding for Diffusion Language Models
Apr 18, 2026Yue Wu, Jian HuangDiffusion Language Model Decoding Diffusion Language Models
Department of Data Science and Artificial Intelligence, The Hong Kong Polytechnic University, Hong Kong, China.

Apr 7, 2026J/K move · Enter open · S save

STDec: Spatio-Temporal Stability Guided Decoding for dLLMs

Yuzhe Chen, Jiale Cao, Xuyang Liu, Jin Xie+2

1Tianjin University · 2Sichuan University · 3Chongqing University

Diffusion Language Model Decoding Diffusion Language Models LLM Inference Acceleration

Open paper AskPDF arXiv

Diffusion Large Language Models (dLLMs) have achieved rapid progress, viewed as a promising alternative to the autoregressive paradigm. However, most dLLM decoders still adopt a global confidence threshold, and do not explicitly model local context from neighboring decoded states or temporal consistency of predicted token IDs across steps. To address this issue, we propose a simple spatio-temporal stability guided decoding approach, named STDec. We observe strong spatio-temporal stability in dLLM decoding: newly decoded tokens tend to lie near decoded neighbors, and their predicted IDs often remain consistent across several denoising steps. Inspired by this stability, our STDec includes spatial-aware decoding and temporal-aware decoding. The spatial-aware decoding dynamically generates the token-adaptive threshold by aggregating the decoded states of nearby tokens. The temporal-aware decoding relaxes the decoding thresholds for tokens whose predicted token IDs remain consistent over denoising steps. Our STDec is training-free and remains compatible with cache-based acceleration methods. Across textual reasoning and multimodal understanding benchmarks, STDec substantially improves throughput while maintaining comparable task performance score. Notably, on MBPP with LLaDA, STDec achieves up to 14.17x speedup with a comparable score. Homepage: https://yzchen02.github.io/STDec.

Abstract

Diffusion Language Model Decoding

Explore similar work

STDec: Spatio-Temporal Stability Guided Decoding for dLLMs

Follow the Latent Roadmap: Navigating Revocable Decoding for Diffusion LLMs with Anchor Tokens

Stability-Weighted Decoding for Diffusion Language Models