Breaking Block Boundaries: Anchor-based History-stable Decoding for Diffusion Large Language Models

Shun Zou; Yong Wang; Zehui Chen; Lin Chen; Chongyang Tao; Feng Zhao; Xiangxiang Chu

cs.CLApr 10, 2026

Breaking Block Boundaries: Anchor-based History-stable Decoding for Diffusion Large Language Models

Authors: Shun Zou, Yong Wang, Zehui Chen, Lin Chen, Chongyang Tao, Feng Zhao, Xiangxiang Chu

Organizations: MoE Key Lab of BIPC, University of Science and Technology of China · AMAP, Alibaba Group

Abstract

Diffusion Large Language Models (dLLMs) have recently become a promising alternative to autoregressive large language models (ARMs). Semi-autoregressive (Semi-AR) decoding is widely employed in base dLLMs and advanced decoding strategies due to its superior performance. However, our observations reveal that Semi-AR decoding suffers from inherent block constraints, which cause the decoding of many cross-block stable tokens to be unnecessarily delayed. To address this challenge, we systematically investigate the identification of stable tokens and present three key findings: (1) naive lookahead decoding is unreliable, (2) token stability closely correlates with convergence trend, and (3) historical information is isolated. Building on these insights, we propose Anchor-based History-stable Decoding (AHD), a training-free, plug-and-play dynamic decoding strategy. Specifically, AHD monitors the stability trend of tokens in real time through dynamic anchors. Once a token reaches stability, it initiates early cross-block decoding to enhance efficiency and performance. Extensive experiments across language, vision-language, and audio-language domains demonstrate that AHD simultaneously improves both performance and inference efficiency. Notably, AHD effectively reverses the performance degradation typically observed in existing advanced decoding acceleration strategies. For instance, on the BBH benchmark, our approach reduces decoding steps by 80% while improving performance by 3.67%.

Figures & tables

Figure 1: Systematic analysis of Semi-AR decoding on dLLMs. (a) Heatmap visualization on a sample. Stars indicate the earliest step at which cross-block tokens reach stability, while black squares denote the finally unmasked step. For instance, position 65 is expected to be decoded no earlier than step 64; however, it stabilizes as early as step 53. (b) Qualitative analysis of cross - block delayed decoding. (c) Distribution of the first stable step.

Figure 2: Dynamics of confidence during the decoding. (a) Local fluctuations persist even after the token has become stable. (b) Prior to stabilization, the confidence remains flat or oscillatory; however, after crossing the first stable step, it exhibits a marked upward trend.

Figure 3: Illustration of Anchor - based History - stable Decoding (AHD). AHD retrospectively tracks historical trajectories from anchor points to dynamically monitor the absolute stability trend. Cross - block stable tokens are unlocked early in the absolute stability trend, reducing decoding steps while further unleashing the potential of parallel diffusion decoding and achieving significant performance gains.

Task	Metric	Vanilla	PC-sampler	Fast-dLLM	KLASS	Saber	Ours
HumanEval	Score ↑	40.85	40.24 (-0.61)	41.46 (+0.61)	40.85 (0.00)	36.59 (-4.26)	43.29 (+2.44)
HumanEval	Step ↓	256	256	78.36 ( $-69\%$ )	131.48 ( $-49\%$ )	82.83 ( $-68\%$ )	77.24 ( $-70\%$ )
MBPP	Score ↑	29.20	31.20 (+2.00)	29.40 (+0.20)	29.20 (0.00)	26.00 (-3.20)	31.20 (+2.00)
MBPP	Step ↓	256	256	70.05 ( $-73\%$ )	113.26 ( $-56\%$ )	71.79 ( $-72\%$ )	66.03 ( $-74\%$ )
BBH	Score ↑	53.11	48.55 (-4.56)	53.17 (+0.06)	53.03 (-0.08)	52.88 (-0.23)	56.78 (+3.67)
BBH	Step ↓	256	256	55.85 ( $-78\%$ )	98.04 ( $-62\%$ )	87.31 ( $-66\%$ )	51.48 ( $-80\%$ )

Table 1: Evaluation results of LLaDA-8B-Instruct on seven benchmarks. Values in parentheses denote the performance change relative to the baseline naive decoding and the reduction in decoding steps. The best performance and lowest decoding steps are highlighted in bold .

Task	Metric	Vanilla	PC-sampler	Fast-dLLM	KLASS	Saber	Ours
HumanEval	Score ↑	42.68	31.88 (-10.80)	43.29 (+0.61)	43.29 (+0.61)	42.07 (-0.61)	43.90 (+1.22)
HumanEval	Step ↓	256	256	75.88 ( $-70\%$ )	131.01 ( $-49\%$ )	82.07 ( $-68\%$ )	75.85 ( $-70\%$ )
MBPP	Score ↑	27.20	25.60 (-1.60)	28.00 (+0.80)	27.20 (0.00)	25.40 (-1.80)	28.60 (+1.40)
MBPP	Step ↓	256	256	73.01 ( $-71\%$ )	119.34 ( $-53\%$ )	77.70 ( $-70\%$ )	72.88 ( $-72\%$ )
BBH	Score ↑	50.35	47.03 (-3.32)	49.85 (-0.50)	50.28 (-0.07)	50.02 (-0.33)	51.90 (+1.55)
BBH	Step ↓	256	256	57.89 ( $-77\%$ )	101.11 ( $-61\%$ )	88.90 ( $-65\%$ )	56.00 ( $-78\%$ )

Table 2: Evaluation results of LLaDA-1.5 on seven benchmarks.

Task	Method	Score ↑	Speed-up ↑
MathVista-mini	MMaDA	32.90	1.00 $\times$
MathVista-mini	Ours	36.00 (+3.10)	2.37 $\times$
MATH-Vision	MMaDA	9.44	1.00 $\times$
MATH-Vision	Ours	10.00 (+0.56)	2.16 $\times$
ScienceQA	MMaDA	48.88	1.00 $\times$
ScienceQA	Ours	49.93 (+1.15)	2.70 $\times$

Table 3: Comparison of performance and inference speedup across different multi-modal benchmarks.

Task	Method	Score ↑	Step ↓
AlpacaEval	DIFFA	3.75	128
AlpacaEval	Ours	3.78 (+0.03)	80.19 ( $-37\%$ )
OpenBookQA	DIFFA	36.50	128
OpenBookQA	Ours	38.50 (+2.00)	28.67 ( $-78\%$ )
CommonEval	DIFFA	2.90	128
CommonEval	Ours	2.94 (+0.04)	72.45 ( $-44\%$ )

Table 4: Evaluation results of DIFFA on VoiceBench.

Figure 4: (a) Computational complexity of related operations; (b) Ablation study on

\varepsilon

; (c) Ablation study on

\mathcal{H}

Method	BBH		HumanEval
Method	Score $\uparrow$	Step $\downarrow$	Score $\uparrow$	Step $\downarrow$
LLaDA	53.28	1024	43.90	1024
PC-sampler	54.03	1024	42.07	1024
Fast-dLLM	53.68	65.54	42.68	184.79
KLASS	53.42	122.66	43.90	309.97
Saber	52.80	213.02	43.90	210.43

Table 5: Long-sequence results of LLaDA-8B-Instruct on two benchmarks (generation length 1024).

Task	Method	128		256		512
Task	Method	Score $\uparrow$	Step $\downarrow$	Score $\uparrow$	Step $\downarrow$	Score $\uparrow$	Step $\downarrow$
MBPP	LLaDA	39.00	128	29.20	256	16.20	512
MBPP	Ours	40.40	47.65	31.20	66.03	16.40	116.75
MMLU-Pro	LLaDA	37.42	128	35.57	256	37.46	512
MMLU-Pro	Ours	38.12	65.51	37.42	133.06	39.00	197.20
BBH	LLaDA	45.71	128	53.11	256	53.14	512

Table 6: Performance and inference speed on three benchmarks at different generation lengths.

Block	Method	MBPP		MMLU-Pro		BBH
length	Method	Score $\uparrow$	Step $\downarrow$	Score $\uparrow$	Step $\downarrow$	Score $\uparrow$	Step $\downarrow$
16	LLaDA	28.60	256	33.74	256	52.51	256
16	Ours	30.80	72.73	37.36	137.02	56.12	59.88
32	LLaDA	29.20	256	35.57	256	53.11	256
32	Ours	31.20	66.03	37.42	133.06	55.69	52.62
64	LLaDA	27.40	256	36.67	256	54.32	256

Table 7: Performance and inference speed on three benchmarks at different block lengths.

Method	16		32		64		128
Method	Score $\uparrow$	Speed Up $\uparrow$	Score $\uparrow$	Speed Up $\uparrow$	Score $\uparrow$	Speed Up $\uparrow$	Score $\uparrow$	Speed Up $\uparrow$
MMaDA	31.3	1.00 $\times$	32.1	1.00 $\times$	32.9	1.00 $\times$	32.5	1.00 $\times$
Ours	33.9	2.01 $\times$	34.0	2.08 $\times$	36.0	2.37 $\times$	32.9	2.04 $\times$

Table 8: Performance and inference speed on Mathvista-mini benchmark at different block lengths.

Method	temperature=0		temperature=0.1		temperature=0.4		temperature=0.7		temperature=1
Method	Score $\uparrow$	Step $\downarrow$	Score $\uparrow$	Step $\downarrow$	Score $\uparrow$	Step $\downarrow$	Score $\uparrow$	Step $\downarrow$	Score $\uparrow$	Step $\downarrow$
LLaDA	35.57	256	35.86	256	35.80	256	35.73	256	35.20	256
Ours	37.42	133.06	37.99	132.77	37.46	132.08	37.22	131.06	36.93	131.62

Table 9: Performance and decoding steps on MMLU-Pro under different temperature coefficients.

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Task	ICL
HumanEval	0-shot
MBPP	3-shot
BBH	3-shot
MMLU-Pro	0-shot
TruthfulQA	0-shot
Math	3-shot

Appendix

Table 10: In-context learning settings for different tasks.

Task	LLaDA-8B-Instruct		LLaDA-1.5
Task	$\epsilon$	$\mathcal{H}$	$\epsilon$	$\mathcal{H}$
HumanEval	0.02	6	0.01	6
MBPP	0.01	6	0.01	4
BBH	0.05	6	0.01	6
MMLU-Pro	0.01	6	0.01	6
TruthfulQA	0.01	6	0.01	6

Appendix

Table 11: Hyperparameters

\epsilon

and

\mathcal{H}

for different tasks.

Method	1024			2048
Method	Score $\uparrow$	TPF $\uparrow$	Step $\downarrow$	Score $\uparrow$	TPF $\uparrow$	Step $\downarrow$
LLaDA	43.90	1.00	1024	44.51	1.00	2048
PC-sampler	42.07	1.00	1024	41.46	1.00	2048
Fast-dLLM	42.68	5.54	184.79	44.51	10.50	195.13
KLASS	43.90	3.30	309.97	44.51	6.39	320.63
Saber	43.90	4.87	210.43	48.17	9.34	219.25

Appendix

Table 12: Long-sequence results of LLaDA-8B-Instruct on HumanEval benchmark.

Method	Score ↑	TPF ↑	Step ↓
LLaDA	79.53	1.00	256
PC-sampler	79.98 (+0.45)	1.00	256
Fast_dLLM	80.29 (+0.76)	3.41 ( $\times$ 3.41)	75.10 ( $-71\%$ )
KLASS	80.06 (+0.53)	1.99 ( $\times$ 1.99)	128.64 ( $-50\%$ )
Saber	79.61 (+0.08)	3.35 ( $\times$ 3.35)	76.49 ( $-70\%$ )
Ours	80.51 (+0.98)	3.41 ( $\times$ 3.41)	75.06 ( $-71\%$ )

Appendix

Table 13: Results of LLaDA - 8B - Instruct on the GSM8K - CoT benchmark.

Task	Method	128		256		512
Task	Method	Score $\uparrow$	Step $\downarrow$	Score $\uparrow$	Step $\downarrow$	Score $\uparrow$	Step $\downarrow$
MMLU-Pro	LLaDA	37.42	128	35.57	256	37.46	512
	PC-sampler	35.59 (-1.83)	128	33.90 (-1.67)	256	35.31 (-2.15)	512
	Fast_dllm	37.36 (-0.06)	67.72 ( $-47\%$ )	35.70 (+0.13)	134.76 ( $-47\%$ )	37.32 (-0.14)	199.31 ( $-61\%$ )
	KLASS	37.43 (+0.01)	88.96 ( $-31\%$ )	35.67 (+0.10)	187.17 ( $-27\%$ )	37.43 (-0.03)	293.96 ( $-43\%$ )
	Saber	36.48 (-0.94)	55.65 ( $-57\%$ )	36.10 (+0.53)	128.83 ( $-50\%$ )	37.16 (-0.30)	195.01 ( $-62\%$ )

Appendix

Table 14: Performance and steps on the MMLU-Pro and BBH benchmarks at various generation lengths.

Block	Method	MMLU-Pro		BBH
length	Method	Score $\uparrow$	Step $\downarrow$	Score $\uparrow$	Step $\downarrow$
16	LLaDA	33.74	256	52.51	256
	PC-sampler	33.98 (+0.24)	256	48.44 (-4.07)	256
	Fast_dllm	33.65 (-0.09)	138.76 ( $-46\%$ )	52.47 (-0.04)	63.37 ( $-75\%$ )
	KLASS	33.73 (-0.01)	192.61 ( $-25\%$ )	52.57 (+0.06)	112.90 ( $-56\%$ )
	Saber	34.55 (+0.81)	125.00 ( $-51\%$ )	51.64 (-0.87)	91.47 ( $-64\%$ )

Appendix

Table 15: Performance and inference speed on the MMLU-Pro and BBH benchmarks at different block lengths.

Diffusion Language Model Decoding

Diffusion language model decoding generates text through iterative parallel refinement, balancing fewer denoising steps with accuracy, coherence, and control.

See all 249 papers

Apr 7, 2026

STDec: Spatio-Temporal Stability Guided Decoding for dLLMs

Diffusion Large Language Models (dLLMs) have achieved rapid progress, viewed as a promising alternative to the autoregressive paradigm. However, most dLLM decoders still adopt a global confidence threshold, and do not explicitly model local context from neighboring decoded states or temporal consistency of predicted token IDs across steps. To address this issue, we propose a simple spatio-temporal stability guided decoding approach, named STDec. We observe strong spatio-temporal stability in dLLM decoding: newly decoded tokens tend to lie near decoded neighbors, and their predicted IDs often remain consistent across several denoising steps. Inspired by this stability, our STDec includes spatial-aware decoding and temporal-aware decoding. The spatial-aware decoding dynamically generates the token-adaptive threshold by aggregating the decoded states of nearby tokens. The temporal-aware decoding relaxes the decoding thresholds for tokens whose predicted token IDs remain consistent over denoising steps. Our STDec is training-free and remains compatible with cache-based acceleration methods. Across textual reasoning and multimodal understanding benchmarks, STDec substantially improves throughput while maintaining comparable task performance score. Notably, on MBPP with LLaDA, STDec achieves up to 14.17x speedup with a comparable score. Homepage: https://yzchen02.github.io/STDec.

Yuzhe Chen, Jiale Cao, Xuyang Liu +3

1Tianjin University · 2Sichuan University · 3Chongqing University

Diffusion Language Model Decoding Diffusion Language Models LLM Inference Acceleration

Ask PDF

Jun 15, 2026

Follow the Latent Roadmap: Navigating Revocable Decoding for Diffusion LLMs with Anchor Tokens

Diffusion Large Language Models (dLLMs) offer a promising avenue for parallel generation but face a trade-off between decoding speed and quality. While revocable decoding strategies attempt to mitigate errors by verifying and remasking tokens, they typically operate within a mixed-quality context. This leads to two critical failures: Error Propagation, where new tokens absorb toxic information from erroneous context, and Local Error Reinforcement, where errors mutually reinforce each other to evade detection. To alleviate these challenges, we propose ASRD (Anchor Supervised Revocable Decoding), a training-free framework that operates within the embedding space. ASRD explicitly decouples the decoding context into trusted Anchor Tokens, which are identified via temporal consistency, and uncertain candidates. Leveraging a dynamic Anchor Tokens Cache, we introduce two complementary mechanisms: (1) Anchor-Guided Generation, which injects entropy-weighted anchor signals into masked positions to implicitly rectify attention toward the reliable global skeleton; and (2) Anchor-Perturbed Verification, which applies orthogonal perturbations to uncertain candidate tokens, destabilizing and remasking errors driven by fragile local consensus. Extensive experiments on math and coding benchmarks demonstrate that ASRD outperforms recent remasking baselines, achieving accuracy improvements of up to 6.4% while accelerating inference throughput by up to 7.2

\times

.The code is available at https://github.com/preordinary/ASRD.

Yizhen Yao, Qinglin Zhu, Runcong Zhao +4

King’s College London · The Chinese University of Hong Kong · The Alan Turing Institute, UK

Diffusion Language Model Inference Diffusion Language Model Decoding Diffusion Language Models Diffusion Model Inference Acceleration

Ask PDF

Apr 18, 2026

Stability-Weighted Decoding for Diffusion Language Models

Diffusion large language models (dLLMs) enable parallel text generation by iteratively denoising a fully masked sequence, unmasking a subset of masked tokens at each step. Existing decoding strategies rely on static confidence metrics computed at a single denoising step, ignoring temporal history and often leading to premature unmasking of unstable tokens. In this work, we theoretically establish that a token's temporal instability, quantified by the KL divergence between consecutive prediction distributions, provides a strict lower bound on its mutual information with the remaining masked context, indicating that temporally unstable tokens are inherently unsafe to unmask. Based on this insight, we propose Stability-Weighted Decoding (SWD), a training-free, plug-and-play strategy that incorporates temporal stability into token scoring and acts as a universal modulator for arbitrary score-based decoding policies. Experiments on code generation and mathematical reasoning benchmarks demonstrate that SWD consistently improves generation accuracy across representative scoring metrics and selection policies, and exhibits exceptional robustness, maintaining a significant performance lead over standard baselines across varying acceleration ratios.

Yue Wu, Jian Huang

Department of Data Science and Artificial Intelligence, The Hong Kong Polytechnic University, Hong Kong, China.

Diffusion Language Model Decoding Diffusion Language Models

Ask PDF

Abstract

Diffusion Language Model Decoding

Explore similar work

STDec: Spatio-Temporal Stability Guided Decoding for dLLMs

Follow the Latent Roadmap: Navigating Revocable Decoding for Diffusion LLMs with Anchor Tokens

Stability-Weighted Decoding for Diffusion Language Models