STDec: Spatio-Temporal Stability Guided Decoding for dLLMs

Yuzhe Chen; Jiale Cao; Xuyang Liu; Jin Xie; Aiping Yang; Yanwei Pang

cs.CLApr 7, 2026

STDec: Spatio-Temporal Stability Guided Decoding for dLLMs

Authors: Yuzhe Chen, Jiale Cao, Xuyang Liu, Jin Xie, Aiping Yang, Yanwei Pang

Organizations: 1Tianjin University · 2Sichuan University · 3Chongqing University

Abstract

Diffusion Large Language Models (dLLMs) have achieved rapid progress, viewed as a promising alternative to the autoregressive paradigm. However, most dLLM decoders still adopt a global confidence threshold, and do not explicitly model local context from neighboring decoded states or temporal consistency of predicted token IDs across steps. To address this issue, we propose a simple spatio-temporal stability guided decoding approach, named STDec. We observe strong spatio-temporal stability in dLLM decoding: newly decoded tokens tend to lie near decoded neighbors, and their predicted IDs often remain consistent across several denoising steps. Inspired by this stability, our STDec includes spatial-aware decoding and temporal-aware decoding. The spatial-aware decoding dynamically generates the token-adaptive threshold by aggregating the decoded states of nearby tokens. The temporal-aware decoding relaxes the decoding thresholds for tokens whose predicted token IDs remain consistent over denoising steps. Our STDec is training-free and remains compatible with cache-based acceleration methods. Across textual reasoning and multimodal understanding benchmarks, STDec substantially improves throughput while maintaining comparable task performance score. Notably, on MBPP with LLaDA, STDec achieves up to 14.17x speedup with a comparable score. Homepage: https://yzchen02.github.io/STDec.

Figures & tables

Figure 1: Throughput comparison with some existing acceleration approaches. Our STDec achieves strong throughput across eight benchmarks, covering textual reasoning tasks with LLaDA and multimodal understanding tasks with LaViDa.

Figure 2: Comparison with existing decoding strategies. In (a)–(c), we show three decoding strategies: top- $k$ ( e.g., top-1) decoding Nie et al. (2025b) , anchor-based decoding Kong et al. (2025) and our STDec.

Panel of Figure 3 — Figure 3: Empirical evidence of spatio-temporal stability in dLLM decoding. (i) Spatial stability. We show the percentage of at least $S$ decoded tokens around masked tokens in a radius of 2 in (a). (ii) Temporal stability. In (b), we present the percentage of decoded tokens already having consistent IDs at least $K$ previous consecutive steps, and also show the averaged confidence score at first ID-stable step and decoded step. On HumanEval and GSM8K, the currently decoded tokens are usually surrounded by already decoded tokens and exhibit stable IDs at previous steps. We call this spatio-temporal stability.

Figure 4: Overview of our STDec. The left gives the overall decoding with our STDec in the dLLM, where STDec progressively decodes tokens across denoising steps. The middle gives the details of our STDec that calculates the token-adaptive thresholds by combining spatial and temporal stability information.

Task	Method	TPS $\uparrow$	Speed $\uparrow$	Score $\uparrow$
Code
MBPP	Vanilla Dream	6.57	1.00 $\times$	51.40
	+ Half-Step	13.11	2.00 $\times$	35.80
	+ dKV-Cache	11.12	1.69 $\times$	49.60
	+ Fast-dLLM (Parallel)	56.88	8.66 $\times$	54.80
	+ LocalLeap	63.65	9.69 $\times$	53.60

Table 1: Comparison on textual reasoning with Dream-7B-Instruct. We report TPS and score. Bold TPS and speedup values indicate the fastest, and grayed scores denote drops relative to original Dream.

Task	Method	TPS $\uparrow$	Speed $\uparrow$	Score $\uparrow$
Code
MBPP	Vanilla LLaDA	5.68	1.00 $\times$	37.60
	+ Half-Step	11.51	2.02 $\times$	34.80
	+ dKV-Cache	10.18	1.79 $\times$	39.00
	+ Fast-dLLM (Parallel)	62.68	11.04 $\times$	37.40
	+ LocalLeap	76.48	13.46 $\times$	37.60

Table 2: Comparison on textual reasoning with LLaDA-8B-Instruct. We report TPS and score. Bold TPS and speedup values indicate the fastest, and grayed scores denote drops relative to original LLaDA.

Task	Method	TPS $\uparrow$	Speed $\uparrow$	Score $\uparrow$
MathVerse	LaViDa w/o Prefix-DLM	5.71	1.00 $\times$	28.30
	+ Prefix-DLM	12.22	2.14 $\times$	27.03
	+ Fast-dLLM (Parallel)	11.94	2.09 $\times$	28.68
	+ LocalLeap	14.31	2.51 $\times$	28.30
	+ STDec	18.22	3.19 $\times$	28.30
MathVision	LaViDa w/o Prefix-DLM	5.34	1.00 $\times$	19.74

Table 3: Comparison on multimodal understanding with LaViDa-Reason. We report TPS and generation score. Bold TPS and speedup values indicate the fastest, and grayed scores denote drops relative to original LaViDa.

Panel of Figure 5 — Figure 5: Combination with cache-based approaches. Our STDec can be integrated with cache-based approaches dKV-Cache Ma et al. (2025) and Prefix-DLM Li et al. (2025b) , which can accelerate the inference speed almost without loss of generation quality.

Task	Method	TPS $\uparrow$	Speed $\uparrow$	Score $\uparrow$
GPQA	LLaDA Instruct	3.53	1.00 $\times$	31.82
GPQA	+ STDec	168.73	47.80 $\times$	31.82
GSM8K	LLaDA Instruct	2.79	1.00 $\times$	77.74
GSM8K	+ STDec	30.58	10.96 $\times$	77.75
MBPP	LLaDA Instruct	4.08	1.00 $\times$	38.20
MBPP	+ STDec	78.32	19.20 $\times$	37.20

Table 4: Comparison on long-sequence generation . We perform the experiment using LLaDA-8B-Instruct.

Setting	TPS $\uparrow$	Speed $\uparrow$	Score $\uparrow$
Baseline	10.82	1.00 $\times$	48.17
+ Temporal	32.89	3.04 $\times$	48.17
+ Spatial	46.77	4.32 $\times$	48.17
+ Both	52.92	4.89 $\times$	48.78

Table 5: Impact of integrating our STDec modules. We integrate our spatial-aware decoding and temporal-aware decoding into the baseline LLaDA-8B-Instruct.

Panel of Figure 6 — Figure 6: Impact of hyperparameters in our STDec, including initial high-value thresholds $\tau_{\text{high}}$ , initial low-value thresholds $\tau_{\text{low}}$ , Gaussian smoothing factor $\sigma$ , and the relaxation constant factor $\alpha$ . The red markers denote our default setting, which achieves a $4.89\times$ speedup over vanilla LLaDA with comparable score.

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Benchmark	Lenth $L$	Steps $T$	Block Size $B$
MBPP	512	512	32
HumanEval	512	512	32
GPQA	128	128	64
GSM8K	256	256	32
MATH	512	512	64

Appendix

Table 6: Evaluation Setup for Textual Reasoning Tasks. We report the detail configuration including generation length

L

, denoising steps

T

and block size

B

used in the main experiments.

Benchmark	Length $L$	Steps $T$	Block Size $B$
MathVerse	512	512	512
MathVision	512	512	512
MathVista	512	512	512

Appendix

Table 7: Evaluation Setup for Multimodal Understanding Tasks. We report the detail configuration including generation length

L

, denoising steps

T

and block size

B

used in the main experiments.

Benchmark	Method	TPS $\uparrow$	Speed $\uparrow$	Score $\uparrow$
GenEval	Lumina-DiMOO	50.47	1.00 $\times$	0.85
GenEval	+ STDec	178.41	3.53 $\times$	0.87
DPGBench	Lumina-DiMOO	49.50	1.00 $\times$	83.38
DPGBench	+ STDec	121.92	2.46 $\times$	85.28

Appendix

Table 8: Comparison on image generation. We integrate our method into Lumina-DiMOO for text-to-image generation.

Panel of Figure 7 — Figure 7: Impact of different hyperparameters in our STDec on GSM8K, including masked-token factor $\tau_{\text{high}}$ , decoded-token factor $\tau_{\text{low}}$ , Gaussian smoothing factor $\sigma$ , and the relaxation constant factor $\alpha$ . The red markers denote our default setting, which achieves a $4.04\times$ speedup over vanilla LLaDA.

Setting	TPS $\uparrow$	Speed $\uparrow$	Score $\uparrow$
Baseline	4.19	1.00 $\times$	78.01
+ Temporal	10.1	2.41 $\times$	79.08
+ Spatial	14.61	3.49 $\times$	78.70
+ Both	16.94	4.04 $\times$	78.01

Appendix

Table 9: Impact of integrating our STDec modules on GSM8K. We integrate our spatial-aware decoding and temporal-aware decoding into the baseline LLaDA.

Smoothing Function	TPS $\uparrow$	Speed $\uparrow$	Score $\uparrow$
Baseline	10.82	1.00 $\times$	48.17
Mean	58.55	5.41 $\times$	40.85
Triangular	57.13	5.28 $\times$	47.56
Gaussian	52.92	4.89 $\times$	48.78

Appendix

Table 10: Smoothing function ablation for spatial smoothing in STDec on HumanEval. We evaluate alternative local smoothing kernels with LLaDA-8B-Instruct, reporting throughput (TPS), TPS speedup over vanilla LLaDA, and task score.

Smoothing Function	TPS $\uparrow$	Speed $\uparrow$	Score $\uparrow$
Baseline	4.19	1.00 $\times$	78.01
Mean	18.45	4.40 $\times$	77.10
Triangular	17.98	4.29 $\times$	77.33
Gaussian	16.94	4.04 $\times$	78.01

Appendix

Table 11: Smoothing function ablation for spatial smoothing in STDec on GSM8K. We repeat the same kernel swap study on GSM8K with LLaDA-8B-Instruct and show the corresponding TPS, speedup, and score.

Figure 8: Case study on multimodal understanding with LaViDa-Reason. Given an input image, LaViDa-Reason generates a detailed scene description. Applying our STDec preserves the semantic content and relative object relations in the response, while reducing decoding time from 99.17 s to 34.41 s ( $2.88\times$ speedup).

Model	Question 1: Lily can run 12 kilometers per hour for 4 hours. After that, she runs 6 kilometers per hour. How many kilometers can she run in 8 hours?
Dream-7B-Instruct 11.59 secs	Lily can run 12 kilometers per hour for 4 hours, so she can run 12 * 4 = 48 kilometers in those 4 hours. After that, she runs 6 kilometers per hour for the next 4 hours, so she can run 6 * 4 = 24 kilometers in those 4 hours. Thus, in 8 hours, she can run 48 + 24 = 72 kilometers. So Lily can run 72 kilometers in 8 hours.
+ STDec 2.30 secs (5.03 $\times$ )	Lily can run 12 kilometers per hour for 4 hours, so she runs 12 * 4 = 48 kilometers in the first 4 hours. After that, she runs 6 kilometers per hour for the next 4 hours, so she runs 6 * 4 = 24 kilometers in the next 4 hours. Thus, in 8 hours, she can run 48 + 24 = 72 kilometers. So Lily can run 72 kilometers in 8 hours.
Question 2: Joy can read 8 pages of a book in 20 minutes. How many hours will it take her to read 120 pages?
Dream-7B-Instruct 11.25 secs	To find out how many hours it will take Joy to read 120 pages, we first need to determine her reading speed in pages per minute. Joy can read 8 pages in 20 minutes, so her reading speed is 8 pages / 20 minutes = 0.4 pages per minute. Now, we need to find out how many minutes it will take her to read 120 pages. Since her reading speed is 0.4 pages per minute, it will take her 120 pages / 0.4 pages per minute = 300 minutes to read 120 pages. Finally, we need to convert the time from minutes to hours. There are 60 minutes in an hour, so 300 minutes is equal to 300 minutes / 60 minutes per hour = 5 hours. Thus, it will take Joy 5 hours to read 120 pages.
+ STDec 3.59 secs (3.13 $\times$ )	To find out how many hours it will take Joy to read 120 pages, we first need to determine her reading speed in pages per minute. Joy can read 8 pages in 20 minutes, so her reading speed is 8 pages / 20 minutes = 0.4 pages per minute. Now, we need to find out how many minutes it will take her to read 120 pages. We can do this by dividing the total number of pages by her reading speed: 120 pages / 0.4 pages per minute = 300 minutes. Finally, we need to convert the time from minutes to hours. Since there are 60 minutes in an hour, we divide the number of minutes by 60: 300 minutes / 60 minutes per hour = 5 hours. So, it will take Joy 5 hours to read 120 pages.

Appendix

Table 12: Case study on GSM8K with Dream-7B-Instruct. Our STDec substantially reduces decoding latency while keeping the generated solution consistent with the baseline output.

Model	Question 1: Lily can run 12 kilometers per hour for 4 hours. After that, she runs 6 kilometers per hour. How many kilometers can she run in 8 hours?
LLaDA-8B-Instruct 12.04 secs	To determine how many kilometers Lily can run in 8 hours, we need to break down the problem into two parts: the distance she runs in the first 4 hours and the distance she runs in the next 4 hours. First, let’s calculate the distance she runs in the first 4 hours: - She runs 12 kilometers per hour for 4 hours. - Distance = Speed × Time - Distance in the first 4 hours = 12 km/h × 4 h = 48 km Next, let’s calculate the distance she runs in the next 4 hours: - She runs 6 kilometers per hour for 4 hours. - Distance = Speed × Time - Distance in the next 4 hours = 6 km/h × 4 h = 24 km Now, we add the distances from both parts to find the total distance she can run in 8 hours: - Total distance = Distance in the first 4 hours + Distance in the next 4 hours - Total distance = 48 km + 24 km = 72 km Therefore, Lily can run a total of 72 kilometers in 8 hours.
+ STDec 2.55 secs (4.72 $\times$ )	To determine how many kilometers Lily can run in 8 hours, we need to break down the problem into two parts: the distance she runs in the first 4 hours and the distance she runs in the next 4 hours. First, let’s calculate the distance she runs in the first 4 hours: - Speed: 12 kilometers per hour - Time: 4 hours - Distance = Speed × Time - Distance = 12 km/h × 4 h - Distance = 48 kilometers Next, let’s calculate the distance she runs in the next 4 hours: - Speed: 6 kilometers per hour - Time: 4 hours - Distance = Speed × Time - Distance = 6 km/h × 4 h - Distance = 24 kilometers Now, we add the distances from both parts to find the total distance she can run in 8 hours: - Total distance = Distance in the first 4 hours + Distance in the next 4 hours - Total distance = 48 km + 24 km - Total distance = 72 kilometers Therefore, Lily can run 72 kilometers in 8 hours.
Question 2: Joy can read 8 pages of a book in 20 minutes. How many hours will it take her to read 120 pages?
LLaDA-8B-Instruct 11.94 secs	To determine how many hours it will take Joy to read 120 pages, we need to follow these steps: Step 1: Calculate Joy’s reading rate in pages per minute. Joy reads 8 pages in 20 minutes. Therefore, her reading rate is: $\frac{8\text{ pages}}{20\text{ minutes}}=0.4\text{ pages per minute}$ Step 2: Determine the total time required to read 120 pages. Since Joy’s reading rate is 0.4 pages per minute, the time required to read 120 pages is: $\frac{120\text{ pages}}{0.4\text{ pages per minute}}=300\text{ minutes}$ Step 3: Convert the total time from minutes to hours. There are 60 minutes in an hour, so: $\frac{300\text{ minutes}}{60\text{ minutes per hour}}=5\text{ hours}$ Therefore, it will take Joy 5 hours to read 120 pages.
+ STDec 2.60 secs (4.59 $\times$ )	To determine how many hours it will take Joy to read 120 pages, we need to follow these steps: Step 1: Calculate Joy’s reading rate in pages per minute. Joy reads 8 pages in 20 minutes. Therefore, her reading rate is: $\text{Reading rate}=\frac{8\text{ pages}}{20\text{ minutes}}=0.4\text{ pages per minute}$ Step 2: Determine the total time required to read 120 pages. Since Joy reads at a rate of 0.4 pages per minute, the total time required to read 120 pages is: $\text{Total time}=\frac{120\text{ pages}}{0.4\text{ pages per minute}}=300\text{ minutes}$ Step 3: Convert the total time from minutes to hours. $\text{Total time in hours}=\frac{300\text{ minutes}}{60\text{ minutes per hour}}=5\text{ hours}$ Final answer: 5

Appendix

Table 13: Case study on GSM8K with LLaDA-8B-Instruct. We apply our STDec to speed up decoding while preserving the original reasoning and final answer.

Diffusion Language Model Decoding

Diffusion language model decoding generates text through iterative parallel refinement, balancing fewer denoising steps with accuracy, coherence, and control.

See all 249 papers

Efficient Diffusion LLMs via Temporal-Spatial Parallel Decoding and Confidence Extrapolation
May 29, 2026Zekai Li, Ji Liu, Yiqing Huang +3Adaptive Computation for Diffusion Models Diffusion Language Model Decoding
Advanced Micro Devices, Inc. (AMD)
$R^2$ -dLLM: Accelerating Diffusion Large Language Models via Spatio-Temporal Redundancy Reduction
Apr 21, 2026Zhenbang Du, Kejing Xia, Xinrui Zhong +6Diffusion Language Model Inference Diffusion Model Fine-Tuning
Georgia Institute of Technology · NVIDIA
Stability-Weighted Decoding for Diffusion Language Models
Apr 18, 2026Yue Wu, Jian HuangDiffusion Language Model Decoding Diffusion Language Models
Department of Data Science and Artificial Intelligence, The Hong Kong Polytechnic University, Hong Kong, China.

May 29, 2026J/K move · Enter open · S save

Efficient Diffusion LLMs via Temporal-Spatial Parallel Decoding and Confidence Extrapolation

Zekai Li, Ji Liu, Yiqing Huang, Ziqiong Liu+2

Advanced Micro Devices, Inc. (AMD)

Adaptive Computation for Diffusion Models Diffusion Language Model Decoding Diffusion Language Models Diffusion Model Inference Acceleration

Open paper AskPDF arXiv

Diffusion-based large language models (dLLMs) support parallel text generation via iterative denoising, yet inference remains latency-heavy because many steps are spent on redundant refinement and repeated remasking of tokens whose final values are already determined. Prior acceleration methods mainly depend on step-local confidence heuristics or fixed schedules, which are sensitive to prompt and task variation and ignore strong positional effects within a sequence. We cast diffusion decoding as a dynamic control problem and show that token-wise denoising trajectories provide the key signal for reliable control. We propose a trace-aware decoding framework with two components. First, Temporal-Spatial Parallel Decoding (TSPD) uses a lightweight temporalspatial controller that consumes per-token trajectory features, including confidence, entropy, and momentum, together with token position, to decide when a token has converged and can be safely fixed. Second, we introduce Confidence Extrapolation (CE), a training-free state-space module that forecasts future logit trends with uncertainty to support proactive decisions, including safe look-ahead and targeted stabilization when trajectories are oscillatory or underconfident. Together, TSPD and CE reduce unnecessary denoising iterations while preserving output quality, and they compose cleanly with system optimizations such as KV caching.

Abstract

Diffusion Language Model Decoding

Explore similar work

Efficient Diffusion LLMs via Temporal-Spatial Parallel Decoding and Confidence Extrapolation

R2R^2R2-dLLM: Accelerating Diffusion Large Language Models via Spatio-Temporal Redundancy Reduction

Stability-Weighted Decoding for Diffusion Language Models

$R^2$ -dLLM: Accelerating Diffusion Large Language Models via Spatio-Temporal Redundancy Reduction