Divide and Conquer: Accelerating Diffusion-Based Large Language Models via Adaptive Parallel Decoding

Xiangzhong Luo; Yilin An; Zhicheng Yu; Weichen Liu; Xu Yang

cs.CLFeb 27, 2026

Divide and Conquer: Accelerating Diffusion-Based Large Language Models via Adaptive Parallel Decoding

Authors: Xiangzhong Luo, Yilin An, Zhicheng Yu, Weichen Liu, Xu Yang

Organizations: Southeast University · Nanyang Technological University

Abstract

Diffusion-based large language models (dLLMs) have shown promising performance across various reasoning tasks, establishing themselves as an alternative to autoregressive large language models (LLMs). Unlike autoregressive LLMs that generate one token per step based on all previous tokens, dLLMs theoretically enable parallel generation of multiple tokens at each decoding step. However, recent dLLMs still favor one-token-per-step generation in practice, as directly decoding multiple masked tokens often leads to degraded generation quality and stability. This reveals a substantial gap between the theoretical parallelism and practical performance of dLLMs. To bridge this gap, we introduce an adaptive parallel decoding approach, namely DiCo, which features a three-phase divide-and-conquer paradigm to unleash the inherent parallelism of dLLMs. During the Divide phase, DiCo first explores the input masked sequence and identifies masked tokens as seed tokens, which are then expanded to construct a set of local clusters. During the Conquer phase, DiCo performs parallel decoding across different local clusters constructed in the Divide phase. The divide-and-conquer process repeatedly alternates between the Divide and Conquer phases until convergence. During the Finalize phase, DiCo decodes the remaining few masked tokens using an effective fine-grained compound decoding scheme to finalize the generation. Extensive experiments demonstrate that DiCo can achieve significant inference speedups while maintaining competitive generation quality.

Figures & tables

Figure 1: Comparisons between autoregressive LLMs and non-autoregressive dLLMs, where each box denotes a specific token.

Figure 2: Illustration of the performance collapse of naive parallel decoding on GSM8K ( left ) and Math-500 ( right ).

Figure 3: Visualization of two representative decoding trajectories of LLaDA-8B-Instruct on GSM8K. The blue points represent the predicted confidence of masked tokens, while the red points denote unmasked tokens. (best viewed when zoomed in)

Figure 4: Visualization of attention patterns in LLaDA-8B-Instruct during the decoding process (accumulated on GSM8K).

Dataset	Method	LLaDA-8B-Instruct ( Nie et al., 2025 )				Dream-7B-Instruct ( Ye et al., 2025 )
		non-AR		semi-AR		non-AR		semi-AR
		Accuracy	Throughput	Accuracy	Throughput	Accuracy	Throughput	Accuracy	Throughput
GSM8K 4-shot	Vanilla (baseline)	56.33	6.94 (1.0 $\times$ )	73.16	6.97 (1.0 $\times$ )	52.01	8.07 (1.0 $\times$ )	35.25	7.91 (1.0 $\times$ )
	Fast-dLLM ( Wu et al., 2025b )	57.01	16.18 (2.3 $\times$ )	72.86	18.53 (2.7 $\times$ )	50.80	17.10 (2.1 $\times$ )	33.51	19.44 (2.5 $\times$ )
	DiCo (ours)	75.13	23.46 (3.4 $\times$ )	75.21	25.63 (3.7 $\times$ )	63.38	31.83 (3.9 $\times$ )	62.93	30.12 (3.8 $\times$ )
Math-500 4-shot	Vanilla (baseline)	25.20	8.77 (1.0 $\times$ )	37.00	8.82 (1.0 $\times$ )	3.80	8.76 (1.0 $\times$ )	21.40	8.97 (1.0 $\times$ )

Table 1: Comprehensive evaluation results on LLaDA-8B-Instruct and Dream-7B-Instruct under both non-AR and semi-AR settings. Bold numbers indicate the best results and numbers in brackets denote the corresponding speedup ratios compared to the Vanilla baseline.

Figure 5: Ablation results of the number of seek tokens $N$ on HumanEval under the non-AR ( left ) and semi-AR ( right ) settings.

Figure 6: Fixed confidence-based parallel decoding vs. adaptive parallel decoding under non-AR ( left ) / semi-AR ( right ) settings.

Figure 7: Visualization of the decoding trajectory of Fast-dLLM ( left , finished in 87 steps) and DiCo ( right , finished in 67 steps).

Method	non-AR		semi-AR
Method	Accuracy	Throughput	Accuracy	Throughput
Vanilla	56.33	6.94 (1.0 $\times$ )	73.16	6.97 (1.0 $\times$ )
DiCo w/o TG	49.28	34.00 (4.9 $\times$ )	68.92	29.60 (4.2 $\times$ )
DiCo w/ TG	75.13	23.46 (3.4 $\times$ )	75.21	25.63 (3.7 $\times$ )

Table 2: Ablation results of trajectory guidance. In this table, TG is the abbreviation of trajectory guidance.

Method	non-AR		semi-AR
Method	Accuracy	Throughput	Accuracy	Throughput
Vanilla	56.33	6.94 (1.0 $\times$ )	73.16	6.97 (1.0 $\times$ )
DiCo w/o LM	74.45	19.45 (2.8 $\times$ )	75.13	22.53(3.2 $\times$ )
DiCo w/ LM	75.13	23.46 (3.4 $\times$ )	75.21	25.63(3.7 $\times$ )

Table 3: Ablation results of fine-grained compound decoding. In this table, LM is the abbreviation of logit margin.

Diffusion Language Model Inference

Diffusion language model inference studies how to generate, guide, and serve text efficiently through iterative denoising and parallel token updates.

See all 165 papers

Jan 25, 2026

Streaming-dLLM: Accelerating Diffusion LLMs via Suffix Pruning and Dynamic Decoding

Diffusion Large Language Models (dLLMs) offer a compelling paradigm for natural language generation, leveraging parallel decoding and bidirectional attention to achieve superior global coherence compared to autoregressive models. While recent works have accelerated inference via KV cache reuse or heuristic decoding, they overlook the intrinsic inefficiencies within the block-wise diffusion process. Specifically, they suffer from spatial redundancy by modeling informative-sparse suffix regions uniformly and temporal inefficiency by applying fixed denoising schedules across all the decoding process. To address this, we propose Streaming-dLLM, a training-free framework that streamlines inference across both spatial and temporal dimensions. Spatially, we introduce attenuation guided suffix modeling to approximate the full context by pruning redundant mask tokens. Temporally, we employ a dynamic confidence aware strategy with an early exit mechanism, allowing the model to skip unnecessary iterations for converged tokens. Extensive experiments show that Streaming-dLLM achieves up to 68.2X speedup while maintaining generation quality, highlighting its effectiveness in diffusion decoding. The code is available at https://github.com/xiaoshideta/Streaming-dLLM.

Zhongyu Xiao, Zhiwei Hao, Jianyuan Guo +4

School of information and Electronics, Beijing Institute of Technology, Beijing · Department of Computer Science, City University of Hong Kong, Hong Kong · School of Computer Science, Wuhan University, Wuhan +2

Adaptive Inference Blockwise Diffusion Language Models Diffusion Language Model Inference Token Pruning Diffusion Language Model Decoding Diffusion Model Inference Acceleration

Ask PDF

Aug 12, 2026

Ripple-Pivot Search: Active Parallel Decoding for Diffusion Large Language Models

Diffusion Large Language Models (dLLMs) have emerged as a competitive alternative to autoregressive language models, offering the potential for substantially faster inference through parallel decoding. Existing parallel decoding schedulers typically commit positions only after they meet a per-position criterion, overlooking how early commitments may benefit subsequent decoding. We identify a ripple effect in dLLM decoding: proactively committing a mid-entropy pivot position can induce a pronounced reduction in uncertainty across the remaining masked positions. This uncertainty reduction allows subsequent steps to unmask more tokens in parallel, thereby accelerating the overall decoding process. To exploit the ripple effect, we propose Ripple-Pivot Search (RPS), a novel training-free decoding method that seeks mid-entropy positions as promising candidate pivots (where to decode), and determines their token assignment that yields the greatest downstream benefit via lookahead evaluation (what to decode). Across 3 dLLMs and 4 reasoning and code-generation benchmarks, RPS achieves 4-10

\times

wall-clock speedup over the standard decoder while preserving generation quality, and improves accuracy over the previous lookahead baseline by up to 5.49% while delivering higher throughput in most settings. When integrated with KV caching, RPS further achieves up to 18

\times

wall-clock speedup over the standard decoder.

Yushi Ye, Xu Chen, Haoyun Jiang +7

Cooperative Medianet Innovation Center, Shanghai Jiao Tong University · Alibaba Group

Language Model Decoding Diffusion Language Model Inference Parallel Decoding Diffusion Language Model Decoding

Ask PDF

Sep 15, 2026

Early-Bird Decoding: Accelerating Diffusion LLMs with Learnable Block Sizes and Parallel Sampling

Diffusion large language models (dLLMs) offer a promising parallel decoding paradigm as an alternative to autoregressive generation through iterative unmasking. However, dLLMs typically require many steps before token confidence reaches the decoding threshold, resulting in inefficient inference even with block-wise KV caching. To accelerate dLLM inference, we for the first time propose an "early-bird (EB)" decoding framework, motivated by the observation that tokens with similarly low entropy tend to cluster and can be jointly decoded earlier, before reaching the confidence threshold. In particular, our EB-Decode framework integrates two key enablers: (1) a learnable network that adaptively groups tokens with similar uncertainty into variable-length blocks, rather than relying on fixed block sizes; (2) a position-aware sampler that learns to unmask tokens in parallel using fewer decoding steps within predicted variable-length blocks. Both components are developed without modifying pretrained dLLM weights and can therefore be directly deployed as plug-ins during serving, with negligible training and inference overhead. Extensive experiments across three models and four benchmarks consistently validate our observation and the effectiveness of EB-Decode, achieving 3.53-18.76

\times

higher throughput than the vanilla decoding method and up to 1.58

\times

higher throughput over the strongest baseline, Fast-dLLM, with comparable accuracy.

Lixuan Wei, Wei Zhou, Jianwen Wu +3

Harvard University · Georgia Institute of Technology · Purdue University

Diffusion Language Model Inference Parallel Decoding Diffusion Language Model Decoding

Ask PDF

Abstract

Diffusion Language Model Inference

Explore similar work

Streaming-dLLM: Accelerating Diffusion LLMs via Suffix Pruning and Dynamic Decoding

Ripple-Pivot Search: Active Parallel Decoding for Diffusion Large Language Models

Early-Bird Decoding: Accelerating Diffusion LLMs with Learnable Block Sizes and Parallel Sampling