Diffusion Large Language Models (dLLMs) have emerged as a promising paradigm for parallel token generation, with block-wise variants garnering significant research interest. Despite their potential, existing dLLMs typically suffer from a rigid accuracy-parallelism trade-off: increasing the number of tokens per forward (TPF) via aggressive parallel decoding often leads to performance degradation and increased generation instability. We identify that this limitation stems from the model's inability to navigate high-parallelism regimes where approximation errors and local corruptions accumulate, ultimately undermining the reliability of parallel generation. To address this, we propose LightningRL, a post-training framework designed to directly optimize the speed-quality Pareto frontier of pre-trained dLLMs. Instead of forcing uniform parallelization, our approach leverages reinforcement learning to identify and reinforce high-parallelism trajectories that maintain generation accuracy. Built upon the Group Relative Policy Optimization (GRPO) framework, LightningRL introduces several enhancements tailored for dLLMs: (1) stabilized training via per-reward decoupled normalization; (2) token-level negative log-likelihood (NLL) regularization on correct trajectories to anchor model performance; and (3) a dynamic sampling strategy with TPF-aware filtering to enhance training efficiency. Experimental results across mathematical and coding benchmarks demonstrate that LightningRL consistently advances the Pareto frontier, achieving competitive task accuracy while significantly increasing parallelism, reaching an average TPF of 7.32 (with a peak of 11.10 on the MBPP dataset). Our code is available at https://github.com/SJTU-DENG-Lab/LightningRL.
Figures & tables
Figure 2: Overview of LightningRL. LightningRL samples a group of decoding trajectories per prompt, applies per-reward decoupled normalization to preserve within-group ranking under heterogeneous scales. The policy is optimized with a GRPO-style objective plus a token-level NLL anchor. The bottom panel shows the resulting shift toward the fastest correct trajectory, improving TPF without degrading accuracy.
Figure 3: Per-reward decoupled normalization improves training stability. It reduces signal collapse (a) and yields more stable reward optimization (b) under the same training setup.
Figure 3
Model
GSM8K
MATH500
MBPP
HumanEval
Acc (%)
TPF
AUP
Acc (%)
TPF
AUP
Acc (%)
TPF
AUP
Acc (%)
TPF
AUP
SDAR ( Cheng et al., 2025 )
88.9
2.85
252.5
63.6
4.81
318.9
58.0
2.44
108.4
73.5
2.39
165.0
Coupled-GRPO ( Gong et al., 2025 )
75.3
4.22
254.7
59.1
4.93
283.1
54.0
2.61
139.9
68.2
2.42
164.6
TraceRL ( Wang et al., 2025b )
76.9
5.04
378.6
60.6
4.82
284.6
57.8
2.50
144.2
75.0
2.29
171.6
DiRL ( Zhu et al., 2026 )
86.6
4.87
414.1
61.6
5.04
301.2
56.8
2.70
152.7
76.0
2.33
176.7
LightningRL
90.3
5.58
507.5
63.0
6.28
409.2
58.3
11.10
412.9
72.6
6.30
421.5
Table 1: Evaluation results of LightningRL and methods on math and code benchmarks. All methods are trained on the SDAR 8B model with block size 32. Compared with the other three algorithms and the SDAR base model, LightningRL delivers substantial improvements across both math and code benchmarks.
Figure 5
Model
GSM8K
MATH500
MBPP
HumanEval
Acc (%)
TPF
AUP
Acc (%)
TPF
AUP
Acc (%)
TPF
AUP
Acc (%)
TPF
AUP
dLLMs
Dream ( Ye et al., 2025 )
83.9
1.00
83.9
39.6
1.00
39.6
57.2
1.00
57.2
55.2
1.00
55.2
Fast-dLLM-Dream ( Wu et al., 2025b )
79.0
1.44
110.4
38.3
1.78
49.0
53.2
1.20
63.6
54.3
1.33
63.2
dParallel-Dream ( Chen et al., 2025 )
82.1
3.02
215.2
38.7
2.94
62.6
55.4
2.24
108.4
54.3
2.57
97.1
d3LLM-Dream ( Qian et al., 2026 )
81.4
4.94
333.6
38.2
3.92
74.6
55.6
2.96
142.1
57.1
3.20
126.6
Table 2: Evaluation results of LightningRL and the baselines on math and code benchmarks. The notation LightningRL-8B-b32 denotes the 8B LightningRL model with a block size of 32. LightningRL consistently advances the speed–quality frontier over its SDAR baseline and prior diffusion and autoregressive baselines, achieving substantially higher AUP at comparable accuracy.
Table 7
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Method
Acc (%)
TPF
AUP
w/ Value Model
80.7
5.14
408.1
LightningRL
90.3
5.58
492.4
Appendix
Table 5: Comparison of Performance with and without Value Model Incorporation on GSM8K. The AUP in this table is computed using each model’s own accuracy at TPF ≈ 1 as ymax .
Scale
BS
Model
GSM8K
MATH500
MBPP
HumanEval
Acc (%)
TPF
AUP
Acc (%)
TPF
AUP
Acc (%)
TPF
AUP
Acc (%)
TPF
AUP
1.7B
32
SDAR
71.5
2.48
176.4
41.2
5.62
226.3
39.0
3.86
149.8
48.8
2.81
136.0
LightningRL
71.7
3.40
251.4
41.2
6.01
241.2
37.3
5.07
185.8
48.2
3.43
165.2
4B
32
SDAR
86.6
3.10
243.8
53.6
5.09
263.9
54.0
1.96
105.9
59.8
3.67
216.8
LightningRL
85.4
4.37
374.7
56.4
6.55
358.7
52.2
3.05
158.3
57.3
4.35
242.3
8B
32
SDAR
88.9
2.85
252.5
63.6
4.81
299.5
58.0
2.44
81.1
73.5
2.39
123.8
Appendix
Table 6: Comparison of SDAR and LightningRL under identical settings. The results are grouped by Model Scale and Block Size (BS) to facilitate direct comparison across four datasets. The AUP in this table is computed using each model’s own accuracy at TPF ≈ 1 as ymax .
Model
H100 TPS
TPF
Acc (%)
Qwen-2.5-7B-it
57.3
1.00
74.1
Fast-dLLM-v2
150.0
2.21
77.5
dParallel-LLaDA
172.2
5.14
72.6
d3LLM-LLaDA
288.9
9.11
73.1
SDAR
105.6
2.85
88.9
LightningRL
336.0
5.58
90.3
Appendix
Table 7: Tokens Per Second (TPS) performance comparison on H100 GPUs. We report on GSM8K datasets. LightningRL achieves superior throughput.
Figure 8: Training dynamics and Accuracy-Parallelism Trade-off on GSM8K. LightningRL avoids the objective drift and reward collapse observed in TraceRL, while sustaining higher decoding throughput and more synchronized termination.
Diffusion-based large language models (dLLMs) have shown promising performance across various reasoning tasks, establishing themselves as an alternative to autoregressive large language models (LLMs). Unlike autoregressive LLMs that generate one token per step based on all previous tokens, dLLMs theoretically enable parallel generation of multiple tokens at each decoding step. However, recent dLLMs still favor one-token-per-step generation in practice, as directly decoding multiple masked tokens often leads to degraded generation quality and stability. This reveals a substantial gap between the theoretical parallelism and practical performance of dLLMs. To bridge this gap, we introduce an adaptive parallel decoding approach, namely DiCo, which features a three-phase divide-and-conquer paradigm to unleash the inherent parallelism of dLLMs. During the Divide phase, DiCo first explores the input masked sequence and identifies masked tokens as seed tokens, which are then expanded to construct a set of local clusters. During the Conquer phase, DiCo performs parallel decoding across different local clusters constructed in the Divide phase. The divide-and-conquer process repeatedly alternates between the Divide and Conquer phases until convergence. During the Finalize phase, DiCo decodes the remaining few masked tokens using an effective fine-grained compound decoding scheme to finalize the generation. Extensive experiments demonstrate that DiCo can achieve significant inference speedups while maintaining competitive generation quality.
Xiangzhong Luo, Yilin An, Zhicheng Yu +2
Southeast University · Nanyang Technological University
Diffusion Large Language Models (dLLMs) offer a promising alternative to autoregressive models, excelling in text generation tasks due to their bidirectional attention mechanisms. However, their computational complexity scales on the order of L cubed with the sequence length L. This poses significant challenges for long-sequence and real-time applications, primarily due to the lack of compatibility with key-value caching and the non-autoregressive nature of denoising steps. Existing acceleration methods rely on static caching or parallel decoding strategies, which fail to account for the dynamic behavior of token properties across layers and decoding steps. We propose Dynamic-dLLM, a training-free framework that enhances dLLM inference efficiency through two components: Dynamic Cache Updating (DCU), which adaptively allocates cache-update budgets based on layer-wise token dynamics, and Adaptive Parallel Decoding (APD), which dynamically calibrates decoding thresholds to balance generation quality and efficiency. Extensive experiments on models like LLaDA-8B-Instruct, LLaDA-1.5, and Dream-v0-7B-Instruct across benchmarks such as MMLU, GSM8K, and HumanEval demonstrate that Dynamic-dLLM significantly improves inference speed. It attains an average speedup exceeding 3 times while maintaining performance. Dynamic-dLLM outperforms state-of-the-art acceleration methods and provides a plug-and-play solution for efficient dLLM deployment without compromising performance. The code is available at https://github.com/TianyiWu233/DYNAMIC-DLLM.
Tianyi Wu, Xiaoxi Sun, Yanhua Jiao +5
*Equal contribution. The work is done during Tianyi’s internship at CBG Celia DeviceAI Team, Huawei. · 1Harbin Institute of Technology, Shenzhen · 2Huawei +1
Diffusion Large Language Models (dLLMs) have emerged as a competitive alternative to autoregressive language models, offering the potential for substantially faster inference through parallel decoding. Existing parallel decoding schedulers typically commit positions only after they meet a per-position criterion, overlooking how early commitments may benefit subsequent decoding. We identify a ripple effect in dLLM decoding: proactively committing a mid-entropy pivot position can induce a pronounced reduction in uncertainty across the remaining masked positions. This uncertainty reduction allows subsequent steps to unmask more tokens in parallel, thereby accelerating the overall decoding process. To exploit the ripple effect, we propose Ripple-Pivot Search (RPS), a novel training-free decoding method that seeks mid-entropy positions as promising candidate pivots (where to decode), and determines their token assignment that yields the greatest downstream benefit via lookahead evaluation (what to decode). Across 3 dLLMs and 4 reasoning and code-generation benchmarks, RPS achieves 4-10× wall-clock speedup over the standard decoder while preserving generation quality, and improves accuracy over the previous lookahead baseline by up to 5.49% while delivering higher throughput in most settings. When integrated with KV caching, RPS further achieves up to 18× wall-clock speedup over the standard decoder.
Yushi Ye, Xu Chen, Haoyun Jiang +7
Cooperative Medianet Innovation Center, Shanghai Jiao Tong University · Alibaba Group