Batch-Paced: Fixed-Budget Batch Decoding in Discrete Diffusion Language Models

Shuai Zhang; Zhanchao Zhou; Huachuan Qiu; Anqi Li; Zhenzhong Lan

cs.CLOct 13, 2026

Batch-Paced: Fixed-Budget Batch Decoding in Discrete Diffusion Language Models

Authors: Shuai Zhang, Zhanchao Zhou, Huachuan Qiu, Anqi Li, Zhenzhong Lan

Organizations: Zhejiang University · Westlake University

Abstract

As generative language models become more capable and widely used, inference demand grows, making efficient serving increasingly important. Discrete diffusion language models (dLLMs) offer a promising path to more efficient inference through parallel token generation. Prior dLLM decoding methods, however, focus on improving inference efficiency for individual sequences. We formulate fixed-budget dLLM batch decoding and introduce Batch-Paced, a training-free method that jointly optimizes the allocation of decoding compute across requests to improve overall batch quality. Across two dLLM backbones, five benchmarks and their representative mixtures, Batch-Paced improves batch performance under matched budgets; gains generally grow with batch size and largely persist across decoding configurations. Analysis of requests' decoding trajectories suggests that these gains arise largely by raising the committed-token confidence of lower-confidence requests. We further show that Batch-Paced's batch decode time remains competitive and that adding a decoding-depth allowance during Batch-Paced decoding can shorten it further at a modest performance cost.

Figures & tables

Figure 1: Batch-Paced uses global compute allocation within the batch rather than independent sequence-wise allocation. After a denoising forward pass assigns confidence scores to masked-token predictions, Uniform commits a fixed number of positions per sequence, whereas Batch-Paced ranks positions across the batch.

Benchmark	LLaDA-8B-Instruct					LLaDA 1.5
	Avg. budget=64		Avg. budget=128		Full=256	Avg. budget=64		Avg. budget=128		Full=256
	Uniform baseline	Batch- Paced	Uniform baseline	Batch- Paced	Full reference	Uniform baseline	Batch- Paced	Uniform baseline	Batch- Paced	Full reference
GSM8K	$67.8_{\scriptscriptstyle\,0.0}$	$77.5_{\scriptscriptstyle\,1.1}^{\ast}$	$76.2_{\scriptscriptstyle\,0.0}$	$81.0_{\scriptscriptstyle\,0.5}^{\ast}$	$81.2_{\scriptscriptstyle\,0.0}$	$70.0_{\scriptscriptstyle\,1.0}$	$80.5_{\scriptscriptstyle\,0.8}^{\ast}$	$77.7_{\scriptscriptstyle\,0.0}$	$83.8_{\scriptscriptstyle\,0.6}^{\ast}$	$83.4_{\scriptscriptstyle\,0.0}$
MATH500	$25.7_{\scriptscriptstyle\,1.0}$	$32.1_{\scriptscriptstyle\,2.9}^{\ast}$	$33.2_{\scriptscriptstyle\,1.7}$	$35.5_{\scriptscriptstyle\,2.1}^{\ast}$	$35.9_{\scriptscriptstyle\,1.7}$	$27.4_{\scriptscriptstyle\,0.6}$	$32.2_{\scriptscriptstyle\,1.6}^{\ast}$	$33.1_{\scriptscriptstyle\,0.6}$	$36.4_{\scriptscriptstyle\,1.3}^{\ast}$	$36.8_{\scriptscriptstyle\,0.8}$
HumanEval	$18.4_{\scriptscriptstyle\,0.3}$	$29.3_{\scriptscriptstyle\,1.2}^{\ast}$	$27.0_{\scriptscriptstyle\,0.3}$	$38.4_{\scriptscriptstyle\,0.4}^{\ast}$	$41.0_{\scriptscriptstyle\,0.3}$	$21.3_{\scriptscriptstyle\,0.0}$	$34.6_{\scriptscriptstyle\,0.9}^{\ast}$	$31.0_{\scriptscriptstyle\,0.3}$	$43.2_{\scriptscriptstyle\,0.9}^{\ast}$	$45.7_{\scriptscriptstyle\,0.0}$
MBPP	$16.9_{\scriptscriptstyle\,0.6}$	$29.3_{\scriptscriptstyle\,1.2}^{\ast}$	$29.3_{\scriptscriptstyle\,0.8}$	$36.2_{\scriptscriptstyle\,0.6}^{\ast}$	$37.9_{\scriptscriptstyle\,0.0}$	$15.9_{\scriptscriptstyle\,1.0}$	$27.0_{\scriptscriptstyle\,1.2}^{\ast}$	$26.6_{\scriptscriptstyle\,1.2}$	$39.8_{\scriptscriptstyle\,0.9}^{\ast}$	$41.8_{\scriptscriptstyle\,0.0}$

Table 1: Batch-Paced improves batch-decoding performance over Uniform at a fixed compute budget. The table compares both policies on individual benchmarks and mixtures across both backbones and budget levels.

Figure 2: Batch-Paced’s largest confidence shifts and performance gains occur in the low-confidence group. The panels show (a) variation in commit allocation across requests and forwards, (b) confidence shifts by Uniform-confidence group, and (c) paired performance differences between Batch-Paced and Uniform.

Figure 3: Batch-Paced gives requests different commit schedules within a fixed-budget batch. Matched panels show the same illustrative mixed

B=16

batch with requests aligned row by row; the shared center separates Uniform’s leftward and Batch-Paced’s rightward token-progress axes. Segment width, height, color, and row-end marks encode per-forward token count, mean confidence, commit size, and final correctness, respectively.

Factor	Setting	LLaDA-8B-Instruct Avg. budget / gen. length		LLaDA 1.5 Avg. budget / gen. length
Factor	Setting	$1/4$	$1/2$	$1/4$	$1/2$
Block length	$H=64$	$43.0_{0.0}/\mathbf{53.8}_{1.5}$	$52.1_{0.0}/\mathbf{58.1}_{1.0}$	$47.0_{0.0}/\mathbf{58.7}_{1.4}$	$55.1_{1.5}/\mathbf{62.4}_{0.7}$
Block length	$H=256$	$43.3_{0.0}/\mathbf{46.6}_{1.5}$	$\mathbf{50.0}_{0.0}/48.3_{0.4}$	$40.7_{0.4}/\mathbf{48.8}_{1.1}$	$49.1_{0.0}/\mathbf{52.8}_{0.5}$
Generation length	$L=128$	$43.0_{0.0}/\mathbf{50.9}_{2.1}$	$52.4_{0.0}/\mathbf{56.5}_{0.4}$	$43.6_{0.0}/\mathbf{52.1}_{1.2}$	$53.4_{0.0}/\mathbf{57.7}_{0.3}$
Generation length	$L=512$	$44.0_{1.2}/\mathbf{59.1}_{2.2}$	$57.1_{1.2}/\mathbf{61.3}_{0.8}$	$45.0_{0.9}/\mathbf{59.6}_{2.1}$	$60.5_{0.3}/\mathbf{63.4}_{0.8}$
Confidence-score function	Top-1/2 margin	$45.2_{1.5}/\mathbf{57.3}_{1.5}^{*}$	$53.4_{0.0}/\mathbf{59.6}_{0.4}^{*}$	$45.4_{0.0}/\mathbf{57.6}_{1.7}^{*}$	$57.3_{0.0}/\mathbf{61.7}_{0.4}^{*}$

Table 2: Batch-Paced largely retains its performance advantage across decoding configurations. The table compares Uniform and Batch-Paced on GSM8K–HumanEval across both models and two budget fractions, varying block and generation lengths and confidence-score functions; entries list Uniform/Batch-Paced mean accuracy.

Figure 4: Batch-Paced’s performance advantage over Uniform generally grows with batch size. Curves show paired accuracy gains across single-task and mixed-task settings for both models at both budgets.

Benchmark	LLaDA-8B-Instruct				LLaDA 1.5
	Avg. budget=64		Avg. budget=128		Avg. budget=64		Avg. budget=128
	Fast- dLLM	Batch- Paced	Fast- dLLM	Batch- Paced	Fast- dLLM	Batch- Paced	Fast- dLLM	Batch- Paced
HumanEval	$36.6_{\scriptscriptstyle\,1.9}^{*}$	$29.3_{\scriptscriptstyle\,1.2}$	$40.9_{\scriptscriptstyle\,0.0}^{*}$	$38.4_{\scriptscriptstyle\,0.4}$	$33.0_{\scriptscriptstyle\,11.5}$	$34.6_{\scriptscriptstyle\,0.9}$	$45.7_{\scriptscriptstyle\,0.0}^{*}$	$43.2_{\scriptscriptstyle\,0.9}$
IFEval	$39.6_{\scriptscriptstyle\,1.9}$	$43.8_{\scriptscriptstyle\,1.7}^{*}$	$50.5_{\scriptscriptstyle\,1.9}$	$51.7_{\scriptscriptstyle\,0.8}$	$40.4_{\scriptscriptstyle\,1.6}$	$47.7_{\scriptscriptstyle\,0.8}^{*}$	$51.7_{\scriptscriptstyle\,0.7}$	$54.5_{\scriptscriptstyle\,0.7}^{*}$
GSM8K	$78.8_{\scriptscriptstyle\,0.4}$	$77.5_{\scriptscriptstyle\,1.1}$	$81.2_{\scriptscriptstyle\,0.0}$	$81.0_{\scriptscriptstyle\,0.5}$	$81.6_{\scriptscriptstyle\,0.9}$	$80.5_{\scriptscriptstyle\,0.8}$	$83.3_{\scriptscriptstyle\,0.1}$	$83.8_{\scriptscriptstyle\,0.6}$
GSM8K + IFEval	$46.0_{\scriptscriptstyle\,2.7}$	$58.2_{\scriptscriptstyle\,1.2}^{*}$	$68.8_{\scriptscriptstyle\,0.7}$	$68.3_{\scriptscriptstyle\,1.0}$	$45.0_{\scriptscriptstyle\,3.7}$	$62.9_{\scriptscriptstyle\,0.8}^{*}$	$71.2_{\scriptscriptstyle\,0.9}$	$71.7_{\scriptscriptstyle\,0.4}$

Table 3: Fast-dLLM and Batch-Paced are broadly comparable. The table compares their accuracy on individual benchmarks and representative mixtures across both backbones and two target denoising budgets.

Figure 5: Depth limits trade Batch-Paced performance for shorter decode time. The plot compares Uniform, Fast-dLLM, and original and depth-limited Batch-Paced across task mixtures and budgets; larger circles denote larger depth allowances. Batch-Paced generally outperforms Uniform at comparable decode time; tighter allowances move it toward Fast-DLLM’s runtime; larger ones recover the original policy’s performance and runtime.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

LLaDA-8B-Instruct
Ratio	Avg. budget=64		Avg. budget=128
	Uniform	Batch-	Uniform	Batch-
	baseline	Paced	baseline	Paced
1:7	$28.3_{\scriptscriptstyle\,1.4}$	$40.9_{\scriptscriptstyle\,0.9}^{*}$	$34.2_{\scriptscriptstyle\,0.9}$	$48.4_{\scriptscriptstyle\,2.1}^{*}$
1:3	$34.4_{\scriptscriptstyle\,1.0}$	$48.8_{\scriptscriptstyle\,2.7}^{*}$	$43.0_{\scriptscriptstyle\,0.8}$	$56.7_{\scriptscriptstyle\,1.6}^{*}$
1:1	$46.1_{\scriptscriptstyle\,0.0}$	$58.4_{\scriptscriptstyle\,1.7}^{*}$	$55.5_{\scriptscriptstyle\,0.0}$	$66.7_{\scriptscriptstyle\,0.9}^{*}$

Appendix

Table 4: Batch-Paced improves performance across task-mixture ratios. The table compares both policies across five GSM8K–HumanEval ratios, backbones, and target budgets.

Figure 6: Batch-Paced outperforms Uniform at every tested average budget. Accuracy as the average denoising budget varies across task settings and backbones.

Panel of Figure 7 — Figure 7: LLaDA-8B-Instruct shows the same pattern, with its largest confidence shifts and performance gains in the low-confidence group. The panels show (a) variation in commit allocation across requests and forwards, (b) confidence shifts by Uniform-confidence group, and (c) paired performance differences between Batch-Paced and Uniform.

Benchmark	Method	LLaDA-8B-Instruct		LLaDA 1.5
		$\bar{T}=64$	$\bar{T}=128$	$\bar{T}=64$	$\bar{T}=128$
HumanEval	Fast-dLLM	$36.6_{\scriptscriptstyle\,1.9}$	$40.9_{\scriptscriptstyle\,0.0}$	$33.0_{\scriptscriptstyle\,11.5}$	$45.7_{\scriptscriptstyle\,0.0}$
	Fast-dLLM + separate sequence budgets	$39.9_{\scriptscriptstyle\,1.5}$	$40.2_{\scriptscriptstyle\,0.0}$	$41.3_{\scriptscriptstyle\,2.1}$	$45.1_{\scriptscriptstyle\,0.0}$
	Fast-dLLM + shared batch budget	$19.9_{\scriptscriptstyle\,10.7}$	$40.9_{\scriptscriptstyle\,0.0}$	$32.7_{\scriptscriptstyle\,11.2}$	$45.7_{\scriptscriptstyle\,0.0}$
	Batch-Paced (ours)	$29.3_{\scriptscriptstyle\,1.2}$	$38.4_{\scriptscriptstyle\,0.4}$	$34.6_{\scriptscriptstyle\,0.9}$	$43.2_{\scriptscriptstyle\,0.9}$
IFEval	Fast-dLLM	$39.6_{\scriptscriptstyle\,1.9}$	$50.5_{\scriptscriptstyle\,1.9}$	$40.4_{\scriptscriptstyle\,1.6}$	$51.7_{\scriptscriptstyle\,0.7}$

Appendix

Table 5: Batch-Paced usually outperforms the budget-enforcement variants on benchmark mixtures. The table compares Fast-dLLM, its two runtime budget-enforcement variants, and Batch-Paced on individual benchmarks and mixtures across both backbones and two average denoising budgets.

Figure 10: Depth limits trade Batch-Paced performance for shorter decode time on LLaDA-8B-Instruct. The plot compares Uniform, Fast-dLLM, and original and depth-limited Batch-Paced on the same three task mixtures and target budgets as Figure 5 ; larger circles denote larger depth allowances.

Diffusion Language Model Decoding

Diffusion language model decoding generates text through iterative parallel refinement, balancing fewer denoising steps with accuracy, coherence, and control.

See all 235 papers

BlockBatch: Multi-Scale Consensus Decoding for Efficient Diffusion Language Model Inference
May 28, 2026Xiaoyou Wu, Cheng-Jhih Shih, Binfei Ji +2Blockwise Diffusion Language Models Diffusion Language Model Inference
Georgia Institute of Technology
Early-Bird Decoding: Accelerating Diffusion LLMs with Learnable Block Sizes and Parallel Sampling
Sep 15, 2026Lixuan Wei, Wei Zhou, Jianwen Wu +3Diffusion Language Model Inference Parallel Decoding
Harvard University · Georgia Institute of Technology · Purdue University
PSD: Pushing the Pareto Frontier of Diffusion LLMs via Parallel Speculative Decoding
May 15, 2026Shengyin Sun, Yiming Li, Renxi Liu +7Parallel Speculative Decoding Masked Diffusion Models
Department of Computer Science City University of Hong Kong · Huawei Technologies · City University of Hong Kong

May 28, 2026J/K move · Enter open · S save

BlockBatch: Multi-Scale Consensus Decoding for Efficient Diffusion Language Model Inference

Xiaoyou Wu, Cheng-Jhih Shih, Binfei Ji, Yong Liu+1

Georgia Institute of Technology

Blockwise Diffusion Language Models Diffusion Language Model Inference Diffusion Language Model Decoding Parallel Decoding for Diffusion Language Models Parallel LLM Inference

Open paper AskPDF arXiv

Diffusion language models (dLLMs) generate text by iteratively denoising multiple token positions in parallel, offering an attractive alternative to strictly autoregressive decoding. In practice, however, block-wise dLLM inference exposes a difficult granularity trade-off: small blocks preserve local conditioning but require many denoising steps, whereas large blocks expose more parallelism but can make premature commitments and accumulate cache error. Existing acceleration methods typically choose a single block size per request, leaving the complementarity among block sizes unused. We show that block size itself is a useful branching dimension. Different block sizes induce related but non-identical KV-cache trajectories: branches often share an initial prefix, bifurcate at semantically decisive positions, and later agree on syntactically lightweight tokens. Motivated by this structure, we propose BlockBatch, a training-free online inference framework that executes multiple block-size branches for the same request inside a batched forward pass. BlockBatch coordinates these branches through confidence-gated token merging, leader-based synchronization, and periodic full-sequence refreshes that re-anchor local block updates to a globally consistent KV state. Across 3 representative dLLMs and 4 datasets, BlockBatch reduces denoising NFEs by 26.6% on average and achieves a 1.33

\times

average end-to-end speedup over Fast-dLLM while preserving accuracy. These results identify block-size diversity as a practical and previously underexplored axis for branch-parallel dLLM inference.

Abstract

Diffusion Language Model Decoding

Explore similar work

BlockBatch: Multi-Scale Consensus Decoding for Efficient Diffusion Language Model Inference

Early-Bird Decoding: Accelerating Diffusion LLMs with Learnable Block Sizes and Parallel Sampling

PSD: Pushing the Pareto Frontier of Diffusion LLMs via Parallel Speculative Decoding