Diffusion Language Models (DLMs) have emerged as effective alternatives to autoregressive models for latency-critical tasks, enabling faster token generation. Recent advances employ block diffusion with self-editing capabilities, where each block refines tokens iteratively before advancing to the next one. Yet each block is only as fast as its slowest token, and the next block cannot start before it. We suggest Blurring the Block Boundaries (B3), a novel method with which two blocks are refined in parallel within a shared forward pass. B3 calibrates itself on the fly, opening the next block only once the model is ready to start generating it. It is a training-free and hyperparameter-free acceleration method which seamlessly integrates within any self-editing block diffusion model. Evaluated on state-of-the-art DLMs across 10 diverse datasets spanning varied tasks and difficulty levels, B3 results in on par generation quality and 11.5-14.5% speedup compared to the base model, pushing the Pareto frontier for latency-quality trade-offs in DLM inference.
Figures & tables
Figure 1: Tokens accepted per denoising step against the share of the block already accepted, for DiffusionGemma 26B on HumanEval+. (a) Vanilla decoding: the orange bands mark low-acceptance regions at the block’s boundaries, which take 40% of the steps but yield only 17% of the tokens. (b) B3 : forward passes shared with the neighbouring block (purple) keep the tokens accepted per step consistent across the block.
Figure 2: Schematic of B3 . Each row is one forward pass. Left: vanilla decoding denoises one block at a time. Right: B3 starts the next block early, sharing forward passes with the current block, and finishes in fewer passes.
Figure 3: B3 attention mask of a shared forward pass; the current block never attends to the future block.
TPS
TPF
Quality
Model
Benchmark
n
Base
B3 (ours)
Δ%
Base
B3 (ours)
Δ%
Base
B3 (ours)
Δpp
DiffusionGemma 26B
GSM8K
1319
136.1
150.8
+10.8
28.74
33.69
+17.2
95.75
96.13
+0.38
MATH500
500
139.9
156.0
+11.6
28.69
34.16
+19.1
95.80
92.40
-3.40
AIME26
30
101.0
121.6
+20.3
20.61
25.48
+23.6
73.33
76.67
+3.33
Omni-MATH
4428
107.6
118.0
+9.6
21.70
25.10
+15.7
61.50
60.68
-0.81
GPQA-D
198
94.4
107.1
+13.4
19.18
22.88
+19.3
73.74
71.72
-2.02
Table 1: Per-benchmark results for B3 on DiffusionGemma 26B, LLaDA 2.1 flash 100B, and LLaDA 2.1 mini 16B.
Figure 4: Sweep of the acceptance criterion. Error bars denote ±1 standard error of the average accuracy due to question sampling, combined from per-benchmark binomial standard errors.
Figure 5: B3 shows a preferable accuracy/speed trade-off compared to increasing the block size. At each shipped operating point, it is better to use B3 instead of increasing the block size.
Block Size
Method
TPS
TPF
Acc
32
Base
111
4.83
68.8
FlowBlock
132 (+18%)
5.77 (+20%)
67.9 (-1.0)
MBD-LMs
131 (+18%)
5.95 (+23%)
67.4 (-1.4)
B3
133 (+20%)
5.88 (+22%)
69.6 (+0.7)
64
Base
127
5.53
68.6
FlowBlock
132 (+4%)
5.89 (+7%)
68.3 (-0.3)
Table 2: LLaDA 2.1 mini 16B: B3 vs. concurrent inter-block methods on the eight light benchmarks described in Section 4.2 .
Figure 6: τ selected by the self-calibrating gate at each spawn, per benchmark and model. Violin: P1–P99; box: P25–P75; line: median.
Figure 7: Ablation on HumanEval+ for our self-calibrating spawning threshold vs. spawning at a fixed fraction of acceptance, θspawn , and at a fixed max entropy gate of τ . Top right is preferable.
Figure 8: Design choices of the spawn gate on HumanEval+. Each numbered point changes one choice of ours (AGG = max, COMBINE = avg, Cτ carried across blocks): (1) Cτ resets every block, (2) AGG = avg, (3) AGG = sum, (4) COMBINE = avg + std, (5) COMBINE = max, (6) COMBINE = max (corrected). AGG and COMBINE are defined in Figure 6 .
Appendix figures & tables16 assets
Supplementary material from the paper’s appendix.
Appendix
Entropy bound
0.001
0.01
0.1
1
10
Method
Base
B3
Base
B3
Base
B3
Base
B3
Base
B3
TPS
GSM8K
113.5
119.2
131.1
141.0
136.1
150.8
142.3
160.6
145.2
167.7
MATH500
125.2
125.1
137.3
142.9
139.9
156.0
144.0
161.2
147.0
171.4
AIME26
93.8
97.7
99.9
108.2
101.0
121.6
114.6
120.0
119.5
129.6
GPQA-D
76.8
84.8
86.3
96.2
94.4
107.1
99.4
114.2
103.6
121.5
Appendix
Table 3: DiffusionGemma 26B, per-benchmark results for Figure 4 .
Confidence threshold
0.3
0.5
0.7
0.85
0.95
Method
Base
B3
Base
B3
Base
B3
Base
B3
Base
B3
TPS
GSM8K
128.1
146.1
133.5
150.1
114.2
124.8
84.5
91.9
58.6
61.5
MATH500
160.6
186.5
171.2
199.5
160.4
186.1
126.5
139.3
95.2
102.9
AIME26
109.6
115.8
131.3
149.9
113.7
125.9
92.6
92.9
70.6
72.1
GPQA-D
65.6
66.2
79.8
87.1
69.5
73.8
54.4
56.1
42.8
44.4
Appendix
Table 4: LLaDA 2.1 mini 16B, per-benchmark results for fig. 4 .
Block
128
256
256
384
512
B3
TPS
GSM8K
90.9
136.1
150.8
171.0
177.8
MATH500
96.6
139.9
156.0
167.8
171.1
AIME26
86.7
101.0
121.6
118.9
112.6
GPQA-D
70.4
94.4
107.1
106.0
104.8
Appendix
Table 5: DiffusionGemma 26B, per-benchmark results for Figure 5 .
Block
32
32
64
64
96
128
B3
B3
TPS
GSM8K
101.5
122.8
114.2
124.8
113.5
91.7
MATH500
133.9
163.9
160.4
186.1
159.7
148.5
AIME26
103.6
118.5
113.7
125.9
121.3
105.4
GPQA-D
65.8
73.4
69.5
73.8
70.5
62.9
Appendix
Table 6: LLaDA 2.1 mini 16B, per-benchmark results for Figure 5 .
Block
32
64
Method
Base
FlowBlock
MBD-LMs
B3
Base
FlowBlock
MBD-LMs
B3
TPS
GSM8K
101.5
153.2
119.9
122.8
114.2
153.8
119.3
124.8
MATH500
133.9
207.1
161.8
163.9
160.4
210.8
176.6
186.1
AIME26
103.6
143.6
116.4
118.5
113.7
147.7
114.0
125.9
GPQA-D
65.8
91.1
74.2
73.4
69.5
83.4
69.7
73.8
Appendix
Table 7: LLaDA 2.1 mini 16B, per-benchmark results for Table 2 .
Figure 9: DiffusionGemma on HumanEval+: tokens accepted per step by block index. Vanilla left, B3 right.
Figure 10: DiffusionGemma on GPQA Diamond: tokens accepted per step by block index. Vanilla left, B3 right.
Figure 11: Share of a block’s tokens accepted in its last 20% (more than 80% of the block accepted), divided by the share of its denoising steps taken there, by the block’s position in the generation (first block 0%, last 100%; 10 bins). Below 1: the end of the block yields fewer tokens than the steps it costs. Solid: HumanEval+; dashed: GPQA Diamond. Dotted: parity.
Figure 12: Spawn points of B3 , pooled over HumanEval+ and GPQA Diamond. Top: share of the current block accepted when the next block spawns; gray: blocks that finished without a spawn. Bottom: τ at spawn. Dashed: median.
Figure 13: DiffusionGemma: spawn points per dataset. Top: share of the current block accepted at spawn; gray: blocks that finished without a spawn. Bottom: τ at spawn. Dashed: median.
Figure 14: LLaDA 2.1 mini: spawn points per dataset. Top: share of the current block accepted at spawn; gray: blocks that finished without a spawn. Bottom: τ at spawn. Dashed: median.
Figure 15: DiffusionGemma: spawn points per block index, pooled over datasets. Top: share of the current block accepted at spawn; gray: blocks that finished without a spawn. Bottom: τ at spawn. Dashed: median.
Figure 16: LLaDA 2.1 mini: spawn points per block index, pooled over datasets. Top: share of the current block accepted at spawn; gray: blocks that finished without a spawn. Bottom: τ at spawn. Dashed: median.
Figure 17: Rightmost position accepted on a denoising step against the share of the block committed after it, vanilla decoding. Lines: mean; bands: P25–P75; dashed: strictly left-to-right.
Figure 18: Step at which each position is first accepted, as a share of the block’s denoising steps that accept at least one token, against its position in the block, vanilla decoding. Lines: mean; bands: P25–P75; dashed: diagonal. Kendall τb : median over blocks of the rank correlation between position and first-acceptance step, and the share of blocks where it is positive.
Accepted in
Tokens
Edited later
Share
DiffusionGemma
Vanilla
237,496
633
0.27%
B3 (future block)
42,426
839
1.98%
LLaDA 2.1 mini
Vanilla
51,520
1,441
2.80%
B3 (future block)
26,250
1,125
4.29%
Appendix
Table 8: Tokens edited after their first acceptance, on HumanEval+.