Sequential block schedules can make token commitment in masked diffusion language models appear strongly left-to-right even when within-block order is random or reversed. For equal contiguous blocks processed without interleaving in positional order, with untied irreversible commitments, we use a Spearman-rank identity to express global position-commitment correlation as a deterministic schedule reference plus a weighted mean within-block correlation. In a reported paired study of LLaDA-8B-Instruct on 500 math and code prompts with 256 generated positions and updates, increasing block size from 64 to 128 lowers mean global correlation from 0.9845 to 0.9149. Uniform-random within-block references are already 0.9375 and 0.7500. After calibration, mean within-block correlations are 0.751 and 0.660, so the larger raw residual at block size 128 does not indicate stronger local positional ordering. We give sharp bounds and derive calibrated intervals from reported summaries, while documenting the limitations of the aggregate-only empirical release. The analysis supports schedule-aware interpretation of commitment order; its empirical scope is one checkpoint and two schedules, without evaluation of output quality or inference speed.
Figures & tables
Figure 1: Reported mean global commitment correlations for the same 500 prompts and the analytic uniform-random within-block references. Error bars reproduce the reported 95% prompt-bootstrap intervals; the interval at σ=64 is smaller than the marker. Lines connect only the two evaluated schedules and do not imply a measured trend at intermediate block sizes.
Quantity
σ=64
σ=128
Global mean μρ
0.984457
0.914898
Reported interval
[0.983510, 0.985360]
[0.908403, 0.921276]
Schedule reference Bσ
0.937514
0.750011
Within-block weight Wσ
0.062486
0.249989
Derived mean residual
0.046943
0.164887
Transformed interval
[0.045996, 0.047846]
[0.158392, 0.171265]
Table 1: Global summaries retained from the original study and algebraically calibrated quantities. Brackets give the reported 95% global interval and its affine transformations for the residual and local correlation. The reference and weight are deterministic, not estimated. Transformed intervals use the exact constants and rounded reported endpoints; no new resampling is performed.
Quantity
σ=64
σ=128
Content-only mean ρ
0.989410
0.941235
Reported 95% interval
[0.988666, 0.990131]
[0.935613, 0.946779]
EOS fraction
0.051664
0.053414
Table 2: Reported terminal-token sensitivity summaries. Content-only correlations exclude terminal tokens and recompute ranks. They are not calibrated with the original equal-block formula.
Open diffusion language models are marketed as parallel, non-autoregressive decoders, yet the order in which a shipped checkpoint actually commits its tokens is almost never measured. We instrument DiffusionGemma 26B, a masked discrete-diffusion mixture-of-experts model built on Gemma 4, hooking its sampler's accept step to record which canvas positions commit, when, and at what confidence. Across a 686-prompt, six-regime probe suite we find that its decoding is neither parallel nor block-autoregressive: it follows a partial left-to-right commit bias whose apparent strength depends almost entirely on the granularity at which you look. Order is weak token by token and strengthens smoothly as the analysis is coarsened, so the model's "block size" turns out to be an artifact of the measuring ruler rather than the architecture. The model commits in large simultaneous batches, leaving much of the within-batch order genuinely undefined rather than merely unobserved. The behaviour is regime-dependent: structured JSON is committed in essentially arbitrary order, and a position's commit confidence tracks correctness on mathematical reasoning but carries no signal on factual recall. Commitment is aggressive, finishing in a short late burst well inside the step budget, while task accuracy matches the model's autoregressive Gemma-4 sibling. Beyond these findings, our central contribution is methodological: measuring decoding order honestly demands handling trailing-EOS padding, within-regime confounding, commit non-monotonicity, block-size sensitivity, and large commit-batch ties, each of which can otherwise manufacture a decoding-order result that is not really there.
Masked diffusion language models (dLLMs) can commit tokens in any order -- a freedom marketed as their core advantage over autoregressive decoding. We show that on reasoning tasks this freedom is instead the axis of failure. Logging every commitment during decoding of LLaDA-8B on GSM8K, we find that unconstrained (pure) decoding commits the final answer at 15-24% of the trajectory while half the reasoning region is still masked, and collapses to answer-only outputs on up to 90% of problems as the canvas grows. The cause is not the model's termination beliefs -- EOS "pressure" is nearly identical across decoders -- but reachability: whether the sampler may act on those beliefs at distant positions. A 2x2 prompt-decoder design shows that chain-of-thought helps only under ordered commitment (interaction +34.8 percentage points, 95% CI [26.8, 42.8]; without reasoning text the decoders are indistinguishable), an interaction we decompose into a collapse channel and an order channel and replicate on Dream-7B and MATH-500. A single-knob intervention -- frontier-gated commitment -- causally recovers the full gap (0.528 to 0.852) while preserving up to 4x parallel decoding, along a measured frontier whose optimal window flips from w=1 at full refinement to unconstrained at 8 tokens/step. Our results reframe existing window-style samplers, previously motivated by efficiency, as the minimal fix for a reasoning pathology they were never designed to address.
Jewon Yeom, Jaewon Sok, Seonghyeon Park +3
Graduate School of Data Science, Seoul National University · Department of Rural Systems Engineering, Seoul National University · Department of Aerospace Engineering, Seoul National University
Diffusion language models enable flexible arbitrary-order generation, but existing sampling methods are mostly designed for early masked diffusion models (MDMs). In this work, we study sampling for recent block diffusion language models (BDLMs). We show empirically and analytically that these models are naturally more aligned with left-to-right decoding than MDMs. Based on this observation, we propose Parallel Autoregressive Decoding (PARD), a simple training-free sampling method that preserves left-to-right unmasking structure while allowing parallel token commitment. Extensive experiments show that PARD consistently outperforms existing parallel samplers in generation quality, while achieving substantial speedups over pure AR decoding with only a small quality gap.
Kai Syun Hou, James Kwok
Department of Computer Science and Engineering The Hong Kong University of Science and Technology