Length-Adaptive Decoding for Masked Diffusion Machine Translation

Yan Zhan; Mengkai Hou; Wanting Zhang; Zhijun Gao

cs.CLAug 23, 2026

Length-Adaptive Decoding for Masked Diffusion Machine Translation

Authors: Yan Zhan, Mengkai Hou, Wanting Zhang, Zhijun Gao

Organizations: Peking University · BYD Company Limited

Abstract

Machine translation tests masked diffusion language models (dLLMs) because every source token must be rendered faithfully, while fixed canvas decoding must choose target length before denoising. Existing masked diffusion decoding work mainly studies token unmasking order, leaving this length decision under-explored despite its direct effect on coverage and redundancy. We introduce Entropy-Valley (EV), a training-free length selector that scores candidate target canvases by mean predictive entropy from all-mask forward passes and selects the canvas the backbone is most prepared to fill. Relative to a baseline using training corpus length statistics, EV recovers 64.9%, 65.3%, and 33.0% of the COMET-22 gain from reference target lengths on En $\to$ Zh, Zh $\to$ En, and En $\to$ De. Our diagnostics show that denoising-friendly lengths need not match reference lengths. Evaluation by three translation experts supports the En $\leftrightarrow$ Zh adequacy gains, with stronger evidence on Zh $\to$ En. Compared with a LLaMA-3-8B autoregressive (AR) model trained on the same fine-tuning data, the EV system ties on En $\to$ Zh and leads on Zh $\to$ En; an oracle-length diagnostic further shows that, in this masked diffusion MT setting, deciding which tokens to reveal first matters less than how the target length is supplied.

Figures & tables

Figure 1: Three En

\to

Zh cases decoded with 32 denoising steps. A short canvas can drop an action verb or a required placeholder; EV chooses a longer canvas in these two cases and recovers the missing content. The third case shows the opposite boundary: the best compression lies below the EV candidate range, so the ratio baseline is better.

Figure 2: Entropy-Valley method overview. Given a source sentence, EV uses five candidate ratios from the fixed set

\mathcal{R}

, yielding up to five distinct target canvases after duplicate lengths are removed. It then probes each candidate length once with an all-mask forward pass through the frozen LoRA-SFT LLaDA-8B model and records the mean predictive entropy

\bar{H}(L)

over all slots except the designated EOS slot. EV selects the entropy valley,

L^{\star}=\arg\min_{L}\bar{H}(L)

, and decodes only this selected canvas with minimum-entropy decoding (MED) for

T{=}32

steps. The procedure adds no parameters, does not use the reference length of the current sentence, and requires no length predictor.

Figure 3: Length-adaptive decoding for masked-diffusion MT. (A) Mean COMET-22 over three runs for a reference-length upper bound, EV, and a corpus-ratio baseline across three directions. EV closes

64.9\%

on En

\to

Zh ,

65.3\%

on Zh

\to

En , and

33.0\%

on En

\to

De . (B) Matched-data AR calibration across three directions. (C) Controlled En

\to

Zh diagnostic: with reference target lengths fixed, the tested source-guided unmasking schedules span

0.0081

COMET. With the schedule fixed to MED, three length choices span

0.0265

COMET. The order-schedule sweep uses reference lengths, so it is a diagnostic rather than a deployable alternative to EV.

Direction	Method	Mean $\pm$ Std	Gap Closure	Mean $\pm$ Std	Gap Closure
		COMET-22		sacreBLEU
En $\to$ Zh	Oracle length	$0.8610\pm 0.0013$	100%	$40.81\pm 0.23$	100%
	Fixed ratio 0.8	$0.8345\pm 0.0017$	0% (baseline)	$36.72\pm 0.22$	0%
	Entropy-Valley	$\mathbf{0.8517\pm 0.0006}$	$\mathbf{64.9\pm 7.4\%}$	$\mathbf{38.57\pm 0.13}$	45.3%
Zh $\to$ En	Oracle length	$0.8519\pm 0.0007$	100%	$27.93\pm 0.23$	100%
	Fixed ratio 1.2	$0.8266\pm 0.0010$	0% (baseline)	$23.65\pm 0.21$	0%

Table 1: Main LLaDA-8B+LoRA results on WMT22 with 32-step MED decoding. Values are mean

\pm

std over three runs. Length Oracle uses the reference target length at decode time and is an upper bound. Gap closure is

(\mathrm{method}-\mathrm{Ratio})/(\mathrm{Oracle}-\mathrm{Ratio})

. Paired tests are in Appendix Table 11 .

LLaDA-8B masked diffusion
System	Size	Setting	En $\to$ Zh	Zh $\to$ En	En $\to$ De
Fixed ratio	8B	LoRA-SFT	83.45	82.66	71.70
Entropy-Valley	8B	LoRA-SFT	85.17	84.31	72.40
Oracle length †	8B	LoRA-SFT	86.10	85.19	73.82
Matched-data autoregressive baseline
LLaMA-3-8B + LoRA-SFT	8B	LoRA-SFT	$85.07{\pm}0.03$	$83.68{\pm}0.07$	$\mathbf{81.19{\pm}0.11}$

Table 2: Matched-data COMET-22 calibration on WMT22. LLaDA rows show the Table 1 means over three runs scaled

\times 100

; the matched AR row reports mean

\pm

std over three runs after training on the same fine-tuning data and evaluating on the same test sets. † Oracle length supplies the reference target length at decode time and is an upper bound, not a deployable method.

Figure 4: Cross-backbone scope check using COMET-22 gap closure (%) for three masked-diffusion backbones and three directions. Gap closure is measured between Ratio and Length Oracle for each backbone–direction pair. Bars show means over three runs; error bars show standard deviations across runs. Full COMET-22 and sacreBLEU values are in Appendix G (Tables 17 and 18 ).

Figure 5: Quality need not track reference length. Target length error and COMET-22 on En

\to

Zh WMT22 (

N{=}2037

). Points show sentence scores; MAE values use the same outputs, and centroid heights use the main results in Table 1 .

Diagnostic	N	Ratio	EV
Placeholder retention	58	66.9%	89.1%
Number retention	290	77.6%	81.3%
$r{\geq}0.8$ COMET	1005	0.8056	0.8402
$r{<}0.6$ COMET	236	0.8449	0.8503

Table 3: Coverage and length buckets on En

\to

Zh. Retention rows measure whether literal source items such as placeholders and numbers are preserved. COMET rows group sentences by

r

, the reference target length divided by the source length under the LLaDA tokenizer. Appendix D gives the definitions, confidence intervals, and Zh

\to

En companion rows.

Direction	$\Delta$ Adequacy	$\Delta$ Fluency	Preference
	EV $-$ Ratio	EV $-$ Ratio	EV / Ratio / Tie
En $\to$ Zh	$+0.18$	$+0.18$	28 / 19 / 53
Zh $\to$ En	$\mathbf{+0.50}$	$+0.11$	45 / 25 / 30

Table 4: Human evaluation on

100

stratified WMT22 sentences per direction. Score differences are EV

-

Ratio, averaged over three bilingual expert annotators; preference is the per-sentence majority vote.

Appendix figures & tables18 assets

Supplementary material from the paper’s appendix.

Appendix

Hyperparameter	Value
Optimizer	AdamW
AdamW betas	$(0.9,0.95)$
Weight decay	$0.01$
Peak learning rate	$2{\times}10^{-4}$
Scheduler	Cosine decay
Warm-up	$5\%$ of training steps

Appendix

Table 5: Training and adaptation hyperparameters for the LLaDA and matched AR LoRA-SFT runs.

En $\to$ Zh (N=200)
Range	K	Ratios	MAE	sBLEU	COMET
Reported (narrow)	5	[0.70, 0.90]	2.71	38.73	0.8562
Medium	7	[0.60, 1.00]	2.84	38.49	0.8594
Wide	9	[0.50, 1.10]	3.06	38.17	0.8588
Full	12	[0.50, 1.20]	3.22	37.58	0.8577
Extra	15	[0.40, 1.50]	3.89	35.98	0.8476

Appendix

Table 6: Candidate-range control for Entropy-Valley on fixed 200-sentence WMT22 diagnostic subsets. The WMT19 training-corpus median provides a reference scale, while diagnostic range comparisons on WMT22 subsets informed the reported ratio grid. The En

\to

Zh grid minimizes length MAE and gives the highest sacreBLEU, while the medium range gives the highest subset COMET; the En

\to

De grid gives the highest subset COMET. The Zh

\to

En subset benefits from a wider range.

En $\to$ Zh (N=500, 32 steps)
Method	sacreBLEU	COMET-22
Oracle	40.40	0.8617
Fixed 0.70	31.73	0.8114
Fixed 0.75	34.59	0.8208
Fixed 0.80	35.84	0.8281
Fixed 0.85	36.84	0.8354

Appendix

Table 7: Fixed-ratio sweep on fixed 500-sentence WMT22 diagnostic subsets. EV is compared with fixed ratios around the corpus median for all three directions. On Zh

\to

En , the best post-hoc fixed ratio is

1.4

0.8390

COMET-22; EV reaches

0.8453

, exceeding it by

+0.0063

COMET and closing

35.4\%

of the oracle-length gap on this subset.

Method	sacreBLEU	COMET-22	Cost
Oracle length	40.40	0.8617	$1.00\times$
Fixed ratio 0.8	35.84	0.8281	$1.00\times$
EV probe+decode	38.48	0.8557	$1.15\times$
Decode 5 neighbours	37.85	0.8404	$\sim 3.5\times$

Appendix

Table 8: Multi-candidate length selection on the same En

\to

Zh WMT22 subset used in Table 7 (N=500, 32 steps). The alternative decodes five candidate canvas lengths obtained as integer offsets around the EV-selected length,

\{L_{\text{EV}}{-}2,L_{\text{EV}}{-}1,L_{\text{EV}},L_{\text{EV}}{+}1,L_{\text{EV}}{+}2\}

(with length collisions deduplicated), to completion and selects by average log-probability; its cost counts all completed decodes.

Dir.	Method	COMET	EV $-$ method	Calls	Slots	Latency (s)
En $\to$ Zh	Ratio	$0.833721$	$+0.010930$	$32.00$	$512.66$	$0.644$
En $\to$ Zh	EV	$\mathbf{0.844651}$	–	$36.09$	$588.32$	$0.724$
En $\to$ Zh	DAEDAL	$0.828043$	$+0.016608\;([0.012954,0.020210];\ p<2{\times}10^{-4})$	$34.58$	$1175.53$	$1.798$
En $\to$ Zh	CAL	$0.837897$	$+0.006754\;([0.003726,0.009880];\ p<2{\times}10^{-4})$	$39.25$	$628.26$	$0.773$
Zh $\to$ En	Ratio	$0.818169$	$+0.016416$	$32.00$	$618.43$	$0.635$
Zh $\to$ En	EV	$\mathbf{0.834585}$	–	$36.43$	$708.98$	$0.721$

Appendix

Table 9: Direct diffusion length comparisons on WMT22 (

N{=}2037

per direction). Confidence intervals and paired bootstrap

p

-values are shown for DAEDAL and CAL. Calls and Slots are per-sentence averages; latency is measured in seconds per sentence in the reported setup. EV is above DAEDAL in both directions and CAL on En

\to

Zh. The EV and CAL interval on Zh

\to

En includes zero. EV uses fewer calls and slots than CAL, with lower latency in this setup.

Figure 6: Cost–quality Pareto. COMET-22 vs decoding steps

T\in\{2,4,8,16,32,64,128\}

on WMT22 (

N{=}2037

, one analyzed run). (A) En

\to

Zh: EV is at least as high as Ratio at every

T

. (B) En

\to

De: EV is higher at small

T

and slightly lower than Ratio at

T\geq 64

. Oracle is a length oracle, not a decode oracle.

	En $\to$ Zh				En $\to$ De
Steps	Oracle	EV	Ratio	Gap%	Oracle	EV	Ratio	Gap%
2	0.6975	0.7082	0.6861	194%	0.3942	0.4030	0.3915	426%
4	0.7856	0.7851	0.7644	98%	0.5074	0.5112	0.4912	123%
8	0.8290	0.8264	0.8058	89%	0.6060	0.6089	0.5862	115%
16	0.8520	0.8446	0.8265	71%	0.6873	0.6752	0.6683	36%
32	0.8592	0.8521	0.8325	73%	0.7417	0.7243	0.7209	16%

Appendix

Table 10: Full cost–quality Pareto on WMT22 (

N{=}2037

, one analyzed run). Gap% is computed relative to the Ratio and Oracle length columns at the same step budget; values can exceed 100% because Oracle fixes reference length but does not optimize decoding under small

T

Dir.	Comp.	$\Delta$	95% CI	Boot. $p$	Wilc. $p$
En $\to$ Zh (A)	EV–Ratio	$+0.0196$	$[+0.0166,+0.0226]$	$<10^{-4}$	$4.1{\times}10^{-32}$
En $\to$ Zh (A)	Oracle–EV	$+0.0071$	$[+0.0046,+0.0096]$	$<10^{-4}$	$1.3{\times}10^{-14}$
En $\to$ Zh (A)	Oracle–Ratio	$+0.0266$	$[+0.0233,+0.0300]$	$<10^{-4}$	$3.4{\times}10^{-58}$
En $\to$ De (A)	EV–Ratio	$+0.0034$	$[-0.0016,+0.0086]$	$0.091$	$8.3{\times}10^{-3}$
En $\to$ De (A)	Oracle–EV	$+0.0173$	$[+0.0121,+0.0225]$	$<10^{-4}$	$1.6{\times}10^{-9}$
En $\to$ De (A)	Oracle–Ratio	$+0.0208$	$[+0.0155,+0.0261]$	$<10^{-4}$	$6.0{\times}10^{-14}$

Appendix

Table 11: Sentence-level paired COMET-22 tests for LLaDA on WMT22 (

N{=}2037

throughout). Positive

\Delta

means the first method is better. Paired bootstrap uses 10k resamples; Wilcoxon uses the signed-rank test with Pratt handling of zeros. The table reports run A for En

\to

Zh and En

\to

De , plus all three Zh

\to

En runs. Oracle rows show the run A gap to the reference-length upper bound.

Direction	Method	T	Probe cap	Fwd cap	COMET-22	sacreBLEU
En $\to$ Zh, one analyzed run, WMT22 N=2037
En $\to$ Zh	Ratio 0.8	32	0	32	0.8325	36.53
En $\to$ Zh	Entropy-Valley	32	$\mathbf{\leq 5}$	$\mathbf{\leq 37}$	$\mathbf{0.8521}$	$\mathbf{38.56}$
En $\to$ Zh	Ratio 0.8	37	0	37	0.8334	36.66
En $\to$ Zh	Ratio 0.8	40	0	40	0.8333	36.71
Zh $\to$ En, one analyzed run, WMT22 N=2037

Appendix

Table 12: Forward-pass budget control. Ratio is evaluated at the default budget (

T{=}32

), EV’s maximum total forward pass budget (

T{=}37

), and a larger budget (

T{=}40

). Duplicate integer candidate lengths can make EV’s realized budget smaller than the cap.

Category	N	Ratio	EV	$\Delta$ (pp)	95% CI (pp)
En $\to$ Zh — error-type preservation rate
Placeholder (#PRS_ORG#)	58	66.9%	89.1%	+22.1 ∗	[+12.4, +32.5]
Number / percentage / decimal	290	77.6%	81.3%	+3.7 ∗	[+1.4, +6.1]
Capitalized named entity (heuristic)	879	11.2%	11.9%	+0.7 ∗	[+0.0, +1.4]
En $\to$ Zh — length-compression buckets (LLaDA token ratio)
Bucket	N	Ratio COMET	EV COMET	$\Delta$ COMET	95% CI

Appendix

Table 13: Coverage and error categories. Top: source-side literal retention rate, averaged over three runs. Middle: En

\to

Zh COMET-22 grouped by

r=|L^{*}|/|\mathbf{x}|

, the reference target length divided by source length under the LLaDA tokenizer. Bottom: Zh

\to

En literal retention.

\Delta

is in percentage points for retention rows; ∗ marks an interval whose unrounded lower endpoint is above zero.

Likely cause	N	Share	Mean $\Delta$ COMET
Overshoot beyond grid	129	21.6%	$-0.0278$
Under-compression	126	21.1%	$-0.0206$
Within-grid wrong pick	339	56.9%	$-0.0247$
Very short source	2	0.3%	—

Appendix

Table 14: Failure buckets for the 596 En

\to

Zh cases where EV trails Ratio in one analyzed run (WMT22,

N{=}2037

). Buckets are assigned by mutually exclusive rules.

Schedule	COMET-22	$\Delta$ vs MED	Paired bootstrap $p$
MED (baseline)	0.8597	—	—
SIG-first	0.8603	$+0.0006$	$0.62$
Reverse-SIG	0.8523	$-0.0075$	$<10^{-4}$
Hybrid MED $\to$ SIG, $p{=}0.25$	0.8582	$-0.0015$	$0.21$
Hybrid MED $\to$ SIG, $p{=}0.50$	0.8575	$-0.0022$	$0.011$
Hybrid MED $\to$ SIG, $p{=}0.75$	0.8591	$-0.0007$	$0.28$

Appendix

Table 15: Reveal-order controls on WMT22 En

\to

Zh (

N{=}2037

T{=}32

) with reference length and the tokens-per-step schedule fixed. The six source-guided variants span

0.0081

COMET. Strict left-to-right and random order are additional controls and both score below MED. A paired bootstrap value is not reported for Strict L2R.

Category	N	SIG-first	MED	$\Delta$ (SIG $-$ MED)
Dates	32	0.8652	0.8665	$-0.0013$
Enumeration	75	0.8560	0.8533	$+0.0027$
Named entities	280	0.8627	0.8629	$-0.0002$
Numbers	123	0.8679	0.8724	$-0.0045$
Overall	400	$\mathbf{0.8640}$	$\mathbf{0.8644}$	$\mathbf{-0.0004}$

Appendix

Table 16: Source-guided order diagnostic on an En

\to

Zh challenge subset (

N{=}400

, one analyzed run,

T{=}32

MED vs. SIG-first under matched oracle length). SIG-first reveals high-source-dependence slots first; MED is the confidence-first schedule used in the main experiments.

LLaDA-8B-Base (paper main results, § 4.2 ; reproduced for cross-backbone comparison)
Backbone	Direction	Oracle COMET	Ratio COMET	EV COMET	$\Delta$ EV $-$ Ratio	COMET Gap closure
LLaDA-8B-Base	En $\to$ Zh	$0.8610{\pm}0.0013$	$0.8345{\pm}0.0017$	$\mathbf{0.8517{\pm}0.0006}$	$+0.0172$	$\mathbf{64.9{\pm}7.4\%}$
LLaDA-8B-Base	Zh $\to$ En	$0.8519{\pm}0.0007$	$0.8266{\pm}0.0010$	$\mathbf{0.8431{\pm}0.0004}$	$+0.0165$	$\mathbf{65.3{\pm}0.8\%}$
LLaDA-8B-Base	En $\to$ De	$0.7382{\pm}0.0090$	$0.7170{\pm}0.0090$	$\mathbf{0.7240{\pm}0.0078}$	$+0.0070$	$\mathbf{33.0{\pm}8.4\%}$
Dream-v0-Base-7B (Qwen2.5-7B continued-pretrained; Qwen2 tokenizer, $152$ k vocab)
Dream-Base	En $\to$ Zh	$0.8414{\pm}0.0020$	$0.8002{\pm}0.0016$	$\mathbf{0.8239{\pm}0.0004}$	$+0.0238$	$\mathbf{57.7{\pm}3.1\%}$

Appendix

Table 17: Cross-backbone COMET-22 results for Entropy-Valley (WMT22,

N{=}2037

, three runs per aggregate cell,

T{=}32

MED). Values are mean

\pm

std across runs. COMET-22 gap closure is

(\text{EV}-\text{Ratio})/(\text{Oracle}-\text{Ratio})

per run, then mean

\pm

std. EV is above Ratio in all six Dream-Base and DiffuLLaMA aggregate COMET cells; paired tests for every run give

p<10^{-3}

in all

18

Dream-Base and DiffuLLaMA comparisons, with positive 95% bootstrap confidence intervals. Absolute scores and closures vary by backbone.

Backbone	Direction	Oracle	Ratio	EV
LLaDA-8B-Base	En $\to$ Zh	$40.81{\pm}0.23$	$36.72{\pm}0.22$	$\mathbf{38.57{\pm}0.13}$
LLaDA-8B-Base	Zh $\to$ En	$27.93{\pm}0.23$	$23.65{\pm}0.21$	$\mathbf{25.28{\pm}0.33}$
LLaDA-8B-Base	En $\to$ De	$22.55{\pm}0.77$	$20.73{\pm}0.68$	$\mathbf{21.55{\pm}0.85}$
Dream-Base	En $\to$ Zh	$36.30{\pm}0.22$	$29.37{\pm}0.20$	$\mathbf{32.83{\pm}0.28}$
Dream-Base	Zh $\to$ En	$27.24{\pm}0.28$	$23.56{\pm}0.04$	$\mathbf{25.78{\pm}0.20}$
Dream-Base	En $\to$ De	$21.64{\pm}0.16$	$17.31{\pm}0.25$	$\mathbf{20.06{\pm}0.17}$

Appendix

Table 18: Companion sacreBLEU view of Table 17 (WMT22,

N{=}2037

, three runs per cell). Values are mean

\pm

std across runs.

Evaluation set	Pool	Ratio	EV	$\Delta$ COMET (95% CI)
WMT19 held out	50k	$0.7428$	$0.7514$	$+0.0086\ [0.0034,0.0139]$
WMT19 held out	200k	$0.7386$	$0.7485$	$+0.0100\ [0.0050,0.0152]$
FLORES devtest	50k	$0.7398$	$0.7481$	$+0.0082\ [0.0020,0.0143]$
FLORES devtest	200k	$0.7315$	$0.7403$	$+0.0088\ [0.0024,0.0152]$

Appendix

Table 19: De

\to

Fr results for two nested training pools. WMT19 held out contains

N{=}1512

sentences, and FLORES uses the full

N{=}1012

\to

Fr devtest. Both pools use a fixed

500

update budget and one candidate set for the direction. EV exceeds Ratio in all four settings, with positive paired confidence intervals; paired bootstrap and Wilcoxon tests are below

0.05

throughout. This is a robustness check under a fixed update budget, not a comparison of scaling with data size.

Figure 7: Human evaluation on En

\to

Zh. (A) Per-annotator adequacy boxplots. (B) Sentence-level

\Delta

COMET-22 vs 3-expert mean

\Delta

Adequacy (EV

-

Ratio); Spearman

\rho=0.637

(

p<0.001

). The corresponding Zh

\to

En correlation is reported in Table 20 .

Panel A: En $\to$ Zh (N=100)
	A	B	C	Agg.
$\Delta$ Adequacy (EV $-$ Ratio), mean	$+0.19$	$+0.17$	$+0.19$	$+0.18$
$\Delta$ Fluency (EV $-$ Ratio), mean	$+0.18$	$+0.20$	$+0.16$	$+0.18$
Preference EV / Ratio / Tie	28/19/53	28/16/56	30/27/43	28/19/53
Wilcoxon $p$ (Adequacy)	$0.038^{\ast}$	$0.070$	$0.065$	—
Wilcoxon $p$ (Fluency)	$0.056$	$0.035^{\ast}$	$0.135$	—

Appendix

Table 20: Detailed human-evaluation statistics. The first three columns report annotator-level results. The aggregate preference uses per-sentence majority vote; aggregate Zh

\to

En adequacy uses the paired Wilcoxon test on the three-expert mean.

Machine Translation

Machine translation develops computational methods for converting text, speech, and visual content between languages while preserving meaning and purpose.

See all 267 papers

Aug 31, 2026

CARVE: Verified Expansion for Variable-Length Generation in Diffusion Language Models

Masked diffusion language models predict tokens from a partially observed response canvas, enabling bidirectional conditioning and parallel token refinement. Yet standard masked-diffusion decoders use a rigid inference interface: the number of masked positions allocated to the answer is fixed before generation begins. Choosing this length is difficult. A short canvas can truncate reasoning or code, while a long canvas wastes computation and can perturb denoising. We introduce CARVE (Counterfactual-Aware Reveal with Verified Expansion), a training-free variable-length algorithm for masked diffusion LMs. Starting from a shorter canvas, CARVE can grow the response during decoding by inserting additional [MASK] positions. Rather than keeping every insertion, CARVE tests a candidate expanded canvas and asks a counterfactual question: would the model make similar predictions for the unresolved positions in the original canvas if the extra masked space were present? The inserted masks are kept only when they induce low Jensen-Shannon (JS) divergence on aligned unresolved positions. This makes length growth a verified stability decision rather than a pure confidence heuristic. CARVE applies without retraining to both full-canvas and blockwise diffusion decoders. Across code generation and mathematical reasoning benchmarks, CARVE consistently improves average performance over fixed-length baselines across all evaluated model families. Crucially, CARVE achieves these accuracy gains while reducing inference cost, reaching half the FLOPs of fixed-length decoding in some settings.

Wail Bouhedja, Amr Mohamed, Guokan Shang

1MBZUAI · 2Sorbonne Université · 3Ecole Polytechnique

Masked Diffusion Models LLM Inference Acceleration Diffusion Language Model Decoding Diffusion Language Models LLM Code Generation LLM Reasoning

Ask PDF

Jun 27, 2026

Masked Diffusion Decoding as $x$ -Prediction Flow

Masked diffusion language models (MDLMs) generate text by iteratively unmasking tokens, but their standard decoder reduces each step to a binary action: a position is either committed to a single token or left fully masked, with no representation of partial belief in between. This all-or-nothing regime discards rich predictive information and forces premature, irrevocable commitments, leading to poor performance under a limited decoding budget. In this paper, we reinterpret mask prediction as clean-state prediction (

x

-prediction) and show that it can be used to induce a continuous flow in input embedding space. Building on this view, we propose a continuous decoding framework for MDLMs where tokens can accumulate partial progress at each diffusion step and remain revisable. To match the uneven contextual constraints across positions in language, we replace the globally synchronous schedule in image diffusion with a confidence-based asynchronous update in which the diffusion progress is token-wise accumulated. Additionally, we introduce a lightweight policy network and formulate its training as a reinforcement learning problem. Applied to pretrained LLaDA, our continuous decoder reaches 97% of its performance on the HumanEval dataset with 25% of decoding budget.

Weitian Wang, Lianlei Shan, Shubham Rai +2

Robert Bosch GmbH, Germany · University of the Chinese Academy of Sciences, China · Ruhr University Bochum, Germany

Masked Diffusion Models Diffusion Language Model Decoding Diffusion Model Inference Acceleration

Ask PDF

May 22, 2026

DiLaDiff: Distilled Latent-Augmented Diffusion for Language Modeling

Diffusion language models intrinsically fail to capture correlations between decoded tokens, which leads to a harsh trade-off between sampling quality and throughput. To solve this issue, we propose DiLaDiff, a variant of masked diffusion language models with three components: (1) a continuous latent space with semantic capabilities, learned by an auto-encoder fine-tuned from an existing masked diffusion language model; (2) a latent diffusion model learning the prior over the encoder distribution; (3) a consistency model distilling the learned prior into a few-step latent generative model. We show that, even without distillation, our latent-guided diffusion model outperforms the masked diffusion baseline while significantly accelerating inference. Consistency distillation further lowers the computational overhead of continuous diffusion, such that the latent is generated in negligible time compared to discrete decoding.

Jean-Marie Lemercier, Tomas Geffner, Karsten Kreis +3

NVIDIA

Diffusion Language Models Diffusion Language Model Distillation Diffusion Language Model Inference Masked Diffusion Models Continuous Diffusion Language Models Diffusion Model Inference Acceleration+1

Ask PDF

Abstract

Machine Translation

Explore similar work

CARVE: Verified Expansion for Variable-Length Generation in Diffusion Language Models

Masked Diffusion Decoding as xxx-Prediction Flow

DiLaDiff: Distilled Latent-Augmented Diffusion for Language Modeling

Masked Diffusion Decoding as $x$ -Prediction Flow