BYOD: Build Your Own DLM - Efficient Autoregressive-to-Diffusion Language Model Conversion

Ruurd J. A. Kuiper; Robert A. Bagheri

cs.CLOct 30, 2026

BYOD: Build Your Own DLM - Efficient Autoregressive-to-Diffusion Language Model Conversion

Authors: Ruurd J. A. Kuiper, Robert A. Bagheri

Organizations: Julius Center for Health Sciences and Primary Care University Medical Center Utrecht · Department of Methodology and Statistics Utrecht University, The Netherlands

Abstract

Unlike autoregressive (AR) models, diffusion language models (DLMs) permit parallel token updates, flexible generation, and bidirectional attention, but leading systems rely on costly pretraining or full-model fine-tuning. We present Build Your Own DLM (BYOD), a generalizable LoRA-only recipe for parameter-efficient AR-to-DLM conversion. Without architecture-specific tuning, BYOD induces diffusion generation in Gemma 2 9B, Llama 3.1 8B, Qwen2.5 7B, and Ministral 8B; to our knowledge, this is the first reported AR-to-DLM conversion of a Mistral model. The adapters comprise 4.1-5.8% of final parameters while every base-model parameter remains frozen; each 25,000-iteration conversion needs at most 102.4M tokens, processed in 3.1-4.0 hours on one Google Colab GPU. All four models generate from fully masked sequences, use bidirectional context in a controlled attention test, and retain substantial parent-model knowledge. BYOD-Gemma is the strongest BYOD conversion on our nine-task evaluation, reaching 50.6% average accuracy compared with 51.9% for LLaDA 8B Instruct in the same harness. Open-generation quality at low network function evaluation (NFE) budgets remains the principal limitation: BYOD perplexity is 8.3-11.1 versus 4.1 for LLaDA and 2.2-3.0 for the AR parents when generating 128 tokens in 64 steps. To enable further experimentation and inspection, we provide a standalone training and evaluation implementation and interactive demos for all four conversions. BYOD thus provides a practical route for extending diffusion generation to other existing model families while identifying the gap that efficient conversion must next close.

Figures & tables

Figure 1: Task-level accuracy profiles from the matched evaluation. (a) LLaDA, BYOD-Llama, and the unchanged Llama AR parent. (b) The four primary BYOD conversions.

System	Diffusion-stage data	Updated weights	Hardware	GPU-h	Cost (USD)	CO 2 e
BYOD (ours)	$\leq$ 102.4M tokens	LoRA	1 $\times$ RTX Pro 6000	3.2	$11	0.9 kg
DiffuLLaMA 7B	65B tokens	Full	64 $\times$ GH200	1,917 †	$9,951	0.65 t
LLaDA 8B	2.3T tokens	Full	H800	130,000	$674,830	44.1 t
Dream 7B	580B tokens + SFT	Full	NR	17,105 †	$88,791	5.81 t
iLLaDA 8B	12T tokens + SFT	Full	NR	404,449 †	$2,099,494	137.3 t
Efficient-DLM 8B	500B tokens	Full	128 $\times$ H100	16,852 †	$87,479	5.72 t

Table 1: Incremental training scale and standardized resource estimates for comparable diffusion systems. GPU-hours marked † are estimated from published model/data scale; unmarked values are reported. We stress that cost and CO 2 e are estimates purely for illustrative purposes, based on a standardized operational scenario (Appendix A ), not reported values. NR: insufficient information.

Diffusion model	AR parent	Total parameters	Trainable	Trainable (%)	Time (h)
BYOD-Gemma	Gemma 2 9B IT	10.732B	572.5M	5.33	4.0
BYOD-Qwen	Qwen2.5 7B Instruct	7.939B	323.0M	4.07	3.1
BYOD-Ministral	Ministral 8B Instruct 2410	8.511B	490.7M	5.77	3.4
BYOD-Llama	Llama 3.1 8B Instruct	8.466B	436.2M	5.15	3.2

Table 2: Diffusion model names matched to parent models, their trainable parameter counts, and measured time for 25,000 updates. All trainable parameters are rank-1024 Q/V LoRA weights.

General knowledge and reasoning (multiple choice)
	Gemma		Qwen		Ministral		Llama			LLaDA
Task	D	AR	D	AR	D	AR	D-25k	D-50k	AR	Ours	Published
ARC-C	.864	.888	.840	.912	.832	.768	.832	.824	.792	.864	.885
GPQA	.320	.280	.288	.344	.272	.368	.296	.272	.320	.312	.333
HellaSwag	.752	.696	.728	.752	.768	.680	.704	.760	.608	.696	.746
MMLU	.688	.648	.568	.656	.576	.544	.592	.616	.536	.560	.655

Table 3: Exact-match/pass@1 accuracy in our matched harness. D denotes a converted diffusion model and AR its unchanged parent. The Llama D-25k and D-50k columns share the same AR parent. Among common-harness diffusion columns, bold marks the best result and underline the best BYOD result (ties retained); published LLaDA is protocol-separated and excluded from highlighting.

Figure 3: Fluency metrics for different models with different NFE budgets. Lines connect each diffusion model’s results at 32, 64, and 128 NFE; crosses to the right show the corresponding AR-parent reference. Perplexity is shown on a logarithmic axis.

Figure 4: BYOD-Llama-50k training diagnostics. (a) Token-weighted Phi-4 perplexity for 128-token generations, shown on a logarithmic y-axis. (b) Mean sliding model-token Distinct-1 for the same generations, reported as a percentage. Color-matched dashed lines are fixed LLaDA references; the dotted gray line is the unchanged Llama AR parent. (c) Validation loss.

Appendix figures & tables15 assets

Supplementary material from the paper’s appendix.

Appendix

System	Parameters	Tokens	Pred. GPU-h	Reported GPU-h	Implied MFU
LLaDA	8B	2.30T	77,519	130,000	23.9%
Fast-dLLM v2	7B	1.31B	123	768	6.4%

Appendix

Table 4: Calibration of the

6ND

estimator against reported accelerator-hours. Predictions use 40% MFU and the stated accelerator’s dense-BF16 peak (989 TFLOP/s H100-class proxy for H800; 312 TFLOP/s for A100) ( NVIDIA, 2023 ; NVIDIA, 2021 ) . “Implied MFU” is the utilization that would reconcile

6ND

with reported time; it is not a measured MFU.

Representation	Parameters	PPL ${}_{32}\downarrow$	PPL ${}_{128}\downarrow$	D-1% $\uparrow$	D-2% $\uparrow$	D-3% $\uparrow$
Adapter loaded	8.466B	19.381	5.081	49.5	78.8	89.9
Merged standalone	8.030B	19.283	4.764	49.8	80.7	91.8

Appendix

Table 5: Open-generation comparison of the adapter-loaded and merged Llama checkpoints. PPL is token-weighted; Distinct-

n

is averaged over answers and reported as a percentage. Both use 128 generated tokens; D-1/2/3 use 32 NFE.

Task	Adapter	Merged	Task	Adapter	Merged
ARC-C	83.2	84.0	MATH	25.6	24.8
GPQA	29.6	30.4	MBPP	43.2	41.6
GSM8K	65.6	63.2	MMLU	59.2	60.8
HellaSwag	70.4	69.6	MMLU-Pro	30.4	30.4
HumanEval	31.2	29.6	Mean	48.7	48.3

Appendix

Table 6: Factual benchmark accuracy (%) before and after merging in the matched harness; the mean is unweighted across the nine tasks.

Setting	Value	Setting	Value
Training examples	400,000	Optimizer updates	25,000
Maximum sequence length	256	Dynamic padding multiple	Llama: 8; others: 16
Batch size	16/16	Gradient accumulation	1
Optimizer	AdamW	Peak learning rate	$10^{-5}$
Schedule	cosine	Warmup	100 updates
Weight decay	0.01	Maximum gradient norm	0.5

Appendix

Table 7: Full common training configuration. Model-specific parent checkpoints, mask strings, and parameter counts appear in Table 2 .

Task	Output tokens	NFE	Block length	Temperature	Terminator handling
Open-ended	128	128/64/32	128	0.7	delayed retention
MMLU	64	64	64	0	ordinary
MMLU-Pro	128	128	128	0	delayed retention
HellaSwag	64	64	64	0	ordinary
ARC-Challenge	64	64	64	0	ordinary
GSM8K	256	256	256	0	delayed retention

Appendix

Table 8: Task-specific masked-diffusion inference settings in the validation notebook. NFE is the number of denoising network evaluations.

Task	Tokens	Task	Tokens
Open-ended	128	MMLU	64
MMLU-Pro	2,048	HellaSwag	64
ARC-Challenge	64	GSM8K	256
MATH-500	256	GPQA-Main	128
HumanEval	256	MBPP	128

Appendix

Table 9: Maximum new tokens for paired greedy AR evaluation.

Model	Benchmark	Invalid / 125	Rate
BYOD-Llama-50k	HumanEval	35	28.0%
Llama AR	HumanEval	0	0.0%
BYOD-Llama-50k	MBPP	11	8.8%
Llama AR	MBPP	0	0.0%

Appendix

Table 10: Parser failures on paired code-generation outputs.

Item	Generated fragment	Parser-level failure
HumanEval/10	defdef make_palindrome(string: str) -> str:	Duplicated def keyword.
HumanEval/2	return number - int(number	Missing closing parenthesis.
MBPP/25	return [t for t in tuples if all(x % k == 0 for x in t]	Square bracket closes an open parenthesis.
MBPP/79	return set(num for for sublist in list_of_lists)	Duplicated for keyword.

Appendix

Table 11: Representative local syntax errors in saved BYOD-Llama-50k outputs. Fragments are reproduced verbatim after the same code extraction used for evaluation.

	Diffusion						AR parent
Model/checkpoint	PPL ${}_{32}\downarrow$	D-1 32 % $\uparrow$	PPL ${}_{64}\downarrow$	D-1 64 % $\uparrow$	PPL ${}_{128}\downarrow$	D-1 128 % $\uparrow$	PPL $\downarrow$	D-1% $\uparrow$
BYOD-Gemma	14.56	44.6	8.27	45.5	4.33	51.2	3.01	63.4
BYOD-Qwen	20.25	43.0	11.12	50.9	6.21	56.0	2.23	69.5
BYOD-Ministral	15.45	44.5	9.42	49.3	5.08	55.5	2.31	65.1
BYOD-Llama-25k	19.38	49.5	10.50	52.7	5.08	51.9	2.49	61.0
BYOD-Llama-50k	17.00	43.5	9.53	49.6	4.83	50.7	2.49	61.0

Appendix

Table 12: Open-ended generation on a 128-token canvas. PPL k is token-weighted Phi-4 perplexity; D-

n

columns report mean per-answer sliding model-token Distinct-

n

k

NFE. Lower PPL and higher Distinct-

n

are better. AR parents generate autoregressively and therefore have no diffusion-NFE setting.

Figure 5: Sliding model-token Distinct-1/2/3 for the final models across diffusion inference budgets. Crosses show the paired AR parents; BYOD-Llama-25k and BYOD-Llama-50k share the same parent. Higher-order metrics show the same broad diversity pattern while compressing differences because longer

n

-grams are less likely to repeat exactly.

Figure 6: Sliding model-token Distinct-1/2/3 throughout BYOD-Llama-50k training. Solid colored lines are BYOD-Llama checkpoints at each NFE budget; matching dashed lines are fixed LLaDA references, and the dotted gray line is Llama AR. All three metrics fluctuate across checkpoints without a strong monotonic trend.

Model	With clue EM	Without clue EM	EM gain	Prediction changed
BYOD-Gemma	100%	0%	+100 pp	100%
BYOD-Qwen	20%	0%	+20 pp	100%
BYOD-Ministral	100%	0%	+100 pp	100%
BYOD-Llama-25k	90%	0%	+90 pp	100%
BYOD-Llama-50k	70%	0%	+70 pp	100%
LLaDA 8B Instruct	100%	0%	+100 pp	100%

Appendix

Table 13: Bidirectional attention test results. “With clue” predicts the earlier mask while the word is visible later; “without clue” removes that later word. Gain is the exact-match difference.

Task	BYOD-Llama (ours)	LLaDA	Dream-Inst.	iLLaDA-Inst.	Efficient-DLM 8B
ARC-C	.832	.885	NR	NR	.619
GPQA	.296	.333	.330	NR	NR
GSM8K	.656	.694	.810	.890	.883
HellaSwag	.704	.746	NR	NR	.725
HumanEval	.312	.494	.555	.659	.683
MATH	.256	.319	.392	.567	.501

Appendix

Table 14: Our Llama result and protocol-separated published diffusion-model accuracies. Bold marks the largest reported value in each row; it does not imply a controlled comparison. No cross-task averages are shown because task coverage differs by model.

Panel of Figure 7 — Figure 7: Confidence-guided BYOD-Gemma-2-9B trajectories at decreasing inference budgets on a 64-token canvas. Red-to-green coloring records each retained token’s sampling probability.

Panel of Figure 8 — Figure 8: Remasking comparison for BYOD-Gemma-2-9B at 64 NFE on a 64-token canvas. Blue intensity records the iteration at which each retained token was predicted.

Diffusion Language Models

Diffusion language models generate and refine text through iterative denoising, enabling bidirectional context and parallel token updates.

See all 488 papers

Context-Tower Conversion Preserves Generation While Freezing Retains Knowledge: Low-Budget AR-to-Diffusion Conversion of MoE LLMs
Oct 2, 2026Wentao Lu, Tianyu Zhu, Jesse ClarkDiffusion Language Models
Celeris, San Francisco, CA
ALoDLM: Adaptively Looped Diffusion Language Models
Oct 3, 2026Liancheng Fang, Zhuowei Li, Youngeun Kim +10Adaptive Computation for Diffusion Models Diffusion Language Models
University of Illinois Chicago · Amazon AGI · Korea University
DiffusionGemma Technical Report
Jul 31, 2026DiffusionGemma Team, Adrien Ali Taïga, James Assiene +41Diffusion Language Model Decoding Blockwise Diffusion Language Models

Oct 2, 2026J/K move · Enter open · S save

Context-Tower Conversion Preserves Generation While Freezing Retains Knowledge: Low-Budget AR-to-Diffusion Conversion of MoE LLMs

Wentao Lu, Tianyu Zhu, Jesse Clark

Celeris, San Francisco, CA

Diffusion Language Models

Open paper AskPDF arXiv

Converting a pretrained autoregressive (AR) model to a diffusion language model (dLLM) enables parallel generation without pretraining a new model. Published conversion methods differ by roughly three orders of magnitude in training data and have not been compared under a common protocol. We compare two conversions of the same 30B Mixture-of-Experts (MoE) parent, holding the corpus, supervised-token budget, trainable parameter set and evaluation harness fixed, each under its own training recipe. The in-place model updates a subset of the parent's weights using denoising and representation-alignment losses; the frozen-tower model instead conditions through cross-attention on a frozen causal copy of the parent. With 1B training tokens, the frozen-tower model scores 71.60 on HumanEval pass@10 against 6.19 for the in-place model, an 11.6x improvement. At the same budget it also keeps 95% of the parent's GSM8K score and 99% of its MMLU-Pro score. A dense-parent experiment reproduces the HumanEval separation. Within the two-tower design at about 500M tokens, freezing the context tower retains substantially more MMLU-Pro performance than training it, while both give similar observed HumanEval scores. Our theoretical analysis establishes that both conversion classes contain an exact sampler for the AR parent under a hard attention mask and left-to-right commitment of one position per round. Under a shared loss, freezing removes the gradient contribution through the context states. Furthermore, evaluation protocol substantially affects a published 500B-token conversion's scores in both directions across tasks, while its AR parent's scores vary by less than three points, so comparing dLLMs needs a common protocol. These results show that, in the tested low-budget regime, the frozen-tower configuration retains substantially more of the parent's generation performance than in-place conversion.

Abstract

Diffusion Language Models

Explore similar work

Context-Tower Conversion Preserves Generation While Freezing Retains Knowledge: Low-Budget AR-to-Diffusion Conversion of MoE LLMs

ALoDLM: Adaptively Looped Diffusion Language Models

DiffusionGemma Technical Report