Enhancing Diffusion Language Models with Autoregressive Post-Training Weights

Yiming Qin; Ke Wang; Amel Abdelraheem; Adam Hazimeh; Pascal Frossard

cs.LGOct 6, 2026

Enhancing Diffusion Language Models with Autoregressive Post-Training Weights

Authors: Yiming Qin, Ke Wang, Amel Abdelraheem, Adam Hazimeh, Pascal Frossard

Organizations: EPFL, Lausanne, Switzerland

Abstract

Diffusion language models (dLLMs) have emerged as a promising alternative to autoregressive (AR) language models, offering flexible token-update orders and parallel decoding. Recent dLLMs are often initialized from pretrained AR models before diffusion conversion in order to inherit their learned representations. After the conversion, however, they typically ignore the extensive post-training ecosystem of their AR ancestors. In this work, we show that these existing AR post-training weight updates can instead be effectively recycled to enhance diffusion models. Despite the changes by AR-to-diffusion conversion, directly adding an AR post-training weight update to a diffusion base model remains effective, bringing its performance close to that achieved by direct diffusion post-training. Notably, AR and diffusion post-training updates are nearly orthogonal in weight space, yet induce substantially more aligned representation changes in the diffusion model. Their distinct updates are also complementary: composing their weights can retain gains from both regimes and further improve the post-trained diffusion model. Based on these findings, we propose A2D, a simple training-free framework for enhancing diffusion models with existing AR post-training resources. A2D can transfer capabilities from AR post-trained models to diffusion base models, and further improve already post-trained diffusion models by composing AR and diffusion post-training updates. Across various dLLMs, including Dream, DreamReasoner, DiffuCoder, Dream-Coder, Nemotron-Labs-Diffusion, and DiffusionGemma, A2D reliably improves instruction following, mathematical reasoning, and coding with both supervised fine-tuning and reinforcement learning updates, without additional training, or inference-time computation.

Figures & tables

Panel of Figure 1 — Figure 1: Illustration of notations and method . (a) Diffusion base models with weights $\bm{D}_{b}$ are often adapted from AR base models with weights $\bm{A}_{b}$ . The two models often undergo separate post-training stages, leading to $\bm{D}_{p}$ and $\bm{A}_{p}$ , with the corresponding task vectors denoted by $\bm{\tau}_{D}$ and $\bm{\tau}_{A}$ , respectively. (b) A2D -Transfer directly adds the AR post-training vector to diffusion base $\bm{D}_{b}$ to create a post-trained diffusion model. A2D -Merge interpolates this model and the diffusion model’s own post-trained checkpoint to further improve downstream capabilities.

Panel of Figure 2 — Figure 2: A2D on Dream. A2D improves diffusion base and post-trained models without training through (a) direct AR update transfer and (b) AR–diffusion update merging.

Panel of Figure 3 — Figure 3: Cross-paradigm transfer of AR post-training. (a) AR task vectors remain effective on the diffusion base. (b) Despite weak parameter alignment, AR and diffusion post-training induce more aligned representation changes than randomized controls.

Panel of Figure 4 — Figure 4: Why AR and diffusion post-training can be composed. (a) Performance over $\bm{D}_{b}+\alpha\bm{\tau}_{A}+\beta\bm{\tau}_{D}$ . The interpolation path between the AR-transferred and diffusion-post-trained endpoints passes through high-performing regions and contains solutions that outperform both endpoints. (b) Among examples failed by $\bm{D}_{b}$ , the two endpoints exhibit both shared and endpoint-specific successes. The best interpolated model preserves most shared successes while retaining gains unique to either endpoint, indicating limited functional interference under composition.

Table 1: Evaluation of A2D across tasks and model families. Headers denote AR donor

\rightarrow

diffusion recipient . A2D (transfer) applies the AR donor’s task vector to the diffusion recipient, while A2D (merge) composes it with diffusion post-training. Anchor tasks for scaling coefficient search are highlighted. All results are in percentages.

Table 6

Appendix figures & tables22 assets

Supplementary material from the paper’s appendix.

Appendix

ID	Ministral Base	Ministral Instruct	Nemotron Base	Nemotron Instruct
10	[IMG]	[IMG]	<\|im_start\|>	<\|im_start\|>
11	<pad>	<pad>	<\|im_end\|>	<\|im_end\|>
12	[IMG_BREAK]	[IMG_BREAK]	<think>	<think>
13	[IMG_END]	[IMG_END]	</think>	</think>
14	[PREFIX]	[PREFIX]	<tool_call>	<tool_call>
15	[MIDDLE]	[MIDDLE]	</tool_call>	</tool_call>

Appendix

Table 5: Special-token mismatches across the four 3B checkpoints. All listed input-embedding rows are masked during task-vector composition.

AR $\rightarrow$ Diffusion	vector type	anchor	transfer $\alpha$	merge $\lambda$
Qwen2.5-7B $\rightarrow$ Dream-7B	instruct (SFT)	IFEval	0.9	0.6
Qwen2.5-Coder $\rightarrow$ Dream-Coder	code (SFT)	HumanEval+	0.6	0.2
Qwen2.5-Coder $\rightarrow$ DiffuCoder	code (SFT)	HumanEval+	0.1	0.2
AceCoder-Rule $\rightarrow$ DiffuCoder	code (RL)	MBPP+	0.2	0.5
Qwen3-8B $\rightarrow$ DreamReasoner	reasoning	MATH-500	0.5	0.2
Ministral-3-3B $\rightarrow$ Nemotron-LD-3B	instruct (SFT)	IFEval	0.9	0.3

Appendix

Table 6: Coefficient selection for each transfer family. Each coefficient is selected on its anchor benchmark and kept fixed on all other benchmarks. For the Nemotron Reasoning-donor row, the anchors are listed in transfer / merge order: IFEval without thinking and MATH-500 with thinking, respectively. Full sweeps are reported in Tables 8 and 7 .

AR $\rightarrow$ Diffusion	Anchor	$\lambda$
		$0$ (parent)	$0.1$	$0.2$	$0.3$	$0.4$	$0.5$	$0.6$	$0.7$	$0.8$	$1.0$
Qwen2.5-7B $\rightarrow$ Dream-7B	IFEval	63.0	65.0	67.6	—	69.8	70.3	72.8	72.6	71.5	56.7
Qwen2.5-Coder $\rightarrow$ Dream-Coder	HumanEval+	77.4	76.2	79.3	76.2	77.4	—	—	—	—	—
Qwen2.5-Coder $\rightarrow$ DiffuCoder	HumanEval+	65.9	70.7	74.4	72.6	71.3	—	—	—	—	—
AceCoder-Rule $\rightarrow$ DiffuCoder (RL)	MBPP+	66.9	—	66.4	69.0	68.3	69.0	66.9	—	—	—
Qwen3-8B $\rightarrow$ DreamReasoner	MATH-500	93.6	93.2	94.8	94.0	94.2	—	—	—	—	—

Appendix

Table 7: Merge ,

D_{b}+(1-\lambda)\tau_{D}+\lambda\tau_{A}

, swept on each family’s anchor benchmark. For DiffusionGemma,

D_{b}

denotes the instruction-tuned reference checkpoint. Shaded = selected

\lambda

, bold = best score. All results are in percentages.

AR $\rightarrow$ Diffusion	Anchor	$\alpha$
		$0$ (base)	$0.1$	$0.2$	$0.3$	$0.4$	$0.5$	$0.6$	$0.7$	$0.8$	$0.9$	$1.0$	$1.1$	$1.3$	$1.4$	$1.5$
Qwen2.5-7B $\rightarrow$ Dream-7B	IFEval	50.5	—	54.1	—	—	58.4	58.7	59.1	58.4	59.3	56.7	—	—	—	—
Qwen2.5-Coder $\rightarrow$ Dream-Coder	HumanEval+	57.9	—	—	—	65.9	65.9	67.1	61.0	59.1	—	—	—	—	—	—
Qwen2.5-Coder $\rightarrow$ DiffuCoder	HumanEval+	61.6	60.4	57.3	—	47.6	—	—	—	—	—	—	—	—	—	—
AceCoder-Rule $\rightarrow$ DiffuCoder (RL)	MBPP+	65.9	66.1	66.9	65.3	65.3	—	—	—	—	—	—	—	—	—	—
Qwen3-8B $\rightarrow$ DreamReasoner	MATH-500	—	—	—	—	74.0	76.0	75.8	—	—	—	—	—	—	—	—

Appendix

Table 8: Transfer ,

D_{b}+\lambda\tau_{A}

: the AR vector on the diffusion base. For DiffusionGemma,

D_{b}

denotes the instruction-tuned reference checkpoint. Shading and bold as above;

\lambda{=}0

is the base. All results are in percentages.

AR $\rightarrow$ Diffusion	Anchor	$\alpha$
		$0$ (parent)	$0.1$	$0.2$	$0.3$	$0.4$	$0.5$	$0.6$	$0.7$	$0.8$	$0.9$	$1.0$	$1.1$	$1.2$
Qwen2.5-7B $\rightarrow$ Dream-7B	IFEval	63.0	—	67.7	—	70.9	—	71.0	73.3	73.5	74.5	74.9	77.1	75.5
Qwen2.5-Coder $\rightarrow$ Dream-Coder	HumanEval+	77.4	—	75.0	76.8	77.4	77.4	75.6	—	—	—	—	—	—
Qwen2.5-Coder $\rightarrow$ DiffuCoder	HumanEval+	65.9	—	66.5	—	68.9	69.5	67.7	60.4	—	—	—	—	—
AceCoder-Rule $\rightarrow$ DiffuCoder (RL)	MBPP+	66.9	—	—	—	66.9	66.9	66.4	—	66.9	—	—	—	—
Qwen3-8B $\rightarrow$ DreamReasoner	MATH-500	93.6	—	93.6	93.6	93.8	94.4	93.6	94.2	—	—	—	—	—

Appendix

Table 9:

D_{p}+\alpha\tau_{A}

: the AR vector added to the post-trained diffusion model. Shading and bold as above. All results are in percentages.

Diffusion (no AR term)	Anchor	$\alpha$
		$0.1$	$0.2$	$0.3$	$0.4$	$0.5$	$0.6$	$0.7$	$0.8$	$0.9$	$1.0$ (parent)
Qwen2.5-7B $\rightarrow$ Dream-7B	IFEval	—	—	—	—	—	61.5	—	61.3	63.4	63.0
Qwen2.5-Coder $\rightarrow$ Dream-Coder	HumanEval+	—	—	—	—	—	—	—	72.0	78.7	77.4
Qwen2.5-Coder $\rightarrow$ DiffuCoder	HumanEval+	—	—	—	68.3	68.9	70.1	65.9	66.5	65.9	65.9
AceCoder-Rule $\rightarrow$ DiffuCoder (RL)	MBPP+	—	65.6	—	66.7	69.3	68.8	69.6	67.5	—	66.9
Qwen3-8B $\rightarrow$ DreamReasoner	MATH-500	—	—	—	—	88.4	89.0	88.8	93.4	89.8	93.6

Appendix

Table 10:

D_{b}+\alpha\tau_{D}

: the model’s own post-training vector rescaled, no AR term. Shading and bold as above. All results are in percentages.

	1,280-token budget		8,192-token budget
$\lambda$	Score	Median length	Score	Median length
0.3	74.03	314	74.07	308
0.4	78.00	332	78.76	336
0.5	75.59	323	76.20	315

Appendix

Table 11: Short-budget IFEval comparison on the 8B A2D merge. Both budgets evaluate the same 541 prompts with thinking off and select

\lambda=0.4

. Scores are percentages; median lengths are generated token counts. Bold marks the best score per budget.

Figure 5: How A2D alters diffusion generation. Left: A2D largely preserves the parent’s decoding order. Right: Prediction changes concentrate on low-confidence tokens.

	Dream	DreamCoder	DiffuCoder	DiffuCoder-RL	DreamReasoner	Nemotron-3B	DiffusionGemma	Avg. rank $\downarrow$	Avg. Acc.
	IFEval	HumanEval+	HumanEval+	MBPP+	MATH-500	IFEval	VQA-RAD	Avg. rank $\downarrow$	Avg. Acc.
Diffusion parent $\ D_{p}$	63.0	77.4	65.9	66.9	93.6	68.0	71.6	3.7	72.3
$+\tau_{A}$ only $\ D_{p}+\lambda\tau_{A}$	77.1	77.4	69.5	66.9	94.4	71.0	74.7	2.4	75.9
$-\tau_{D}$ only $\ D_{p}-\lambda\tau_{D}$	63.4	78.7	70.1	69.6	93.4	72.0	72.5	2.4	74.2
A2D merge $\ D_{p}+\lambda(\tau_{A}-\tau_{D})$	72.8	79.3	74.4	69.0	94.8	72.9	74.5	1.4	76.8

Appendix

Table 12: Sources of the A2D gain. Best-performing point for each construction, with coefficients selected on the anchor task. Avg. rank is the mean within-pair rank (1 = best) and Avg. Acc. is the mean over the seven pairs. All results are in percentages.

	Dream-7B $\rightarrow$ Qwen2.5-7B				DiffuCoder $\rightarrow$ Qwen2.5-Coder		Nemotron-LD-3B $\rightarrow$ Ministral-3-3B
	IFEval	IFBench	GSM8K	MATH-500	HumanEval+	MBPP+	IFEval	MATH-500	MBPP
Diffusion donor	63.0	19.0	83.0	38.4	65.9	65.9	68.0	76.8	73.0
AR base	40.5	16.3	62.5	17.4	74.4	62.2	45.4	60.2	64.3
A2D (transfer, $\alpha{=}1.0$ / $0.1$ / $0.9$ )	46.7	19.0	56.0	16.2	74.4	64.8	55.7	35.0	55.8
AR instruct	75.3	31.7	83.0	56.0	86.6	71.7	67.0	82.4	76.2
A2D (merge, $\lambda{=}0.1$ / $0.2$ / $0.2$ )	77.3	30.7	82.9	56.4	86.6	71.7	69.0	81.4	77.0

Appendix

Table 13: Reverse direction: diffusion post-training vector into the AR model. AR’s rows use their native budget. All results are in percentages.

	A2D-Transfer	A2D-Merge
Full model	40.8	45.8
w/o AR attention	40.8	43.0
w/o AR MLP	30.4	45.6

Appendix

Table 14: Module-wise ablation of the transferred AR update. MATH-500 accuracy on Dream-7B. For A2D-Merge, the ablated module is reverted to the diffusion parent

D_{p}

, while all remaining modules use the merged weights. All results are in percentages.

	Recipient	Benchmark	$\alpha$ in $D_{b}+\alpha\tau_{\mathrm{CPT}}$
			$0$	$0.01$	$0.05$	$1.0$
Math ( $82\times$ ), Qwen2.5-Math $-$ Qwen2.5	Dream-7B-Base	GSM8K	76.6	76.6	74.8	0.0
Code ( $70\times$ ), Qwen2.5-Coder $-$ Qwen2.5	Dream-Coder-Base	HumanEval	63.4	61.6	57.3	0.0

Appendix

Table 15: Continued pre-training vectors do not transfer. Adding the AR continued-pretraining vector to the diffusion base does not improve performance, even at small coefficients. These vectors are

70

–

82\times

larger than the post-training vectors used in our main experiments. Bold denotes the best

\alpha{>}0

result; all results are percentages.

	Qwen2.5-7B $\rightarrow$ Dream-7B
	IFEval	IFBench	GSM8K	MATH-500	HumanEval+	MBPP+
AR donor	75.3	31.7	83.0	36.6	76.8	69.0
Base	49.7	14.0	66.5	20.8	48.8	59.3
A2D (transfer)	59.2	25.0	81.7	40.2	54.9	62.7
Instruct	63.0	19.0	83.0	38.4	51.8	57.1
A2D (merge)	72.8	22.3	83.7	45.0	59.8	58.7

Appendix

Table 16: Qwen2.5-7B

\rightarrow

Dream-7B on six tasks. One transfer model (

\alpha{=}0.9

) and one merged model (

\lambda{=}0.6

), both selected on IFEval, improve every task over the corresponding baseline. All results are in percentages.

	Mode	GSM8K	MATH-500	HumanEval	MBPP	IFEval	MMLU	GPQA	AIME24	AIME25	LCB-CPP	Mean
AR donor	AR	91.3	83.0	80.5	75.9	65.1	67.9	40.9	30.0	13.3	27.1	56.7
Base (raw protocol)	DLM	59.1	40.0	64.6	70.9	23.2	62.0	35.4	0.0	0.0	9.9	38.0
Base (Instruct protocol)	DLM	78.0	34.6	39.6	38.6	25.8	55.1	26.3	0.0	0.0	4.0	30.7
A2D (transfer)	DLM	88.1	52.6	72.6	74.3	60.6	62.8	33.3	3.3	3.3	14.1	44.9
Base (raw protocol)	AR	52.1	50.4	71.3	72.8	21.1	58.0	32.8	10.0	13.3	14.5	41.7
A2D (transfer)	AR	89.2	60.4	73.8	74.6	59.0	62.6	33.3	6.7	0.0	14.5	46.1

Appendix

Table 17: Ministral-3-3B-Instruct

\rightarrow

Nemotron-LD-3B. DLM and AR denote diffusion and autoregressive decoding of the same checkpoint. Mean excludes IFEval. Bold marks the better result within each Instruct-merge pair. All results are in percentages.

	Mode	GSM8K	MATH-500	HumanEval	MBPP	IFEval	MMLU	GPQA	AIME24	AIME25	LCB-CPP	Mean
AR donor	AR	93.2	87.8	77.4	77.8	69.5	76.7	50.0	30.0	26.7	34.8	61.6
Base (raw protocol)	DLM	64.7	51.2	77.4	82.0	28.2	38.4	34.3	13.3	20.0	17.0	44.3
Base (Instruct protocol)	DLM	78.2	59.8	4.9	13.2	21.5	—	9.1	6.7	10.0	2.2	23.0
A2D (transfer)	DLM	89.5	71.6	75.0	81.7	65.7	72.4	37.4	13.3	6.7	11.2	51.0
Base (raw protocol)	AR	63.8	60.2	76.2	79.9	26.0	53.6	33.3	13.3	20.0	13.4	46.0
A2D (transfer)	AR	89.7	69.8	74.4	79.4	62.6	70.1	39.9	6.7	13.3	12.8	50.7

Appendix

Table 18: Ministral-3-8B

\rightarrow

Nemotron-LD-8B. DLM and AR denote diffusion and autoregressive decoding of the same checkpoint. Mean excludes IFEval and, when unavailable, MMLU, which under some settings exhibits repetitive generation and requires over four days to complete. Bold marks the better result within each corresponding pair. All results are in percentages.

Figure 6: Generation-length mismatch in Nemotron-LD. The 8B model, unlike the 3B one, exhibits longer, reasoning-like generations even with thinking disabled. Enabling thinking substantially reduces the length mismatch between the parent and merged models.

	Gemma 4 $\rightarrow$ DiffusionGemma
	VQA-RAD	SLAKE	VQA-Med
AR donor	58.09	80.68	37.20
Base	57.43	63.24	43.20
A2D (transfer)	56.54	77.10	37.20
Post-trained	60.75	83.13	39.60
A2D (merge)	61.64	83.32	40.00

Appendix

Table 19: AR SLAKE post-training transfer from Gemma to DiffusionGemma. AR donor denotes the AR checkpoint providing the task vector. Base denotes the instruction-tuned diffusion checkpoint. Anchor task used for hyperparameter selection is highlighted. Bold marks the best result compared to the corresponding baseline. All results are in percentages.

	MATH-500			MBPP
Model	Acc. $\uparrow$	TPF $\uparrow$	Rel. time $\downarrow$	Acc. $\uparrow$	TPF $\uparrow$	Rel. time $\downarrow$
Ministral-3-3B-Instruct $\rightarrow$ Nemotron-LD-3B, thinking off
Instruct parent	76.6	4.96	1.00	69.8	3.71	1.00
A2D (merge)	80.8	5.04	0.87	74.9	3.90	0.80
Ministral-3-8B-Instruct $\rightarrow$ Nemotron-LD-8B, thinking off
Instruct parent	87.0	5.34	1.00	68.0	3.42	1.00

Appendix

Table 20: Accuracy and efficiency of self-speculative decoding. TPF is

\sum_{i}T_{i}/\sum_{i}\mathrm{NFE}_{i}

, including thinking tokens. Relative time is normalized generation time per token within each dataset, model size, and thinking setting; lower is better. Timings are observational and exclude model loading and final grading. Accuracy results are percentages (MATH-500 accuracy; MBPP base-test pass@1), while TPF and relative time are ratios. Accuracy is in percentages.

diffusion $\leftarrow$ AR	post-training	$\\|\tau_{A}\\|$	$\\|\tau_{D}\\|$	$\\|\Delta_{b}\\|$	$\\|\Delta_{p}\\|$	$\cos(\Delta_{b},\Delta_{p})$	$\cos(\tau_{A},\tau_{D})$
Qwen2.5-7B $\rightarrow$ Dream-7B	instruct (SFT)	23.0	31.5	729	732	$+0.999$	$+0.013$
Qwen2.5-Coder $\rightarrow$ Dream-Coder	code (SFT)	87.0	84.6	589	602	$+0.980$	$+0.023$
Qwen3-8B $\rightarrow$ DreamReasoner	reasoning (SFT / SFT+RL)	189.2	295.8	459	556	$+0.803$	$+0.118$
Ministral-3 $\rightarrow$ Nemotron-LD-3B	instruct (SFT), cross-arch.	43.1	17.1	280.9	271.4	$+0.987$	$+0.075$
Ministral-3 $\rightarrow$ Nemotron-LD-8B	instruct (SFT), cross-arch.	42.2	16.3	387.0	380.4	$+0.994$	$+0.070$
Qwen2.5-Coder $\rightarrow$ DiffuCoder	code (SFT)	87.0	33.3	627	630	$+0.989$	$+0.005$

Appendix

Table 21: Geometry of post-training and paradigm-conversion directions across model pairs. Measurements use backbone parameters and, for Nemotron, compatible input embeddings after key mapping, excluding protected rows. Output heads are excluded throughout.

Figure 7: Correction increases with the AR contribution while error introduction remains limited over the effective interpolation range. We report accuracy, correction rate (parent wrong

\to

A2D correct), and error introduction rate (parent correct

\to

A2D wrong) along the A2D interpolation path. Dashed lines denote the diffusion parent, and stars mark the best-performing coefficient.

Figure 8: A2D keeps the successes of both endpoints. For every item of the benchmark, the inner ring records which of the diffusion parent and the AR donor solves it. The outer ring records whether the merged model solves it.

Figure 9: Distinct parameter updates induce aligned representation changes for released checkpoints.

Explore similar work

Data-Efficient Autoregressive-to-Diffusion Language Models via On-Policy Distillation
Jun 4, 2026Xingyu Su, Jacob Helwig, Shubham Parashar +6Diffusion Language Models Autoregressive Model
Department of Computer Science and Engineering, Texas A&M University · Department of Bioinformatics and Systems Medicine,2026 University of Texas Health Science Center at Houston · Department of Electrical and Computer Engineering, Texas A&M UniversityJun 4
Beyond Fully Random Masking: Attention-Guided Denoising and Optimization for Diffusion Language Models
Jun 10, 2026Jia Deng, Junyi Li, Wayne Xin Zhao +3Diffusion Language Models
Gaoling School of Artificial Intelligence, Renmin University of China · Department of Data Science, City University of Hong Kong · Beijing Key Laboratory of Research on Large Models and Intelligent Governance +2
DARE: Diffusion Language Model Activation Reuse for Efficient Inference
May 1, 2026Natalia Frumkin, Bokun Wang, Hung-Yueh Chiang +3Diffusion Language Models Efficient Inference
Chandra Family Department of Electrical and Computer Engineering The University of Texas at Austin · Cornell University

Jun 4, 2026, cs.CLJ/K move · Enter open · S save

Data-Efficient Autoregressive-to-Diffusion Language Models via On-Policy Distillation

Xingyu Su, Jacob Helwig, Shubham Parashar, Atharv Chagi+5

Department of Computer Science and Engineering, Texas A&M University · Department of Bioinformatics and Systems Medicine,2026 University of Texas Health Science Center at Houston · Department of Electrical and Computer Engineering, Texas A&M UniversityJun 4

Diffusion Language Models Autoregressive Model Training-Inference Mismatch

Open paper AskPDF arXiv

We study the transformation of autoregressive models (ARLMs) into diffusion language models (DLMs). Rather than pretraining from scratch, prior work replaces the causal attention in ARLMs with bidirectional attention and then trains the resulting model using a DLM objective. However, these approaches incur two distribution shifts. First, transitioning from a next-token prediction objective to a DLM objective can discard knowledge acquired by the ARLM during training. Second, standard DLMs suffer from a train-inference mismatch, as the training loss is defined on randomly masked sequences rather than the trajectories encountered at inference produced by confidence-based decoding. To address both challenges, we introduce an On-Policy Diffusion Language Model (OPDLM) in which On-Policy Distillation (OPD) is employed for ARLM-to-DLM transformation. Specifically, OPDLM is trained via self-OPD, where the student, an ARLM with bidirectional attention, generates its own trajectories, and the teacher, the original frozen ARLM, distills its knowledge by providing target logits on these trajectories. By training directly in an on-policy manner, OPDLM eliminates the train-inference mismatch in DLMs, while distillation from the original model enhances knowledge retention from the ARLM. Empirical results demonstrate that OPDLM requires 15x to 7,000x fewer training tokens with strong performance across a wide variety of tasks. OPDLM avoids the prohibitive cost of DLM pretraining and positions DLM transformation as a form of ARLM post-training.

Abstract

Explore similar work

Data-Efficient Autoregressive-to-Diffusion Language Models via On-Policy Distillation

Beyond Fully Random Masking: Attention-Guided Denoising and Optimization for Diffusion Language Models

DARE: Diffusion Language Model Activation Reuse for Efficient Inference