On-Policy Residual Learning for Diffusion Language Models

Yifan Yu; Yuqing Jian; Jisen Li; Yuezhou Hu; Tianyi Zhang; Xiaoxia Wu; Rui-Jie Zhu; Junxiong Wang

cs.LGOct 14, 2026

On-Policy Residual Learning for Diffusion Language Models

Authors: Yifan Yu, Yuqing Jian, Jisen Li, Yuezhou Hu, Tianyi Zhang, Xiaoxia Wu, Rui-Jie Zhu, Junxiong Wang,

Organizations: Together AI · University of Illinois Urbana-Champaign · University of California, Berkeley · University of California, Santa Cruz ·

Abstract

Residual context is key to unlocking the speed-quality Pareto frontier of diffusion language models (DLMs) by carrying forward predictive distributions across denoising steps. Yet turning this seemingly simple mechanism into an effective learning paradigm is far from simple: there is no clear recipe for how residual context should be learned. We introduce ORL, a framework for on-policy residual learning in DLMs. Algorithmically, we identify that residual learning is inherently on-policy. ORL learns from model-induced residual trajectories on the fly and introduces an entropy-contraction objective that sharpens predictions as the model revisits its own predictions. System-wise, exact residual supervision requires capturing, retaining, and reprocessing many trajectory-dependent states. ORL co-designs the algorithm with asynchronous residual capture and transport, specialized block-diffusion attention, and packed compute-balanced training, making it practical at scale. On a 4B DLM converted from Qwen3-4B, ORL improves tokens per forward (TPF) by up to 6.2x and, with our dlmrt engine, end-to-end throughput by approximately 6.9x at comparable or better quality. In single-request serving on one H100, this mode achieves a stronger speed-quality Pareto frontier than autoregressive models with EAGLE3 and DFlash speculative decoding under both strict and relaxed verification, running 1.74x faster than DFlash at matched accuracy. ORL also transfers to existing DLMs: on LLaDA-2.1-mini, 36 H100-hours and 4.2M self-generated tokens, about 10x less data than prior LLaDA recipes, yield up to 1.5x higher TPF and up to 3x higher throughput at comparable quality, and on DiffusionGemma, ORL raises TPF by 1.1-1.2x. With dlmrt and NVFP4 quantization, ORL models decode a single request at up to 5,046 tokens/s on one B200, 7,708 on eight B200s, and 2,788 on a consumer RTX 5090, with comparable quality.

Figures & tables

Figure 1: (a) On-policy residual training. Residual predictions

r_{k}

are carried across denoising steps. ORL trains on the fresh residuals produced by the current rollout (on-policy) rather than on cached residuals that no longer match the rollout (offline), and enforces entropy contraction across steps, applied only where entropy stalls, so that more tokens are committed. (b) A DLM target that learns to refine its residuals reaches the speculative-decoding frontier. Average GSM8K/HumanEval quality.

Panel of Figure 2 — Figure 2: Three observations behind ORL . (a) Only fresh trajectories jointly improve accuracy and TPF. (b) Standard residual supervision leaves entropy nearly flat across denoising steps. (c) Exact on-policy training incurs substantial time and memory overhead.

Figure 3: Overview of ORL . The current model generates residual-conditioned trajectories with exact state capture; verified states are packed and load-balanced across GPUs, replayed with staircase attention, and optimized using CE and AEC.

Model / training	GSM8K		MATH-500		HumanEval		MBPP
	Acc. $\uparrow$	TPF $\uparrow$	Acc. $\uparrow$	TPF $\uparrow$	pass@1 $\uparrow$	TPF $\uparrow$	pass@1 $\uparrow$	TPF $\uparrow$
Qwen-based models and other DLM baselines
d3LLM–Dream	81.4	4.9	38.2	3.9	57.1	3.2	55.6	3.0
Fast-dLLM-v2-7B	77.5	2.2	48.7	2.6	61.7	2.6	50.1	2.0
RCD–SDAR-4B-b64	87.0	—	68.4	—	—	—	—	—
TraDo-4B (TraceRL, dynamic)	90.3	—	71.8	—	—	—	—	—

Table 1: Generation quality and parallel decoding efficiency. We report benchmark scores (%) and tokens per forward (TPF). Our results evaluate a single checkpoint across all benchmarks. Base and ORL use their selected family-specific decoders (Appendix E.1 ), so arrows compare complete model–decoder systems rather than weights alone. Within each model-family block, the best result is bolded and the starting point for ORL is underlined . Red arrows indicate ORL’s TPF improvement.

Dataset	LLaDA Base	LLaDA + ORL	DG Base	DG + ORL
AIME-24	50.0 /5.31	48.3/ 7.77	55.8/18.15	56.3 / 22.02
MMLU-Pro	57.7/3.60	58.2 / 4.88	78.6 /18.57	78.1/ 22.60
GPQA-D	49.5 /3.05	46.1/ 4.21	65.7 /17.50	65.3/ 21.27
LCB-v6	28.0/5.90	30.3 / 7.43	53.5 /19.96	51.4/ 24.09
IFEval	84.1 /1.53	82.3/ 1.74	66.9 /9.90	66.9 / 11.68

Table 2: Generalization across domains. Post-training of LLaDA-2.1-mini and DiffusionGemma (DG) across additional benchmarks. Each entry reports quality / TPF .

Figure 4: Common-task quality–throughput sweeps: mean GSM8K/HumanEval accuracy versus clean-request TPS for every Base and ORL setting. Arrows mark the highlighted comparable-accuracy pairs, with

5.60\times

(QwenDLM) and

2.95\times

(LLaDA-2.1-mini) clean-TPS gains. Appendix G.4 lists the settings and reports completed-output TPS. All settings are measured with SGLang.

(a) Training-design ablation			(b) Hyperparameter sensitivity
Configuration	GSM8K	MATH-500	Configuration	GSM8K	MATH-500
Full	94.5 / 6.25	90.0 / 5.64	Default	94.5 / 6.25	90.0 / 5.64
$-$ AEC	92.6 / 4.48	92.6 / 4.38	$\lambda_{\mathrm{AEC}}=12.5$	94.6 / 5.53	92.8 / 5.06
$-$ entropy-gated RKL	87.8 / 3.29	76.8 / 3.46	$\lambda_{\mathrm{AEC}}=200$	92.6 / 8.53	88.2 / 7.42
$-$ residual pathway	92.0 / 3.68	75.0 / 3.56	RKL cap $=0.1$	93.6 / 6.51	92.2 / 5.83
			RKL cap $=20$	93.8 / 6.80	93.2 / 5.89

Table 3: Each component matters, and performance is robust to hyperparameters. Accuracy (%) / TPF on GSM8K and MATH-500. All variants train QwenDLM with ORL on GSM8K.

		GSM8K		HumanEval
Model		Prior	dlmrt	Prior	dlmrt
QwenDLM	Base	210	328 $\uparrow$ $1.56\times$	164	249 $\uparrow$ $1.52\times$
QwenDLM	+ ORL	718	987 $\uparrow$ $1.37\times$	1148	1609 $\uparrow$ $1.40\times$
LLaDA-2.1-mini	Base	716	918 $\uparrow$ $1.28\times$	1302	1698 $\uparrow$ $1.30\times$
LLaDA-2.1-mini	+ ORL	949	1281 $\uparrow$ $1.35\times$	1530	2124 $\uparrow$ $1.39\times$
DiffusionGemma	Base	990	1271 $\uparrow$ $1.28\times$	963	1196 $\uparrow$ $1.24\times$

Table 5: Clean-request throughput (tokens/s) of the prior engines and dlmrt for a single request on one H100, with the same checkpoints and decoding settings; red arrows give the speedup of dlmrt over the prior engine. The prior engine is SGLang for QwenDLM and LLaDA-2.1-mini and vLLM for DiffusionGemma.

Model	Precision	GSM8K	MATH-500	HumanEval	MBPP
LLaDA-2.1-mini
+ ORL	BF16	92.0 / 1281	87.0 / 1627	89.0 / 2124	77.3 / 1539
+ ORL	NVFP4	92.0 / 2953 $\uparrow$ $2.31\times$	84.0 / 3882 $\uparrow$ $2.39\times$	89.0 / 5046 $\uparrow$ $2.38\times$	75.9 / 3564 $\uparrow$ $2.32\times$
DiffusionGemma
+ ORL	BF16	96.0 / 1456	92.3 / 1415	91.9 / 1378	87.4 / 1361
Base	NVFP4 (NVIDIA)	96.1 / 3627	92.7 / 3198	91.3 / 3464	87.1 / 2991

Table 6: ORL models in BF16 on one H100 and with NVFP4 quantization on one B200, all throughput measured with dlmrt . Each cell reports accuracy (%) / TPS; red arrows give the speedup of NVFP4 on B200 over BF16 on H100, which combines the newer GPU with the lower precision. BF16 accuracy is that of Table 1 and BF16 TPS that of dlmrt in Table 5 . NVFP4 (NVIDIA) is NVIDIA’s NVFP4 checkpoint of the DiffusionGemma base model ( NVIDIA, 2026 ) ; NVFP4, TP8 runs DiffusionGemma + ORL with tensor parallelism across eight B200s (generation seed 1234). Appendix F.2 adds TPF.

Model	GSM8K	MATH-500	HumanEval	MBPP
LLaDA-2.1-mini + ORL	1648	2081	2788	2053
DiffusionGemma + ORL	1540	1408	1490	1465

Table 7: Throughput (tokens/s) of the ORL models with NVFP4 quantization on one consumer GeForce RTX 5090, measured with dlmrt (DiffusionGemma: generation seed 1234).

Appendix figures & tables18 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 10 — Figure 10: Residual-learning motivation. Left: top- $1$ /top- $5$ agreement exceeds the accepted-and-final fraction early in denoising. Right: sparse, GPU-resident capture avoids the baseline’s growing host footprint over the measured range.

Benchmark	Tasks	Eval. seeds LL/DG
AIME-24	30	8 / 8
GPQA-Diamond	198	3 / 3
MMLU-Pro	12,032	1 / 3
LiveCodeBench-v6	175	1 / 3
IFEval	541	1 / 3

Appendix

Table 8: Transfer evaluation. Evaluation-seed counts are generation-seed settings per trained checkpoint (LL: LLaDA; DG: DiffusionGemma). ORL metrics average independent training runs and repeated evaluations; starting-model baselines average repeated evaluations of their reference checkpoints. Each evaluation generates one completion per task. AIME-24 scores 30 tasks from a shared 60-task, two-year generation input.

Figure 11: Cumulative LLaDA inference optimizations in SGLang, HumanEval Quality mode. Each side uses four one-H100 engines, pooled completion TPS, and removes the first warmup request per shard. The ORL checkpoint differs from the main-table checkpoint. The 864-to-1,324 transition had 153/164 identical outputs and accuracy

0.896\to 0.890

, not exact parity. Later changes pass the final gate. The last two normalized gains are 0.60%/1.18%; their raw paired comparison is

1610.2\to 1629.2

TPS.

Projection	cuBLAS BF16	DeepGEMM BF16	HBM floor
QKV ( $2560\to 6144$ )	15.7	15.6	9.4
O ( $4096\to 2560$ )	13.6	12.1	6.3
gate_up ( $2560\to 19456$ )	46.2	45.1	29.7
down ( $9728\to 2560$ )	30.8	26.8	14.9

Appendix

Table 9: Time per call (

\mu

s) of the QwenDLM-4B projections at 256 rows on one H100, against the time to read the BF16 weights at peak HBM bandwidth.

Model	GPU, precision	Round (ms)
QwenDLM + ORL	H100, BF16	5.04
LLaDA-2.1-mini + ORL	H100, BF16	4.99
	B200, NVFP4 experts	2.03
DiffusionGemma + ORL	H100, BF16	16.96
	B200, NVFP4 experts	5.59

Appendix

Table 10: Latency of one denoising round in dlmrt at an empty context on one GPU (median over replays of one captured round).

Model	Configuration	GSM8K	MATH-500	HumanEval	MBPP
LLaDA-2.1-mini-ORL	BF16 (H100)	92.0 / 8.5 / 1281	87.0 / 10.2 / 1627	89.0 / 13.1 / 2124	77.3 / 10.3 / 1539
	NVFP4 (Local-Hessian, B200)	92.0 / 7.54 / 2953	84.0 / 9.72 / 3882	89.0 / 12.23 / 5046	75.9 / 9.79 / 3564
DiffusionGemma-ORL	BF16 (H100)	96.0 / 30.0 / 1456	92.3 / 27.4 / 1415	91.9 / 27.9 / 1378	87.4 / 28.7 / 1361
	NVFP4 (Diagonal-Hessian, B200)	95.6 / 29.25 / 4214	93.4 / 26.69 / 3805	91.9 / 27.32 / 3949	86.3 / 27.96 / 3898
DiffusionGemma	NVFP4 (NVIDIA, B200)	96.1 / 24.81 / 3627	92.7 / 22.79 / 3198	91.3 / 23.91 / 3464	87.1 / 21.60 / 2991

Appendix

Table 11: BF16 and NVFP4 evaluation of ORL models. Each cell reports accuracy (%) / tokens per forward (TPF) / tokens per second (TPS). Each configuration runs on one GPU of the indicated type, with throughput measured with dlmrt . H100 rows list the single-H100 BF16 results: accuracy and TPF as in Table 1 , TPS as in Table 5 . B200 rows report NVFP4 measurements; DiffusionGemma results are averaged over generation seeds 1234–1236, with the ORL decoder on all four tasks. NVFP4 configurations use an FP8 KV cache. The throughput gains in Section 4.3 compare NVFP4 on B200 with BF16 on H100.

Role	Task	Acc. (%)	$L/N$	Length clean/all	Clean TPS	All TPS	Completed TPS
QwenDLM
Base	GSM8K	90.30	1/1319	1723/1759	209.7	213.5	209.0
ORL	GSM8K	92.42	1/1319	1140/1164	718.4	724.8	709.3
Base	HumanEval	82.93	6/164	4929/5948	164.1	196.0	156.5
ORL	HumanEval	89.63	4/164	5621/6283	1148.2	1153.7	1007.0
LLaDA-2.1-mini

Appendix

Table 12: Full-task main runs.

L/N

counts length failures over all tasks. Lengths are mean returned tokens on clean/all requests. TPS columns use Equations 8 and 9 : clean, all-output, and completed-output/all-request-time, respectively. Quality always includes all tasks, with length failures incorrect. All runs use SGLang.

Target / method	Profile	$\tau$	Mean acc. (%)	Clean TPS	Length failures
Qwen / AR	QA	—	93.40	240.1	2
Qwen / EAGLE3	QE01	1	93.47	352.7	3
	QE02	0.1	94.23	404.7	3
	QE03	0.03	88.39	432.3	23
	QE04	0.01	81.08	462.6	69
	QE05	0.003	65.58	483.4	220

Appendix

Table 13: Speculative-decoding results on GSM8K1319 and HumanEval164 (Figure 1 b). Accuracy and clean TPS each average the two benchmark-level metrics. Length failures count as incorrect and are excluded from clean TPS; counts cover both tasks.

\tau

is the verification threshold: a drafted token

v

is accepted when the target model’s probability satisfies

p(v)\geq\tau\max_{u}p(u)

, so

\tau=1

is strict verification. High-failure profiles remain visible, but clean TPS does not charge their failed-request time.

Figure 13: Accuracy–throughput on GSM8K and HumanEval with additional baselines. Both axes average the two benchmark-level metrics. Marker shape denotes the method and color the model family; orange arrows connect each Base DLM to its ORL model. LLaDA-Q and LLaDA-S denote Quality and Speed modes. TPS for CForce, LightningRL, and Nemotron-Diff is estimated as described above. Base and ORL DLM throughput is measured with dlmrt .

Figure 14: Common-task quality–throughput sweeps. Axes average GSM8K and HumanEval; all settings remain visible. Top: clean TPS; bottom: completed-output TPS; charging failed-request time. The highlighted Qwen pair (Base threshold 0.90, ORL entropy budget 0.80) gives

86.09/86.92\%

mean accuracy and

5.60/5.09\times

clean/completed TPS gains. The highlighted LLaDA pair (Base L14, ORL L11) gives

90.49\%

on both sides, with

2.95/3.39\times

gains. Lines connect points in order of increasing threshold or entropy budget; matching refers to observed mean accuracy. All settings are measured with SGLang.

Figure 15: LLaDA Quality-mode Base versus ORL, with all operating points. Lines are independent log-logistic fits within observed ranges; bars are pointwise 95% percentile intervals from 2,000 paired task-vector resamples, conditional on the archived evaluation records. They quantify task-resampling uncertainty, excluding between-run variation and simultaneous coverage across operating points. The HumanEval arrow compares Base profile L14 (threshold 0.99) with ORL profile L11 (threshold 0.85); lower strips show length failures. Profiles are defined in Table 15 .

Figure 16: QwenDLM system ablation at response cap 2,048. Top: median phase times; bottom: median costs per 1k supervised tokens, 100 denoising forwards, and replay row. Medians use post-warmup steps; the main figure instead uses mean step times. Raw runs can differ in sampled workload.

Panel of Figure 17 — Figure 17: Transport and memory diagnostics. Paired handoff bars show median rollout/replay/update times; memory panels show peaks. All GPU configurations fit an 80 GB H100.

Panel of Figure 18 — Figure 18: Residual-path stability and block-size sensitivity. Rescaling primarily changes residual, not backbone, update magnitudes.

Figure 19: QwenDLM residual-to-embedding norm ratio, rising from 0.91 on the first on-policy rollouts to 1.08 during training. The dashed RCD-SDAR-4B reference is a separately measured entropy-weighted diagnostic. The offline-SFT RMS ratio of 1.06 at the same initializer uses a different population and statistic, so the values are not directly comparable. These traces describe residual use, not its causal benefit.

Model	GPUs	Per job (h)	Total (h)	GPU-hours
QwenDLM	8	$5.00+0.66+1.79$	7.45	60
LLaDA-2.1-mini	8	$3.25+1.27$	4.52	36
DiffusionGemma	16	$8.22+9.55$	17.77	284

Appendix

Table 14: Post-training compute for the reported checkpoints. Per-job hours follow chain order; GPU-hours count all allocated H100 80GB GPUs and are rounded.

		HumanEval		GSM8K
Profile	Threshold	$\Delta a$	$R$	$\Delta a$	$R$
L01	0.00	+37.20	1.048	+73.77	0.920
L02	0.05	+37.20	1.096	+73.69	1.380
L03	0.10	+25.61	1.344	+60.05	2.190
L04	0.20	+12.80	1.368	+19.94	2.111
L05	0.30	+16.46	1.392	+10.46	2.159

Appendix

Table 15: LLaDA Base–ORL comparisons across all fourteen shared profiles. Threshold: minimum top-token probability for accepting a position.

\Delta a

: ORL minus Base accuracy in percentage points.

R

: clean completion -TPF ratio, not padded canvas or wall TPS.

	One point out		Adjacent pair out
	Log-logistic	Isotonic	Log-logistic	Isotonic
HE / Base	4.46	3.66	6.65	5.76
HE / ORL	1.69	1.42	2.39	1.31
GSM / Base	6.91	8.28	7.44	9.66
GSM / ORL	0.46	0.44	0.59	0.89

Appendix

Table 16: Conditional accuracy-prediction RMSE (percentage points), including endpoint folds. This holds out operating points, not tasks or seeds.

Diffusion Language Models

Diffusion language models generate and refine text through iterative denoising, enabling bidirectional context and parallel token updates.

See all 473 papers

Jul 31, 2026

CORA-Diff: Confidence-Oriented Residual Acceptance for Efficient Diffusion Language Model Inference

Diffusion language models (DLMs) update many tokens in parallel, yet practical decoders often use a fixed denoising horizon. Many predictions stabilize early, but blockwise decoding continues until all positions are resolved, causing repeated dense forward passes. Existing accelerators often rely on learned filters, modified scores, dependency models, or cache-specific mechanisms. We ask whether native trajectory signals can identify residual positions likely to match the deterministic dense endpoint. We propose CORA-Diff, a training-free method that preserves the original transfer rule and applies confidence-and-persistence gating only to positions that rule leaves unresolved. Accepted tokens remain visible as context, and the block terminates once all positions are resolved. This requires no backbone change, learned acceptance model, or logit modification. Our theory explains why high-confidence, persistent predictions are more likely to match the fixed-horizon dense endpoint, and paired post-intervention trajectories provide direct empirical support. We select one operating point on a separate GSM8K calibration subset and freeze it for all evaluations. Under a matched Learn2PD-style LLaDA protocol, CORA-Diff has the lowest measured runtime in all eight task-length settings. Task scores match or exceed dense decoding in five settings, and the largest observed drop is 1.22 points. Its incremental speedups over EOS-aware dense decoding are 2.70x and 3.32x on GSM8K and HumanEval. It also reaches 13.14x under the fixed-horizon 1024/1024 mechanism-isolation protocol and transfers to Dream without retuning at 3.18x-3.53x. These results show that native confidence and persistence enable reliable residual acceptance, reducing repeated denoising computation while preserving task quality.

Yifan Wu, Yufeng Zhang, Kenli Li

College of Computer Science, Hunan University Changsha 410082, China

Diffusion Model Serving Diffusion Language Model Inference Diffusion Language Model Decoding Diffusion Language Models

Ask PDF

Jun 4, 2026

Data-Efficient Autoregressive-to-Diffusion Language Models via On-Policy Distillation

We study the transformation of autoregressive models (ARLMs) into diffusion language models (DLMs). Rather than pretraining from scratch, prior work replaces the causal attention in ARLMs with bidirectional attention and then trains the resulting model using a DLM objective. However, these approaches incur two distribution shifts. First, transitioning from a next-token prediction objective to a DLM objective can discard knowledge acquired by the ARLM during training. Second, standard DLMs suffer from a train-inference mismatch, as the training loss is defined on randomly masked sequences rather than the trajectories encountered at inference produced by confidence-based decoding. To address both challenges, we introduce an On-Policy Diffusion Language Model (OPDLM) in which On-Policy Distillation (OPD) is employed for ARLM-to-DLM transformation. Specifically, OPDLM is trained via self-OPD, where the student, an ARLM with bidirectional attention, generates its own trajectories, and the teacher, the original frozen ARLM, distills its knowledge by providing target logits on these trajectories. By training directly in an on-policy manner, OPDLM eliminates the train-inference mismatch in DLMs, while distillation from the original model enhances knowledge retention from the ARLM. Empirical results demonstrate that OPDLM requires 15x to 7,000x fewer training tokens with strong performance across a wide variety of tasks. OPDLM avoids the prohibitive cost of DLM pretraining and positions DLM transformation as a form of ARLM post-training.

Xingyu Su, Jacob Helwig, Shubham Parashar +6

Department of Computer Science and Engineering, Texas A&M University · Department of Bioinformatics and Systems Medicine,2026 University of Texas Health Science Center at Houston · Department of Electrical and Computer Engineering, Texas A&M UniversityJun 4

Language Model Distillation Diffusion Language Models Diffusion Language Model Distillation Autoregressive Diffusion On-Policy Distillation Diffusion Language Model Training+1

Ask PDF

Oct 3, 2026

ALoDLM: Adaptively Looped Diffusion Language Models

Diffusion language models (DLMs) enable fast generation by predicting multiple tokens in parallel, but their practical adoption remains limited by a persistent quality gap relative to comparably sized autoregressive (AR) models. We attribute this gap to a computation-difficulty mismatch: within a partially observed sequence, some unknown tokens are easy to predict, while others require substantially more computation. Existing DLMs nevertheless apply uniform computational depth to all unknown positions at each denoising step. We introduce ALoDLM, which replaces uniform computation with token-adaptive latent recurrence. At each denoising step, ALoDLM iteratively refines latent representations and allocates computation according to token difficulty. Tokens ready to commit are fed back as discrete context, while unresolved tokens retain and further refine their latent states through additional recurrent passes. To learn token prediction and computation allocation jointly, we formulate token-wise computation schedules as latent variables and derive a conditional negative evidence lower bound (NELBO). We train ALoDLM at 1.7B and 8B parameter scales. Across eleven benchmarks, ALoDLM outperforms all evaluated DLMs and the corresponding AR baselines in average benchmark score at both scales. ALoDLM also retains fast parallel decoding, yielding a strong quality-efficiency trade-off among evaluated autoregressive and diffusion models under optimized inference engines.

Liancheng Fang, Zhuowei Li, Youngeun Kim +10

University of Illinois Chicago · Amazon AGI · Korea University

Adaptive Computation for Diffusion Models Diffusion Language Models Diffusion Language Model Inference Diffusion Language Model Decoding Diffusion Language Model Training Parallel Decoding for Diffusion Language Models

Ask PDF

Abstract

Diffusion Language Models

Explore similar work

CORA-Diff: Confidence-Oriented Residual Acceptance for Efficient Diffusion Language Model Inference

Data-Efficient Autoregressive-to-Diffusion Language Models via On-Policy Distillation

ALoDLM: Adaptively Looped Diffusion Language Models