Reasoning with Continuous Latent Diffusion

Xiang Cheng

cs.AISep 28, 2026

Reasoning with Continuous Latent Diffusion

Authors: Xiang Cheng

Organizations: Duke University Department of Electrical and Computer Engineering

Abstract

Continuous diffusion generates complete reasoning solutions through iterative refinement in latent space. We introduce the Continuous Embedding Diffusion Reasoner (CEDR), an ELF-based training and inference recipe. Our experiments show that accurate decoding alone does not ensure strong reasoning performance. We therefore learn compact representations from multiple layers of a strong autoregressive teacher. Their decomposition also enables asynchronous denoising at different rates. We show that prompt encodings need only preserve the information required for the correct text-conditional score, rather than exactly match teacher features, and use a staged curriculum to learn a compact prompt encoder that replaces the teacher Transformer at inference. We adapt DiffusionNFT to learned self-conditioning guidance and incorporate gold-solution endpoints to supplement sparse rewards. Our supervised models outperform reported results from recent continuous-diffusion baselines at comparable backbone scales on mathematical reasoning and HumanEval code generation. With a 638M-parameter denoising backbone and learned prompt conditioning, post-NFT CEDR-L achieves 63.74% pass@1 on GSM8K and 24.6% on MATH500 at 64 denoising steps, and 32.85% on HumanEval and 30.18% on HumanEval+ at 128 denoising steps. Code will be available at: https://github.com/chengxiang/CEDR.

Figures & tables

Figure 1: The CEDR training and inference pipeline. Left: Learn projections of Qwen layers 16/24/32 through teacher cross-entropy, then concatenate their outputs (Section 4.1 ). Center: Train the ELF backbone with flow and token-decoding losses and staged prompt learning (Sections 4 and 4.2 ). Right: Denoise the representation groups under asynchronous clocks, then decode the answer in parallel (Section 4.3 ).

Figure 2: Representation and conditioning affect reasoning. CEDR-B learning curves on GSM8K, with 32 steps and

t=\tau^{2}

. The first three variants retain Qwen conditioning; the fourth jointly trains a randomly initialized prompt encoder. Protocol: Appendix C.4 ; values and SDs: Table 12 (Appendix C.4 ).

Figure 3: Qwen layer structure. Linear CKA of teacher-forced activations on the combined discovery and confirmation panels. White markers select layers 16, 24, and 32. The transition after layer 16 motivates retaining that layer, followed by regularly spaced deeper features.

Figure 4: From teacher conditioning to learned prompts. Top: GSM8K; bottom: MATH500. Circles/squares denote CEDR-B/L; open/filled markers denote Qwen/learned prompt encoding. Red line: epoch-12 switch; arrows: immediate MSE-swap loss. Values and settings: Appendix B.3 .

Task / model	Frozen-Qwen flow epochs	Prompt-MSE epochs	Joint epochs (flow endpoint)	Reported NFT update
GSM8K / CEDR-B	12	60	6 (18)	300
GSM8K / CEDR-L	12	30	9 (21)	500
MATH / CEDR-L	12	10	9 (21)	600
OpenCodeInstruct / CEDR-L	12	10	1 (13)	100

Table 1: Training stages by task. Flow-training epochs count passes over answer rows; prompt-MSE epochs count passes over unique prompts and do not advance the flow-training epoch. Joint epochs are additional to the first 12 flow epochs. The last column gives the NFT checkpoint used for reporting, in successful optimizer updates, rather than a further epoch count.

$(\gamma_{16},\gamma_{24},\gamma_{32})$	GSM8K	MATH500
$(1,1,1)$	52.96 ( $\pm 1.13$ )	15.50 ( $\pm 1.27$ )
$(2,2,2)$	57.51 ( $\pm 0.80$ )	17.40 ( $\pm 1.41$ )
$(1.5,2,2.5)$	58.07 ( $\pm 0.64$ )	17.60 ( $\pm 1.98$ )
$(2.5,2,1.5)$	59.25 ( $\pm 0.16$ )	17.90 ( $\pm 1.27$ )

Table 2: Inference clocks change accuracy without retraining. CEDR-L epoch 21, 32 steps; mean pass@1 (%). Protocol and full sweep: Appendix D .

Autoregressive
Model	Embedding Params (M)	Other Params (M)	NFE	GSM8K	MATH500
Qwen3-0.6B-Base ( Yang et al., 2025 )	156	440	—	59.59	—
Qwen3-0.6B, thinking ( Yang et al., 2025 )	156	440	—	—	77.60
Discrete diffusion / blockwise diffusion
LLaDA-8B-Base ( Nie et al., 2025 )	1,040	6,980	1,024	70.30	—
Dream-v0-Base-7B ( Ye et al., 2025 )	1,090	6,530	256	77.20	—

Table 3: Mathematical reasoning across model families, mean pass@1 (%). Parameters are in millions (three significant digits). Embedding Params count vocabulary input/output maps, tied weights once; Other Params count remaining weights, with

(+X)

used once per response. Frozen components are included.

{}^{*}N

denotes

N-1

denoising calls for ELF-REG. Bold/underline mark first/second within each sub-billion continuous-backbone group.

	GSM8K ( $\sim$ 100M)				GSM8K ( $\sim$ 650M)				MATH500 ( $\sim$ 650M)
Method / NFE	8	16	32	64	8	16	32	64	8	16	32	64
FMLM+ Init	15.10	26.10	31.80	33.60	—	—	—	—	—	—	—	—
ELF-REG	∗ 26.20	∗ 34.42	∗ 37.15	∗ 38.11	∗ 47.34	∗ 52.89	∗ 55.04	∗ 55.96	∗ 9.76	∗ 11.53	∗ 12.40	∗ 13.31
CEDR, pre-NFT	26.40	34.48	37.80	39.17	49.41	55.33	57.88	58.67	15.23	17.93	19.65	21.18
CEDR, post-NFT	30.08	37.77	41.13	42.16	54.84	60.55	62.47	63.74	17.45	20.93	22.60	24.68

Table 4: Reasoning at lower NFE, mean pass@1 (%). Table 3 gives model parameter counts. Bold/underline mark first/second within each benchmark and backbone-scale block; ∗ marks one fewer denoising call than the heading. External results follow Agarwal et al. (2026) ; Li et al. (2026) ; Appendix F gives source and protocol details.

Figure 5: CEDR-L before and after NFT on GSM8K and MATH500, with SCCFG 2. Top: oracle pass@

k

; bottom: majority-vote accuracy. Colors denote denoising NFE per sample; the budget is

kS

. Numerical values appear in Appendix G.3 ; Appendix G.2 gives the corresponding CEDR-B curves.

Model	NFE	$k$	$k\times\mathrm{NFE}$	Pass@ $k$ (%)
ELF-REG-L ( Li et al., 2026 )	63	8	504	40.41
ELF-REG-L ( Li et al., 2026 )	63	10	630	44.04
CEDR-L, pre-NFT	64	8	512	47.20
CEDR-L, post-NFT	64	8	512	52.60

Table 5: MATH500 oracle accuracy at higher sample counts. CEDR-L uses learned prompt conditioning, SCCFG 2, and eight saved draws. ELF-REG-L retains its external prompt encoder and reports the two sample counts below. NFE counts denoising calls per sample;

k\times\mathrm{NFE}

makes the differing total budgets explicit. These are cross-paper reported results, with protocol details in Appendix F . Table 17 gives our uncertainty estimates.

Standard scoring: no function-name repair
Model / setting	Denoising NFE	HE	HE+	MBPP-378 base	MBPP+ full
PlaidQ ( Peng et al., 2026b )	128	19.02	18.05	22.18	—
PlaidQ+CFG ( Peng et al., 2026b )	256	22.04	19.70	24.58	—
CEDR-L / frozen Qwen	128	33.69	30.87	38.72	33.99
CEDR-L / prompt MSE 10	128	27.97	25.30	19.61	16.96
CEDR-L, pre-NFT (ours)	128	29.57	26.91	18.45	16.40

Table 6: Code generation under separate scoring conventions. Mean pass@1 (%). HE/HE+ use base/full extended tests; both MBPP columns use 378 tasks. PlaidQ’s “MBPP+” uses base tests. Frozen Qwen and prompt MSE 10 share the epoch-12 backbone; pre-NFT is joint epoch 13, and post-NFT is update 100. Our rows use eight seeds.

{}^{*}128

denotes 127 denoising calls for ELF-REG; all budgets exclude terminal decoding. The alias block allows single-function entry-point repair; rows across blocks should not be ranked together. Appendix H gives remaining protocol differences and seed SDs.

Model	NFE	$k$	HE	HE+	MBPP-378 base	MBPP+ full
Standard scoring: no function-name repair
PlaidQ ( Peng et al., 2026b )	128	10	25.72	24.03	34.33	—
PlaidQ+CFG ( Peng et al., 2026b )	256	10	39.87	36.74	44.79	—
CEDR-L, pre-NFT	128	8	54.27	47.56	39.95	35.98
CEDR-L, post-NFT	128	8	54.27	49.39	43.39	37.83
Single-function name aliasing

Table 7: Oracle code accuracy with multiple attempts. Entries are pass@

k

(%) at the stated

k

, with denoising NFE per sample. Our learned-conditioning rows use eight draws; external pass@10 values remain labeled as such. PlaidQ uses token-prefix conditioning and ELF-REG-L an external contextual encoder. Separate blocks retain standard versus alias scoring, and both MBPP columns use the same 378 tasks. External values are from PlaidQ Table 1 and ELF-REG Table 16; generation protocols and total compute differ.

Appendix figures & tables20 assets

Supplementary material from the paper’s appendix.

Appendix

Component	CEDR-B	CEDR-L
ELF backbone: blocks / width / heads	12 / 768 / 12	32 / 1,280 / 16
ELF input bottleneck	512	512
Prompt Transformer: blocks / width / heads	6 / 768 / 12	6 / 1,280 / 16
Prompt frontend	$2560\to 512\to 768$	$2560\to 512\to 1280$
Prompt output width	1,024	1,024
Frozen Qwen token lookup	$151{,}936\times 2{,}560$

Appendix

Table 8: CEDR architectures. Both flow models use the ELF backbone, shared between denoising and token decoding. The prompt encoder is a separate bidirectional Transformer; its Qwen vocabulary lookup is frozen from initialization onward.

Source	Retained subset	Training rows
OpenMathInstruct-2	gsm8k	292,485
	augmented_gsm8k	1,841,419
Orca-Math	Eligible train rows	147,464
MetaMathQA	GSM types	152,962
Total		2,434,330

Appendix

Table 9: GSM8K training mixture after filtering and question-group splitting. Counts are answer rows, shared by CEDR-B and CEDR-L. All sources originate from their original training splits.

Epoch	Prompt	Correct 42	Correct 123	Mean (SD)	95% CI
GSM8K, CEDR-B; $N=1319$
4	Qwen	92	86	6.75 (0.32)	$[5.76,7.81]$
8	Qwen	401	424	31.27 (1.23)	$[29.11,33.43]$
12	Qwen	514	503	38.55 (0.59)	$[36.28,40.83]$
12	MSE swap	398	381	29.53 (0.91)	$[27.29,31.69]$
16	Joint	482	462	35.78 (1.07)	$[33.55,38.02]$

Appendix

Table 10: Numerical values for Figure 4 . Correct counts for seeds 42/123 and the benchmark size

N

specify the exact per-seed accuracies. Mean pass@1 and sample seed SD (parentheses) are percentages; brackets give the plotted 95% problem-bootstrap intervals.

Probe input	Token accuracy (%)	CE (nats/token)
Fixed layer-32 PCA	99.24 [99.19, 99.28]	0.0548 [0.0502, 0.0597]

Appendix

Table 11: Clean-token decoding from fixed PCA coordinates after two epochs, using final raw probe weights. Brackets give 95% prompt-cluster bootstrap intervals. The input is the representation used by the single-layer generation ablation.

Figure 6: Identity-clock robustness. The four models from Figure 2 , evaluated with

t=\tau

and 32 steps. Points average seeds 42/123 (bars: seed SD). Each epoch uses one trained checkpoint per arm. Numerical values appear in Table 12 .

$t=\tau^{2}$
Epoch	Learned 16/24/32	PCA 16/24/32	PCA layer 32	Epoch-0 joint prompt training
1	3.75 (0.05)	4.85 (0.75)	2.65 (0.11)	2.46 (0.05)
2	12.40 (0.27)	11.18 (0.16)	5.50 (0.70)	3.98 (1.02)
3	20.17 (0.54)	16.38 (0.32)	8.64 (0.43)	3.68 (0.16)
4	26.61 (0.86)	21.49 (0.80)	7.47 (0.05)	7.77 (1.02)
5	29.95 (0.21)	25.85 (0.21)	9.82 (0.16)	7.54 (1.45)

Appendix

Table 12: Numerical learning curves and endpoint summary for Figures 2 and 6 : mean GSM8K accuracy (%) with sample seed SD in parentheses. Curves use two seeds (42/123); the final block reports the four-seed epoch-six comparison (42/123/456/789). The final column trains the prompt encoder jointly from epoch zero; the other three retain Qwen prompt conditioning.

	Validation		Test
Clock powers	GSM8K	MATH	GSM8K	MATH500
$(1,1,1)$	65.00 (0.99)	15.40 (0.28)	52.96 (1.13)	15.50 (1.27)
$(2,2,2)$	72.70 (0.85)	17.80 (0.57)	57.51 (0.80)	17.40 (1.41)
$(1.5,2,2.5)$	72.80 (1.13)	18.30 (0.14)	58.07 (0.64)	17.60 (1.98)
$(2.5,2,1.5)$	72.15 (0.35)	18.70 (0.14)	59.25 (0.16)	17.90 (1.27)

Appendix

Table 13: Four-schedule CEDR-L epoch-21 comparison: mean pass@1 (%) with sample seed SD in parentheses, over seeds 42/123 at 32 denoising steps and SCCFG 2. Validation sets contain 1,000 GSM8K and 500 MATH problems. Test columns include all entries and SDs from Table 2 . Bold marks each column maximum, including ties.

Figure 7: Layer groups expose distinct reconstruction sensitivities. CEDR-B epoch 18 on 64 GSM8K validation problems. Solid curves perturb actual layer groups; dashed curves average eight random partitions with matched widths, times, and injected Gaussian energy. Colors identify the corresponding layer or control slot. Decoder CE changes are relative to the same pass’s clean-input prediction: first without answer self-conditioning (left), then after one refinement on the same corrupted input (right). Bars show 95% CIs; shading is the range of random-partition means, not a confidence interval. Table 14 gives every plotted value.

First prediction
Layer	$\eta$	Actual group [95% CI]	Random mean [min, max]
16	0	$0.000\ [0.000,0.000]$	$0.000\ [0.000,0.000]$
16	0.8	$0.151\ [-0.006,0.315]$	$-0.015\ [-0.053,0.064]$
16	1.6	$0.327\ [0.045,0.633]$	$0.008\ [-0.057,0.136]$
16	2	$0.439\ [0.080,0.848]$	$0.026\ [-0.075,0.209]$
16	3	$0.651\ [0.158,1.272]$	$0.071\ [-0.087,0.340]$

Appendix

Table 14: Numerical values for Figure 7 , in nats/token. Actual-group entries give

\Delta

CE and its 95% problem-bootstrap CI; random-group entries give the mean over eight matched partitions and the minimum/maximum partition means. Noise strength is

\eta

; the layer identifies the matched group width.

GSM8K, CEDR-B, SCCFG 2
Model / stage	8 NFE	16 NFE	32 NFE	64 NFE
Sampling seeds $n$	64	32	16	8
Pre-NFT	$26.40\pm 0.87$	$34.48\pm 0.94$	$37.80\pm 1.20$	$39.17\pm 0.63$
Post-NFT	$30.08\pm 0.87$	$37.77\pm 0.84$	$41.13\pm 0.94$	$42.16\pm 0.88$
GSM8K, CEDR-L, SCCFG 2
Pre-NFT	$49.41\pm 0.86$	$55.33\pm 0.91$	$57.88\pm 1.03$	$58.67\pm 0.69$

Appendix

Table 15: Mean accuracy

\pm

standard deviation across sampling seeds, in percentage points. Each seed evaluates the complete benchmark at the same fixed checkpoint. Columns use 64, 32, 16, and 8 seeds, respectively. The 64-NFE column supplies the results in Table 3 ; SCCFG 3 supplies the MATH500 rows in Tables 3 and 4 . SCCFG 2 MATH500 rows correspond to Figure 5 .

Figure 8: GSM8K CEDR-B before and after NFT. Upper panels show oracle pass@

k

; lower panels show majority-vote accuracy. Colors denote denoising NFE per sample. The same 64/32/16/8 sample cohorts and total budget of 512 denoising calls are used as in Figure 5 . Table 18 gives every plotted value with problem-bootstrap SDs.

			Oracle pass@ $k$		Majority vote
NFE $S$	$k$	$kS$	Pre-NFT	Post-NFT	Pre-NFT	Post-NFT
8	1	8	49.41 (1.04)	54.84 (1.09)	49.41 (1.04)	54.84 (1.09)
8	2	16	60.21 (1.08)	63.89 (1.11)	49.41 (1.04)	54.84 (1.09)
8	4	32	68.84 (1.05)	70.85 (1.08)	56.11 (1.15)	60.04 (1.17)
8	8	64	76.01 (0.99)	76.53 (1.02)	59.92 (1.18)	62.69 (1.20)
8	16	128	81.85 (0.92)	81.34 (0.95)	62.05 (1.21)	64.07 (1.24)

Appendix

Table 16: GSM8K, CEDR-L: every point in Figure 5 , with SCCFG 2. Entries are accuracy percentages with problem-bootstrap SDs in parentheses, not across-seed SDs (Appendix F.4 ). Each denoising depth uses

n=512/S

saved draws per problem;

k

-subset averaging uses all

n

			Oracle pass@ $k$		Majority vote
NFE $S$	$k$	$kS$	Pre-NFT	Post-NFT	Pre-NFT	Post-NFT
8	1	8	13.88 (0.99)	16.31 (1.11)	13.88 (0.99)	16.31 (1.11)
8	2	16	21.10 (1.29)	23.98 (1.40)	13.88 (0.99)	16.31 (1.11)
8	4	32	29.74 (1.55)	32.65 (1.63)	16.87 (1.24)	19.64 (1.37)
8	8	64	39.29 (1.73)	41.82 (1.79)	20.00 (1.44)	22.68 (1.56)
8	16	128	49.27 (1.86)	50.94 (1.90)	22.23 (1.58)	24.59 (1.69)

Appendix

Table 17: MATH500, CEDR-L: every point in Figure 5 , with SCCFG 2. Entries are accuracy percentages with problem-bootstrap SDs in parentheses, not across-seed SDs (Appendix F.4 ). Each denoising depth uses

n=512/S

saved draws per problem;

k

-subset averaging uses all

n

			Oracle pass@ $k$		Majority vote
NFE $S$	$k$	$kS$	Pre-NFT	Post-NFT	Pre-NFT	Post-NFT
8	1	8	26.40 (0.86)	30.08 (0.95)	26.40 (0.86)	30.08 (0.95)
8	2	16	36.10 (1.02)	39.21 (1.08)	26.40 (0.86)	30.08 (0.95)
8	4	32	45.58 (1.11)	47.77 (1.14)	31.20 (1.02)	34.14 (1.09)
8	8	64	54.40 (1.15)	55.75 (1.16)	34.55 (1.11)	36.50 (1.16)
8	16	128	62.54 (1.15)	63.20 (1.15)	36.49 (1.18)	37.77 (1.22)

Appendix

Table 18: GSM8K, CEDR-B: every point in Figure 8 , with SCCFG 2. Entries are accuracy percentages with problem-bootstrap SDs in parentheses, not across-seed SDs (Appendix F.4 ). Each denoising depth uses

n=512/S

saved draws per problem;

k

-subset averaging uses all

n

Checkpoint / conditioner	Seeds	HE	HE+	MBPP-378 base	MBPP+
Epoch 4 / Qwen	2	23.17 (3.45)	21.04 (2.16)	26.85 (0.19)	23.68 (0.19)
Epoch 8 / Qwen	2	27.74 (6.47)	25.61 (5.17)	34.92 (2.24)	30.42 (1.87)
Epoch 12 / Qwen	4	27.90 (1.75)	25.61 (2.49)	37.76 (0.70)	32.34 (0.79)
Epoch 12 / prompt MSE 1	4	19.82 (2.08)	18.60 (2.31)	8.93 (1.02)	7.61 (1.23)
Epoch 12 / prompt MSE 10	4	26.37 (1.44)	24.70 (2.08)	17.59 (2.61)	15.15 (2.16)

Appendix

Table 19: Coding training and prompt-substitution results at 32 denoising steps. Mean pass@1 (%) with generation-seed SD in parentheses; no aliases. Flow-training epochs 4/8 use seeds 42/123; epoch 12 and both prompt endpoints add 456/789. All rows use SCCFG 3. The prompt-MSE rows retain the same epoch-12 backbone weights.

Stage	Benchmark	pass@1	pass@2	pass@4	pass@8	pass@1 seed SD
Frozen Qwen	HumanEval	33.69 (2.78)	44.66 (3.21)	54.13 (3.46)	62.20 (3.75)	2.11
Frozen Qwen	HumanEval+	30.87 (2.70)	41.53 (3.18)	50.82 (3.45)	59.15 (3.77)	2.09
Frozen Qwen	MBPP-378 base	38.72 (2.01)	49.17 (2.21)	58.26 (2.32)	66.40 (2.48)	1.37
Frozen Qwen	MBPP+	33.99 (2.00)	43.26 (2.25)	51.46 (2.40)	58.73 (2.60)	1.62
Prompt MSE	HumanEval	27.97 (2.84)	35.98 (3.27)	42.87 (3.58)	48.78 (3.90)	1.44
Prompt MSE	HumanEval+	25.30 (2.81)	32.19 (3.25)	37.65 (3.55)	42.07 (3.85)	1.84

Appendix

Table 20: Standard scoring without aliases. CEDR-L with 128 denoising calls, SCCFG 3, eight seeds. Frozen Qwen and prompt MSE use epoch 12; pre-NFT is joint epoch 13; post-NFT is update 100. Pass@

k

entries are accuracies (%) with problem-bootstrap standard errors in parentheses; the final column gives the separate generation-seed SD of pass@1. Both uncertainty measures are in percentage points (Appendix F.4 ). Higher-

k

values average all seed subsets.

Stage	Benchmark	pass@1	pass@2	pass@4	pass@8	pass@1 seed SD
Frozen Qwen	HumanEval	33.69 (2.78)	44.66 (3.21)	54.13 (3.46)	62.20 (3.75)	2.11
Frozen Qwen	HumanEval+	30.87 (2.70)	41.53 (3.18)	50.82 (3.45)	59.15 (3.77)	2.09
Frozen Qwen	MBPP-378 base	43.45 (2.06)	53.97 (2.22)	62.90 (2.28)	70.63 (2.39)	1.21
Frozen Qwen	MBPP+	37.93 (2.07)	47.21 (2.28)	55.07 (2.40)	61.90 (2.55)	1.43
Prompt MSE	HumanEval	28.05 (2.84)	36.11 (3.27)	43.05 (3.59)	48.78 (3.90)	1.38
Prompt MSE	HumanEval+	25.38 (2.81)	32.32 (3.25)	37.82 (3.56)	42.07 (3.85)	1.78

Appendix

Table 21: Scoring with single-function aliases. CEDR-L with 128 denoising calls, SCCFG 3, eight seeds. Frozen Qwen and prompt MSE use epoch 12; pre-NFT is joint epoch 13; post-NFT is update 100. Pass@

k

k

values average all seed subsets.

Scoring	Benchmark	$\Delta$ pass@1 [95% CI]	$\Delta$ pass@8 [95% CI]
Standard	HumanEval	$+1.60$ $[-0.61,+3.73]$	$+5.49$ $[+0.00,+11.59]$
Standard	HumanEval+	$+1.60$ $[-0.53,+3.58]$	$+5.49$ $[+0.61,+10.98]$
Standard	MBPP-378 base	$-1.16$ $[-2.38,+0.03]$	$-3.97$ $[-7.67,-0.53]$
Standard	MBPP+	$-0.56$ $[-1.69,+0.53]$	$-0.79$ $[-4.23,+2.38]$
Alias	HumanEval	$+1.68$ $[-0.53,+3.81]$	$+5.49$ $[+0.00,+11.59]$
Alias	HumanEval+	$+1.68$ $[-0.46,+3.73]$	$+5.49$ $[+0.61,+10.98]$

Appendix

Table 22: Joint epoch 13 minus pre-joint prompt MSE. Differences in percentage points at 128 denoising calls and SCCFG 3, using the same eight seeds. Brackets give paired 95% problem-bootstrap intervals from 4,000 replicates. Both models use backbone EMA

0.9999

; prompt EMA is

0.999

before joint training and

0.9999

afterward. Differences are computed before rounding the stage accuracies.

Scoring	Benchmark	$\Delta$ pass@1 [95% CI]	$\Delta$ pass@8 [95% CI]
Standard	HumanEval	$+3.28$ $[+1.22,+5.34]$	$+0.00$ $[-5.49,+5.49]$
Standard	HumanEval+	$+3.28$ $[+1.30,+5.34]$	$+1.83$ $[-3.05,+6.71]$
Standard	MBPP-378 base	$+3.80$ $[+2.61,+5.09]$	$+3.44$ $[+0.00,+7.14]$
Standard	MBPP+	$+3.21$ $[+2.05,+4.43]$	$+1.85$ $[-1.32,+5.03]$
Alias	HumanEval	$+3.43$ $[+1.30,+5.56]$	$+0.61$ $[-4.27,+6.10]$
Alias	HumanEval+	$+3.20$ $[+1.14,+5.34]$	$+1.83$ $[-3.05,+6.71]$

Appendix

Table 23: NFT update 100 minus joint epoch 13. Differences in percentage points with paired 95% problem-bootstrap intervals, over eight matched seeds at 128 denoising calls and SCCFG 3. Backbone/prompt EMAs are

0.9999/0.9999

before NFT and

0.9/0.9

afterward. Intervals use 4,000 problem resamples and condition on the selected endpoints.

Figure 10: GSM8K generation trajectory. Steps 20, 40, 49, 60, 62, and 64, from top to bottom, for the GSM8K example in Figure 9 . CEDR-L NFT update 500; backbone/prompt EMA

.99/.99

; seed 42; async

(2.5,2,1.5)

; 64 denoising steps; CFG 1 and SCCFG 2. Light blue identifies tokens matching the final answer at the same position.

Explore similar work

ELF-REG: Scaling Continuous Diffusion Language Models to Reasoning Tasks
Sep 24, 2026Zeyu Michael Li, William Xingxu Chen, Bingshuo Qian +2Diffusion Language Models Reasoning Skills
Duke University · Tsinghua University
Next Thoughts Are Distributions: Generative Autoregressive Reasoning in the Latent Space
Sep 27, 2026Yang Li, Yi Wang, Shiyuan Huang +3Thoughts Reasoning Skills
Rutgers University · Amazon · University of Illinois at Urbana-Champaign
Why Struggle with Continuous Latents? Interpretable Discrete Latent Reasoning via Rendered Compression
Jun 29, 2026Shuochen Chang, Qingyang Liu, Shaobo Wang +8Efficient Latent Reasoning Intermediate Latent States
Shanghai Jiao Tong University · Tongji University

Sep 24, 2026, cs.CLJ/K move · Enter open · S save

ELF-REG: Scaling Continuous Diffusion Language Models to Reasoning Tasks

Zeyu Michael Li, William Xingxu Chen, Bingshuo Qian, Jiayin Liu+1

Duke University · Tsinghua University

Diffusion Language Models Reasoning Skills Iterative Denoising Methods

Open paper AskPDF arXiv

Fully continuous diffusion language models (dLMs) denoise continuous representations without intermediate discretization, then decode all response tokens in parallel at the final step. Their performance on challenging reasoning tasks remains less established than that of autoregressive (AR) LLMs and masked dLMs. We scale Embedded Language Flows (ELF) to mathematical reasoning and code generation on GSM8K, MATH-500, HumanEval, and MBPP. We introduce ELF-REG, which improves learning with representation alignment and entanglement (REPA+REG), where a frozen AR teacher supervises intermediate denoiser features and supplies a global representation that is jointly denoised with the response. ELF-REG-L achieves 55.96% pass@1 on GSM8K at 64 network function evaluations (NFE), and 13.39% on MATH-500 and 22.56% on HumanEval at 128 NFE. It outperforms the evaluated comparable-scale dLMs in pass@1 on GSM8K and code, and improves MATH-500 pass@1 from 10.55% for the ELF-L baseline to 13.39% with ELF-REG-L. Without few-step training, the same task-specific checkpoints support strong low-NFE performance through early-stop, which decodes an intermediate clean prediction without completing the denoising trajectory. At 16 NFE, ELF-REG-L reaches 41.21% HumanEval pass@10, outperforming recent continuous dLMs of comparable scale.

Abstract

Explore similar work

ELF-REG: Scaling Continuous Diffusion Language Models to Reasoning Tasks

Next Thoughts Are Distributions: Generative Autoregressive Reasoning in the Latent Space

Why Struggle with Continuous Latents? Interpretable Discrete Latent Reasoning via Rendered Compression