Think First, Diffuse Fast: Improving Diffusion Language Model Reasoning via Autoregressive Plan Conditioning

Earl J St Sauver

cs.AIFeb 20, 2026

Think First, Diffuse Fast: Improving Diffusion Language Model Reasoning via Autoregressive Plan Conditioning

Authors: Earl J St Sauver

Organizations: Independent Researcher

Abstract

Diffusion large language models (dLLMs) generate text via iterative denoising but consistently underperform on multi-step reasoning. We hypothesize this gap stems from a coordination problem: AR models build coherence token-by-token, while diffusion models must coordinate all positions simultaneously. We propose plan conditioning, a training-free method that prepends a short (~100-token) natural-language plan from an AR model to the diffusion model's prompt. The plan serves as a frozen scaffold -- globally visible context that every token position can attend to from the first denoising step. On GSM8K, plan conditioning improves LLaDA-8B-Instruct from 75.6% to 87.2% (+11.6 percentage points), matching a same-size AR model (LLaMA 3.1 8B, 87.7%) despite a 6.4pp weaker baseline. On HumanEval, the gain is +12.8pp (37.2% to 50.0%), showing plans generalize to code. The same plans improve LLaMA by only +5.7pp on GSM8K and +1.3pp on HumanEval -- diffusion models benefit 2-10x more, supporting the coordination-problem hypothesis. Across 5 random seeds, plan-conditioned GSM8K accuracy has zero standard deviation, making diffusion inference highly stable. Ablations reveal the model follows plan strategy (wrong-strategy plans cause -16.3pp) but is robust to plan values (perturbed numbers: -1.1pp), and that planner quality has a sharp threshold: smaller Llama-class plans hurt (-1.6 to -6.8pp) while frontier plans provide the full lift. Attention analysis confirms the mechanism: plan tokens receive 1.8x excess attention during early denoising, declining to uniform as completion tokens solidify. Plan conditioning costs ~$0.002 per problem and adds ~2s of latency.

Figures & tables

Format	Description	Example snippet
Strategy	High-level approach in 1–2 sentences	“Calculate daily production, subtract consumed, find remaining”
Outline	Step-by-step procedure without computing answers	“Step 1: Identify total chickens. Step 2: …”
Constraints	Key facts, pitfalls, expected answer form	“Daily production: 16 eggs. Pitfall: don’t double-count…”
Hybrid	Strategy sentence + key constraints	“Track eggs through pipeline. Key: 16/day produced, 3/day consumed…”

Table 1: Plan format taxonomy. Each format provides different guidance density per token. Hybrid combines high-level strategy with specific numeric anchors.

Figure 1: Accuracy across benchmarks and model–condition pairs. On GSM8K, plan conditioning improves LLaDA by +11.6pp (vs. +5.7pp for LLaMA with the same plans), closing the diffusion–AR gap entirely. HumanEval shows the largest diffusion advantage: LLaDA gains +12.8pp while LLaMA gains only +1.3pp. On Countdown, both architectures start at identical baselines and benefit identically (+12.1pp).

Benchmark	Baseline	Strategy	Outline	Constraints	Hybrid	Best $\Delta$
	128 / 256	128 / 256	128 / 256	128 / 256	128 / 256
GSM8K	69.0 / 75.6	77.7 / 85.7	78.2 / 84.9	76.8 / 86.0	82.0 / 87.2	+11.6
MATH500	27.4 / 35.8	30.6 / 37.4	34.4 / 36.6	30.0 / 37.6	32.6 / 41.8	+6.0
HumanEval	25.0 / 37.2	32.3 / 50.0	42.1 / 48.8	28.0 / 36.0	32.9 / 48.2	+12.8
Countdown	21.1 / 19.1	29.7 / 28.5	18.4 / 21.5	22.3 / 14.5	28.1 / 31.2	+12.1
Sudoku	11.6 / 6.9	5.7 / 7.9	6.3 / 6.1	7.3 / 7.7	7.2 / 6.2	+1.0

Table 2: LLaDA-8B accuracy (%) by plan format and generation length. Bold indicates best per row. Baseline values are at generation lengths 128/256;

\Delta

is computed from the 256-token baseline. Hybrid plans win on 3/5 benchmarks at gen-256; strategy wins on HumanEval and Sudoku.

Figure 2: Accuracy improvement (pp) over baseline by plan format and benchmark at generation length 256. Hybrid plans (rightmost column) dominate on 3/4 benchmarks. Sudoku (bottom row) shows near-zero or negative deltas across all formats—plans do not help spatial constraint propagation. The blue cell (Countdown–Constraints, $-$ 4.6pp) shows that constraint-only plans hurt when they lack procedural structure.

GSM8K
Condition	LLaMA 3.1 8B (AR)	LLaDA-8B (Diffusion)
Bare	82.0	75.6
+ Sonnet plan	87.7 (+5.7)	87.2 (+11.6)
+ Haiku plan	84.9 (+2.9)	82.3 (+6.7)
MATH500
Bare	43.0	35.8

Table 3: AR vs. diffusion comparison. Accuracy (%) with per-condition lift in parentheses. On GSM8K, LLaDA benefits 2

\times

more from Sonnet plans than LLaMA (+11.6 vs. +5.7pp). On Countdown, both architectures benefit identically (+12.1pp). On HumanEval, the diffusion advantage is largest: LLaDA gains +12.8pp while LLaMA gains only +1.3pp.

Plan Budget	GSM8K		Countdown
	Accuracy	Lift	Accuracy	Lift
0 (bare)	75.6	—	19.1	—
25 tokens	79.5	+3.9	17.6	$-$ 1.5
50 tokens	83.0	+7.4	16.8	$-$ 2.3
100 tokens	87.2	+11.6	31.2	+12.1
150 tokens	88.0	+12.4	39.5	+20.4

Table 4: LLaDA accuracy (%) by plan token budget (hybrid format). GSM8K shows smooth saturation with diminishing returns past 100 tokens. Countdown exhibits a sharp performance threshold: plans hurt at 25–50 tokens, then jump +14.5pp from 50

\to

100. Both benchmarks plateau by 150–200 tokens.

Figure 3: LLaDA accuracy vs. plan token budget (hybrid format). Left: GSM8K shows smooth saturation with diminishing returns past 100 tokens. Right: Countdown exhibits a sharp performance threshold—plans hurt at 25–50 tokens, then jump +14.5pp from 50

\to

100. Dashed line marks the no-plan baseline.

Category	Accuracy (%)	$N$
No leakage (answer absent from plan)	80.1	1,022
False-positive leak (answer already in problem)	83.5	85
True leakage (answer new in plan)	88.7	212
Baseline (no plan)	75.6	1,319

Table 5: Leakage decomposition for GSM8K hybrid plans (

N{=}1{,}319

). “False-positive leak” denotes problems where the answer number already appears in the problem text (e.g., “20 chickens” when the answer is 20), so its presence in the plan is not informative. Even excluding all leaked problems, accuracy exceeds the baseline by +4.5pp. Maximum leakage contribution: 1.4pp out of 11.6pp (12%). Note: per-category accuracies are from the gen=128 ablation run (overall 82.0%); the qualitative conclusion—that leakage accounts for at most

{\sim}

12% of the lift—holds for the gen=256 run (87.2%) as well.

Planner	GSM8K	MATH500	Countdown	Cost/plan
None (baseline)	75.6	35.8	19.1	$0
LLaDA self-plan	74.9 ( $-$ 0.7)	—	—	$0
Llama 3.2 3B	68.8 ( $-$ 6.8)	30.0 ( $-$ 5.8)	16.4 ( $-$ 2.7)	$0.00005
Llama 3.1 8B	74.0 ( $-$ 1.6)	31.6 ( $-$ 4.2)	13.3 ( $-$ 5.8)	$0.0001
Haiku 4.5	82.3 (+6.7)	34.4 ( $-$ 1.4)	24.2 (+5.1)	$0.0006
Sonnet 4.5	87.2 (+11.6)	41.8 (+6.0)	31.2 (+12.1)	$0.002

Table 6: LLaDA accuracy (%) by planner capability. Plan quality has a sharp threshold: below Haiku-class capability, plans hurt across all benchmarks.

Figure 4: GSM8K accuracy by planner capability. There is a sharp quality cliff between Haiku (helpful, +6.7pp) and Llama 8B (marginally harmful,

-

1.6pp). Self-plans (LLaDA planning for itself) are indistinguishable from no plan.

Benchmark	Rescue rate (hard)	Retention rate (easy)	Fix:Break ratio
GSM8K	71.4%	92.3%	3.0:1
MATH500	21.5%	78.2%	1.8:1
Countdown	28.0%	44.9%	2.1:1

Table 7: Per-difficulty analysis of hybrid plan conditioning. “Rescue rate” is the fraction of baseline-incorrect problems that plans fix; “retention rate” is the fraction of baseline-correct problems that remain correct. Plans are not a free lunch: they break 8–55% of previously correct answers, but fix even more.

Category	Count	% of failures
Execution error (right plan, wrong math)	100	59.2
Format failure (right answer, wrong format)	31	18.3
Plan wrong (plan misled the model)	27	16.0
No answer extracted	11	6.5

Table 8: Error taxonomy for 169 GSM8K hybrid failures. The dominant failure mode (59%) is execution error: the plan is correct but the model makes arithmetic mistakes. Zero failures were classified as “plan ignored.”

Condition	GSM8K (%)	$\Delta$	What it tests
Wrong strategy	59.3	$-$ 16.3	Incorrect mathematical approach
Mismatched plan	71.0	$-$ 4.6	Plan for a different problem
Perturbed numbers	74.5	$-$ 1.1	Correct strategy, wrong values
Baseline (no plan)	75.6	—
Random tokens	78.2	+2.6	Token count only
Shuffled plan	80.0	+4.4	Vocabulary preserved

Table 9: Plan quality spectrum on GSM8K. The model follows plan strategy (wrong strategy:

-

16.3pp) but is robust to plan values (perturbed numbers:

-

1.1pp). 61% of the improvement comes from semantic content; 23% from extra tokens; 16% from domain vocabulary.

\dagger

The “real plan” control in this ablation run scored 86.9% vs. 87.2% in the main sweep (Table 2 ); the 0.3pp difference reflects single-seed variance.

Figure 5: Plan quality spectrum on GSM8K. Wrong-strategy plans cause catastrophic failure (

-

16.3pp), while perturbed-numbers plans are nearly neutral (

-

1.1pp). The model follows the plan’s reasoning approach but computes its own values.

Figure 6: Attention from completion tokens to sequence regions across denoising steps. (a) Plan tokens receive 43% of attention at step 0 despite comprising only 24% of the sequence (dotted line = uniform baseline). By step 63, attention converges to near-uniform. (b) Excess plan attention (ratio over uniform) is 1.8

\times

at step 0, declining smoothly to 1.0

\times

. (c) The effect is consistent across all layers; early layers (layer 0) show the strongest absolute attention to plans.

Figure 7: Excess plan attention across plan quality conditions. (a) All conditions show elevated plan attention during early denoising, declining toward uniform. (b) Step-0 excess attention scales with plan coherence (Sonnet

>

Llama 3B

>

wrong strategy

>

random) but not with downstream accuracy impact. The model reads plans indiscriminately; quality discrimination is downstream.

Condition	GSM8K	MATH500	Rel. FLOPs
Baseline (gen=256, steps=128)	75.6	35.8	1.0 $\times$
Longer gen (gen=384, steps=192)	78.9 (+3.3)	34.4 ( $-$ 1.4)	${\sim}$ 1.5 $\times$
More steps (gen=256, steps=256)	79.3 (+3.7)	34.4 ( $-$ 1.4)	${\sim}$ 2.0 $\times$
Plan conditioning	87.2 (+11.6)	41.8 (+6.0)	${\sim}$ 1.0 $\times$ *

Table 10: Plan conditioning vs. compute-matched controls. Doubling denoising steps or increasing generation length 50% yields modest GSM8K gains (+3–4pp) and hurts MATH500. Plan conditioning provides 3–4

\times

more improvement at negligible additional diffusion FLOPs. * Plan generation is external; diffusion FLOPs are unchanged.

Benchmark	Baseline	+ Sonnet plan	$\Delta$
GSM8K	75.51 $\pm$ 0.15	88.02 $\pm$ 0.00	+12.5
MATH500	35.76 $\pm$ 0.08	41.52 $\pm$ 0.56	+5.8
Countdown	17.73 $\pm$ 1.01	37.11 $\pm$ 2.60	+19.4

Table 11: Five-seed evaluation (mean

\pm

s.d. across seeds). Baseline variance is remarkably low on GSM8K (

\pm

0.15pp) and MATH500 (

\pm

0.08pp), confirming single-seed estimates are representative. All improvements are significant (

p<0.001

, paired bootstrap). Plan-conditioned GSM8K has zero variance across seeds.

Figure 8: Five-seed evaluation with error bars (1 s.d.). Plan conditioning produces large, statistically significant improvements across all three benchmarks. On GSM8K, plans reduce variance to zero—the same problems are solved regardless of the random seed.

Benchmark	LLaDA	+ Sonnet Plan	$\Delta$	Berr. BL	Berr. Text	Berr. Latent $\dagger$
DART5	11.0	17.2	+6.2	15.0	27.0	54.0
ARC-Challenge	71.3	75.2	+3.9	87.5	91.0	81.0
AIME24	0.0	3.3	+3.3	1.5	1.5	14.0

Table 12: Plan conditioning on benchmarks from Berrayana et al. (2025) . “Berr. BL” is their LLaDA-only baseline. “Berr. Text” is their best text-space collaboration result (any direction). “Berr. Latent” is their best latent-space DDLM

\to

ARM result.

\dagger

Uses a trained Linear-GELU-Linear projector on 35K domain-specific samples. On ARC-C, text-space collaboration (91.0%) outperforms latent (81.0%), but on DART5 and AIME the latent projector provides dramatically higher accuracy.

Diffusion Language Models

Diffusion language models generate and refine text through iterative denoising, enabling bidirectional context and parallel token updates.

See all 443 papers

Oct 17, 2026

DiffuPlan: Editable Planning State for Diffusion Language Model Agents

Diffusion language models (dLLMs) are emerging as an alternative to autoregressive (AR) language models, yet dLLM agents largely inherit an interaction protocol designed for AR generation: once intermediate state is written, later observations can only be incorporated by appending more text. This is a poor match to a defining capability of dLLMs -- reconstructing earlier text conditioned on context that appears both before and after it. We ask whether this capability can support a different agent-state paradigm in which intermediate planning state is directly revisable as evidence arrives. We introduce DiffuPlan, a diffusion search agent that maintains a single editable planning state and reconstructs only the affected steps when later evidence resolves an entity or changes the search route. To support variable-length revisions, we introduce Expansion-Partition Marginalization (EPM), which marginalizes over equivalent expansion paths. We further adapt diffusion trajectory RL to multi-turn search. Across four multi-hop QA benchmarks, DiffuPlan achieves 50.24 average F1, outperforming strong AR search agents. More importantly, under identical data and training, editable planning state improves the diffusion agent by 2.44 F1 versus only 0.47 for its AR source model. These results show that dLLMs need not imitate append-only AR agents: revising committed state is a distinct agentic advantage.

Qisheng Zhou, Zhen Xiong, Qiaoyu Tan

New York University Shanghai · New York University

LLM Agents LLM Agent Planning Diffusion Language Models

Ask PDF

Feb 26, 2026

Test-Time Scaling with Diffusion Language Models via Reward-Guided Stitching

Reasoning with large language models often benefits from generating multiple chains-of-thought, but existing aggregation strategies are typically trajectory-level (e.g., selecting the best trace or voting on the final answer), discarding useful intermediate work from partial or "nearly correct" attempts. We propose Stitching Noisy Diffusion Thoughts, a self-consistency framework that turns cheap diffusion-sampled reasoning into a reusable pool of step-level candidates. Given a problem, we (i) sample many diverse, low-cost reasoning trajectories using a masked diffusion language model, (ii) score every intermediate step with an off-the-shelf process reward model (PRM), and (iii) stitch these highest-quality steps across trajectories into a composite rationale. This rationale is then used to recompute only the final answer. This modular pipeline separates exploration (diffusion) from evaluation and solution synthesis, avoiding monolithic unified hybrids while preserving broad search. Across math reasoning benchmarks, we find that step-level recombination is most beneficial on harder problems, and ablations highlight the importance of the final solver in converting stitched but imperfect rationales into accurate answers. Using low-confidence diffusion sampling with parallel, independent rollouts, our training-free framework improves average accuracy by up to 23.8% across six math and coding tasks. At the same time, it achieves up to a 1.8x latency reduction relative to both traditional diffusion models (e.g., Dream, LLaDA) and unified architectures (e.g., TiDAR). The code is available https://github.com/roymiles/diffusion-stitching

Roy Miles, Aysim Toker, Andreea-Maria Oncescu +2

Huawei London Research Center · MVP Lab

Structured Reasoning in Language Models Test-Time Scaling Diffusion Language Models

Ask PDF

Oct 7, 2026

Plan-and-Patch: Diffusion Language Models for Agentic Planning

Planning is increasingly important for long-horizon agents, where successful execution requires coordinating subgoals, tool use, and intermediate outcomes over many steps. Yet assumptions made during planning may be invalidated by the environment, tools may return unexpected results, or actions may fail. Effective agents must therefore not only generate plans, but also revise them. Such revisions often affect only part of a plan, leaving the preceding and subsequent structure intact. Rather than regenerate the entire plan and risk unnecessary changes, repair can regenerate the affected region conditioned on the preserved prefix and suffix. We introduce Plan-and-Patch, a plan-and-act framework in which a diffusion language model (dLLM) generates a structured, program-like plan through parallel unmasking and repairs it by filling in selected regions while keeping the surrounding steps fixed. We compare DreamReasoner-8B and Qwen3-8B as diffusion and autoregressive (AR) planners. On Natural Plan without task-specific training, diffusion (53.7%) achieves nearly twice the plan repair success rate of AR (27.0%). After task-specific training on agentic benchmarks, ALFWorld and TextCraft, the planners achieve similar observed success in plan generation, while diffusion reduces mean plan-generation latency by 39-46% relative to AR. Our results show that Plan-and-Patch provides a framework for faster plan generation and effective plan repair in long-horizon agents.

Syamantak Kumar, Jiang Guo, Hassan Hamad +6

University of Texas at Austin · Amazon

LLM Planning Diffusion-Based Planning Long-Horizon LLM Agents Diffusion Language Models

Ask PDF

Abstract

Diffusion Language Models

Explore similar work

DiffuPlan: Editable Planning State for Diffusion Language Model Agents

Test-Time Scaling with Diffusion Language Models via Reward-Guided Stitching

Plan-and-Patch: Diffusion Language Models for Agentic Planning