DiffuPlan: Editable Planning State for Diffusion Language Model Agents

Qisheng Zhou; Zhen Xiong; Qiaoyu Tan

cs.CLOct 17, 2026

DiffuPlan: Editable Planning State for Diffusion Language Model Agents

Authors: Qisheng Zhou, Zhen Xiong, Qiaoyu Tan

Organizations: New York University Shanghai · New York University

Abstract

Diffusion language models (dLLMs) are emerging as an alternative to autoregressive (AR) language models, yet dLLM agents largely inherit an interaction protocol designed for AR generation: once intermediate state is written, later observations can only be incorporated by appending more text. This is a poor match to a defining capability of dLLMs -- reconstructing earlier text conditioned on context that appears both before and after it. We ask whether this capability can support a different agent-state paradigm in which intermediate planning state is directly revisable as evidence arrives. We introduce DiffuPlan, a diffusion search agent that maintains a single editable planning state and reconstructs only the affected steps when later evidence resolves an entity or changes the search route. To support variable-length revisions, we introduce Expansion-Partition Marginalization (EPM), which marginalizes over equivalent expansion paths. We further adapt diffusion trajectory RL to multi-turn search. Across four multi-hop QA benchmarks, DiffuPlan achieves 50.24 average F1, outperforming strong AR search agents. More importantly, under identical data and training, editable planning state improves the diffusion agent by 2.44 F1 versus only 0.47 for its AR source model. These results show that dLLMs need not imitate append-only AR agents: revising committed state is a distinct agentic advantage.

Figures & tables

Figure 1: Three ways to represent evolving search state. (a) An implicit reasoning trace scatters intermediate state across appended thoughts and actions. (b) An explicit planner appends a regenerated plan after each revision, so superseded versions remain in the context. (c) DiffuPlan maintains one planning state $P_{t}$ and reconstructs only the affected steps in place using evidence retrieved later. Green marks revised content. The reported numbers compare DiffuPlan with an append-only diffusion agent trained with the same data and post-training recipe (Section 5.1 ).

Figure 2: DiffuPlan runtime and training. (a) Action Mode searches and requests an update or replan; Edit Mode then reconstructs the selected earlier plan span using the evidence retrieved after it and commits the revision at the same position. (b) The shared dLLM uses block-causal attention

M^{\mathrm{act}}

for forward action generation and full-canvas attention

M^{\mathrm{edit}}

for cross-turn state reconstruction. (c) Joint SFT trains both views. Dynamic expansion supports variable-length edits, while EPM marginalizes over equivalent expansion paths. Multi-turn Diffusion Search with Anchored Stabilization (MDSAS) then optimizes multi-turn Action-Mode decisions while regularizing toward the frozen SFT policy.

Inference-only methods
	HotpotQA		2Wiki		MuSiQue		Bamboogle		Avg.
System	F1	ACC L	F1	ACC L	F1	ACC L	F1	ACC L	F1	ACC L
Direct	25.72	29.20	29.04	28.80	11.26	15.60	21.51	20.80	21.88	23.60
CoT	21.95	26.40	32.55	31.40	11.70	11.20	40.88	40.80	26.77	27.45
Standard RAG	39.30	46.40	31.96	30.60	10.60	11.40	25.06	29.60	26.73	29.50
SuRe	37.68	45.60	26.90	27.60	12.15	12.40	29.44	35.20	26.54	30.20

Table 1: End-to-end QA performance (%) on four multi-hop benchmarks. Each dataset reports token F1 and LLM-as-Judge accuracy (ACC L ); Avg. is the unweighted mean across the four datasets. Bold marks the best score in each column.

Planning state	F1	ACC L	Tokens
dLLM (LLaDA2.0-mini)
Editable	50.24	55.30	338
Append	47.80	52.30	411
AR LLM (Ling-mini-2.0)
Editable	50.01	53.70	357
Append	49.54	52.60	425

Table 2: Editable versus append-only planning state, trained with the same data and recipe. Token F1 and ACC L (%) are averaged over the four datasets, and Tokens is model-generated tokens per question.

Figure 3: Editable versus append-only planning state on the 1,625 test questions, grouped by the append agent’s edit requests (the last two groups are subsets of the second). The F1 gain grows with the number of state edits, and the editable agent generates fewer tokens in every group. Full numbers are in Table 10 .

Figure 4: (a) EPM versus single-path cross-entropy (CE) and (b) dynamic expansion versus a fixed 96-token canvas, at SFT epoch 1 on the 256 validation questions under the same interaction budget. Valid. is format validity.

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Query $\,\backslash\,$ key	$x_{\sigma}^{0}$	$x_{\sigma}^{1}$	$x_{\sigma}^{2}$	$x_{0}^{0}$	$x_{0}^{1}$	$x_{0}^{2}$
$x_{\sigma}^{0}$	1	$\cdot$	$\cdot$	$\cdot$	$\cdot$	$\cdot$
$x_{\sigma}^{1}$	$\cdot$	1	$\cdot$	1	$\cdot$	$\cdot$
$x_{\sigma}^{2}$	$\cdot$	$\cdot$	1	1	1	$\cdot$
$x_{0}^{0}$	$\cdot$	$\cdot$	$\cdot$	1	$\cdot$	$\cdot$
$x_{0}^{1}$	$\cdot$	$\cdot$	$\cdot$	1	1	$\cdot$
$x_{0}^{2}$	$\cdot$	$\cdot$	$\cdot$	1	1	1

Appendix

Table 3: Block-level Action mask; 1 marks a visible key block.

Stage	Serialization	Direct loss positions	Conditioning only
Action-Mode SFT	$[x_{\sigma};x_{0}]$	Corrupted tokens in the newly generated Action-Mode segment	Prompt, history, information, uncorrupted target tokens, clean lane, padding
Edit-Mode SFT	$[x_{t}^{\mathrm{edit}};\mathrm{PAD}^{N}]$	Controller-owned unresolved masks, using EPM	All other canvas tokens, resolved edit tokens, non-owned plan spans, dummy half
MDSAS	$x_{g,k}$	Generated Action-Mode tokens committed in slice $k$	Prefix, observations, clean helper blocks, padding

Appendix

Table 4: Prediction targets for the two runtime views and MDSAS.

Partition $(\kappa_{1},\kappa_{2})$	Chunk assignment	Immediate mask labels
$(1,3)$	$(y_{1})\mid(y_{2},y_{3},y_{4})$	$(y_{1},\chi_{\mathrm{exp}})$
$(2,2)$	$(y_{1},y_{2})\mid(y_{3},y_{4})$	$(\chi_{\mathrm{exp}},\chi_{\mathrm{exp}})$
$(3,1)$	$(y_{1},y_{2},y_{3})\mid(y_{4})$	$(\chi_{\mathrm{exp}},y_{4})$

Appendix

Table 5: All EPM partitions for two masks and four target tokens.

Record family	Count	Breakdown
Teacher trajectories	6,486	—
Action records	26,539	12,722 search; 6,769 update; 6,486 answer; 562 replan
Edit records	83,519	70,031 update states; 13,488 replan states

Appendix

Table 6: Canonical teacher trajectories and derived SFT records.

Setting	SFT	MDSAS
Initialization	LLaDA2.0-mini MoE merged checkpoint	SFT epoch 1
Optimizer	AdamW	AdamW
Weight decay	$0.1$	$0.01$
Learning rate	peak $2\times 10^{-6}$	constant $1\times 10^{-6}$
Schedule	3% linear warmup, cosine decay to $10^{-7}$	no warmup or scheduler
Gradient clipping	$5.0$	$1.0$

Appendix

Table 7: Optimization configuration.

Figure 5: MDSAS and its unanchored variant. (a) Unsmoothed 2Wiki validation F1 and (b) hard-invalid rollout rates over the first 50 training steps.

	HotpotQA		2Wiki		MuSiQue		Bamboogle		Avg.
Planning state	F1	ACC L	F1	ACC L	F1	ACC L	F1	ACC L	F1	ACC L	Tokens
LLaDA2.0-mini
Editable	55.34	65.00	55.03	56.00	31.72	34.60	58.86	65.60	50.24	55.30	338
Append	54.25	64.00	51.82	53.40	28.73	34.20	56.40	57.60	47.80	52.30	411
Ling-mini-2.0
Editable	55.99	65.20	54.30	54.80	33.59	35.60	56.16	59.20	50.01	53.70	357

Appendix

Table 8: Per-dataset results for the backbone and protocol comparison of Table 2 . Each dataset reports token F1 and ACC L (%); Avg. averages the four datasets. Tokens is model-generated tokens per question. Bold marks the best value in each column.

Dataset	Update	Replan	Upd./q	Rep./q	Exp.
HotpotQA	71.2	9.6	1.04	0.10	88.3
2Wiki	83.2	16.4	1.52	0.16	92.9
MuSiQue	81.8	18.8	1.20	0.19	90.4
Bamboogle	91.2	4.8	0.98	0.05	89.1
All	79.7	14.2	1.23	0.14	90.7

Appendix

Table 9: Edit usage of DiffuPlan on the test set. Update and Replan are the percentages of questions with at least one such edit; Upd./q and Rep./q are edits per question; Exp. is the percentage of executed edits that invoke expansion.

Append-agent requests	$n$	%	Editable	Append	$\Delta$	Editable	Append
			F1			Tokens
No edit request	223	13.7	39.40	39.20	+0.20	359	379
$\geq$ 1 edit request	1,402	86.3	49.65	46.87	+2.79	335	416
$\geq$ 2 updates	433	26.6	55.38	50.75	+4.63	383	546
$\geq$ 1 replan	272	16.7	30.18	23.06	+7.12	433	585

Appendix

Table 10: Paired F1 (%) and model-generated tokens per question on the 1,625 test questions, grouped by the edit requests of the LLaDA2.0-mini append agent. The last two groups are subsets of the second.

	Training rollout	Checkpoint selection
Block size / denoising steps	32 / 32	32 / 32
Temperature	0.8	0
Top- $p$ / top- $k$	0.95 / 0	1.0 / 0
Confidence threshold	0.90	0.95
Max new tokens per segment	256	256

Appendix

Table 11: Action-Mode sampling parameters.

System	EM	F1	SubEM	ACC L
Search-o1	29.45	39.78	39.50	46.25
Search-R1	27.50	35.33	33.90	37.75
R1-Searcher	38.40	48.71	46.55	52.60
DLLM-Searcher	0.05	7.25	36.25	36.65
DiffuPlan	38.15	50.24	48.45	55.30

Appendix

Table 12: Answer metrics averaged over the four datasets (%).

LLM Agents

Large language model agents use language models to plan, use tools, and act across multi-step tasks and environments.

See all 681 papers

Plan-and-Patch: Diffusion Language Models for Agentic Planning
Oct 7, 2026Syamantak Kumar, Jiang Guo, Hassan Hamad +6LLM Planning Diffusion-Based Planning
University of Texas at Austin · Amazon
Think First, Diffuse Fast: Improving Diffusion Language Model Reasoning via Autoregressive Plan Conditioning
Feb 20, 2026Earl J St SauverDiffusion Language Models LLM Planning
Independent Researcher
Latent-DARM: Bridging Discrete Diffusion And Autoregressive Models For Reasoning
Mar 10, 2026Lina Berrayana, Ahmed Heakl, Abdullah Sohail +3Multi-Agent Reasoning Multi-Agent LLMs
EPFL · MBZUAI · ETH Zürich +1

Oct 7, 2026J/K move · Enter open · S save

Plan-and-Patch: Diffusion Language Models for Agentic Planning

Syamantak Kumar, Jiang Guo, Hassan Hamad, Hideo Kobayashi+5

University of Texas at Austin · Amazon

LLM Planning Diffusion-Based Planning Long-Horizon LLM Agents Diffusion Language Models

Open paper AskPDF arXiv

Planning is increasingly important for long-horizon agents, where successful execution requires coordinating subgoals, tool use, and intermediate outcomes over many steps. Yet assumptions made during planning may be invalidated by the environment, tools may return unexpected results, or actions may fail. Effective agents must therefore not only generate plans, but also revise them. Such revisions often affect only part of a plan, leaving the preceding and subsequent structure intact. Rather than regenerate the entire plan and risk unnecessary changes, repair can regenerate the affected region conditioned on the preserved prefix and suffix. We introduce Plan-and-Patch, a plan-and-act framework in which a diffusion language model (dLLM) generates a structured, program-like plan through parallel unmasking and repairs it by filling in selected regions while keeping the surrounding steps fixed. We compare DreamReasoner-8B and Qwen3-8B as diffusion and autoregressive (AR) planners. On Natural Plan without task-specific training, diffusion (53.7%) achieves nearly twice the plan repair success rate of AR (27.0%). After task-specific training on agentic benchmarks, ALFWorld and TextCraft, the planners achieve similar observed success in plan generation, while diffusion reduces mean plan-generation latency by 39-46% relative to AR. Our results show that Plan-and-Patch provides a framework for faster plan generation and effective plan repair in long-horizon agents.

Abstract

LLM Agents

Explore similar work

Plan-and-Patch: Diffusion Language Models for Agentic Planning

Think First, Diffuse Fast: Improving Diffusion Language Model Reasoning via Autoregressive Plan Conditioning

Latent-DARM: Bridging Discrete Diffusion And Autoregressive Models For Reasoning