Diffusion language models (dLLMs) are emerging as an alternative to autoregressive (AR) language models, yet dLLM agents largely inherit an interaction protocol designed for AR generation: once intermediate state is written, later observations can only be incorporated by appending more text. This is a poor match to a defining capability of dLLMs -- reconstructing earlier text conditioned on context that appears both before and after it. We ask whether this capability can support a different agent-state paradigm in which intermediate planning state is directly revisable as evidence arrives. We introduce DiffuPlan, a diffusion search agent that maintains a single editable planning state and reconstructs only the affected steps when later evidence resolves an entity or changes the search route. To support variable-length revisions, we introduce Expansion-Partition Marginalization (EPM), which marginalizes over equivalent expansion paths. We further adapt diffusion trajectory RL to multi-turn search. Across four multi-hop QA benchmarks, DiffuPlan achieves 50.24 average F1, outperforming strong AR search agents. More importantly, under identical data and training, editable planning state improves the diffusion agent by 2.44 F1 versus only 0.47 for its AR source model. These results show that dLLMs need not imitate append-only AR agents: revising committed state is a distinct agentic advantage.
Figures & tables
Figure 1: Three ways to represent evolving search state. (a) An implicit reasoning trace scatters intermediate state across appended thoughts and actions. (b) An explicit planner appends a regenerated plan after each revision, so superseded versions remain in the context. (c) DiffuPlan maintains one planning state Pt and reconstructs only the affected steps in place using evidence retrieved later. Green marks revised content. The reported numbers compare DiffuPlan with an append-only diffusion agent trained with the same data and post-training recipe (Section 5.1 ).
Figure 2: DiffuPlan runtime and training. (a) Action Mode searches and requests an update or replan; Edit Mode then reconstructs the selected earlier plan span using the evidence retrieved after it and commits the revision at the same position. (b) The shared dLLM uses block-causal attention Mact for forward action generation and full-canvas attention Medit for cross-turn state reconstruction. (c) Joint SFT trains both views. Dynamic expansion supports variable-length edits, while EPM marginalizes over equivalent expansion paths. Multi-turn Diffusion Search with Anchored Stabilization (MDSAS) then optimizes multi-turn Action-Mode decisions while regularizing toward the frozen SFT policy.
HotpotQA
2Wiki
MuSiQue
Bamboogle
Avg.
System
F1
ACC L
F1
ACC L
F1
ACC L
F1
ACC L
F1
ACC L
Inference-only methods
Direct
25.72
29.20
29.04
28.80
11.26
15.60
21.51
20.80
21.88
23.60
CoT
21.95
26.40
32.55
31.40
11.70
11.20
40.88
40.80
26.77
27.45
Standard RAG
39.30
46.40
31.96
30.60
10.60
11.40
25.06
29.60
26.73
29.50
SuRe
37.68
45.60
26.90
27.60
12.15
12.40
29.44
35.20
26.54
30.20
Table 1: End-to-end QA performance (%) on four multi-hop benchmarks. Each dataset reports token F1 and LLM-as-Judge accuracy (ACC L ); Avg. is the unweighted mean across the four datasets. Bold marks the best score in each column.
Planning state
F1
ACC L
Tokens
dLLM (LLaDA2.0-mini)
Editable
50.24
55.30
338
Append
47.80
52.30
411
AR LLM (Ling-mini-2.0)
Editable
50.01
53.70
357
Append
49.54
52.60
425
Table 2: Editable versus append-only planning state, trained with the same data and recipe. Token F1 and ACC L (%) are averaged over the four datasets, and Tokens is model-generated tokens per question.
Figure 3: Editable versus append-only planning state on the 1,625 test questions, grouped by the append agent’s edit requests (the last two groups are subsets of the second). The F1 gain grows with the number of state edits, and the editable agent generates fewer tokens in every group. Full numbers are in Table 10 .
Figure 4: (a) EPM versus single-path cross-entropy (CE) and (b) dynamic expansion versus a fixed 96-token canvas, at SFT epoch 1 on the 256 validation questions under the same interaction budget. Valid. is format validity.
Appendix figures & tables11 assets
Supplementary material from the paper’s appendix.
Appendix
Query \ key
xσ0
xσ1
xσ2
x00
x01
x02
xσ0
1
⋅
⋅
⋅
⋅
⋅
xσ1
⋅
1
⋅
1
⋅
⋅
xσ2
⋅
⋅
1
1
1
⋅
x00
⋅
⋅
⋅
1
⋅
⋅
x01
⋅
⋅
⋅
1
1
⋅
x02
⋅
⋅
⋅
1
1
1
Appendix
Table 3: Block-level Action mask; 1 marks a visible key block.
Stage
Serialization
Direct loss positions
Conditioning only
Action-Mode SFT
[xσ;x0]
Corrupted tokens in the newly generated Action-Mode segment
Table 6: Canonical teacher trajectories and derived SFT records.
Setting
SFT
MDSAS
Initialization
LLaDA2.0-mini MoE merged checkpoint
SFT epoch 1
Optimizer
AdamW
AdamW
Weight decay
0.1
0.01
Learning rate
peak 2×10−6
constant 1×10−6
Schedule
3% linear warmup, cosine decay to 10−7
no warmup or scheduler
Gradient clipping
5.0
1.0
Appendix
Table 7: Optimization configuration.
Figure 5: MDSAS and its unanchored variant. (a) Unsmoothed 2Wiki validation F1 and (b) hard-invalid rollout rates over the first 50 training steps.
HotpotQA
2Wiki
MuSiQue
Bamboogle
Avg.
Planning state
F1
ACC L
F1
ACC L
F1
ACC L
F1
ACC L
F1
ACC L
Tokens
LLaDA2.0-mini
Editable
55.34
65.00
55.03
56.00
31.72
34.60
58.86
65.60
50.24
55.30
338
Append
54.25
64.00
51.82
53.40
28.73
34.20
56.40
57.60
47.80
52.30
411
Ling-mini-2.0
Editable
55.99
65.20
54.30
54.80
33.59
35.60
56.16
59.20
50.01
53.70
357
Appendix
Table 8: Per-dataset results for the backbone and protocol comparison of Table 2 . Each dataset reports token F1 and ACC L (%); Avg. averages the four datasets. Tokens is model-generated tokens per question. Bold marks the best value in each column.
Dataset
Update
Replan
Upd./q
Rep./q
Exp.
HotpotQA
71.2
9.6
1.04
0.10
88.3
2Wiki
83.2
16.4
1.52
0.16
92.9
MuSiQue
81.8
18.8
1.20
0.19
90.4
Bamboogle
91.2
4.8
0.98
0.05
89.1
All
79.7
14.2
1.23
0.14
90.7
Appendix
Table 9: Edit usage of DiffuPlan on the test set. Update and Replan are the percentages of questions with at least one such edit; Upd./q and Rep./q are edits per question; Exp. is the percentage of executed edits that invoke expansion.
F1
Tokens
Append-agent requests
n
%
Editable
Append
Δ
Editable
Append
No edit request
223
13.7
39.40
39.20
+0.20
359
379
≥ 1 edit request
1,402
86.3
49.65
46.87
+2.79
335
416
≥ 2 updates
433
26.6
55.38
50.75
+4.63
383
546
≥ 1 replan
272
16.7
30.18
23.06
+7.12
433
585
Appendix
Table 10: Paired F1 (%) and model-generated tokens per question on the 1,625 test questions, grouped by the edit requests of the LLaDA2.0-mini append agent. The last two groups are subsets of the second.
Training rollout
Checkpoint selection
Block size / denoising steps
32 / 32
32 / 32
Temperature
0.8
0
Top- p / top- k
0.95 / 0
1.0 / 0
Confidence threshold
0.90
0.95
Max new tokens per segment
256
256
Appendix
Table 11: Action-Mode sampling parameters.
System
EM
F1
SubEM
ACC L
Search-o1
29.45
39.78
39.50
46.25
Search-R1
27.50
35.33
33.90
37.75
R1-Searcher
38.40
48.71
46.55
52.60
DLLM-Searcher
0.05
7.25
36.25
36.65
DiffuPlan
38.15
50.24
48.45
55.30
Appendix
Table 12: Answer metrics averaged over the four datasets (%).
Planning is increasingly important for long-horizon agents, where successful execution requires coordinating subgoals, tool use, and intermediate outcomes over many steps. Yet assumptions made during planning may be invalidated by the environment, tools may return unexpected results, or actions may fail. Effective agents must therefore not only generate plans, but also revise them. Such revisions often affect only part of a plan, leaving the preceding and subsequent structure intact. Rather than regenerate the entire plan and risk unnecessary changes, repair can regenerate the affected region conditioned on the preserved prefix and suffix. We introduce Plan-and-Patch, a plan-and-act framework in which a diffusion language model (dLLM) generates a structured, program-like plan through parallel unmasking and repairs it by filling in selected regions while keeping the surrounding steps fixed. We compare DreamReasoner-8B and Qwen3-8B as diffusion and autoregressive (AR) planners. On Natural Plan without task-specific training, diffusion (53.7%) achieves nearly twice the plan repair success rate of AR (27.0%). After task-specific training on agentic benchmarks, ALFWorld and TextCraft, the planners achieve similar observed success in plan generation, while diffusion reduces mean plan-generation latency by 39-46% relative to AR. Our results show that Plan-and-Patch provides a framework for faster plan generation and effective plan repair in long-horizon agents.
Diffusion large language models (dLLMs) generate text via iterative denoising but consistently underperform on multi-step reasoning. We hypothesize this gap stems from a coordination problem: AR models build coherence token-by-token, while diffusion models must coordinate all positions simultaneously. We propose plan conditioning, a training-free method that prepends a short (~100-token) natural-language plan from an AR model to the diffusion model's prompt. The plan serves as a frozen scaffold -- globally visible context that every token position can attend to from the first denoising step. On GSM8K, plan conditioning improves LLaDA-8B-Instruct from 75.6% to 87.2% (+11.6 percentage points), matching a same-size AR model (LLaMA 3.1 8B, 87.7%) despite a 6.4pp weaker baseline. On HumanEval, the gain is +12.8pp (37.2% to 50.0%), showing plans generalize to code. The same plans improve LLaMA by only +5.7pp on GSM8K and +1.3pp on HumanEval -- diffusion models benefit 2-10x more, supporting the coordination-problem hypothesis. Across 5 random seeds, plan-conditioned GSM8K accuracy has zero standard deviation, making diffusion inference highly stable. Ablations reveal the model follows plan strategy (wrong-strategy plans cause -16.3pp) but is robust to plan values (perturbed numbers: -1.1pp), and that planner quality has a sharp threshold: smaller Llama-class plans hurt (-1.6 to -6.8pp) while frontier plans provide the full lift. Attention analysis confirms the mechanism: plan tokens receive 1.8x excess attention during early denoising, declining to uniform as completion tokens solidify. Plan conditioning costs ~$0.002 per problem and adds ~2s of latency.
Most multi-agent systems rely exclusively on autoregressive language models (ARMs) that are based on sequential generation. Although effective for fluent text, ARMs limit global reasoning and plan revision. On the other hand, Discrete Diffusion Language Models (DDLMs) enable non-sequential, globally revisable generation and have shown strong planning capabilities, but their limited text fluency hinders direct collaboration with ARMs. We introduce Latent-DARM, a latent-space communication framework bridging DDLM (planners) and ARM (executors), maximizing collaborative benefits. Across mathematical, scientific, and commonsense reasoning benchmarks, Latent-DARM outperforms text-based interfaces on average, improving accuracy from 27.0% to 36.0% on DART-5 and from 0.0% to 14.0% on AIME2024. Latent-DARM approaches the results of state-of-the-art reasoning models while using less than 2.2% of its token budget. This work advances multi-agent collaboration among agents with heterogeneous models.