Anchors Enhance Controllability and Reasoning of Diffusion Language Models

Feiyu Qu; Zheng Huang; Andrea Zanette

cs.LGOct 10, 2026

Anchors Enhance Controllability and Reasoning of Diffusion Language Models

Authors: Feiyu Qu, Zheng Huang, Andrea Zanette

Organizations: Department of Computer Science, Dartmouth College · Department of Electrical and Computer Engineering, Carnegie Mellon University

Abstract

Diffusion language models (DLMs) offer new opportunities for controllable text generation. However, their controllability, defined as the ability to satisfy task constraints while preserving their original capabilities, remains underexplored. DLMs' bidirectional attention allows control tokens to guide generation from arbitrary positions in the output sequence, but effective guidance depends on the compatibility between their content and placement. To this end, we propose AnchorDLM, a framework in which an anchor generator produces an anchor consisting of a short output fragment and its position for each query. The anchor is placed as control tokens at the specified position to initialize the output sequence of a frozen DLM and guide subsequent denoising. We train the anchor generator with reinforcement learning (RL) to jointly optimize anchor content and placement, using rewards computed from the DLM's generated outputs based on answer correctness and constraint satisfaction. Moreover, through the lens of discrete flow matching (DFM), we interpret anchor placement as helping to reshape the initial velocity field that guides generation. Experiments on code generation and mathematical reasoning across five DLM backbones show that AnchorDLM improves controllability and further yields gains in reasoning performance across diverse benchmarks.

Figures & tables

Figure 1: Overview of AnchorDLM framework. (Right) Decoding order averaged over 50 samples.

Figure 2: Training pipeline and DFM perspective of AnchorDLM. T1–T2 construct training data. T3 performs supervised fine-tuning(SFT). T4 further refines the adapter with reinforcement learning(RL). (Right) From a discrete flow matching perspective, an anchor reshapes the initial coordinate of the discrete flow, pulling the generation trajectory from the uncontrolled region toward the controlled region of the discrete state space.

Backbone	Method	Controllability						Reasoning
		GSM8K		HumanEval		MBPP		GSM8K	MATH	HE	MBPP
		Fmt.	Bwd.	Cmt.	Str.	Cmt.	Str.	Acc	Acc	Pass@1	Pass@1
Llama 3-8B-Base	Vanilla	13.92	10.02	23.49	44.13	25.82	45.65	43.59	24.80	33.54	41.40
Llama 3-8B-Base	Prompt	24.66	21.67	36.73	46.97	39.61	47.41	56.03	28.00	36.59	43.80
LLaDA-8B-Base	Vanilla	5.01	5.01	14.59	42.32	29.77	43.16	66.03	30.20	34.15	39.80
	Prompt	7.99	7.40	23.29	45.04	31.16	47.27	66.57	31.80	35.37	40.20

Table 1: Main results with the anchor generator trained for controllability (left) and reasoning (right). Controllability is measured by G-mean, and reasoning by accuracy or pass@1. Bold indicates the best result for each backbone.

Model	Qwen3					Llama 3.3
Size	0.6B	1.7B	4B	8B	32B	70B
HumanEval	24.39	33.54	38.41	38.41	35.98	40.24
MBPP	28.60	37.80	37.60	38.00	42.20	43.20

Table 2: Pass@1 (%) with anchors extracted from different models using the same heuristic.

Figure 3: Effect of anchor placement with content held fixed: G-mean (left) and performance (right).

Figure 4: Anchor position distribution and density across datasets.

Variant	GSM8K		MBPP		Avg. $\Delta$
Variant	Formal	Acc	Comt.	Pass@1	Avg. $\Delta$
LLaDA-8B-Base (no anchor)	5.01	66.03	29.77	39.80	–
Full AnchorDLM	69.33 $\uparrow$ 64.32	67.17 $\uparrow$ 1.14	40.05 $\uparrow$ 10.28	43.60 $\uparrow$ 3.80	+19.89
w/o WPO (replace with DPO)	60.42 $\uparrow$ 55.41	66.79 $\uparrow$ 0.76	38.06 $\uparrow$ 8.29	40.20 $\uparrow$ 0.40	+16.22
w/o RL (SFT only)	47.16 $\uparrow$ 42.15	65.96 $\downarrow$ 0.07	36.88 $\uparrow$ 7.11	40.00 $\uparrow$ 0.20	+12.35
w/o query-anchor augmentation	22.97 $\uparrow$ 17.96	58.98 $\downarrow$ 7.05	31.73 $\uparrow$ 1.96	28.00 $\downarrow$ 11.80	+0.27

Table 3: Ablation study of AnchorDLM on GSM8K and MBPP.

Figure 5: Case study on MBPP sample (idx=389), with anchor highlighted.

	HumanEval	MBPP	GSM8K	MATH500
Anchor length (tokens)	11.08	11.45	27.85	26.63
Anchor content ratio (%)	4.33	4.47	10.88	10.86

Table 4: Average anchor length and content ratio across benchmarks.

DLM	Method	GSM8K			HumanEval		MBPP
DLM	Method	Formal	Backw.	Acc.	Cmt.	Pass@1	Cmt.	Pass@1
LLaDA-8B-Base	Vanilla	5.01	5.01	66.03	14.59	34.15	29.77	39.80
LLaDA-8B-Base	AnchorDLM	69.33 (+64.32)	71.08 (+66.07)	67.17 (+1.14)	43.89 (+29.30)	37.20 (+3.05)	40.05 (+10.28)	43.60 (+3.80)
LLaDA-8B-Instruct	Vanilla	17.08	30.94	56.10	29.83	39.02	28.25	40.80
LLaDA-8B-Instruct	AnchorDLM (transferred)	59.51 (+42.43)	62.34 (+31.40)	58.23 (+2.13)	45.02 (+15.19)	39.63 (+0.61)	47.80 (+19.55)	41.00 (+0.20)

Table 5: Transfer from LLaDA-8B-Base to LLaDA-8B-Instruct without retraining the anchor generator. Formal, Backw., and Cmt. report controllability measured by G-mean; Acc. and Pass@1 report task performance. Parentheses indicate absolute gains over each DLM’s vanilla baseline.

Diffusion Language Models

Diffusion language models generate and refine text through iterative denoising, enabling bidirectional context and parallel token updates.

See all 475 papers

DELA: Learning Controllable Edits for Diffusion Language Models
Oct 23, 2026Xinsong Feng, Chenan Wang, Leshu Tang +4Diffusion Language Models Diffusion Language Model Decoding
Georgia Institute of Technology · William & Mary · UCLA +1
Dynamic Infilling Anchors for Format-Constrained Generation in Diffusion Large Language Models
Jun 3, 2026Boyan Han, Yiwei Wang, Yi Song +2Controllable Text Generation Diffusion Language Model Decoding
AGI Lab, Westlake University, China · University of California, Merced, USA · Teeni AI, China +1
Time-Anchored Diffusion Language Models: Latent-Space Caching for Fast Generation
Sep 29, 2026Joel Anto Paul, Litu Rout, Aditya Akella +1Diffusion Language Model Training Diffusion Language Model Inference
UT Austin

Oct 23, 2026J/K move · Enter open · S save

DELA: Learning Controllable Edits for Diffusion Language Models

Xinsong Feng, Chenan Wang, Leshu Tang, Johnny Liu+3

Georgia Institute of Technology · William & Mary · UCLA+1

Diffusion Language Models Diffusion Language Model Decoding

Open paper AskPDF arXiv

Diffusion language models (DLMs) support bidirectional denoising and iterative refinement, but generation remains largely controlled by token-level masking schedules. This leaves draft-level refinement implicit and limits explicit control over when to insert missing content, delete redundant spans, revisit uncertain regions, or stop editing. We introduce Diffusion Editing Language Adapter (DELA), a plug-in framework that enables controllable draft refinement with a frozen DLM. DELA operates on clean, variable-length drafts using four explicit editing actions: Insert, Delete, Remask, and Stop. The controller determines when and where to edit the draft, while the frozen DLM fills masked regions through denoising, separating high-level edit control from token-level generation. A naive single-phase editor would be computationally expensive because arbitrary interior edits can disrupt prefix stability and limit reliable KV-cache reuse. To address this, DELA uses a two-phase decoding procedure: a fast, append-only drafting phase that supports KV-cache reuse, followed by a global refinement phase that enables targeted non-monotonic edits to the full draft. We train the controller using a staged curriculum that begins with learning local edit priors from synthetic recovery tasks and mined actions that improve draft quality. Subsequent stages refine edit selection through rollout-based outcome preferences and alignment with downstream task feedback. Experiments on mathematical reasoning and code generation show that DELA achieves gains of up to 5.5 percentage points over direct diffusion decoding while keeping the backbone frozen. DELA thus provides a modular approach to explicit draft-level control in diffusion language models.

Abstract

Diffusion Language Models

Explore similar work

DELA: Learning Controllable Edits for Diffusion Language Models

Dynamic Infilling Anchors for Format-Constrained Generation in Diffusion Large Language Models

Time-Anchored Diffusion Language Models: Latent-Space Caching for Fast Generation