GRADS: Graph-Regulated Agentic Data Synthesis with Controllable Difficulty

Gaocheng Yang; Ruijie Tao; Kai Ye; Senhua Zhang; Zhengqin Zhu; Ziyang Li

cs.AIOct 15, 2026

GRADS: Graph-Regulated Agentic Data Synthesis with Controllable Difficulty

Authors: Gaocheng Yang, Ruijie Tao, Kai Ye, Senhua Zhang, Zhengqin Zhu, Ziyang Li

Organizations: Johns Hopkins University · Nanjing University · Amazon

Abstract

Deep research agents are increasingly deployed to retrieve precise information from large corpora, yet multi-hop, long-horizon tasks remain challenging. Synthetic data generation has emerged as a scalable solution, with existing pipelines typically expanding seed tasks to increase their apparent depth and breadth. However, this expansion does not reliably translate into necessary reasoning: newly introduced evidence may be only weakly coupled to the answer, creating a mismatch between intended and empirical difficulty. To address this problem, we introduce GRADS, a graph-guided agentic data synthesis framework with explicit difficulty control. GRADS runs a closed loop of seed selection, evidence expansion, question--answer construction, and verification while maintaining a graph of the evidence and inference dependencies required to derive each answer. At each step, it conditions generation on a relevant subgraph, preserving indispensable inference without accumulating irrelevant context. Experiments across security, software-engineering, and scientific-document domains show that GRADS consistently generates verified tasks with empirically separated difficulty distributions, as measured by independent solvers. Moreover, under matched training budgets, post-training on this data delivers consistent gains at every level: SFT raises the solution-termination rate to over 95% while cutting tool calls and turns by up to 75%, RL trims tool use by up to a further 19.5%, and on our held-out, domain-specific benchmarks, the resulting 4B model matches Sonnet~4 on software engineering and surpasses it on the scientific and security domains.

Figures & tables

Figure 1: Graph expansion. Exploration adds provenance-linked claim nodes and necessary-dependency edges across rounds; reasoning depth accumulates until the requested band is reached, at which point a subgraph of the target depth is selected and verbalized into one multi-hop question.

Figure 2: Data synthesis agent flow. A stateless Brain drives the loop from the selected context. (1) Beginning: the Sampler draws a seed and the Analyst sets the generation target. (2) Main Workflow: each round, Explorer , Analyst , Respondent , Questioner , and Verifier build and check one QA item (green

=

information, red

=

feedback). (3) Context Management: the Graph Builder , Graph Verifier , and Selector maintain the evidence graph and return the target-shaped subgraph as the next context. An append-only Long History Memory and a rewritten Short Graph Memory back the loop.

Figure 3: Difficulty is controllable and empirical only for GRADS: delivered depth (bottom row) and solver score (top row) by requested band. For more details, check Appendix Table 1 .

Figure 4: SFT on GRADS data (Ours) dominates every synthesis baseline in every domain; dashed line is the raw model. Check more details in Appendix Table 5 .

Figure 7: Every capability tier degrades from Easy

\to

Hard; capability is a level shift. For more details, check Appendix Table 4 .

Appendix figures & tables13 assets

Supplementary material from the paper’s appendix.

Appendix

		Solver score			Reasoning depth
Method	Domain	Easy	Medium	Hard	Easy	Medium	Hard
GRADS	GitHub	0.778	0.722	0.522	3.53	5.50	11.03
	Physics	0.715	0.450	0.374	3.47	4.83	10.82
	Security	0.725	0.671	0.559	2.57	4.17	9.10
TaskCraft	GitHub	1.000	0.950	0.950	1.20	1.80	2.55
	Physics	0.800	1.000	0.950	1.05	1.40	1.45

Appendix

Table 1: Data behind Figure 3 . Solver score (rubric-weighted,

\in[0,1]

;

n=20

per cell) and delivered reasoning depth (longest necessary-inference chain) by method

\times

domain

\times

requested band. Only GRADS shows a monotone Easy

>

Medium

>

Hard solver-score gradient together with depth that grows Easy

\to

Hard.

Axis	Symbol	GitHub	Physics	Security	Pooled
Depth	$d(G)$	$-0.50$	$-0.51$	$-0.49$	$-0.45$
Information flow	$c_{\mathrm{info}}(G)$	$-0.38$	$-0.36$	$-0.39$	$-0.39$
Size	$\|V\|$	$-0.26$	$-0.26$	$-0.35$	$-0.38$
Breadth	$b(G)$	$-0.12$	$-0.07$	$-0.10$	$-0.17$
Dispersion	–	$-0.44$	$-0.34$	$-0.44$	$-0.38$

Appendix

Table 2: Spearman

r_{s}

between rubric-graded solver score and each term of

\mathcal{C}(G)

, per domain and pooled. Depth is the strongest predictor on GitHub, Physics, and pooled; dispersion is the number of distinct source records.

Domain	Arm	Ckpt 1	Ckpt 2	Ckpt 3	Best	$\Delta$ raw	$\Delta$ ours
Physics	raw Qwen3.5-4B	–			0.2319	–	–
	Ours	0.2932 (10)	0.3390 (20)	0.3139 (24)	0.3390	$+0.1071$	–
	Claude-Code	0.2936 (12)	0.2884 (20)	0.3011 (24)	0.3011	$+0.0692$	$-0.0379$
	TaskCraft	0.2507 (12)	0.2399 (20)	0.2407 (24)	0.2507	$+0.0188$	$-0.0883$
	WebShaper	0.2850 (12)	0.2602 (16)	0.2548 (20)	0.2850	$+0.0531$	$-0.0540$
GitHub	raw Qwen3.5-4B	–			0.1149	–	–

Appendix

Table 3: SFT benchmark score at three checkpoints per arm (step in parentheses; best in bold), with the best score and its change versus the raw model and versus ours. Ours is the best arm in every domain.

Model	Domain	Easy	Medium	Hard
Opus-4.8 (frontier)	GitHub	0.778	0.722	0.522
	Physics	0.715	0.450	0.374
	Security	0.725	0.671	0.559
Sonnet-4 (medium)	GitHub	0.421	0.362	0.183
	Physics	0.381	0.279	0.170
	Security	0.446	0.340	0.266

Appendix

Table 4: Data behind Figure 7 . Mean solver score (

\in[0,1]

, claim-weighted judge held constant at Opus 4.8) by model

\times

domain

\times

band, for a frontier (Opus 4.8), medium (Sonnet 4), and light (Qwen3.5-4B) solver. Every model degrades monotonically Easy

>

Medium

>

Hard in every domain.

Domain	Arm	Ckpt 1	Ckpt 2	Ckpt 3
Physics	Ours	0.2932	0.3390	0.3139
	ClaudeCode	0.2936	0.2884	0.3011
	TaskCraft	0.2507	0.2399	0.2407
	WebShaper	0.2850	0.2602	0.2548
GitHub	Ours	0.2574	0.2835	0.2838
	ClaudeCode	0.1372	0.1202	0.1586

Appendix

Table 5: Data behind Figure 4 . Benchmark score at three checkpoints per arm and domain (dash where an arm has fewer saved points). Raw Qwen3.5-4B: Physics

0.2319

, GitHub

0.1149

, Security

0.1414

Domain	step 9	step 18	step 24	step 30	SFT start	raw
Physics	0.3130	0.3226	0.3185	0.3407	0.3390	0.2319
GitHub	0.2540	0.2732	0.2855	0.3069	0.2835	0.1149
Security	0.4119	0.4488	0.4663	0.4543	0.4277	0.1414

Appendix

Table 6: GRPO benchmark score at four checkpoints (steps

9/18/24/30

), the SFT starting policy, and the raw model, behind Figure 6 . Best-RL vs. best-SFT deltas all fall inside the

\pm 0.043

noise floor.

Model	GitHub	Physics	Security
Sonnet-4 (mid solver)	0.308	0.266	0.342
Post-trained 4B (best RL)	0.307	0.341	0.466
$\Delta$ (4B $-$ Sonnet-4)	$-0.001$	$+0.075$	$+0.124$

Appendix

Table 7: Post-training closes the capability gap to a mid-sized solver. Benchmark mean score (

30

30

40

-weighted) of the Sonnet-4 solver (from Table 4 ) versus the post-trained Qwen3.5-4B (best RL checkpoint, Table 6 ). After post-training the 4B matches Sonnet-4 on GitHub and surpasses it on Physics and Security.

Domain	Model	Tool calls	Turns	boxed	Ans. length	Cap-hits
Physics	raw	16.35	17.08	6	6662	28
	SFT	10.30	11.27	97	5528	4
	RL	8.29	9.29	100	6656	–
GitHub	raw	21.96	22.27	5	3172	69
	SFT	10.00	10.95	95	5776	5
	RL	8.95	9.94	99	5465	–

Appendix

Table 8: Data behind Figure 6 and the tool analysis. Per-item means for the raw model, best SFT, and best RL checkpoint; \boxed{} count and items-at-25-turn-cap are counts out of 100. SFT’s invariant effect is the jump in boxed rate and the drop in tool calls/turns; answer length moves in domain-specific directions.

Band	API calls	Input tok	Output tok	Total tok
Easy	$17.5$	$149$ k	$31$ k	$180$ k
Medium	$30.2$	$271$ k	$56$ k	$327$ k
Hard	$40.3$	$358$ k	$76$ k	$434$ k

Appendix

Table 9: GRADS synthesis cost per delivered item, by requested difficulty (Opus 4.8). Both API calls and tokens rise monotonically with depth.

Method	API calls / item	Tokens / item
GRADS	28	$302$ k
Claude-Code	$33.7$	$236$ k
TaskCraft	${\sim}278$	${>}0.84$ M (floor)
WebShaper	${\sim}2.9$	${\sim}22$ k (est.)

Appendix

Table 10: Synthesis cost per item, GRADS versus baselines. Cost is only meaningful alongside delivered depth (Table 1 ): WebShaper is cheapest because it barely explores (and stays shallow), TaskCraft is by far the most expensive yet still shallow, and Claude-Code spends about as much as GRADS but never reaches the target depth. Measurement notes: GRADS (Opus 4.8) and Claude-Code are exact; TaskCraft calls are exact but its tokens are a floor (metered on ReAct agent-steps only,

{\sim}3.5

k of

{\sim}25.6

k calls, so true tokens are higher); WebShaper calls are exact but its tokens are estimated.

Type	Intent
information_lookup	Retrieve a specific grounded fact.
troubleshooting	Diagnose a symptom to its root cause.
how_to_process	Recover a procedure or sequence of steps.
summarization_analysis	Summarize and analyze a body of evidence.
domain_knowledge	Apply domain-specific concepts to the records.
code_explanation	Explain what a piece of code or a change does.

Appendix

Table 11: The ten question types and their intent.

Domain	Arm	$N$	Steps	Tokens (k)	Sup. tokens (k)	Turns	Tool calls	Score
Physics	Raw Qwen3.5-4B	–	–	–	–	–	–	0.2319
	Ours	61	24	905.1	169.0	8.22	7.22	0.3390
	ClaudeCode	59	24	938.6	156.4	10.22	9.22	0.3011
	TaskCraft	60	24	626.6	84.5	4.98	3.98	0.2507
	WebShaper	60	24	715.2	98.5	5.43	4.43	0.2850
GitHub	Raw Qwen3.5-4B	–	–	–	–	–	–	0.1149

Appendix

Table 12: SFT training data and benchmark score per arm, for the same arms as Table 3 .

N

: trajectories per arm; Steps: optimizer steps (6 epochs, 16 sequences per step); Tokens: all tokens per epoch; Sup. tokens: tokens receiving loss; Turns / Tool calls: means per trajectory; Score: reported checkpoint. † Built against an augmented retrieval index, because the method’s security questions cite documents it generated itself, so its score is a lower bound.

Setting	Value
Initialization	SFT ckpt. (physics 20, GitHub 56, security 16)
Training / validation pairs	240 / 40 per domain (in-loop validation: seeded 10-item subset)
Advantage estimator	GRPO, group size $n{=}8$
Batch	40 prompts $\times$ 8 rollouts $=$ 320 trajectories per step; mini-batch $=$ batch
Schedule	6 steps/epoch $\times$ 5 epochs $=$ 30 steps (9,600 rollouts)
Learning rate	$10^{-6}$ , constant after 10% linear warmup

Appendix

Table 13: GRPO configuration, shared across domains except where noted. Training truncates observations to 3,000 characters and allows 12–14 turns, versus 12,000 characters and 25 turns at evaluation.

Synthetic Data Generation for LLM Agents

See all 30 papers

GRADUS: Discovering and Recovering Agent Lessons from the Environment's Latent Syllabus
Oct 20, 2026Qing Ping, Wei Yang, Panpan Xu +4LLM Agent Training LLM Agent Trajectory Synthesis
AWS AI · University of Southern California · BMW Group Research and Technology
QUEST: Training Frontier Deep Research Agents with Fully Synthetic Tasks
May 22, 2026Jian Xie, Tianhe Lin, Zilu Wang +16Deep Research Agents LLM Agent Training
1The Ohio State University · 2Amazon AGI SF Lab
Data World: Scaling Data Environments from Scratch
Oct 15, 2026Zhanli Li, Zeyuan Xu, Chaoxu Pang +16Synthetic Data Generation for LLM Agents Data Analysis Agents
Longcat Interaction Team, Meituan · Zhongnan University of Economics and Law · Hunan University +7

Oct 20, 2026J/K move · Enter open · S save

GRADUS: Discovering and Recovering Agent Lessons from the Environment's Latent Syllabus

Qing Ping, Wei Yang, Panpan Xu, Johannes Kirmayr+3

AWS AI · University of Southern California · BMW Group Research and Technology

LLM Agent Training LLM Agent Trajectory Synthesis Agent Skill Learning Curriculum Learning Tool Use

Open paper AskPDF arXiv

Training LLM agents for policy-constrained tool environments, such as an airline backend with booking, refund, and cancellation policies, requires tasks that exercise the environment's rules individually and in combination. Existing synthesis pipelines let a model propose tasks and retain only those it immediately solves, biasing training data toward easier behaviors. We introduce Gradus (Graded Rungs of Ascending Difficulty from an Underlying System), which decouples what a task should teach from the concrete task and solution used to teach it. Without seed tasks, Gradus treats a fixed environment as a latent syllabus, extracting guarded behavioral operators that capture how actions depend on current conditions and how their effects reshape subsequent decisions. It composes these operators into increasingly demanding tasks while using corpus-level feedback to target missing outcome combinations. Crucially, Gradus jointly calibrates tasks and solutions against the environment: execution feedback identifies which side to refine while preserving the intended behavioral requirements. Verification thus becomes a recovery signal rather than a terminal filter, coupling coverage-guided exploration with the preservation of difficult examples. From off-the-shelf backbones, the resulting curriculum lifts a 30B model from 59% to 89% on

τ^2

-Bench and from 47% to 83% on CAR-Bench with an Opus 4.5 proposer, and to 76% and 62%, respectively, with an open-weight Qwen3-235B proposer. These results show the value of jointly controlling which behaviors are explored and which survive into verified training data.

Abstract

Synthetic Data Generation for LLM Agents

Explore similar work

GRADUS: Discovering and Recovering Agent Lessons from the Environment's Latent Syllabus

QUEST: Training Frontier Deep Research Agents with Fully Synthetic Tasks

Data World: Scaling Data Environments from Scratch