GRADUS: Discovering and Recovering Agent Lessons from the Environment's Latent Syllabus

Qing Ping; Wei Yang; Panpan Xu; Johannes Kirmayr; Lukas Stappen; Junjie Tang; Jun Huan

cs.AIOct 20, 2026

GRADUS: Discovering and Recovering Agent Lessons from the Environment's Latent Syllabus

Authors: Qing Ping, Wei Yang, Panpan Xu, Johannes Kirmayr, Lukas Stappen, Junjie Tang, Jun Huan

Organizations: AWS AI · University of Southern California · BMW Group Research and Technology

Abstract

Training LLM agents for policy-constrained tool environments, such as an airline backend with booking, refund, and cancellation policies, requires tasks that exercise the environment's rules individually and in combination. Existing synthesis pipelines let a model propose tasks and retain only those it immediately solves, biasing training data toward easier behaviors. We introduce Gradus (Graded Rungs of Ascending Difficulty from an Underlying System), which decouples what a task should teach from the concrete task and solution used to teach it. Without seed tasks, Gradus treats a fixed environment as a latent syllabus, extracting guarded behavioral operators that capture how actions depend on current conditions and how their effects reshape subsequent decisions. It composes these operators into increasingly demanding tasks while using corpus-level feedback to target missing outcome combinations. Crucially, Gradus jointly calibrates tasks and solutions against the environment: execution feedback identifies which side to refine while preserving the intended behavioral requirements. Verification thus becomes a recovery signal rather than a terminal filter, coupling coverage-guided exploration with the preservation of difficult examples. From off-the-shelf backbones, the resulting curriculum lifts a 30B model from 59% to 89% on $τ^2$ -Bench and from 47% to 83% on CAR-Bench with an Opus 4.5 proposer, and to 76% and 62%, respectively, with an open-weight Qwen3-235B proposer. These results show the value of jointly controlling which behaviors are explored and which survive into verified training data.

Figures & tables

Figure 1: Overview of Gradus . (1) Environment artifacts are compiled into guarded operators. (2) Coverage-guided evolution searches uncovered syllabus regions. (3) Joint task-solution refinement recovers unresolved candidates, preserving difficult lessons while accumulating verified examples.

Frontier models
		$\tau^{2}$ -Bench			CAR-Bench
Model	Proposer	Air	Ret	Tel	Base	Disamb	Halluc	Avg
Opus 4.5	–	$0.88$	$0.89$	$0.92$	$0.85$	$0.68$	$0.64$	$0.81$
GPT-5.2	–	$0.82$	$0.88$	$0.89$	$0.79$	$0.52$	$0.88$	$0.80$
Kimi-K2-Thinking	–	$0.84$	$0.88$	$0.79$	$0.56$	$0.42$	$0.67$	$0.69$
DeepSeek-V3.2	–	$0.78$	$0.91$	$0.76$	$0.64$	$0.48$	$0.57$	$0.69$

Table 1: Task success on

\tau^{2}

-Bench and CAR-Bench, with the six-split average (Avg) at right. GRADUS (ours) rows are shaded; bold marks the best result among all rows outside the Frontier models block.

Remove one component	Tel.	Dis.	Add validity cumulatively	Tel.	Dis.
Full (Ours)	$0.75$ $\pm 0.02$	$0.60$ $\pm 0.05$	No validity	$0.65$ $\pm 0.05$	$0.40$ $\pm 0.04$
$-$ Guarded operators	$0.64$ $\pm 0.03$	$0.45$ $\pm 0.03$	$+$ Solution: parallel	$0.68$ $\pm 0.02$	$0.43$ $\pm 0.03$
$-$ Global ledger	$0.54$ $\pm 0.03$	$0.52$ $\pm 0.04$	$+$ Solution: reflection	$0.71$ $\pm 0.04$	$0.55$ $\pm 0.05$
$-$ Structural evolution	$0.63$ $\pm 0.03$	$0.55$ $\pm 0.05$	$+$ Task recovery (full)	$0.75$ $\pm 0.02$	$0.60$ $\pm 0.05$
$-$ Interaction variation	$0.67$ $\pm 0.02$	$0.46$ $\pm 0.04$

Table 2: Ablation of GRADUS components with Qwen3-235B-A22B as proposal model and Qwen3-30B-A3B as student backbone. SFT task success (mean

\pm

std) on

\tau^{2}

-Bench telecom and Car-Bench disambiguation. Left: leave-one-out ablation; right: cumulative validity loop.

Figure 2: Dataset Coverage by Environment Dimensions (1,2: Airline; 3,4: Retail). Panel-1,3 : Policy-Outcome Coverage. Panel-2,4 : Action Combinations.

Figure 3: From intermittent frontier-model capability into reliable student performance.

Figure 4: Curriculum preservation retains difficult lessons. (a) Student zero-shot success (averaged over 8 runs) versus evolution depth. (b) Replay, reflection and recovery dynamics per loop. (c) Verified tasks grouped by the round in which they were accepted, split by difficulty class (Easy/Medium/Medium-Hard/Hard).

	Airline ( $\tau^{2}$ )		Retail ( $\tau^{2}$ )		Base (CAR)
Model	Original	Evolved	Original	Evolved	Original	Evolved
GPT-5.2	$0.82$ $\pm 0.05$	$0.64$ $\pm 0.08$	$0.88$ $\pm 0.02$	$0.62$ $\pm 0.08$	$0.79$ $\pm 0.02$	$0.44$ $\pm 0.04$
Opus 4.5	$0.88$ $\pm 0.01$	$0.52$ $\pm 0.08$	$0.89$ $\pm 0.01$	$0.60$ $\pm 0.02$	$0.85$ $\pm 0.02$	$0.43$ $\pm 0.03$
DeepSeek-V3.2	$0.78$ $\pm 0.03$	$0.43$ $\pm 0.08$	$0.91$ $\pm 0.02$	$0.57$ $\pm 0.04$	$0.64$ $\pm 0.02$	$0.26$ $\pm 0.04$
Kimi-K2-Thinking	$0.84$ $\pm 0.03$	$0.40$ $\pm 0.10$	$0.88$ $\pm 0.01$	$0.49$ $\pm 0.16$	$0.56$ $\pm 0.03$	$0.17$ $\pm 0.03$

Table 3: Task success on the original benchmarks and the frozen frontier exams (mean

\pm

std over five runs). Results are reported on

\tau^{2}

-Bench (Airline and retail), and CAR-bench (base). Opus 4.5 serves as the scout; all other models are held out during the evolved exam synthesis.

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Method	Source	Adaptive coverage	Drop	Revise	Demo	Human	Reason.	Cross-dom.	Transfer
			Failed-lesson handling			Interaction challenges
Simia ( Li et al., 2025 )	Seed	$\times$	$\sim$	$\times$	$\times$	$\times$	$\times$	$\times$	Xfer
CoVe ( Chen et al., 2026b )	System	$\times$	$\checkmark$	$\times$	$\times$	$\sim$	$\times$	$\times$	Direct
WRIT ( Gu et al., 2026 )	System	$\times$	$\checkmark$	$\times$	$\times$	$\checkmark$	$\sim$	$\times$	Direct
AgentSkiller ( Sun et al., 2026 )	World	$\times$	$\checkmark$	$\sim$	$\times$	$\sim$	$\times$	$\checkmark$	Xfer
EnvScaler ( Song et al., 2026 )	World	$\times$	$\sim$	$\times$	$\times$	$\times$	$\times$	$\times$	Xfer

Appendix

Table 4: Curriculum-design comparison.

\checkmark

present;

\sim

partial;

\times

absent. Direct and Xfer indicate construction in the target system and transfer from other systems.

Domain	SFT tasks	SFT rows	RL tasks	SFT tasks	SFT rows	RL tasks
	Opus 4.5 proposer			Qwen3-235B proposer
Airline	698	8,468	341	699	6,848	267
Retail	585	7,971	381	448	5,096	249
Telecom	414	6,145	367	370	5,162	168
CAR (combined)	1,901	11,091	1,227	986	5,117	1,018

Appendix

Table 5: Final curriculum size per domain and proposer. SFT tasks and RL tasks are verified task–reference pairs; a task supplies either supervised demonstrations or an RL reference.

Opus 4.5 proposer
Depth	Airline	Retail	Telecom	CAR (combined)
1	156	128	178	718
2	352	278	363	900
3	267	223	192	660
4	156	190	48	458
5	108	147	–	392

Appendix

Table 6: Verified curriculum tasks by evolution depth. Each entry is the total number of accepted tasks at that depth (supervised and RL tasks combined). The Opus proposer evolves to depth

4

on airline, retail, and CAR; telecom saturates at depth

4

. The Qwen proposer is reported over depths

1

4

. CAR combines the base, disambiguation, and hallucination splits.

Round	Verification finding	Recovery
0	The supposed flown segment was actually airline-cancelled, and the replacement request admitted valid alternatives.	Re-ground the same blueprint with a user having a truly landed segment, sold-out direct economy inventory, and reference actions consistent with the executable state.
1	The flown-segment defect was repaired, but the user could still accept a connecting economy itinerary because only basic economy had been excluded.	Clarify that the replacement request is for a direct economy flight and that connecting and basic-economy alternatives must be declined.
2	The predicted and reference database hashes match: agent_db_match=true .	Accept the recovered task into the final task set.

Appendix

Table 7: Task-side recovery preserves the evolved lesson while repairing grounding and ambiguity defects.

Model	Stage	Proposer	Airline	Retail	Telecom
Qwen3-8B (Ours)	SFT	Qwen3-235B	$0.71$ $\pm 0.02$	$0.74$ $\pm 0.02$	$0.64$ $\pm 0.01$
Qwen3-14B (Ours)	SFT	Qwen3-235B	$0.69$ $\pm 0.04$	$0.85$ $\pm 0.02$	$0.69$ $\pm 0.03$
Qwen3-30B-A3B (Ours)	SFT	Qwen3-235B	$0.69$ $\pm 0.05$	$0.85$ $\pm 0.02$	$0.75$ $\pm 0.02$
Qwen3-8B (Ours)	SFT	Opus 4.5	$0.71$ $\pm 0.05$	$0.85$ $\pm 0.02$	$0.92$ $\pm 0.01$
Qwen3-14B (Ours)	SFT	Opus 4.5	$0.75$ $\pm 0.01$	$0.85$ $\pm 0.01$	$0.85$ $\pm 0.03$
Qwen3-30B-A3B (Ours)	SFT	Opus 4.5	$0.75$ $\pm 0.05$	$0.86$ $\pm 0.03$	$0.90$ $\pm 0.02$

Appendix

Table 8: Student architecture and capacity comparison on

\tau^{2}

-Bench.

Model	Stage	Proposer	Base	Disamb.	Halluc.
Qwen3-8B (Ours)	SFT	Qwen3-235B	$0.69$ $\pm 0.04$	$0.45$ $\pm 0.04$	$0.51$ $\pm 0.03$
Qwen3-14B (Ours)	SFT	Qwen3-235B	$0.68$ $\pm 0.03$	$0.50$ $\pm 0.06$	$0.55$ $\pm 0.04$
Qwen3-30B-A3B (Ours)	SFT	Qwen3-235B	$0.72$ $\pm 0.04$	$0.60$ $\pm 0.05$	$0.55$ $\pm 0.04$
Qwen3-8B (Ours)	SFT	Opus 4.5	$0.76$ $\pm 0.02$	$0.58$ $\pm 0.06$	$0.74$ $\pm 0.04$
Qwen3-14B (Ours)	SFT	Opus 4.5	$0.80$ $\pm 0.02$	$0.60$ $\pm 0.03$	$0.74$ $\pm 0.02$
Qwen3-30B-A3B (Ours)	SFT	Opus 4.5	$0.80$ $\pm 0.04$	$0.74$ $\pm 0.02$	$0.74$ $\pm 0.02$

Appendix

Table 9: Student architecture and capacity comparison on Car-Bench.

Frontier models
Model	Proposer	Airline	Retail	Telecom	Avg	Avg. $\mathrm{pass}^{3}$
Opus 4.5	–	$0.88$ $\pm 0.01$	$0.89$ $\pm 0.01$	$0.92$ $\pm 0.01$	$0.90$	$0.82$
GPT-5.2	–	$0.82$ $\pm 0.05$	$0.88$ $\pm 0.02$	$0.89$ $\pm 0.01$	$0.86$	$0.76$
Kimi-K2-Thinking	–	$0.84$ $\pm 0.03$	$0.88$ $\pm 0.01$	$0.79$ $\pm 0.03$	$0.84$	$0.73$
DeepSeek-V3.2	–	$0.78$ $\pm 0.03$	$0.91$ $\pm 0.02$	$0.76$ $\pm 0.04$	$0.82$	$0.66$
Open-weight backbones

Appendix

Table 10: Detailed task success on

\tau^{2}

-Bench. Cells report mean

\pm

standard deviation over five runs. Avg averages the three domains.

\mathrm{pass}^{k}

credits a task only when all

k

independent runs succeed;we report

k=3

. Proposer names the curriculum-construction model where applicable. GRADUS rows are shaded; bold marks the best value outside the Frontier models block.

Frontier models
Model	Proposer	Base	Disamb	Halluc	Avg	Avg. $\mathrm{pass}^{3}$
Opus 4.5	–	$0.85$ $\pm 0.02$	$0.68$ $\pm 0.06$	$0.64$ $\pm 0.03$	$0.72$	$0.61$
GPT-5.2	–	$0.79$ $\pm 0.02$	$0.52$ $\pm 0.08$	$0.88$ $\pm 0.01$	$0.73$	$0.64$
Kimi-K2-Thinking	–	$0.56$ $\pm 0.03$	$0.42$ $\pm 0.05$	$0.67$ $\pm 0.03$	$0.55$	$0.43$
DeepSeek-V3.2	–	$0.64$ $\pm 0.02$	$0.48$ $\pm 0.07$	$0.57$ $\pm 0.03$	$0.56$	$0.41$
Open-weight backbones

Appendix

Table 11: Detailed task success on CAR-Bench. Cells report mean

\pm

standard deviation over five runs. Avg averages the three splits.

\mathrm{pass}^{k}

credits a task only when all

k

independent runs succeed; we report

k=3

. External curriculum baselines (Simia, CoVe, WRIT) do not report CAR-Bench and are omitted. Proposer names the curriculum-construction model where applicable. GRADUS rows are shaded; bold marks the best value outside the Frontier models block.

Domain	Evolved	Executable	Valid	$\leq 5/8$	Both correct
Airline ( $\tau^{2}$ )	212	84	82	32	25
Retail ( $\tau^{2}$ )	399	225	222	26	26
CAR-Bench	512	472	324	79	70

Appendix

Table 12: Construction and human-audit counts.

Domain	Correctness ( Yes/No/CT )	Realism ( Nat./Unusual/Contr. )	Strictness ( No/Partly/Yes )
Airline	87.5 / 10.9 / 1.6	26.6 / 53.1 / 20.3	18.8 / 70.3 / 10.9
Retail	100.0 / 0.0 / 0.0	46.2 / 32.7 / 21.2	53.8 / 38.5 / 7.7
CAR	90.5 / 8.2 / 1.3	88.6 / 11.4 / 0.0	81.0 / 13.3 / 5.7

Appendix

Table 13: Pooled annotation distributions (%).

Domain	Dimension	Raw agreement	Gwet’s AC1 [95% CI]
Airline	Correctness	81.2%	0.789 [0.586, 0.933]
	Realism	59.4%	0.399 [0.056, 0.680]
	Oracle strictness	78.1%	0.728 [0.461, 0.897]
Retail	Correctness	100.0%	1.000 [1.000, 1.000]
	Realism	73.1%	0.596 [0.239, 0.860]
	Oracle strictness	84.6%	0.821 [0.584, 0.960]

Appendix

Table 14: Inter-annotator agreement.

LLM Agent Training

Large language model agent training teaches models to plan, use tools, and learn from multi-step interaction with environments.

See all 567 papers

GRADS: Graph-Regulated Agentic Data Synthesis with Controllable Difficulty
Oct 15, 2026Gaocheng Yang, Ruijie Tao, Kai Ye +3Synthetic Data Generation for LLM Agents Synthetic Data Generation for Language Models
Johns Hopkins University · Nanjing University · Amazon
AgentGrad: Intervention-guided Prompt Optimization for Multi Agent Systems
Sep 8, 2026Jaewon Chu, Jinwoo Seo, Jaewon Cho +4Multi-Agent LLM Systems Multi-Agent System Optimization
Korea University · KAIST · Meta AI +1
From Agent Failures to Text Policies: What Works and What Breaks
Jul 22, 2026Jaideep Ray, Ankit GoyalPolicy Optimization LLM Agent Self-Improvement
Independent Researcher

Oct 15, 2026J/K move · Enter open · S save

GRADS: Graph-Regulated Agentic Data Synthesis with Controllable Difficulty

Gaocheng Yang, Ruijie Tao, Kai Ye, Senhua Zhang+2

Johns Hopkins University · Nanjing University · Amazon

Synthetic Data Generation for LLM Agents Synthetic Data Generation for Language Models Synthetic Data Generation Synthetic Benchmark Generation LLM Agents Synthetic Data Evaluation

Open paper AskPDF arXiv

Deep research agents are increasingly deployed to retrieve precise information from large corpora, yet multi-hop, long-horizon tasks remain challenging. Synthetic data generation has emerged as a scalable solution, with existing pipelines typically expanding seed tasks to increase their apparent depth and breadth. However, this expansion does not reliably translate into necessary reasoning: newly introduced evidence may be only weakly coupled to the answer, creating a mismatch between intended and empirical difficulty. To address this problem, we introduce GRADS, a graph-guided agentic data synthesis framework with explicit difficulty control. GRADS runs a closed loop of seed selection, evidence expansion, question--answer construction, and verification while maintaining a graph of the evidence and inference dependencies required to derive each answer. At each step, it conditions generation on a relevant subgraph, preserving indispensable inference without accumulating irrelevant context. Experiments across security, software-engineering, and scientific-document domains show that GRADS consistently generates verified tasks with empirically separated difficulty distributions, as measured by independent solvers. Moreover, under matched training budgets, post-training on this data delivers consistent gains at every level: SFT raises the solution-termination rate to over 95% while cutting tool calls and turns by up to 75%, RL trims tool use by up to a further 19.5%, and on our held-out, domain-specific benchmarks, the resulting 4B model matches Sonnet~4 on software engineering and surpasses it on the scientific and security domains.

Abstract

LLM Agent Training

Explore similar work

GRADS: Graph-Regulated Agentic Data Synthesis with Controllable Difficulty

AgentGrad: Intervention-guided Prompt Optimization for Multi Agent Systems

From Agent Failures to Text Policies: What Works and What Breaks