Know What You Don't Know: Agentic Data Synthesis by Evolving Generation Space for Closed-Loop Learning

Wei Yang; Qing Ping; Youzhi Luo; Yawei Wang; Abhishek Kumar; Panpan Xu; Jun Huan

cs.AIOct 8, 2026

Know What You Don't Know: Agentic Data Synthesis by Evolving Generation Space for Closed-Loop Learning

Authors: Wei Yang, Qing Ping, Youzhi Luo, Yawei Wang, Abhishek Kumar, Panpan Xu, Jun Huan

Organizations: University of Southern California · AWS AI

Abstract

The continued improvement of large language models increasingly depends on synthetic experience, yet effective data synthesis for interactive agents requires not only executable and verifiable trajectories, but also interactions that provide learning value for the current task agent. Existing methods increasingly incorporate environment feedback, data validation, or learner failures, but still face a feedback-to-decision gap: observed weaknesses do not directly specify how future synthesis should change, or whether they can be represented by the current synthesis space. To address this gap, we propose EvoSyn, a learning-feedback-driven multi-agent framework that organizes synthesis around a revisable state specifying what interaction factors can be deliberately generated and where synthesis effort should be allocated. A role-specialized multi-agent system translates learner feedback into synthesis revisions, while a Designer allocates synthesis directions and a Planner grounds them into executable experience. This process enables within-space adaptation toward weak or under-covered regions and synthesis-space expansion that promotes newly discovered factors into explicit generation controls. Experiments on $τ^2$ -Bench and CAR-Bench show that EvoSyn improves average task success by 13.3 points over the strongest synthesis baseline, while producing higher-quality and structurally broader synthetic experience and discovering controllable factors missed without learner feedback.

Figures & tables

Figure 1: From static synthesis to closed-loop self-evolving agentic synthesis. Existing paradigms improve data quality or adapt within a fixed synthesis space. In contrast, EvoSyn uses learner feedback to revise both the allocation over known factors and the synthesis space itself by introducing new controllable interaction factors.

Figure 2: Overview of EvoSyn . Each round instantiates a closed-loop synthesis–learning process in which a revisable synthesis state mediates learner diagnosis and generation control, role-specialized agents realize verified experience, and feedback drives within-space adaptation or synthesis-space expansion to co-evolve synthesis with task-agent learning.

Method	Synthesis LLM	$\tau^{2}$ -Bench			CAR-Bench	Avg.
Method	Synthesis LLM	Airline	Retail	Telecom	CAR-Bench	Avg.
Frontier Closed-source Models
Claude Opus 4.5	–	79.3	91.8	88.0	82.0	85.3
GPT-5.2	–	83.0	81.6	89.7	74.0	82.1
Gemini-3 Pro	–	80.5	75.9	91.0	67.3	78.7
Open-Source Foundation Models

Table 1: Overall task success rate (%) on agentic benchmarks. For a consistent comparison across synthesis approaches, all synthesis methods use Qwen3-8B as the downstream task agent. EvoSyn -R1 uses first-round data, while EvoSyn -Final-SFT uses accumulated multi-round data.

Method	$\tau^{2}$ -Bench Airline					CAR-Bench Base
Method	Success Rate	Factor Coverage	Factor Balance	Interaction Coverage	Trajectory Diversity	Success Rate	Factor Coverage	Factor Balance	Interaction Coverage	Trajectory Diversity
EvoSyn	0.713	0.891	0.450	0.455	0.769	0.726	0.893	0.508	0.373	0.938
w/o Learning Feedback	0.660	0.778	0.368	0.403	0.538	0.613	0.787	0.468	0.351	0.750
w/o Reflector	0.673	0.862	0.413	0.437	0.615	0.633	0.803	0.483	0.350	0.800
w/o Designer	0.613	0.766	0.387	0.412	0.462	0.540	0.634	0.450	0.339	0.733
Single-Agent Synthesis	0.547	0.365	0.165	0.133	0.077	0.406	0.465	0.425	0.287	0.467

Table 2: Ablation study on

\tau^{2}

-Bench Airline and CAR-Bench Base. All variants use Opus 4.5 as the synthesis LLM, Qwen3-8B as the task agent, Final-SFT as the training protocol, and the same candidate generation budget. Intrinsic data-quality metrics are computed only on the final verified synthetic samples and normalized to

[0,1]

, where higher is better.

Figure 3: Closed-loop synthesis-space evolution on CAR-Bench. As learner weaknesses shift across rounds, EvoSyn promotes newly discovered interaction factors into explicit generation controls and redirects synthesis toward the evolving capability boundary, progressing from write-intensive execution to increasingly read-intensive decision making.

Figure 4: Closed-loop evolution of task performance and synthetic-data quality on CAR-Bench. Unlike static accumulation, EvoSyn progressively improves both downstream success and the structural quality of synthesized experience across successive synthesis rounds.

Figure 5: Capability-gap-driven synthesis adaptation on CAR-Bench. The three panels trace the closed-loop transition from pre-round capability gaps to next-round synthesis reallocation and subsequent capability changes, showing how EvoSyn redirects synthesis toward learner weaknesses.

Appendix figures & tables21 assets

Supplementary material from the paper’s appendix.

Appendix

Method	Interactive Setting	Executable & Verifiable	Task-Agent Feedback	Learning-Effect Feedback	Structured Task Space	Known-Factor Adaptation	New-Factor Discovery	Closed-Loop Evolution
Reasoning and instruction synthesis
MATRIX / MATRIX-Gen	–	–	–	–	✓	–	–	–
Socratic-Zero	–	–	✓	–	–	✓	–	✓
R-Zero	–	–	✓	–	–	✓	–	✓
SwS	–	–	✓	–	✓	✓	–	–
Interactive and agentic data synthesis

Appendix

Table 3: Comparison of representative data-synthesis methods for post-training LLMs and agents. ✓ indicates that the capability is explicitly supported and demonstrated by the method, while – indicates that it is absent or not demonstrated. Partial or implicit support is conservatively marked as –.

Method	Synthesis LLM	Task Agent	$\tau^{2}$ -Bench			CAR-Bench	Avg.
Method	Synthesis LLM	Task Agent	Airline	Retail	Telecom	CAR-Bench	Avg.
Frontier Closed-source Models
Claude Haiku 4.5	–	–	60.0	75.1	42.4	53.3	57.7
Claude Sonnet 4.5	–	–	83.3	88.3	62.0	71.3	76.2
Claude Opus 4.5	–	–	79.3	91.8	88.0	82.0	85.3
GPT-5.2	–	–	83.0	81.6	89.7	74.0	82.1

Appendix

Table 4: Direct foundation-model performance and task-agent scaling results of EvoSyn . For EvoSyn , Claude Opus 4.5 is used as the synthesis LLM, while Qwen3-8B, Qwen3-14B, and Qwen3-30B serve as downstream task agents. EvoSyn -R1 uses first-round synthesized data, whereas EvoSyn -Final-SFT trains on accumulated multi-round data.

Figure 6: Role of the Designer in task-space exploration on $\tau^{2}$ -Bench Airline. A Planner–Generator pipeline without explicit high-level allocation collapses toward flight_change , whereas introducing the Designer separates global synthesis allocation from local task construction and restores coverage across all six task families.

Figure 7: Structural complementarity of EvoSyn relative to domain-matched synthetic-data baselines on Airline. Left: fraction of each method’s distinct decisive interaction support that is absent from the pooled support of all other methods under matched data budgets. EvoSyn provides the largest unique structural contribution. Right: fraction of EvoSyn ’s structural support that is absent from the union of all matched baseline corpora across synthesis rounds. Iterative synthesis-space evolution progressively moves generation toward interaction structures not represented by existing synthesis pipelines.

Figure 8: Order-sensitive contrastive structure in synthetic data. Left: On Telecom, EvoSyn exhibits substantially higher tool-order contrastive support than the domain-matched AReaL-SEA baseline ( $53.1\%$ vs. $9.6\%$ , $5.5\times$ ). Right: Conceptual illustration of the metric. Repeated use of the same ordering provides little order-level contrast, whereas multiple orderings over the same or overlapping action composition create richer order-sensitive structural variation.

Figure 9: Balanced structural diversity of synthetic agent data. Left: On Retail, EvoSyn achieves the highest trajectory-shape evenness among five domain-matched synthetic corpora under the same data budget, exceeding the strongest competing method by $1.35\times$ and AReaL-SEA by $3.12\times$ . Right: On Telecom, where the compared ground-truth traces have directly comparable decisive-action structure, EvoSyn provides higher factor balance, interaction coverage, trajectory richness, and trajectory evenness than AReaL-SEA. Together, these results show that EvoSyn distributes synthesis mass across a broader and more balanced set of interaction structures rather than concentrating on a small number of dominant trajectory patterns.

Transition	Targeted Slice	Pre-round SR	Capability Gap	$\Delta$ Prop	Post-round Gain	Targeted Improvement
R1 $\rightarrow$ R2	multi_write	0.429	0.571	+0.340	+0.262	✓
R1 $\rightarrow$ R2	high_write	0.333	0.667	+0.145	+0.389	✓
R2 $\rightarrow$ R3	navigation	0.463	0.537	+0.690	+0.148	✓
	multi_read	0.410	0.590	+0.804	+0.205	✓
	read_then_set	0.590	0.410	+0.354	+0.103	✓
R1 $\rightarrow$ R2 aggregate		Allocation–Gain $\rho=0.571$		Targeted Improvement Rate $=2/2$

Appendix

Table 5: Targeted capability evolution on CAR-Bench Base. We report the promoted or directly targeted slices in each transition.

\Delta

Prop measures the change in verified synthetic-data proportion from the current round to the next round, and Gain measures the corresponding held-out success-rate change after training.

Figure 10: Capability-gap-driven synthesis adaptation on CAR-Bench from R2 to R3. As the learner’s residual weakness shifts toward read-intensive decision making, EvoSyn substantially increases the verified-data share of navigation , multi_read , and read_then_set , all of which subsequently improve after training.

Transition	Empirical Feedback Signal	Feedback-Conditioned Discovery	Adoption Evidence	No-Feedback Proposals	Control Outcome
R1 $\rightarrow$ R2	Tasks with 5+ write actions have high failure, indicating that write-action quantity is a missing difficulty coordinate beyond the coarse tool_use_pattern dimension.	Promotes num_write with buckets {0, 1, 2, 3–4, 5+}. The factor is novel, evidence-grounded, actionable, and adopted.	num_write is added to the next synthesis interface. The verified multi_write share increases from 0.232 to 0.571.	temporal_reasoning_depth, entity_resolution_complexity, constraint_interaction.	Misses the adopted write-volume factor; 0/3 proposals recover the adopted write-volume factor.
R2 $\rightarrow$ R3	Tasks with 3–4 read actions have high failure and low coverage, exposing an under-sampled read-heavy decision region.	Promotes num_read with buckets {0, 1, 2, 3–4, 5+}. The factor is novel, evidence-grounded, actionable, and adopted.	num_read is added to the next synthesis interface. The verified multi_read share increases from 0.056 to 0.860.	temporal_reasoning, information_synthesis_depth, target_resolution.	Misses the adopted read-volume factor; 0/3 proposals recover the adopted read-volume factor.
Overall	Feedback-conditioned Reflector: 2 proposed factors, both evidence-grounded and both adopted.		No-feedback control: 6 proposed factors, none grounded in observed failure or coverage evidence, and none matching the adopted write/read-volume factors.		Adopted-factor recovery: 2/2 vs. 0/6.

Appendix

Table 6: Detailed no-feedback control for Reflector breadth on CAR-Bench Base. The feedback-conditioned Reflector uses empirical failure and coverage evidence, while the no-feedback control receives only the ontology, environment description, and tool schema.

Domain	Synthesis LLM	Exact Match	Spearman $\rho$
CAR-Bench	Claude Opus 4.5	99.6%	0.998
Retail	Claude Opus 4.5	100.0%	1.000
Airline	Qwen3-30B	100.0%	1.000
CAR-Bench	Qwen3-235B	99.7%	0.997

Appendix

Table 7: Controllability of the synthesis coordinate across R2 synthesis runs. We compare the value requested in the synthesis plan with the value independently recomputed from the realized ground-truth trajectory.

Figure 11: Capability-gap-driven synthesis reallocation on CAR-Bench from R2 to R3. Each point denotes an interaction slice. The pre-round capability gap ( $1-\mathrm{SR}_{R2}$ ) is strongly aligned with the subsequent change in synthesis allocation (Spearman $\rho=0.964$ , 95% CI $[0.70,1.00]$ , permutation $p=0.003$ ), showing that EvoSyn shifts synthesis effort toward regions where the current learner remains weak.

Domain	Method	R1	R2	R3
CAR-Bench	EvoSyn	270	497	952
	Static	276	398	415
Airline	EvoSyn	28	111	122
	Static	23	27	36

Appendix

Table 8: Structural-support growth across synthesis rounds. We report the cumulative number of distinct ground-truth tool-call sequences covered by the verified synthetic data.

Figure 12: Size-matched structural support at the final synthesis round. The cumulative EvoSyn pool is subsampled to match the number of accepted examples in the Static baseline, and structural support is measured by the number of distinct ground-truth tool-call sequences. After controlling for data volume, EvoSyn still covers $1.32\times$ more interaction sequences on CAR-Bench and $2.19\times$ more on Airline.

Domain	Promoted Factor	$\mathbf{I(Y_{\mathrm{fail}};d_{\mathrm{new}}\mid\mathcal{O}_{r})}$ (nats)
Airline	num_write	0.122
CAR-Bench	num_write	0.168
CAR-Bench	num_read	0.134

Appendix

Table 9: Residual learner-failure information captured by promoted synthesis factors. All statistics are computed using the learner before the corresponding factor is introduced. Conditional mutual information measures the additional failure information provided by the promoted factor after conditioning on the existing synthesis space

\mathcal{O}_{r}

Figure 13: Pre-promotion learner failure rate across num_write buckets on Airline. Tasks requiring more state-changing write actions are substantially more failure-prone before num_write is introduced as an explicit synthesis coordinate, with a positive task-level association between write complexity and learner failure (Spearman $\rho=0.459$ , 95% CI $[0.32,0.59]$ ).

Domain	Promoted Factor $z$	$\mathbf{\mathrm{NMI}(z,g)}$	$\mathbf{I(Y_{\mathrm{fail}};g)}$	$\mathbf{I(Y_{\mathrm{fail}};z\mid g)}$
Airline	num_write	0.216	0.032	0.241
CAR-Bench	num_read	0.420	0.109	0.075

Appendix

Table 10: Promoted synthesis factors capture learner-failure structure beyond generic interaction complexity. We use the total number of tool calls as a domain-agnostic complexity variable

g

. All information quantities are reported in nats.

Transition	$\mathbf{\Delta}$ SR	Added Examples	$\mathbf{\Delta}$ SR / 1K
R1 $\rightarrow$ R2	+0.020	0.97K	+0.021
R2 $\rightarrow$ R3	+0.120	0.78K	+0.153
Final endpoint efficiency relative to the base agent
EvoSyn R3	+0.400	1.996K	+0.200
Static R3	+0.210	1.287K	+0.163

Appendix

Table 11: Observed marginal learning gain on CAR-Bench. Round-wise gain is normalized by the number of newly added verified examples; endpoint efficiency is measured relative to the base Qwen3-8B agent (

\mathrm{SR}=0.360

Setting	Round	Factor Coverage	Factor Balance	Interaction Coverage	Trajectory Pattern Diversity	Success Rate
Static Multi-Round	R1	0.640	0.492	0.339	0.688	0.607
	R2	0.705	0.470	0.343	0.735	0.627
	R3	0.787	0.468	0.351	0.750	0.613
EvoSyn	R1	0.640	0.492	0.339	0.688	0.607
	R2	0.760	0.479	0.351	0.750	0.655
	R3	0.893	0.508	0.373	0.938	0.726

Appendix

Table 12: Intrinsic quality of synthetic data over rounds on CAR-Bench Base. Each row is computed on the cumulative data up to the corresponding round. All intrinsic metrics are normalized to

[0,1]

, where higher is better.

Transition	Feedback Evidence	Reflector Decision	Designer Focus	Observed Synthesis Change
R1 $\rightarrow$ R2	R1 evaluation shows that the hardest cases concentrate in high-write vehicle-control tasks: the 5+ write bucket has failure rate 0.92. The R1 data is skewed toward lower write counts (1 write: 54%; 2 writes: 14%; 3–4 writes: 16%; 5+ writes: 16%). The existing tool_use_pattern dimension captures action-sequence shape, but not the quantity of state-changing writes.	Promote num_write_actions into a new controllable dimension num_write . This separates write-action quantity from the existing tool-pattern dimension and introduces a new axis for execution-heavy tasks.	Increase synthesis budget for high-write regions, especially num_write =3–4 and 5+, under the base-only CAR environment with a weak macro prior.	R2 generates multi-write vehicle-control tasks that the initial interface could not directly target. The round accepts 971 verified tasks, bringing the cumulative verified set to 1212.
R2 $\rightarrow$ R3	After R2, the num_write axis is validated: failure on 5+ write tasks drops from 0.92 to 0.53, and the failure gradient becomes monotonic across write-count buckets. New feedback shows that read-heavy tasks remain both difficult and under-covered: num_read =3–4 has failure rate 0.63 but only 3.9% coverage.	Promote num_write from experimental to standard. Promote num_read into a new controllable axis with buckets 0/1/2/3–4/5+. The new axis is considered orthogonal to num_write , since read actions correspond to information gathering and decision preparation, while write actions correspond to execution.	Shift macro budget toward the under-covered high-failure read region, especially num_read =3–4, while keeping num_write as a standard seed dimension.	R3 produces read-heavy trajectories: the average number of reads increases to 5.02 per task, compared with 0.97 in R2. The round accepts 784 verified tasks and expands the data toward read–decide–act interactions.

Appendix

Table 13: Mechanistic trace of learning-guided synthesis evolution on CAR-Bench Base. Each row reports the concrete feedback evidence used by the Reflector, the resulting ontology decision, the Designer’s budget shift, and the observed synthesis change.

Setting	Round	Factor Coverage	Factor Balance	Interaction Coverage	Trajectory Pattern Diversity	Success Rate
Static Multi-Round	R1	0.742	0.264	0.311	0.462	0.648
	R2	0.760	0.309	0.330	0.538	0.633
	R3	0.778	0.368	0.403	0.538	0.660
EvoSyn	R1	0.742	0.264	0.311	0.462	0.648
	R2	0.785	0.453	0.399	0.692	0.693
	R3	0.891	0.450	0.455	0.769	0.713

Appendix

Table 14: Intrinsic quality of synthetic data over rounds on

\tau^{2}

-Bench Airline. Each row is computed on the cumulative data up to the corresponding round. All metrics are normalized to

[0,1]

, where higher is better.

Transition	Feedback Evidence	Reflector Decision	Designer Focus	Observed Synthesis Change
R1 $\rightarrow$ R2	R1 data is highly skewed: flight_change accounts for 93.6% of verified tasks. Failure probes further show that the 3–4 write-action bucket has failure rate 1.00, indicating that high-write state-changing interactions are difficult but not explicitly controllable in the initial interface.	Promote num_write_actions into a new controllable axis num_write with buckets 0/1/2/3–4/5+. The axis is introduced as a failure-driven experimental dimension because the probe shows strong failure correlation and is generator-realizable.	Increase macro budget for the previously under-covered high-write buckets, especially num_write =3–4 and 5+. Lower write-count buckets remain medium priority.	R2 blueprints shift strongly toward high-write synthesis: the 3–4 and 5+ buckets together account for 57% of realized blueprints. Accepted tasks span all five write-count buckets, showing that num_write becomes an operational synthesis control rather than only a diagnostic label.
R2 $\rightarrow$ R3	After R2, num_write is validated as a meaningful factor: the 5+ write bucket remains difficult with failure rate 0.86, while write-count coverage becomes healthier. A new probe, num_numeric_fields , shows failure rate 1.00 at value 1, revealing difficulty with numerical or derived tool arguments before execution.	Promote num_write from experimental to standard. Then promote num_numeric_fields into a new semantic axis computation , with values none , aggregate , constrained_opt , and conditional .	Tilt macro budget toward all four buckets of the new computation axis. The goal is to systematically explore different forms of computation-before-action behavior while retaining num_write as a standard dimension.	R3 blueprints become approximately balanced across computation types, with each computation bucket covering about one quarter of the realized blueprint distribution. The synthesis focus therefore shifts from only controlling write-action complexity to also covering numerical decision-making and conditional tool execution.

Appendix

Table 15: Mechanistic trace of learning-guided synthesis evolution on

\tau^{2}

-Bench Airline. Each row reports the concrete feedback evidence used by the Reflector, the resulting ontology decision, the Designer’s budget shift, and the observed synthesis change.

Synthetic Data Generation

Synthetic data generation creates artificial examples, simulations, and datasets to train, test, or protect machine-learning systems.

See all 625 papers

SynCo: Data Synthesis Co-Training for Self-Evolving LLMs via Multi-Agent Reinforcement Learning
Oct 8, 2026Wei Yang, Shawn Li, Yuehan Qin +8Self-Evolving Agents Synthetic Data Generation for Language Models
University of Southern California · The George Washington University · Georgia Institute of Technology
ADE: Agentic Data Evolution Framework for Human-Centered Objectives
Aug 24, 2026Yang Yu, Yilin Jiang, Zexuan Fei +6LLM Alignment Synthetic Data Selection
East China Normal University, Shanghai, China · The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China · Shanghai Innovation Institute, Shanghai, China
Make LLM Learn to Synthesize from Streaming Experiences through Feedback
May 28, 2026Zhenlin Hu, Yan Wang, Zhen Bi +7Continual Learning for LLMs Synthetic Data Generation
Huzhou Normal University · Alibaba Group · Zhejiang Key Laboratory of Intelligent Education Technology and Application +1

Oct 8, 2026J/K move · Enter open · S save

SynCo: Data Synthesis Co-Training for Self-Evolving LLMs via Multi-Agent Reinforcement Learning

Wei Yang, Shawn Li, Yuehan Qin, Yawei Wang+7

University of Southern California · The George Washington University · Georgia Institute of Technology

Self-Evolving Agents Synthetic Data Generation for Language Models LLM Reasoning RL for Language Model Reasoning

Open paper AskPDF arXiv

Self-evolving LLM agents promise to improve autonomously through continual interaction and learning, reducing their dependence on manually curated supervision. Realizing this promise requires not only updating the agent, but also evolving its training experience as its capabilities change. However, most existing pipelines rely on static datasets or separately updated synthesis models, causing previously useful tasks to become trivial while overly difficult tasks remain uninformative. This growing mismatch between agent capability and training experience limits sustained self-improvement. To address this problem, we propose SynCo, an agentic data synthesis co-training framework for self-evolving LLMs based on multi-agent reinforcement learning. SynCo jointly optimizes two independently parameterized agents: a Synthesizer that constructs training tasks from the Reasoner's evolving capability state, and a Reasoner that learns from the resulting experience. Each synthesized task induces multiple Reasoner rollouts whose outcomes provide complementary rewards to both agents. Correctness feedback improves the Reasoner, while task quality, answer reliability, and outcome-grounded teachability guide the Synthesizer. Their updates are fed back into subsequent synthesis rounds, allowing the task-solving policy and its training distribution to evolve together. Extensive experiments across eight mathematical reasoning benchmarks demonstrate that SynCo substantially outperforms a broad range of existing synthetic-data methods and controlled baselines, achieving the strongest overall performance while deriving most of its gains from previously unsolved problems.

Abstract

Synthetic Data Generation

Explore similar work

SynCo: Data Synthesis Co-Training for Self-Evolving LLMs via Multi-Agent Reinforcement Learning

ADE: Agentic Data Evolution Framework for Human-Centered Objectives

Make LLM Learn to Synthesize from Streaming Experiences through Feedback