The continued improvement of large language models increasingly depends on synthetic experience, yet effective data synthesis for interactive agents requires not only executable and verifiable trajectories, but also interactions that provide learning value for the current task agent. Existing methods increasingly incorporate environment feedback, data validation, or learner failures, but still face a feedback-to-decision gap: observed weaknesses do not directly specify how future synthesis should change, or whether they can be represented by the current synthesis space. To address this gap, we propose EvoSyn, a learning-feedback-driven multi-agent framework that organizes synthesis around a revisable state specifying what interaction factors can be deliberately generated and where synthesis effort should be allocated. A role-specialized multi-agent system translates learner feedback into synthesis revisions, while a Designer allocates synthesis directions and a Planner grounds them into executable experience. This process enables within-space adaptation toward weak or under-covered regions and synthesis-space expansion that promotes newly discovered factors into explicit generation controls. Experiments on τ2-Bench and CAR-Bench show that EvoSyn improves average task success by 13.3 points over the strongest synthesis baseline, while producing higher-quality and structurally broader synthetic experience and discovering controllable factors missed without learner feedback.
Figures & tables
Figure 1: From static synthesis to closed-loop self-evolving agentic synthesis. Existing paradigms improve data quality or adapt within a fixed synthesis space. In contrast, EvoSyn uses learner feedback to revise both the allocation over known factors and the synthesis space itself by introducing new controllable interaction factors.
Figure 2: Overview of EvoSyn . Each round instantiates a closed-loop synthesis–learning process in which a revisable synthesis state mediates learner diagnosis and generation control, role-specialized agents realize verified experience, and feedback drives within-space adaptation or synthesis-space expansion to co-evolve synthesis with task-agent learning.
Method
Synthesis LLM
τ2 -Bench
CAR-Bench
Avg.
Airline
Retail
Telecom
Frontier Closed-source Models
Claude Opus 4.5
–
79.3
91.8
88.0
82.0
85.3
GPT-5.2
–
83.0
81.6
89.7
74.0
82.1
Gemini-3 Pro
–
80.5
75.9
91.0
67.3
78.7
Open-Source Foundation Models
Table 1: Overall task success rate (%) on agentic benchmarks. For a consistent comparison across synthesis approaches, all synthesis methods use Qwen3-8B as the downstream task agent. EvoSyn -R1 uses first-round data, while EvoSyn -Final-SFT uses accumulated multi-round data.
Method
τ2 -Bench Airline
CAR-Bench Base
Success Rate
Factor Coverage
Factor Balance
Interaction Coverage
Trajectory Diversity
Success Rate
Factor Coverage
Factor Balance
Interaction Coverage
Trajectory Diversity
EvoSyn
0.713
0.891
0.450
0.455
0.769
0.726
0.893
0.508
0.373
0.938
w/o Learning Feedback
0.660
0.778
0.368
0.403
0.538
0.613
0.787
0.468
0.351
0.750
w/o Reflector
0.673
0.862
0.413
0.437
0.615
0.633
0.803
0.483
0.350
0.800
w/o Designer
0.613
0.766
0.387
0.412
0.462
0.540
0.634
0.450
0.339
0.733
Single-Agent Synthesis
0.547
0.365
0.165
0.133
0.077
0.406
0.465
0.425
0.287
0.467
Table 2: Ablation study on τ2 -Bench Airline and CAR-Bench Base. All variants use Opus 4.5 as the synthesis LLM, Qwen3-8B as the task agent, Final-SFT as the training protocol, and the same candidate generation budget. Intrinsic data-quality metrics are computed only on the final verified synthetic samples and normalized to [0,1] , where higher is better.
Figure 3: Closed-loop synthesis-space evolution on CAR-Bench. As learner weaknesses shift across rounds, EvoSyn promotes newly discovered interaction factors into explicit generation controls and redirects synthesis toward the evolving capability boundary, progressing from write-intensive execution to increasingly read-intensive decision making.
Figure 4: Closed-loop evolution of task performance and synthetic-data quality on CAR-Bench. Unlike static accumulation, EvoSyn progressively improves both downstream success and the structural quality of synthesized experience across successive synthesis rounds.
Figure 5: Capability-gap-driven synthesis adaptation on CAR-Bench. The three panels trace the closed-loop transition from pre-round capability gaps to next-round synthesis reallocation and subsequent capability changes, showing how EvoSyn redirects synthesis toward learner weaknesses.
Appendix figures & tables21 assets
Supplementary material from the paper’s appendix.
Appendix
Method
Interactive Setting
Executable & Verifiable
Task-Agent Feedback
Learning-Effect Feedback
Structured Task Space
Known-Factor Adaptation
New-Factor Discovery
Closed-Loop Evolution
Reasoning and instruction synthesis
MATRIX / MATRIX-Gen
–
–
–
–
✓
–
–
–
Socratic-Zero
–
–
✓
–
–
✓
–
✓
R-Zero
–
–
✓
–
–
✓
–
✓
SwS
–
–
✓
–
✓
✓
–
–
Interactive and agentic data synthesis
Appendix
Table 3: Comparison of representative data-synthesis methods for post-training LLMs and agents. ✓ indicates that the capability is explicitly supported and demonstrated by the method, while – indicates that it is absent or not demonstrated. Partial or implicit support is conservatively marked as –.
Method
Synthesis LLM
Task Agent
τ2 -Bench
CAR-Bench
Avg.
Airline
Retail
Telecom
Frontier Closed-source Models
Claude Haiku 4.5
–
–
60.0
75.1
42.4
53.3
57.7
Claude Sonnet 4.5
–
–
83.3
88.3
62.0
71.3
76.2
Claude Opus 4.5
–
–
79.3
91.8
88.0
82.0
85.3
GPT-5.2
–
–
83.0
81.6
89.7
74.0
82.1
Appendix
Table 4: Direct foundation-model performance and task-agent scaling results of EvoSyn . For EvoSyn , Claude Opus 4.5 is used as the synthesis LLM, while Qwen3-8B, Qwen3-14B, and Qwen3-30B serve as downstream task agents. EvoSyn -R1 uses first-round synthesized data, whereas EvoSyn -Final-SFT trains on accumulated multi-round data.
Figure 6: Role of the Designer in task-space exploration on τ2 -Bench Airline. A Planner–Generator pipeline without explicit high-level allocation collapses toward flight_change , whereas introducing the Designer separates global synthesis allocation from local task construction and restores coverage across all six task families.
Figure 7: Structural complementarity of EvoSyn relative to domain-matched synthetic-data baselines on Airline. Left: fraction of each method’s distinct decisive interaction support that is absent from the pooled support of all other methods under matched data budgets. EvoSyn provides the largest unique structural contribution. Right: fraction of EvoSyn ’s structural support that is absent from the union of all matched baseline corpora across synthesis rounds. Iterative synthesis-space evolution progressively moves generation toward interaction structures not represented by existing synthesis pipelines.
Figure 8: Order-sensitive contrastive structure in synthetic data. Left: On Telecom, EvoSyn exhibits substantially higher tool-order contrastive support than the domain-matched AReaL-SEA baseline ( 53.1% vs. 9.6% , 5.5× ). Right: Conceptual illustration of the metric. Repeated use of the same ordering provides little order-level contrast, whereas multiple orderings over the same or overlapping action composition create richer order-sensitive structural variation.
Figure 9: Balanced structural diversity of synthetic agent data. Left: On Retail, EvoSyn achieves the highest trajectory-shape evenness among five domain-matched synthetic corpora under the same data budget, exceeding the strongest competing method by 1.35× and AReaL-SEA by 3.12× . Right: On Telecom, where the compared ground-truth traces have directly comparable decisive-action structure, EvoSyn provides higher factor balance, interaction coverage, trajectory richness, and trajectory evenness than AReaL-SEA. Together, these results show that EvoSyn distributes synthesis mass across a broader and more balanced set of interaction structures rather than concentrating on a small number of dominant trajectory patterns.
Transition
Targeted Slice
Pre-round SR
Capability Gap
Δ Prop
Post-round Gain
Targeted Improvement
R1 → R2
multi_write
0.429
0.571
+0.340
+0.262
✓
high_write
0.333
0.667
+0.145
+0.389
✓
R2 → R3
navigation
0.463
0.537
+0.690
+0.148
✓
multi_read
0.410
0.590
+0.804
+0.205
✓
read_then_set
0.590
0.410
+0.354
+0.103
✓
R1 → R2 aggregate
Allocation–Gain ρ=0.571
Targeted Improvement Rate =2/2
Appendix
Table 5: Targeted capability evolution on CAR-Bench Base. We report the promoted or directly targeted slices in each transition. Δ Prop measures the change in verified synthetic-data proportion from the current round to the next round, and Gain measures the corresponding held-out success-rate change after training.
Figure 10: Capability-gap-driven synthesis adaptation on CAR-Bench from R2 to R3. As the learner’s residual weakness shifts toward read-intensive decision making, EvoSyn substantially increases the verified-data share of navigation , multi_read , and read_then_set , all of which subsequently improve after training.
Transition
Empirical Feedback Signal
Feedback-Conditioned Discovery
Adoption Evidence
No-Feedback Proposals
Control Outcome
R1 → R2
Tasks with 5+ write actions have high failure, indicating that write-action quantity is a missing difficulty coordinate beyond the coarse tool_use_pattern dimension.
Promotes num_write with buckets {0, 1, 2, 3–4, 5+}. The factor is novel, evidence-grounded, actionable, and adopted.
num_write is added to the next synthesis interface. The verified multi_write share increases from 0.232 to 0.571.
Misses the adopted read-volume factor; 0/3 proposals recover the adopted read-volume factor.
Overall
Feedback-conditioned Reflector: 2 proposed factors, both evidence-grounded and both adopted.
No-feedback control: 6 proposed factors, none grounded in observed failure or coverage evidence, and none matching the adopted write/read-volume factors.
Adopted-factor recovery: 2/2 vs. 0/6.
Appendix
Table 6: Detailed no-feedback control for Reflector breadth on CAR-Bench Base. The feedback-conditioned Reflector uses empirical failure and coverage evidence, while the no-feedback control receives only the ontology, environment description, and tool schema.
Domain
Synthesis LLM
Exact Match
Spearman ρ
CAR-Bench
Claude Opus 4.5
99.6%
0.998
Retail
Claude Opus 4.5
100.0%
1.000
Airline
Qwen3-30B
100.0%
1.000
CAR-Bench
Qwen3-235B
99.7%
0.997
Appendix
Table 7: Controllability of the synthesis coordinate across R2 synthesis runs. We compare the value requested in the synthesis plan with the value independently recomputed from the realized ground-truth trajectory.
Figure 11: Capability-gap-driven synthesis reallocation on CAR-Bench from R2 to R3. Each point denotes an interaction slice. The pre-round capability gap ( 1−SRR2 ) is strongly aligned with the subsequent change in synthesis allocation (Spearman ρ=0.964 , 95% CI [0.70,1.00] , permutation p=0.003 ), showing that EvoSyn shifts synthesis effort toward regions where the current learner remains weak.
Domain
Method
R1
R2
R3
CAR-Bench
EvoSyn
270
497
952
Static
276
398
415
Airline
EvoSyn
28
111
122
Static
23
27
36
Appendix
Table 8: Structural-support growth across synthesis rounds. We report the cumulative number of distinct ground-truth tool-call sequences covered by the verified synthetic data.
Figure 12: Size-matched structural support at the final synthesis round. The cumulative EvoSyn pool is subsampled to match the number of accepted examples in the Static baseline, and structural support is measured by the number of distinct ground-truth tool-call sequences. After controlling for data volume, EvoSyn still covers 1.32× more interaction sequences on CAR-Bench and 2.19× more on Airline.
Domain
Promoted Factor
I(Yfail;dnew∣Or) (nats)
Airline
num_write
0.122
CAR-Bench
num_write
0.168
CAR-Bench
num_read
0.134
Appendix
Table 9: Residual learner-failure information captured by promoted synthesis factors. All statistics are computed using the learner before the corresponding factor is introduced. Conditional mutual information measures the additional failure information provided by the promoted factor after conditioning on the existing synthesis space Or .
Figure 13: Pre-promotion learner failure rate across num_write buckets on Airline. Tasks requiring more state-changing write actions are substantially more failure-prone before num_write is introduced as an explicit synthesis coordinate, with a positive task-level association between write complexity and learner failure (Spearman ρ=0.459 , 95% CI [0.32,0.59] ).
Domain
Promoted Factor z
NMI(z,g)
I(Yfail;g)
I(Yfail;z∣g)
Airline
num_write
0.216
0.032
0.241
CAR-Bench
num_read
0.420
0.109
0.075
Appendix
Table 10: Promoted synthesis factors capture learner-failure structure beyond generic interaction complexity. We use the total number of tool calls as a domain-agnostic complexity variable g . All information quantities are reported in nats.
Transition
Δ SR
Added Examples
Δ SR / 1K
R1 → R2
+0.020
0.97K
+0.021
R2 → R3
+0.120
0.78K
+0.153
Final endpoint efficiency relative to the base agent
EvoSyn R3
+0.400
1.996K
+0.200
Static R3
+0.210
1.287K
+0.163
Appendix
Table 11: Observed marginal learning gain on CAR-Bench. Round-wise gain is normalized by the number of newly added verified examples; endpoint efficiency is measured relative to the base Qwen3-8B agent ( SR=0.360 ).
Setting
Round
Factor Coverage
Factor Balance
Interaction Coverage
Trajectory Pattern Diversity
Success Rate
Static Multi-Round
R1
0.640
0.492
0.339
0.688
0.607
R2
0.705
0.470
0.343
0.735
0.627
R3
0.787
0.468
0.351
0.750
0.613
EvoSyn
R1
0.640
0.492
0.339
0.688
0.607
R2
0.760
0.479
0.351
0.750
0.655
R3
0.893
0.508
0.373
0.938
0.726
Appendix
Table 12: Intrinsic quality of synthetic data over rounds on CAR-Bench Base. Each row is computed on the cumulative data up to the corresponding round. All intrinsic metrics are normalized to [0,1] , where higher is better.
Transition
Feedback Evidence
Reflector Decision
Designer Focus
Observed Synthesis Change
R1 → R2
R1 evaluation shows that the hardest cases concentrate in high-write vehicle-control tasks: the 5+ write bucket has failure rate 0.92. The R1 data is skewed toward lower write counts (1 write: 54%; 2 writes: 14%; 3–4 writes: 16%; 5+ writes: 16%). The existing tool_use_pattern dimension captures action-sequence shape, but not the quantity of state-changing writes.
Promote num_write_actions into a new controllable dimension num_write . This separates write-action quantity from the existing tool-pattern dimension and introduces a new axis for execution-heavy tasks.
Increase synthesis budget for high-write regions, especially num_write =3–4 and 5+, under the base-only CAR environment with a weak macro prior.
R2 generates multi-write vehicle-control tasks that the initial interface could not directly target. The round accepts 971 verified tasks, bringing the cumulative verified set to 1212.
R2 → R3
After R2, the num_write axis is validated: failure on 5+ write tasks drops from 0.92 to 0.53, and the failure gradient becomes monotonic across write-count buckets. New feedback shows that read-heavy tasks remain both difficult and under-covered: num_read =3–4 has failure rate 0.63 but only 3.9% coverage.
Promote num_write from experimental to standard. Promote num_read into a new controllable axis with buckets 0/1/2/3–4/5+. The new axis is considered orthogonal to num_write , since read actions correspond to information gathering and decision preparation, while write actions correspond to execution.
Shift macro budget toward the under-covered high-failure read region, especially num_read =3–4, while keeping num_write as a standard seed dimension.
R3 produces read-heavy trajectories: the average number of reads increases to 5.02 per task, compared with 0.97 in R2. The round accepts 784 verified tasks and expands the data toward read–decide–act interactions.
Appendix
Table 13: Mechanistic trace of learning-guided synthesis evolution on CAR-Bench Base. Each row reports the concrete feedback evidence used by the Reflector, the resulting ontology decision, the Designer’s budget shift, and the observed synthesis change.
Setting
Round
Factor Coverage
Factor Balance
Interaction Coverage
Trajectory Pattern Diversity
Success Rate
Static Multi-Round
R1
0.742
0.264
0.311
0.462
0.648
R2
0.760
0.309
0.330
0.538
0.633
R3
0.778
0.368
0.403
0.538
0.660
EvoSyn
R1
0.742
0.264
0.311
0.462
0.648
R2
0.785
0.453
0.399
0.692
0.693
R3
0.891
0.450
0.455
0.769
0.713
Appendix
Table 14: Intrinsic quality of synthetic data over rounds on τ2 -Bench Airline. Each row is computed on the cumulative data up to the corresponding round. All metrics are normalized to [0,1] , where higher is better.
Transition
Feedback Evidence
Reflector Decision
Designer Focus
Observed Synthesis Change
R1 → R2
R1 data is highly skewed: flight_change accounts for 93.6% of verified tasks. Failure probes further show that the 3–4 write-action bucket has failure rate 1.00, indicating that high-write state-changing interactions are difficult but not explicitly controllable in the initial interface.
Promote num_write_actions into a new controllable axis num_write with buckets 0/1/2/3–4/5+. The axis is introduced as a failure-driven experimental dimension because the probe shows strong failure correlation and is generator-realizable.
Increase macro budget for the previously under-covered high-write buckets, especially num_write =3–4 and 5+. Lower write-count buckets remain medium priority.
R2 blueprints shift strongly toward high-write synthesis: the 3–4 and 5+ buckets together account for 57% of realized blueprints. Accepted tasks span all five write-count buckets, showing that num_write becomes an operational synthesis control rather than only a diagnostic label.
R2 → R3
After R2, num_write is validated as a meaningful factor: the 5+ write bucket remains difficult with failure rate 0.86, while write-count coverage becomes healthier. A new probe, num_numeric_fields , shows failure rate 1.00 at value 1, revealing difficulty with numerical or derived tool arguments before execution.
Promote num_write from experimental to standard. Then promote num_numeric_fields into a new semantic axis computation , with values none , aggregate , constrained_opt , and conditional .
Tilt macro budget toward all four buckets of the new computation axis. The goal is to systematically explore different forms of computation-before-action behavior while retaining num_write as a standard dimension.
R3 blueprints become approximately balanced across computation types, with each computation bucket covering about one quarter of the realized blueprint distribution. The synthesis focus therefore shifts from only controlling write-action complexity to also covering numerical decision-making and conditional tool execution.
Appendix
Table 15: Mechanistic trace of learning-guided synthesis evolution on τ2 -Bench Airline. Each row reports the concrete feedback evidence used by the Reflector, the resulting ontology decision, the Designer’s budget shift, and the observed synthesis change.
Self-evolving LLM agents promise to improve autonomously through continual interaction and learning, reducing their dependence on manually curated supervision. Realizing this promise requires not only updating the agent, but also evolving its training experience as its capabilities change. However, most existing pipelines rely on static datasets or separately updated synthesis models, causing previously useful tasks to become trivial while overly difficult tasks remain uninformative. This growing mismatch between agent capability and training experience limits sustained self-improvement. To address this problem, we propose SynCo, an agentic data synthesis co-training framework for self-evolving LLMs based on multi-agent reinforcement learning. SynCo jointly optimizes two independently parameterized agents: a Synthesizer that constructs training tasks from the Reasoner's evolving capability state, and a Reasoner that learns from the resulting experience. Each synthesized task induces multiple Reasoner rollouts whose outcomes provide complementary rewards to both agents. Correctness feedback improves the Reasoner, while task quality, answer reliability, and outcome-grounded teachability guide the Synthesizer. Their updates are fed back into subsequent synthesis rounds, allowing the task-solving policy and its training distribution to evolve together. Extensive experiments across eight mathematical reasoning benchmarks demonstrate that SynCo substantially outperforms a broad range of existing synthetic-data methods and controlled baselines, achieving the strongest overall performance while deriving most of its gains from previously unsolved problems.
Wei Yang, Shawn Li, Yuehan Qin +8
University of Southern California · The George Washington University · Georgia Institute of Technology
Aligning large language models to human-centered objectives is difficult when targets are non-executable and context-dependent, limiting reliable verification and scalable supervision. Although synthetic data expands coverage, weak verification shifts the bottleneck from generation to selection. Noisy signals destabilize iterative refinement and can cause silent regressions. We propose Agentic Data Evolution (ADE), a data-centric framework that organizes synthetic supervision as evolving data snapshots. ADE improves data snapshots through a closed-loop Observation-Variation-Selection (OVS) procedure, where a steady-state admission mechanism acts as a quality ratchet that conservatively gates updates for sustained cross-round improvement. We validate these improvements through complementary intrinsic trend tracking and extrinsic post-training evaluation. On DEV300, ADE raises the intrinsic win rate from 50% to 75.81% and the extrinsic win rate from 55.20% to 68.86%, consistent performance gains across diverse benchmarks. Blind expert evaluation further confirms this, with a 66.11% preference for evolved answers. These gains extend across post-training methods, model scales, and tasks beyond the target weakly verifiable educational objectives. Resources are available at https://github.com/ZeroLoss-Lab/Agentic-Data-Evolution.
Yang Yu, Yilin Jiang, Zexuan Fei +6
East China Normal University, Shanghai, China · The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China · Shanghai Innovation Institute, Shanghai, China
Large language models (LLMs) have been widely adopted for synthetic data generation, significantly reducing annotation costs. However, most existing studies treat synthesis as a set of isolated tasks and overlook a more fundamental question: whether a model can learn to synthesize by accumulating experience from past tasks and transferring it to future ones. In this work, we introduce StreamSynth, a new setting in which synthesis tasks arrive sequentially and experience from historical tasks provides informative signals for future synthesis. To address this setting, we propose SynLearner, a general framework that enables synthesis models to acquire reusable synthesis experience over a task stream. Instead of generating data independently for each task, SynLearner encourages the model to explore diverse synthesis patterns, learn from feedback, and balance sample quality with set-level diversity as tasks evolve. Extensive experiments across multiple benchmarks show that SynLearner effectively leverages experience from earlier tasks to improve synthesis performance on later ones, exhibiting consistent cross-task transferability. These findings provide evidence for the feasibility of StreamSynth and highlight synthetic data generation as an experience-driven process that can benefit from task streams.
Zhenlin Hu, Yan Wang, Zhen Bi +7
Huzhou Normal University · Alibaba Group · Zhejiang Key Laboratory of Intelligent Education Technology and Application +1