TopoCurate:Modeling Interaction Topology for Tool-Use Agent Training

Jinluan Yang; Yuxin Liu; Zhengyu Chen; Chengcheng Han; Yueqing Sun; Qi Gu; Hui Su; Xunliang Cai

cs.LGMar 2, 2026

TopoCurate:Modeling Interaction Topology for Tool-Use Agent Training

Authors: Jinluan Yang, Yuxin Liu, Zhengyu Chen, Chengcheng Han, Yueqing Sun, Qi Gu, Hui Su, Xunliang Cai,

Organizations: Zhejiang University, Hangzhou, China · Meituan, Beijing, China.

Abstract

Training tool-use agents typically relies on outcome-based filtering: Supervised Fine-Tuning (SFT) on successful trajectories and Reinforcement Learning (RL) on pass-rate-selected tasks. However, this paradigm ignores interaction dynamics: successful trajectories may lack error recovery or exhibit redundancy, while pass rates fail to distinguish structurally informative tasks from trivial ones. We propose TopoCurate, an interaction-aware framework that projects multi-trial rollouts from the same task into a unified semantic quotient topology. By merging equivalent action-observation states, this projection transforms scattered linear trajectories into a structured manifold that explicitly captures how tool invocations and environmental responses drive the divergence between effective strategies and failure modes. Leveraging this representation, we introduce a dual-selection mechanism: for SFT, we prioritize trajectories demonstrating reflective recovery, semantic efficiency, and strategic diversity to mitigate covariate shift and mode collapse; for RL, we select tasks with high error branch ratios and strategic heterogeneity, maximizing gradient Signal-to-Noise Ratio to address vanishing signals in sparse-reward settings. Evaluations on BFCLv3 and Tau2 Bench show that TopoCurate achieves consistent gains of 4.2% (SFT) and 6.9% (RL) over state-of-the-art baselines. We will release the code and data soon for further investigations.

Figures & tables

Figure 1: Overview of the TopoCurate Framework. Our method operates in three stages: (Left) Topological Modeling transforms disjoint rollouts into a unified state-transition graph by defining states via action-observation tuples and aggregating semantically equivalent turns; (Middle) Trajectory Selection for SFT applies three process-aware metrics—Reflective Recovery (resilience), Semantic Efficiency (economy), and Strategic Diversity (exploration)—to prioritize high-quality trajectories; (Right) Task Selection for RL evaluates tasks using Error Branch Ratio and Strategy Heterogeneity to select structurally complex tasks that maximize gradient efficiency.

Method	Trajectory Selection (SFT)	Task Selection (RL)
APIGEN-MT Prabhakar et al. (2025)	Outcome ( $r=1$ )	N/A
MUA Zhao et al. (2025)	Outcome ( $r=1$ )	Pass Rate
Simia-Tau Li et al. (2025b)	Outcome ( $r=1$ )	N/A
TopoCurate (Ours)	Outcome ( $r=1$ ) $\downarrow$ Topological $S_{ref},S_{eff},S_{rare}$	Pass Rate $\downarrow$ Topological $V_{struct},V_{div}$

Table 1: Baseline Comparison. All methods start with outcome-based filtering; TopoCurate further refines via topological metrics.

Model	Tau2 Bench (IID)			BFCL V3 Multiturn (OOD)
Model	Airline	Retail	Telecom	Base	Long Context	Miss Func	Miss Param
32B Size Models
Qwen3-32B-Thinking	0.435	0.525	0.248	0.340	0.210	0.155	0.245
APIGen-MT-SFT	0.347	0.592	0.202	0.375	0.215	0.175	0.205
MUA-SFT	0.311	0.582	0.193	0.350	0.195	0.210	0.285
MUA-RL	0.454	0.673	0.283	0.420	0.215	0.200	0.300

Table 2: Performance comparison on Tau2 Bench (IID) and BFCL v3 Multiturn (OOD). Models are trained on Tau2 environments. Gray rows denote TopoCurate variants. Bold indicates best results.

Panel of Figure 2 — Figure 2: Pass@k Comparison across three settings : the Qwen3-Instruct backbone (Thinking mode), a baseline SFT model trained on standard outcome-filtered data (w/o Topology), and our TopoCurate-SFT. The results demonstrate that TopoCurate consistently achieve higher pass@k across all domains and model scales.

Method	Components			Tau2 Pass@1
Method	Ref	Div	Eff	Airline	Retail	Telecom	Avg
TopoCurate-SFT (w/o Topology)	$\times$	$\times$	$\times$	0.436	0.500	0.418	0.451
w/o Reflection	$\times$	$\checkmark$	$\checkmark$	0.436	0.510	0.422	0.456
w/o Diversity	$\checkmark$	$\times$	$\checkmark$	0.393	0.472	0.428	0.431
w/o Efficiency	$\checkmark$	$\checkmark$	$\times$	0.401	0.468	0.435	0.435
TopoCurate-SFT (Ours)	$\checkmark$	$\checkmark$	$\checkmark$	0.445	0.529	0.435	0.470

Table 3: Ablation Study (SFT). Impact of topological metrics on Pass@1 accuracy (Qwen3-8B-Thinking as backbone). “Ref", “Div", and “Eff" denote Reflection, Diversity, and Efficiency.

Method	Components		Tau2 Pass@1
Method	Struct	Div	Airline	Retail	Telecom	Avg
TopoCurate-RL (w/o Topology)	$\times$	$\times$	0.531	0.643	0.721	0.632
w/o Structure	$\times$	$\checkmark$	0.542	0.661	0.748	0.650
w/o Diversity	$\checkmark$	$\times$	0.547	0.668	0.756	0.657
TopoCurate-RL (Ours)	$\checkmark$	$\checkmark$	0.558	0.687	0.784	0.676

Table 4: Ablation Study (RL). Impact of topological metrics on Pass@1 accuracy (Qwen3-32B-Thinking as backbone). “Struct" and “Div" denote Error Branch Ratio and Strategic Heterogeneity.

Appendix figures & tables13 assets

Supplementary material from the paper’s appendix.

Appendix

Model	Params	Dim	Context	Tier Agreement	Spearman $\rho$ $\dagger$
Jina-v2-base-en	137M	768	8,192	—	—
Owen3-Embedding-0.6B	600M	1,024	32,768	87.8%	0.945

Appendix

Table 5: Embedding Model Selection. Comparison of different embedding models used in the Tier construction phase.

\dagger

represents the correlation coefficient between the selected trajectories and the ground truth ranking.

Weight Settings $(\alpha,\beta,\gamma)$	Airline	Retail	Telecom	Avg
$(0.4,0.3,0.3)$	0.445	0.529	0.435	0.470
$(0.4,0.2,0.4)$	0.413	0.549	0.420	0.461
$(0.3,0.4,0.4)$	0.415	0.517	0.424	0.452

Appendix

Table 6: Sensitivity Analysis of Trajectory Selection Metric Weights. The weight settings are denoted as

(\alpha,\beta,\gamma)

, corresponding to the weights for Efficiency, Diversity, and Reflection metrics, respectively.

Panel of Figure 3 — Figure 3: Model Behavior Analysis. We compare the behavioral patterns of TopoCurate-SFT (Ours) vs. the baseline SFT (w/o Topology). (a) Ours demonstrates significantly higher reflective recovery rates, particularly in the complex Telecom domain. (b) In terms of the number of interaction turns (lower is better), Ours achieves higher efficiency overall, while maintaining success rate across all domains. (c) Ours consistently generates more diverse tool chains (Unique Chain Ratio), mitigating mode collapse.

Figure 4: Evaluation Accuracy on Tau2 for RL. (Left) Impact of SFT initialization quality on downstream RL performance across Airline, Retail, and Telecom domains. Models initialized with topological SFT exhibit consistently higher accuracy. (Right) Impact of RL task selection on final performance. Topologically-selected tasks lead to superior convergence across all domains, with the most pronounced gains in Telecom.

Figure 5: Training Reward Curves for RL. (Left) Comparison of models initialized with topologically-curated SFT (w/ Topology) vs. outcome-filtered SFT (w/o Topology), using the same RL task pool. (Right) Comparison of models trained on topologically-selected RL tasks (w/ Topology) vs. uniformly-sampled tasks (w/o Topology) from the same SFT checkpoint. Topological curation improves both initialization quality and task selection efficiency.

Method	NON_LIVE	LIVE	MULTI_TURN	AGENTIC	HALLUCINATION
Qwen3-32B-Thinking (Base)	0.475	0.700	0.215	0.197	0.724
TopoCurate-SFT (w/o Topology)	0.531	0.799	0.253	0.201	0.703
TopoCurate-SFT	0.550	0.790	0.261	0.227	0.713
TopoCurate-RL (w/o Topology)	0.539	0.798	0.301	0.255	0.783
TopoCurate-RL	0.562	0.805	0.325	0.261	0.803

Appendix

Table 7: BFCL v4 Results on Qwen3-32B. Evaluation across multiple sub-tasks. The best result in each column is highlighted in bold.

Domain	Avg Rollouts/Task	Raw Turns (Mean)	Merged Nodes (Mean)	Compression Ratio
Airline	16	392	87	4.5 $\times$
Retail	16	428	102	4.2 $\times$
Telecom	16	526	138	3.8 $\times$

Appendix

Table 8: Topology construction statistics across Tau2 domains. We report the average number of rollouts per task, raw turns before merging, merged nodes after quotient construction, and the resulting compression ratio.

Method (RL task selection)	Airline	Retail	Telecom	Avg
w/o Topology (uniform / pass-rate)	0.531	0.643	0.721	0.632
DUMP (advantage-UCB curriculum)	0.540	0.656	0.744	0.647
TopoCurate-RL (Ours)	0.558	0.687	0.784	0.676

Appendix

Table 9: Comparison with curriculum RL / DUMP on Qwen3-32B (Tau2 Pass@1). The best result in each column is highlighted in bold.

Selection Ratio	Outcome-based	TopoCurate-SFT	Relative Gain
5%	0.312	0.389	+24.7%
10%	0.368	0.421	+14.4%
20%	0.405	0.448	+10.6%

Appendix

Table 10: Robustness of TopoCurate under low selection ratios on Qwen3-8B. The best result in each row is highlighted in bold.

	Telecom	Retail	Airline
Construction_Topology	38.3 min	33.3 min	40.1 min
Outcome_Reward_Filtering	300.5 min	233.6 min	227.2 min

Appendix

Table 11: Time consumption comparison for topology construction and filtering on three domains.

LLM Agent Training

Large language model agent training teaches models to plan, use tools, and learn from multi-step interaction with environments.

See all 585 papers

Trajectory2Gym: Decision-Grounded Compositional Interaction Synthesis for Tool-Use Agents
Oct 24, 2026Yihao Hu, Zhihao Wen, Yitong Lin +5Tool-Using Agents LLM Agent Trajectory Synthesis
Westlake University (westlake.edu) · Ant International (ant-intl.com)
CoVe: Training Interactive Tool-Use Agents via Constraint-Guided Verification
Mar 2, 2026Jinpeng Chen, Cheng Gong, Hanbo Li +9Tool-Using Agents LLM Agent Training
Huawei Research · Independent Researcher
PACT: Privileged Trace Co-Training for Multi-Turn Tool-Use Agents
Jun 15, 2026Zhenbang Du, Jun Luo, Zhiwei Zheng +8Learning from Demonstration LLM Agent Training
Georgia Institute of Technology · Ohio State University · University of Pennsylvania +1

Oct 24, 2026J/K move · Enter open · S save

Trajectory2Gym: Decision-Grounded Compositional Interaction Synthesis for Tool-Use Agents

Yihao Hu, Zhihao Wen, Yitong Lin, Siyuan Lu+4

Westlake University (westlake.edu) · Ant International (ant-intl.com)

Tool-Using Agents LLM Agent Trajectory Synthesis Supervised Fine-Tuning

Open paper AskPDF arXiv

Successful tool-use trajectories are not necessarily useful supervision. Even successful interactions may underrepresent multi-step dependencies or contain actions that are unjustified by the information visible to the agent at decision time. We identify two coupled requirements for effective trajectory supervision: compositional coverage, so that important observation-action dependencies are represented, and decision grounding, so that each supervised action is supported by the learner-visible interaction history. We introduce Trajectory2Gym, which constructs executable interactions around under-supported compositional structures and derives supervision only from grounded decisions in their realized executions. Structural-support selection allocates the resulting experience, while parent-normalized supervised fine-tuning preserves the semantic interaction as the unit of training weight. This reframes trajectory generation as the deliberate construction of learnable interaction experience, rather than the collection of successful trajectories. Across five Qwen3 scales, Trajectory2Gym improves Tau1, Tau2, and BFCL at every tested scale, increasing the equal-weight three-benchmark mean by 16.9-29.8 percentage points. The gains are strongest in stateful and multi-turn interaction, where later decisions depend on earlier observations and feedback. Our results suggest that scaling tool-use agents depends not only on collecting more successful trajectories, but on constructing interactions whose dependencies are represented and whose decisions are learnable from the agent's visible history.

Abstract

LLM Agent Training

Explore similar work

Trajectory2Gym: Decision-Grounded Compositional Interaction Synthesis for Tool-Use Agents

CoVe: Training Interactive Tool-Use Agents via Constraint-Guided Verification

PACT: Privileged Trace Co-Training for Multi-Turn Tool-Use Agents