While most efforts to improve LLM-based tool-using agents focus on the agent itself - through larger models, better prompting, or fine-tuning - agent performance increasingly plateaus due to the quality of the tool interfaces these agents consume. Tool descriptions are often written for human developers and tolerate ambiguity that agents cannot resolve, particularly as the number of candidate tools grows. Existing approaches to improving tool interfaces (1) require re-running a multi-stage per-tool pipeline - synthesizing queries, executing an agent to collect trajectories, annotating trajectories, and prompting a strong LLM multiple times - for every API that enters the catalog, and (2) typically optimize each tool independently, limiting scalability and generalization to unseen tools. We propose Trace-Free+, a curriculum learning framework that progressively transfers supervision from trace-rich settings to trace-free deployment, encouraging the model to internalize reusable patterns of what makes a tool description effective. To support this approach, we construct a large-scale dataset of high-quality tool interfaces derived from real-world APIs through a principled data synthesis workflow. Experiments on widely adopted benchmarks show that Trace-Free+ improves robustness as tool catalogs scale to 150+ candidates - in scaling experiments, reducing accuracy degradation by 29.23% and improving average query-level success by 60.89% on StableToolBench - generalizes across domains without retraining, and provides complementary gains on top of agent fine-tuning.
Figures & tables
Figure 1: An illustration of the proposed tool interface improvement pipeline. Compared to the original description ( D0 ), the learned description generator produces more effective tool descriptions that lead to better tool usage.
Figure 2: The data synthesis pipeline.
Multi-Step (G2+G3)
Single-Step (G1)
G2 Category
G2 Instruction
G3 Instruction
Avg
G1 Category
G1 Instruction
G1 Tool
Avg
SL
QL
SL
QL
SL
QL
SL
QL
SL
QL
SL
QL
SL
QL
SL
QL
Trace-Free+
68.7 ± 0.7
50.8 ± 0.0
71.4 ± 1.1
47.0 ± 0.1
63.9 ± 1.8
36.1 ± 0.1
68.0
44.6
73.8 ± 0.7
65.6 ± 1.0
72.6 ± 1.5
60.0 ± 1.0
70.0 ± 1.4
56.4 ± 1.6
72.1
60.7
Trace-Free
66.4 ± 1.2
44.1 ± 0.0
69.2 ± 4.0
46.4 ± 3.6
59.9 ± 1.2
41.8 ± 1.2
65.2
44.1
71.8 ± 2.3
62.7 ± 2.1
70.8 ± 0.1
60.8 ± 0.9
68.5 ± 2.7
53.6 ± 2.8
70.4
59.0
D1
67.9 ± 1.2
48.5 ± 0.1
67.4 ± 1.0
45.5 ± 0.1
45.2 ± 4.2
30.6 ± 0.1
60.2
41.5
75.5 ± 1.3
64.9 ± 1.3
74.7 ± 0.7
66.1 ± 0.6
68.0 ± 0.6
53.7 ± 0.5
72.7
61.6
D0
68.4 ± 0.3
39.0 ± 0.0
67.8 ± 0.8
43.9 ± 0.1
50.7 ± 0.2
17.6 ± 0.1
62.3
33.5
73.0 ± 0.3
62.4 ± 1.5
72.8 ± 0.4
62.3 ± 0.9
71.0 ± 0.9
52.8 ± 1.1
72.3
59.2
Table 1: Trace-free evaluation on StableToolBench for multi-step (G2+G3) and single-step (G1) subsets. SL/QL: subtask/query-level success rate. GPT-4.1 is the tool-using agent.
RestBench
TMDB
Spotify
Method
SL
QL
SL
QL
Trace-Free+
88.1 ± 0.4
74.9 ± 0.5
68.1 ± 0.3
49.3 ± 0.6
Trace-Free
78.4 ± 1.1
57.7 ± 1.2
65.0 ± 1.6
44.7 ± 2.8
D1
78.2 ± 0.1
58.0 ± 0.8
65.1 ± 0.8
45.7 ± 1.8
D0
69.8 ± 0.2
49.5 ± 0.2
57.1 ± 2.9
34.9 ± 2.1
Table 2: Trace-free evaluation results on RestBench – TMDB and Spotify. SL: subtask-level, QL: query-level. No tool execution traces are available at inference time.
Figure 3: Scaling experiment results on the more challenging G2-G3 subsets of StableToolBench. We report query-level (QL) results.
Setting
StableToolBench
RestBench
Base + D0
37.4
54.1
Base + Trace-Free+
40.1 (+ 10.7%)
62.1 (+ 14.8%)
Fine-tuned + D0
40.0 (+ 10.7%)
55.0 (+ 1.0%)
Fine-tuned + Trace-Free+
41.8 (+ 11.7%)
62.9 (+ 16.3%)
Table 3: Query-level success rate with base and fine-tuned Qwen3-4B-Instruct agents.
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
Dataset
Queries
Tools
Tools per Query
RestBench
TMDB
100
54
54
Spotify
57
40
40
StableToolBench
G1 Category
153
364
4.21
G1 Instruction
163
820
5.29
Appendix
Table 4: Descriptive statistics of evaluation benchmarks. For StableToolBench, we only consider their solvable queries ( Guo et al., 2024 ) .
Dataset
Queries
Tools
Tools per Query
Synthesized SFT Data
Ats
4,726
4,585
6.81
Atr
2,189
991
9.21
Appendix
Table 5: Statistics of the dataset we created in Sec 3.2 for tool interface improvement. To ensure evaluation on entirely unseen tools, all tools appearing as candidates in StableToolBench test queries are assigned to the test set Ats and the remainder form Atr .
Parameter
SFT
Base Model
Qwen3-4B-Instruct-2507
Hardware
1 × 8-A100 GPU Node
Optimizer
AdamW
Learning Rate
5.0×10−5
LR Scheduler
Cosine
Training Epochs
2.0
Appendix
Table 6: Hyperparameters for SFT.
#
Category
Level / Key Test
Explanation
Example
1
Tool selection scope
Tool-level. Does it help an agent decide between multiple options?
Explicitly states when to use this tool vs. another, when NOT to use it, or compares it to a similar tool. Must go beyond a simple purpose statement.
“Use this API when you need to retrieve a list of artworks by search query. Do not use it if you require detailed metadata such as dimensions or provenance. For more detail, use Detect Features instead.”
2
Cross-tool dependencies
Tool-level. Does it name a specific upstream endpoint?
A parameter value must come from calling a specific named tool or endpoint first. The description explicitly names the other tool/endpoint — not just general prior state.
“The hash parameter must be extracted from the Check Status API response when movie_status equals DONE, and vsid must come directly from the Start Movie Session API response.”
3
Output description
Tool-level. Does it enumerate response fields or explicitly state what is NOT returned?
Explicitly enumerates response fields, data types, or structure — or explicitly states what is not included. Does not apply to general purpose statements that merely describe what the tool retrieves.
“Returns all static metadata including logo, description, official website URL, and social links. Does not include videos, subcategories, or category-specific details.”
4
Parameter constraints
Parameter-level. Does it restrict valid values or structure?
Restricts what values a parameter can take or specifies how it must be structured: enums, numeric ranges, case sensitivity rules, date formats, encoding schemes, separator conventions, or array structure.
“Must be one of en (English) or nl (Dutch). limit must be a positive integer between 1 and 100. date must be in YYYY-MM-DD format. image must be a Base64-encoded string.”
5
Cross-param dependencies
Parameter-level. Does it constrain one parameter based on another?
Constraints between parameters within the same tool — parameters that must be paired together or are mutually exclusive.
“ longitude (requires lat ).”
Appendix
Table 7: Definitions of the five interface pattern categories used to classify tool description improvements. Categories 1–3 are tool-level (apply to the description as a whole); Categories 4–5 are parameter-level (apply per parameter).
#
Category
D0
D1
D2
Trace-Free+
1
Tool selection scope
3.3%
99.7%
100.0%
97.2%
2
Cross-tool dependencies
0.3%
32.0%
21.1%
30.0%
3
Output description
11.6%
98.6%
100.0%
27.0%
4
Parameter constraints
9.9%
87.9%
89.5%
94.2%
5
Cross-param dependencies
0.3%
8.5%
21.1%
17.1%
Appendix
Table 8: Percentage of tool descriptions covering each of the five interface pattern categories, comparing D0 , D1 , D2 , and Trace-Free+. D0 descriptions are consistently sparse across all categories; all three improved variants close most of the gap on scope and parameter constraints, with differences concentrated on cross-tool dependencies and output description.
D0 (Original)
Trace-Free+ (Ours)
API
Walk Score (provider: Walk Score )
Parameters
lat (str), lon (str), address (str), wsapikey (str), bike (str, opt.), transit (str, opt.), format (str, opt.)
Description
Get Walk Score. address : “The URL encoded address.” lat / lon : no range stated. format : “Type of result to return: (movie, series, episode).”
Retrieves Walk, Transit, and Bike scores for a location. lat must be a decimal in [−90,90] ; lon in [−180,180] . address must not be URL-encoded. bike and transit must be exactly ’1’ — not ’true’ or ’on’ . format must be ’json’ or empty (XML default); any other value is rejected. Do not use for real-time traffic data or historical trends.
Patterns
—
Parameter constraints ( lat / lon numeric ranges; exact-value flags for bike , transit , format ; format correction for address ); Tool selection scope (exclusions: not for real-time traffic or historical trends)
Table 9: Three examples of Trace-Free+-generated descriptions for unseen tools compared against D0 . In all cases the model infers constraints without execution traces.
StableToolBench
G1 Category
G1 Instruction
G1 Tool
G2 Category
G2 Instruction
G3 Instruction
Ratios
SL
QL
SL
QL
SL
QL
SL
QL
SL
QL
SL
QL
(0.1, 0.9)
73.8 ± 0.7
65.6 ± 1.0
72.6 ± 1.5
60.0 ± 1.0
70.0 ± 1.4
56.4 ± 1.6
68.7 ± 0.7
47.5 ± 0.8
71.4 ± 1.1
48.3 ± 1.6
63.9 ± 1.8
46.4 ± 4.1
(0.3, 0.7)
73.7 ± 1.9
63.6 ± 2.8
73.2 ± 2.3
61.3 ± 4.3
68.3 ± 1.1
52.8 ± 0.7
68.6 ± 0.0
47.9 ± 0.6
69.1 ± 2.9
45.4 ± 3.6
61.1 ± 3.0
41.8 ± 3.5
(0.5, 0.5)
75.8 ± 0.0
66.0 ± 0.0
71.1 ± 0.0
64.3 ± 0.0
69.2 ± 0.0
55.6 ± 0.0
65.9 ± 0.0
48.3 ± 0.0
68.9 ± 0.0
44.9 ± 0.0
59.8 ± 0.0
39.3 ± 0.0
Trace-Free
71.8 ± 2.3
62.7 ± 2.1
70.8 ± 0.1
60.8 ± 0.9
68.5 ± 2.7
53.6 ± 2.8
66.4 ± 1.2
44.1 ± 0.0
69.2 ± 4.0
46.4 ± 3.6
59.9 ± 1.2
41.8 ± 1.2
Appendix
Table 10: Parameter Study for Trace-Free+ in Trace-free Evaluation
Non-Live
Live
Model
D0
+ Trace-Free+
D0
+ Trace-Free+
GPT-4.1
88.56
89.04
79.28
80.63
Claude Sonnet 4.5
63.11
65.50
52.40
53.45
Gemini-3-pro-preview
91.24
91.55
84.98
86.41
Appendix
Table 11: Transfer evaluation on BFCLv2. We apply Trace-Free+ to improve tool descriptions and evaluate on the Non-Live and Live splits.
Large language model (LLM) agents must generalize from tools seen during training to unseen tools at deployment. A key challenge is tool trialing, i.e., excessive trials waste the interaction budget, whereas selective trials enable exploration of unfamiliar tools. Existing outcome-based post-training leaves wasteful trials unguided, while turn-level supervision may suppress necessary exploration. We introduce ToolCompass, a post-training framework that guides tool trialing by organizing tool-call representations according to shared functions. Specifically, ToolCompass models each function class as a von Mises--Fisher distribution and jointly reduces intra-function variation across domains and increases inter-function separation. This structure transfers experience from seen tools to functionally similar unseen tools, directing exploration away from unrelated alternatives. ToolCompass requires no ground-truth call traces or unseen-tool access and incurs no inference overhead. Experiments on AppWorld and FTRL show consistent gains across GRPO, RFT, and DMPO. improves AppWorld OOD task success by up to 10.71 percentage points over vanilla post-training and performs best among competitive baselines on both benchmarks.
Junlin Fang, Chong Zhang, Do Nguyen-Thanh +3
College of Computing and Data Science, Nanyang Technological University · Zhejiang University · Australian Artificial Intelligence Institute, University of Technology Sydney +1
Tool-augmented language-model agents rely on interface text: tool descriptions, argument schemas, and the candidate tool list shown at inference time. These interfaces often change after deployment even when the underlying tool semantics remain fixed, creating a practical reliability problem for function-calling systems. We study this failure mode as interface drift and evaluate whether a lightweight, training-free repair layer can recover robustness without retraining the base model. Our method, SchemaShield-Lite, combines schema validation, candidate-set retry for missing tool names, and one-shot constrained repair. On Qwen3.5-9B, it raises DICE-300 accuracy from 0.717 to 0.813 and BFCL-200 accuracy from 0.580 to 0.805, recovering 19/22 and 33/45 drift-induced failures with no observed repair harms. Naive retry does not explain the gains. Ablations show that candidate-set drift, especially distractor tools, accounts for most observed damage, while description and schema edits alone are less harmful. The results position validation-and-repair as a practical robustness layer for deployed LLM-agent tool use.
Aniruddha Chattopadhyay, Kanishka Halder, Partha Sarathi Roy
Large language model (LLM) agents augmented with external tools often struggle as number of tools grow large and become domain-specific. In such settings, ambiguous tool descriptions and under-specified agent instructions frequently lead to tool mis-selection and incorrect slot/value instantiation. We hypothesize that this is due to two root causes: generic, one-size-fits-all prompts that ignore tool-specific nuances, and underspecified tool schemas that lack clear guidance on when and how to use each tool and how to format its parameters. We introduce Joint Tool-Prompt Reflective Optimization (JTPRO), a framework for improving tool-calling reliability in trace-supervised settings by iteratively using rollout-driven reflection to co-optimize global instructions and per-tool schema/argument descriptions for accurate tool selection and argument instantiation in large tool inventories. JTPRO is designed to preserve only tool-local cues needed for correct disambiguation and slot filling. We evaluate JTPRO across multi-tool benchmarks, which account for different number of tools using three metrics: Tool Selection Accuracy (TSA), Slot Filling Accuracy(SFA), and Overall Success Rate(OSR) (correct tool + correct slots + correct values). JTPRO consistently outperforms strong baselines, including CoT-style agents, and reflective prompt optimizers such as GEPA by 5%-20% (relative) on OSR. Ablations show that joint optimization of instructions and tool schemas is more effective and robust than optimizing either component in isolation.