Organizations: Department of Computer Science, Brown University, Providence, RI 02912, USA · Department of Computer Science, University of Southern California, Los Angeles, CA 0089, USA
Large language model agents increasingly retrieve tools from large catalogs, yet most retrievers rank each tool independently by request similarity. For multi tool tasks, this recovers the goal matching tool while omitting prerequisite tools required by the workflow. We introduce LIBRA, a framework for Library Induction by Runtime Assembly that turns a flat tool catalog into an executable tool library. An LLM proposes per tool contracts of candidate invocations, reusable outputs and state changes, and success conditions; LIBRA then tests these compositions in a sandbox and retains the relations supported at runtime. Constructed before user queries, the library requires no dependency annotations, tool use trajectories, or task specific training, and supports both one-shot and agentic retrieval. On AppWorld, BFCL, and SciToolEval, LIBRA improves one-shot Recall and NDCG over the same dense retriever by 7.9 and 3.7 percentage points on average. Within ToolReAGt, it yields corresponding gains of 4.5 and 2.9 points after ten retrieval iterations.
Figures & tables
Method
Trajectory free
Training free
Execution validation
ToolNet ( Liu et al., 2024 )
✗
✓
✗
TGR ( Gao et al., 2025 )
✓
✗
✗
GTool ( Chen et al., 2026 )
✗
✗
✗
Ego-Trip ( Bansal et al., 2025 )
✓
✓
✗
LIBRA
✓
✓
✓
Table 1: Comparison of methods for learning tool dependencies. Prior methods either depend on existing tool use trajectories or training, or retain metadata inferred relations without runtime validation. LIBRA performs trajectory free, training free dependency induction and validates candidate tool compositions through execution in the target environment.
Figure 1: Overview of LIBRA . Stage 1: Before user queries arrive, an LLM proposes an executable contract of each tool. LIBRA tests each candidate composition in the target runtime. Successful compositions are retained to construct the executable tool library, while unsuccessful ones are removed. For example, although card and trans_id share the same integer type, a payment card identifier cannot be used as a transaction identifier. Stage 2: Given the request “Find Alice in my contact and I will make the transfer,” dense retrieval returns Venmo.Search_users . LIBRA performs one-hop propagation and recovers its prerequisite tool Venmo.Login .
AppWorld@15
SciToolEval@10
BFCL@15
Method
P
R
NDCG
P
R
NDCG
P
R
NDCG
BM25 ( Robertson and Zaragoza, 2009 )
10.9
20.0
18.8
12.0
86.4
54.1
26.2
75.5
69.5
QTS
15.5
28.7
32.8
11.3
84.1
83.9
28.6
81.8
76.2
ToolREX ( Lu et al., 2026 )
14.5
26.8
30.7
11.1
83.1
83.8
27.1
77.7
71.9
TGR-Adapt Gao et al. (2025)
19.7
36.3
39.9
10.2
76.2
75.1
28.5
81.4
73.6
Ego-Trip ( Bansal et al., 2025 )
15.8
29.2
33.2
11.3
84.1
83.9
30.4
85.7
78.4
Table 2: One-shot retrieval results without query time reranking. Dataset@ K denotes the evaluation cutoff. Values are percentages. The best result in each column is highlighted, and the second best is underlined.
Figure 2: Agentic retrieval after 10 iterations. Each panel reports Recall and NDCG for one agent LLM and benchmark. Points show the mean over three random seeds, and error bars show standard error. LIBRA attains the highest Recall in 8 of the 9 settings and the highest NDCG on SciToolEval and BFCL for all three agent LLMs.
Figure 3: We projected tool embeddings separately for each benchmark using t-SNE, with background colors indicating different clusters. In each example, Dense retrieval ranks the consumer (blue) within the top K but misses its provider (orange): (a) phone.update_alarm (#1) depends on phone.login (#28); (b) tail (#3) depends on touch (#16); and (c) GetNPR2 (#1) depends on SELFIEStoSMILES (#169). LIBRA follows the retained relation and recovers the provider. For SciToolEval and AppWorld, we observed runtime dependencies connecting tools across different clusters.
Benchmark
Dense misses ( n )
No graph edge (%)
Consumer not seeded (%)
Provider not selected (%)
Below cutoff (%)
Recovered (%)
AppWorld
860
65.1
13.0
3.4
0.2
18.3
BFCL
224
2.7
30.4
12.1
21.4
33.5
SciToolEval
85
38.8
0.0
2.4
0.0
58.8
Table 3: Recovery outcomes for query specific gold tool occurrences outside the top K under Dense retrieval. Each occurrence is assigned to exactly one outcome; percentages sum to 100 within each benchmark.
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
QTS
Schema-Only
LIBRA
AppWorld@15 R / NDCG
28.7 / 32.8
29.2 / 33.1
40.7 / 39.1
SciToolEval@10 R / NDCG
84.1 / 83.9
84.1 / 83.9
91.1 / 86.6
BFCL@15 R / NDCG
81.8 / 76.2
81.4 / 76.0
86.4 / 78.4
Appendix
Table 4: Ablation of graph quality with fixed dense retrieval. All variants use the same encoder over original tool metadata, differing only in the retrieval graph.
AppWorld
SciToolEval
BFCL
Catalog tools
473
477
128
Candidate compositions
581,690
163,608
12,762
Retained relations (value based / state)
1,946 / 0
732 / 0
595 / 77
Wall clock (min:s)
260:40
20:45
4:28
Seconds per 1,000 candidates
26.9
7.6
21.0
Appendix
Table 5: Library construction statistics. Candidate compositions are schema compatible combinations of provider exports and consumer inputs before runtime filtering. A single tool pair may contribute multiple candidate compositions. Relation counts report value based relations and state dependencies included in the benchmark specific retrieval graph. Normalized time is calculated from the recorded wall clock time and candidate composition count.
Setting
Variant
AppWorld@15
SciToolEval@10
BFCL@15
Provider selection
Field-best only
40.3 / 38.9
92.0 / 87.0
86.0 / 78.2
Dense-best only
39.7 / 38.5
88.0 / 85.3
84.8 / 77.7
Both (default)
40.7 / 39.1
91.1 / 86.6
86.4 / 78.4
All retained
38.6 / 38.5
76.4 / 75.0
85.3 / 78.0
Seed Ks
10
40.8 / 39.6
91.9 / 86.9
86.2 / 78.5
15 (default)
40.7 / 39.1
91.1 / 86.6
86.4 / 78.4
Appendix
Table 6: Sensitivity to retrieval-time settings. The induced library is fixed throughout. Values are R / NDCG (%).
Figure 17: Agentic retrieval across ToolReAGt iterations. Each subfigure reports Recall and NDCG for three agent LLMs on AppWorld, SciToolEval, and BFCL at their benchmark-specific evaluation depths. Results show the mean over three random seeds, and error bars show standard error.
Figure 18: Agentic retrieval across ToolReAGt iterations (continued).
School of Mathematical Sciences, Shanghai Jiao Tong University · Department of Data Science and Artificial Intelligence, The Hong Kong Polytechnic University