ToolOmni: Enabling Open-World Tool Use via Agentic learning with Proactive Retrieval and Grounded Execution

Shouzheng Huang; Meishan Zhang; Baotian Hu; Min Zhang

cs.CLApr 15, 2026

ToolOmni: Enabling Open-World Tool Use via Agentic learning with Proactive Retrieval and Grounded Execution

Authors: Shouzheng Huang, Meishan Zhang, Baotian Hu, Min Zhang

Organizations: Harbin Institute of Technology (Shenzhen)

Abstract

Large Language Models (LLMs) enhance their problem-solving capability by utilizing external tools. However, in open-world scenarios with massive and evolving tool repositories, existing methods relying on static embedding retrieval or parameter memorization of tools struggle to align user intent with tool semantics or generalize to unseen tools, respectively, leading to suboptimal accuracy of open-world tool retrieval and execution. To address these, we present ToolOmni, a unified agentic framework that enables LLMs for open-world tool use by proactive retrieval and grounded execution within a reasoning loop. First, we construct a cold-start multi-turn interaction dataset to instill foundational agentic capabilities via Supervised Fine-Tuning (SFT). Then, we introduce open-world tool learning based on a Decoupled Multi-Objective GRPO algorithm, which simultaneously optimizes LLMs for both tool retrieval accuracy and execution efficacy in online environments. Extensive experiments demonstrate that ToolOmni achieves state-of-the-art performance both in retrieval and execution, surpassing strong baselines by a significant margin of +10.8% in end-to-end execution success rate, while exhibiting exceptional robustness and generalization capabilities.

Figures & tables

Figure 1: Motivation for ToolOmni in Open-World Scenarios: Embedding retrieval methods struggle with Massive tools , often resulting in low retrieval accuracy due to shallow matching; Parameter memory methods fail to adapt to Evolving tools , suffering from poor generalization to unseen tools. ToolOmni overcomes these limitations via a unified agentic framework that couples Proactive Retrieval with Grounded Execution , enabling effective open-world tool use.

Figure 2: Overview of the ToolOmni framework. The pipeline operates in two decoupled phases: Proactive Retrieval : The agent iteratively interacts with the retrieval server to curate a candidate tool set. Grounded Execution : With retrieval results, the agent performs reasoning and tool invocation to generate the final answer.

Method	I1			I2			I3			Avg.
Method	@1	@3	@5	@1	@3	@5	@1	@3	@5	Avg.
In-Domain
BM25*	29.46	31.12	33.27	24.13	25.29	27.65	32.00	25.88	29.78	28.73
EmbSim*	63.67	61.03	65.37	49.11	42.27	46.56	53.00	46.40	52.73	53.35
Re-Invoke*	69.47	-	61.10	54.56	-	53.79	59.65	-	59.55	59.69
IterFeedback	71.64	71.29	76.31	62.65	55.58	60.62	73.85	64.06	69.02	67.22

Table 1: Main results of tool retrieval performance (NDCG@

k

%) on ToolBench. ToolOmni achieves the best performance across most metrics. The best performance is boldfaced, while the second-best performance is underlined. Note : Methods marked with * report results cited from ToolGen Wang et al. (2025) .

Method	Solvable Pass Rate (SoPR)				Solvable Win Rate (SoWR)
Method	I1	I2	I3	Avg.	I1	I2	I3	Avg.
With Ground-Truth Tools
GPT-3.5	56.60	47.80	54.64	53.01	-	-	-	-
ToolLlama-v2	43.60	45.80	39.30	42.90	38.65	55.66	32.79	42.37
ToolLlama-v1	24.20	29.20	12.30	21.90	28.22	42.45	16.39	29.02
ToolGen (3B)	40.80	43.40	21.30	35.17	39.26	47.17	14.75	33.73

Table 2: End-to-end execution performance on ToolBench. We report both Solvable Pass Rate (SoPR %) and Solvable Win Rate (SoWR %) . ToolOmni demonstrates robust superiority, significantly outperforming GPT-3.5 and ToolLlama-v2 in end-to-end settings.

Method	Tool Gen.	Category Gen.	Average
GPT-3.5	50.60	42.10	46.35
ToolLlama-v2	39.20	39.40	39.30
ToolGen (7B)	33.20	40.10	36.65
ToolOmni	52.20	55.95	54.08

Table 3: Generalization performance (SoPR %) in the End-to-End setting. We report performance on Tool Gen. and Category Gen.. ToolOmni achieves the best generalization robustness.

Figure 5: Ablation study of RL components on the ToolBench. Decoupled optimization and trajectory filtering are crucial for execution performance.

Figure 6: Sensitivity analysis of the format reward weight on the ToolBench.

Metric	ToolLlama	ToolGen	ChatGPT	ToolOmni
Search	1.00	-	1.00	3.02
Tool	2.40	2.75	3.98	2.47

Table 6: Average number of search times and tool execution calls per query on the ToolBench Benchmark.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Complexity Level	Number of Queries
I1 (Single-tool)	796
I2 (Intra-category)	573
I3 (Intra-collection)	218
Total	1587

Appendix

Table 7: Statistics of the ToolBench Retrieval Test Set . The dataset is categorized by instruction complexity (I1: Single-tool, I2: Intra-category, I3: Intra-collection).

Difficulty	Subset Name	Count
G1	Instruction Gen. (I1)	163
	Tool Gen. (I1)	158
	Category Gen. (I1)	153
G2	Instruction Gen. (I2)	106
G2	Category Gen. (I2)	124
G3	Instruction Gen. (I3)	61

Appendix

Table 8: Statistics of the StableToolBench Solvable Test Set . Queries are stratified by generalization difficulty (G1, G2, G3) and type.

Method	GPT-5	Gemini-3	Qwen-32B	Mean
ChatGPT	41.68	32.18	54.60	42.82
Llama-v2	42.22	30.28	53.00	41.83
Llama-v1	28.47	21.92	46.55	32.31
ToolGen-3B	35.62	21.68	36.62	31.31
ToolGen-7B	34.73	15.62	32.83	27.73
ToolOmni	54.08	31.95	79.15	55.06

Appendix

Table 9: Average SoPR (%) on ToolBench judged by different LLM backbones.

Panel of Figure 8 — Figure 8: Qualitative comparison (Case Study). Case 2 highlights the robustness of ToolOmni against execution failures. While ToolLlama gets trapped in a repetitive error loop due to rigid parameter usage, ToolOmni demonstrates adaptive planning: after tool failures, it dynamically pivots its strategy—switching from general search to specific verification—to successfully fulfill the user request. Note: The process is condensed for clarity.

Panel of Figure 9 — Figure 9: System prompts used for Retrieval (Left) and Execution (Right) phases. The retrieval prompt guides the agent to proactively search and select tools, while the execution prompt instructs it to perform grounded reasoning and tool invocation.

LLM Tool Use

Large language model tool use studies how models select, call, and coordinate external tools to extend their reasoning and take actions.

See all 295 papers

Harness Engineering in LLM Tool Use via Agent-Native Reusable Tool Primitives
Sep 1, 2026Haibo Jin, Suijin Wang, Xucheng Yu +2Tool-Augmented Language Model Agents LLM Agent Orchestration
School of Information Sciences University of Illinois at Urbana-Champaign Champaign, IL 61820 · Independent Researcher Starc Institute
ToolSearcher: Optimizing Tool Selection at Scale via Reinforcement Learning
Sep 25, 2026Zhenlong Dai, Xujie Song, Zitong Wang +7Tool Selection LLM Tool Use
Zhejiang University · Ant Group
Tools Are Not Islands: Set-Level Tool Retrieval for LLM Agents via Query-Conditioned Hyperedge Prediction
Jul 28, 2026Xinyi Hong, Pinjun Dong, Xinyang Yu +1Tool-Augmented Language Model Agents LLM Tool Use
School of Mathematical Sciences, Shanghai Jiao Tong University · Department of Data Science and Artificial Intelligence, The Hong Kong Polytechnic University

Sep 1, 2026J/K move · Enter open · S save

Harness Engineering in LLM Tool Use via Agent-Native Reusable Tool Primitives

Haibo Jin, Suijin Wang, Xucheng Yu, Haojing Luo+1

School of Information Sciences University of Illinois at Urbana-Champaign Champaign, IL 61820 · Independent Researcher Starc Institute

Tool-Augmented Language Model Agents LLM Agent Orchestration LLM Tool Use Tool-Using Language Agents LLM Agent Harnesses

Open paper AskPDF arXiv

Large language models (LLMs) augmented with external tools have demonstrated remarkable capability in solving complex real-world tasks. However, existing approaches suffer from two key challenges: brittle multi-step and multi-turn reasoning caused by incompatible tool output types and API schemas, and performance degradation under large tool catalogues. To address these, we introduce Tool Primitives, a design that replaces rigid API schema-based invocation with natural language as the interface for tool calling, where each tool is wrapped with an LLM interface that handles schema resolution and execution internally, enabling natural inter-tool communication for nested and multi-turn tool calling. Building on Tool Primitives, we host ToolFace, a centralized repository of 25,519 functions from which LLMs dynamically retrieve only the relevant tools at inference time, eliminating the need to enumerate raw API schemas in context. To orchestrate Tool Primitives and ToolFace reliably in complex settings, we further propose HEART, a Harness Engineering framework via Agent-native, Reusable Tool Primitives, comprising a Planner, Router, and Verifier that jointly support dynamic tool invocation planning, multi-step execution, and feedback-driven recovery. Experiments on five benchmarks demonstrate that HEART outperforms SFT-based models by

10\%

on average and surpasses GPT-5.4, Claude-4.6-Sonnet, and Gemini-3.1-Pro by

6\%

on average while reducing API cost by up to

85\%

. On 50 real-world tasks, HEART achieves

84\%

task completion,

3.8\times

the average of three frontier commercial models (

22\%

Abstract

LLM Tool Use

Explore similar work

Harness Engineering in LLM Tool Use via Agent-Native Reusable Tool Primitives

ToolSearcher: Optimizing Tool Selection at Scale via Reinforcement Learning

Tools Are Not Islands: Set-Level Tool Retrieval for LLM Agents via Query-Conditioned Hyperedge Prediction