Try, Check and Retry: A Divide-and-Conquer Framework for Boosting Long-context Tool-Calling Performance of LLMs

Kunfeng Chen; Qihuang Zhong; Juhua Liu; Bo Du; Dacheng Tao

cs.CLMar 12, 2026

Try, Check and Retry: A Divide-and-Conquer Framework for Boosting Long-context Tool-Calling Performance of LLMs

Authors: Kunfeng Chen, Qihuang Zhong, Juhua Liu, Bo Du, Dacheng Tao

Organizations: School of Computer Science, National Engineering Research Center for Multimedia Software, Institute of Artificial Intelligence and Hubei Key Laboratory of Multimedia and Network Communication Engineering, Wuhan University, China · Nanyang Technological University, Singapore

Abstract

Tool-calling empowers Large Language Models (LLMs) to interact with external environments. However, current methods often struggle to handle massive and noisy candidate tools in long-context tool-calling tasks, limiting their real-world application. To this end, we propose Tool-DC, a Divide-and-Conquer framework for boosting tool-calling performance of LLMs. The core of Tool-DC is to reduce the reasoning difficulty and make full use of self-reflection ability of LLMs via a "Try-Check-Retry" paradigm. Specifically, Tool-DC involves two variants: 1) the training-free Tool-DC (TF), which is plug-and-play and flexible; 2) the training-based Tool-DC (TB), which is more inference-efficient. Extensive experiments show that both Tool-DC methods outperform their counterparts by a clear margin. Tool-DC (TF) brings up to +25.10% average gains against the baseline on BFCL and ACEBench benchmarks, while Tool-DC (TB) enables Qwen2.5-7B to achieve comparable or even better performance than proprietary LLMs, e.g., OpenAI o3 and Claude-Haiku-4.5.

Figures & tables

Figure 1: Performance comparison on BFCL Patil et al. (2025) with different candidate tool scales. We see that as the number of candidate tools increases, all models’ performance degrades significantly, whereas our Tool-DC method can effectively mitigate this issue.

Figure 2: Overview of our Tool-DC framework. The training-free strategy employs a “Try-Check-Retry” pipeline to reduce the reasoning difficulty, while the training-based strategy leverages the prior reasoning trajectories to internalize this divide-and-conquer paradigm into model parameters via fine-tuning.

Figure 3: Illustration of Try stage in Tool-DC (TF) . By splitting the total candidate tools into several parallel groups, Tool-DC (TF) can reduce the length of context and reasoning difficulty effectively.

		Standard Setting					Extended Setting
Methods		BFCL			ACEBench	Avg.	BFCL			ACEBench	Avg.
		Non-Live	Live	Overall	Overall	Avg.	Non-Live	Live	Overall	Overall	Avg.
Qwen2.5-1.5B-Instruct
	GT_Funs	70.96	62.77	66.87	47.92	57.40	70.96	62.77	66.87	47.92	57.40
	All_Funs	69.96	58.18	64.07	46.33	55.20	40.69	33.23	36.96	22.00	29.48
	Top- $K$	76.35	63.80	70.08	44.52	57.30 ${}_{\uparrow\textbf{2.10}}$	60.31	43.60	51.96	38.58	45.27 ${}_{\uparrow\textbf{15.79}}$

Table 1: Performance comparison of Qwen2.5 models with different training-free strategies on BFCL and ACEBench. The best and second-best results are in bold and underlined , respectively. The subscript results denote the relative performance gains against the All_Funs baseline.

Figure 4: Performance comparison of other LLMs with different training-free strategies on the BFCL benchmark. Here, we mainly compare our Tool-DC (TF) method with the All_Funs baseline.

Models	Non-Live	Live	Overall
Close-source Models
OpenAI o3	81.94	73.21	77.58
DeepSeek-V3.2-Exp	85.52	76.02	80.77
Claude-Haiku-4.5	86.50	78.68	82.59
Gemini-3-Pro-Preview	90.65	83.12	86.89
Open-source Models

Table 2: Performance comparison between our tuned models and other proprietary models on BFCL . Notably, the results of proprietary models are from the official leaderboard of BFCL.

Figure 5: Comparison between base models and tuned models using Tool-DC (TB) on Standard ACEBench.

Methods	Modules			Performance
Methods	Try	Check	Retry	Non-Live	Live	Overall
Base	✗	✗	✗	62.73	44.34	53.54
Tool-DC (TF)	✓	✓	✓	71.79	57.74	64.77
w/o Try	✗	✓	✓	44.19	29.39	36.79
w/o Check	✓	✗	✓	52.06	52.41	52.24
w/o Retry	✓	✓	✗	7.33	3.18	5.26

Table 3: Ablation study on Tool-DC (TF) . Here, we use the Qwen2.5-3B-Instruct as the base model and evaluate on the Extended Setting of BFCL.

Figure 6: Impact of Group Count

K

. We evaluate Qwen2.5-3B-Instruct using different training-free methods on the Extended Setting of BFCL.

Figure 7: Scalability and Efficiency Analysis. Here, we use the Qwen2.5-3B-Instruct as the base model, and compare our Tool-DC (TF) with the All_Funs baseline method on the Extended Setting of BFCL.

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Hyperparameter	Value
Batch Size	16
Learning Rate	2e-5
LR Scheduler	cosine
Warmup Ratio	0.1
Epochs	2

Appendix

Table 4: Training hyperparameters for all models.

Figure 8: Sensitivity analysis of group count

K

across Qwen2.5 series. All models are evaluated on the Extended Setting of BFCL.

Methods	Non-Live				Live				Overall
Methods	Simple	Multiple	Parallel	Parallel Multiple	Simple	Multiple	Parallel	Parallel Multiple	Non-Live	Live	Overall
Qwen2.5-7B-Instruct-1M
GT_Funs	74.92	95.00	90.50	85.50	80.23	77.30	62.50	66.67	86.48	77.50	81.99
All_Funs	68.67	85.50	84.00	71.00	63.18	65.81	62.50	58.33	77.29	65.14	71.22
Tool-DC (TF) (Ours)	73.17	89.00	88.50	79.50	65.89	69.71	62.50	66.67	82.54	68.84	75.69
InternLM2.5-7B-Chat-1M

Appendix

Table 5: Performance of long-context LLMs on the Extended Setting of BFCL.

Methods	Non-Live				Live				Overall
Methods	Simple	Multiple	Parallel	Parallel Multiple	Simple	Multiple	Parallel	Parallel Multiple	Non-Live	Live	Overall
GT_Funs	70.92	91.50	79.00	75.50	67.83	70.37	56.25	37.50	79.23	69.13	74.18
All_Funs	64.42	71.00	62.50	53.00	43.41	44.92	43.75	29.17	62.73	44.34	53.54
bge-reranker-v2-m3
-w Top- $K$	67.42	83.50	74.50	61.00	52.71	64.10	50.00	37.50	71.60	61.29	66.45
-w Tool-DC (TF)	68.08	82.50	75.50	68.50	57.75	64.29	50.00	41.67	73.65	62.47	68.06

Appendix

Table 6: Performance of Qwen2.5-3B-Instruct using different retrievers on the Extended Setting of BFCL.

Panel of Figure 9 — Figure 9: The prompt template of “Try” stage in Tool-DC (TF).

Panel of Figure 10 — Figure 10: The prompt template of “Retry” stage in Tool-DC (TF).

Panel of Figure 11 — Figure 11: The prompt template for constructing the CoT data in Tool-DC (TB).

		Non-Live				Live				Overall
Methods		Simple	Multiple	Parallel	Parallel Multiple	Simple	Multiple	Parallel	Parallel Multiple	Non-Live	Live	Overall
Standard Setting
Qwen2.5-1.5B-Instruct
	GT_Funs	63.33	88.50	68.50	63.50	65.12	63.25	43.75	29.17	70.96	62.77	66.87
	All_Funs	63.33	84.00	68.00	64.50	65.12	57.26	43.75	33.33	69.96	58.18	64.07
	Top- $K$	66.42	83.50	79.50	76.00	68.99	63.15	50.00	45.83	76.35	63.80	70.08

Appendix

Table 7: Detailed results of different training-free methods on BFCL, which is a full version of Table 1 .

Methods	Non-Live				Live				Overall
Methods	Simple	Multiple	Parallel	Parallel Multiple	Simple	Multiple	Parallel	Parallel Multiple	Non-Live	Live	Overall
Qwen2.5-7B-Instruct	75.33	94.50	91.50	84.50	76.74	74.93	62.50	70.83	86.46	67.44	76.95
-w Vanilla SFT	70.92	95.50	88.50	82.00	77.52	73.69	81.25	58.33	84.23	74.24	79.24
-w Tool-DC (TB)	71.17	95.00	93.00	87.50	83.72	79.58	56.25	54.17	86.67	79.64	83.16
Llama-3.2-3B-Instruct	70.67	92.50	88.50	79.00	58.33	65.12	57.64	25.00	82.67	58.33	70.50
-w Vanilla SFT	71.25	92.00	88.50	81.50	70.93	61.35	56.25	54.17	83.31	62.99	73.15

Appendix

Table 8: Detailed results of different training-based methods on BFCL, which is a full version of Table 2 .

Methods	Standard Setting						Extended Setting
Methods	Atom	Single Turn	Multiple Turn	Similar API	Profile	Overall	Atom	Single Turn	Multiple Turn	Similar API	Profile	Overall
Qwen2.5-1.5B-Instruct
GT_Funs	56.00	38.50	28.00	52.00	54.00	47.92	56.00	38.50	28.00	52.00	54.00	47.92
All_Funs	56.67	35.00	29.00	50.00	38.00	46.33	27.67	15.00	5.00	32.00	26.00	22.00
Top- $K$	57.33	37.00	10.12	52.00	44.00	44.52	52.67	28.00	8.50	40.00	34.00	38.58
HiTEC-ICL	54.33	34.00	24.00	52.00	48.00	45.17	28.00	21.50	13.00	24.00	44.00	25.42

Appendix

Table 9: Detailed results of different training-free methods on ACEBench, which is a full version of Table 1 . “Atom”, “Single Turn”, “Multiple Turn”, “Similar API”, and “Profile” are subsets of ACEBench.

Methods	Atom	Single Turn	Multiple Turn	Similar API	Profile	Overall
Qwen2.5-7B-Instruct	62.33	63.50	45.00	64.00	64.00	59.92
-w Vanilla SFT	67.67	65.50	7.14	70.00	64.00	57.11
-w Tool-DC (TB)	76.33	47.00	22.94	74.00	64.00	61.32
Llama-3.2-3B-Instruct	33.33	21.00	8.00	34.00	50.00	28.50
-w Vanilla SFT	65.00	43.00	5.22	52.00	48.00	48.87
-w Tool-DC (TB)	72.67	43.00	12.71	56.00	38.00	53.45

Appendix

Table 10: Detailed results of different training-based methods on Standard ACEBench, referred to Figure 5 .

LLM Tool Use

Large language model tool use studies how models select, call, and coordinate external tools to extend their reasoning and take actions.

See all 295 papers

The Bitter Lesson of Tool Calling
Aug 6, 2026Ishan Patel, Sahil Sen, Elias Lumer +1LLM Evaluation Tool-Augmented Language Model Agents
Commercial Technology and Innovation Office PricewaterhouseCoopers, U.S.A
ToolSpec: Accelerating Tool Calling via Schema-Aware and Retrieval-Augmented Speculative Decoding
Apr 15, 2026Heming Xia, Yongqi Li, Cunxiao Du +2LLM Inference Acceleration LLM Tool Use
Department of Computing, The Hong Kong Polytechnic University · Peking University
Gecko: A Simulation Environment with Stateful Feedback for Refining Agent Tool Calls
Feb 22, 2026Zeyu Zhang, Guohao Li, Zhenchang Xing +3LLM Tool Use LLM Agent Reliability
Australian National University · CSIRO’s Data61 · Aipotheosis Labs

Aug 6, 2026J/K move · Enter open · S save

The Bitter Lesson of Tool Calling

Ishan Patel, Sahil Sen, Elias Lumer, Vamse Kumar Subbiah

Commercial Technology and Innovation Office PricewaterhouseCoopers, U.S.A

LLM Evaluation Tool-Augmented Language Model Agents LLM Tool Use

Open paper AskPDF arXiv

Tool use transforms LLMs into agents that act beyond their training data, and for code-capable models, programmatic tool calling extends this further by replacing rigid JSON calls with scripts that chain and parallelize naturally. However, a systematic evaluation of tools as code on an established benchmark across current and prior model generations under real-world task conditions has not been conducted. In this work, we empirically compare programmatic tool calling (PTC) to native JSON tool calling across 14 language models on BFCL v4. In the programmatic tool calling paradigm, tools are exposed as typed Python stubs that the model invokes through code, with execution and results handled in a single agent turn. Programmatic tool calling matches or exceeds native JSON tool calling in 11 of 14 models on BFCL v4, with the GPT-5.6 family achieving a 10.6% improvement over the JSON tool calling baseline. Further, it matches or outperforms baseline in 13 of 14 models under parallel fan-out, and holds stable under context rot conditions where baseline degrades 2.3% on average. Our results demonstrate that programmatic tool calling is a viable and robust alternative to JSON tool calling, with performance tracking model capability across release generations.

Abstract

LLM Tool Use

Explore similar work

The Bitter Lesson of Tool Calling

ToolSpec: Accelerating Tool Calling via Schema-Aware and Retrieval-Augmented Speculative Decoding

Gecko: A Simulation Environment with Stateful Feedback for Refining Agent Tool Calls