CARE-PO: Controllable Tool-Call Efficiency for LLM Agents
Authors: Xiaoyang Ye, Xiao Yao, Hairui Zhu, Yiying Yang, Bei Gan, Jinliang Lu, Tengjin Weng, Yuan Yao, Wenhao Jiang
Organizations: Guangdong Laboratory of Artificial Intelligence and Digital Economy (SZ) · Shenzhen University · China Southern Power Grid Company Limited
External tool calls are a major running cost of an LLM agent, while its training objectives align almost exclusively with task completion. In a step-level diagnosis of Qwen3.5 trajectories at two model sizes on GAIA, a long-horizon tool-use benchmark, we find that most contain removable calls, at a similar rate at both sizes. Some methods add an efficiency reward to the objective to reduce the call count, but introduce problems of their own: the anchor they price calls against is computed from the policy's own rollouts (e.g., the group's shortest correct rollout) and so moves with the policy; training can drift toward a collapsed policy that issues no tool call at all, a degeneracy seen in published reports and in our 27B baseline; and a training run delivers a single operating point, so the user cannot trade efficiency against capability. We present CARE-PO (Controllable Anchored Relative Efficiency Policy Optimization) to mitigate these problems. The efficiency reward prices calls against a reference budget anchored to the task's own difficulty and fixed before training. Injecting a successful trajectory from the training data into each online GRPO rollout group and charging for evidence a trajectory never gathers ease the drift toward a collapsed call count. Shaping the cost curve by an effort tag in the prompt yields an efficiency and a capability operating point from a single training run. On GAIA, five of the six operating points across three model sizes improve on their base model in success rate and call count at once. On four held-out multi-hop benchmarks the result is size-dependent: at 27B and 4B success rate is held or raised at a lower call count, while at 9B both operating points trade success rate for fewer calls. The call count remains well above zero throughout training at all three sizes.
Figures & tables
Figure 1: (a) Success rate against average tool calls (AC) at 9B on GAIA’s 165-question validation split; error bars are the s.d. over three runs. Vanilla is the untrained model; CARE-PO (low) and (high) are two operating points of one set of weights; the baselines are defined in Section 4 . (b) On one GAIA task at 27B, all three answer correctly, with 15, 3 and 7 calls (Appendix B ).
Figure 2: CARE-PO overview. The budgets 3 and 10 in (b) are illustrative. Cost u covers more than calls: Eq. ( 3 ) also charges a repeated call twice and charges evidence the trajectory never obtained. The 0.17 marked in (d) is the largest gap between the tiers, and the dashed middle curve there is the unconditioned tier (no tag). Definitions in Sections 3.2 – 3.5 .
Method
SR ↑
AC ↓
HV ↑
Qwen3.5-9B
Vanilla ReAct
40.2 ± 0.7
8.26 ± 0.14
0.236 ± 0.005
Budget-Prompt
35.4 ± 0.3 ▼ 4.8
8.50 ± 0.05 ▲ 2.9%
0.203 ± 0.001 ▼ 14.0%
DEPO
34.7 ± 0.9 ▼ 5.5
7.54 ± 0.05 ▼ 8.7%
0.217 ± 0.005 ▼ 8.1%
OTC-PO
40.6 ± 1.2 ▲ 0.4
5.01 ± 0.13 ▼ 39.3%
0.304 ± 0.012 ▲ 28.8%
CARE-PO (low)
39.8 ± 2.3 ▼ 0.4
3.42 ± 0.30 ▼ 58.6%
0.330 ± 0.015 ▲ 39.8%
Table 1: Main results on GAIA-165 (mean ± sample s.d. over three independent runs; best per metric and size in bold). Below the dashed line: CARE-PO (low), the efficiency point, and CARE-PO (high), the capability point, from one set of weights. Arrows give the change from Vanilla ReAct at the same size (percentage points for SR, relative for AC and HV); green marks an improvement.
2Wiki
HotpotQA
MuSiQue
Bamboogle
Method
SR ↑
AC ↓
HV ↑
SR ↑
AC ↓
HV ↑
SR ↑
AC ↓
HV ↑
SR ↑
AC ↓
HV ↑
Qwen3.5-9B
Vanilla ReAct
75.8
5.64
0.544
48.6
4.87
0.368
29.0
6.59
0.194
76.0
4.05
0.606
OTC-PO
70.6
2.24
0.627
48.0
1.59
0.442
25.6
2.23
0.227
68.8
1.38
0.640
CARE-PO (low)
63.2
1.94
0.571
41.4
1.09
0.391
21.4
1.55
0.197
43.2
0.50
0.421
CARE-PO (high)
70.4
2.19
0.627
46.2
1.51
0.427
25.2
2.10
0.226
60.8
1.20
0.572
Table 2: Results on four held-out multi-hop benchmarks (deterministic single run; question counts in Section 4 ). Best per column and size in bold.
Configuration
SR ↑
AC ↓
HV ↑
Note
Trained at t=1 , low tag
39.2
4.07
0.312
same price at every tier
Trained at t=1 , high tag
40.6
4.91
0.306
as above
Same weights, no tag at inference
39.8
4.29
0.313
the unconditioned tier from training
Untrained base + low tag
37.0
7.01
0.240
no training, prompt only
Untrained base + high tag
40.6
7.77
0.248
no training, prompt only
CARE-PO (low)
39.8
3.42
0.330
efficiency operating point
Table 3: Ablation of the effort conditioning (GAIA-165, 9B).
Figure 3: Training dynamics at 27B for OTC-PO and our method, excluding the injected source trajectory; solid curves are running means of the per-step values (dots). (a) The running means at six steps; bold values follow the collapse. (b) Group success rate on the training distribution. (c) Mean tool calls per rollout.
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 4: A real trajectory of the 9B base model on GAIA task 0a3cd321 (Level 2), with its redundant calls labelled step by step by the diagnosis judge. A call may match several forms; the label gives its principal one. The form (iii) cue is a World Bank DataBank table with per-country values for each year of 2001–2010: its link appears in the search results of steps 1, 7, 9, 10, 14 and 19 (snippets omitted), yet the table is never opened.
Figure 5: Call trajectories of three policies on GAIA task a0068077 , with arguments and observations abbreviated. All three answer correctly; they differ in how many calls they need to reach the same numeric answer. The highlighted steps 4–7 in (c) are the capability point’s extra checking calls.
Method
AC
Turns
Generated
Observations
Context processed
Vanilla ReAct
8.26
9.1
1.91k
13.1k
104.4k
OTC-PO
5.01
6.0
1.41k
8.90k
53.2k
CARE-PO (low)
3.42
4.4
1.04k
6.54k
34.6k
CARE-PO (high)
4.44
5.4
1.24k
8.11k
45.3k
Appendix
Table 4: Per-question cost on GAIA-165 at 9B, averaged over the three runs of Table 1 . Tokens are counted with the Qwen3.5 tokenizer over the stored traces; context processed is the total prompt length over the model calls of a trajectory (system prompt, question and all preceding messages).
Figure 6: Training dynamics of our method at three model sizes, excluding the injected source trajectory; curves are centred running means over 25 optimization steps. (a) Group success rate on the training distribution. (b) Mean tool calls per rollout. (c) Mean reward. (d) The efficiency factor minus one, i.e. λtanh(γe(1−u)) in Eq. ( 2 ), which is zero (dashed line) at u=1 , where cost is exactly one reference budget. (e) Difference in mean call count between the high and low tiers. (f) Sample standard deviation of the call count within a group.
Deciding whether to call a tool is a core competence of an LLM agent, and a costly one to get wrong: needless calls add latency, accrue cost, and may trigger irreversible side effects, while missing calls leave the model confidently wrong on questions it could only answer through tool-calls. Models manage this balance poorly, both over-using and under-using tools. Existing methods such as post-training and prompt engineering are expensive and difficult to modify at inference time. We show that whether an instruction-tuned model calls a tool can be controlled by a single linear direction in its residual stream, extracted without any training from the model's own tool-use preference signal and turned into an inference-time intervention with no prompt change. Adding the direction with strength α moves the call rate monotonically from near 0% to over 90% while keeping calls well-formed. The steering works in both directions: dialing it down suppresses calls, and dialing it up induces new calls that land precisely on the questions the model cannot answer from its own knowledge. We also show that the direction generalizes to unseen tools with strength comparable to each tool's own direction and without favoring any specific tool choice. With live tool execution, a single sweep of the steering traces a cost/accuracy Pareto frontier and nearly doubles open-domain QA accuracy (0.29→0.56); the same recipe transfers across a diverse range of models spanning dense, MoE, and multimodal architectures, without any training. Our code is publicly available at https://github.com/YuqiChen4188/Steering-Tool-Use-Propensity.
Yuqi Chen, Vincent Siu, Yang Liu +2
Department of Computer Science and Engineering, UC Santa Cruz · Department of Computer Science, UC Berkeley
Tool-calling is a central component of modern large language model (LLM) agents, equipping them with skills beyond their parametric knowledge. This paper studies tool-calling along two complementary axes: effectiveness, i.e., how this capability is measured, and efficiency, i.e., how it is learned. On effectiveness, we systematically analyze tool-calling evaluation pipelines and show that results can be highly sensitive to seemingly minor, often undocumented implementation choices including the random seed, system prompt, multi-turn template construction, and how prior interaction/reasoning history is carried forward. These choices can lead to substantial differences in reported performance, especially in multi-turn settings where without rigorous standardization, leaderboard rankings are unreliable. On efficiency, we examine standard reinforcement learning (RL) for tool-calling and identify two sources of computational waste: (i) during rollouts, many prompts produce no learning signal, and (ii) during policy updates, optimization incurs high computational cost. Guided by these findings, we introduce two techniques that accelerate RL-based tool-calling training, achieving substantial wall-clock speedup without degrading performance.
Tong Liu, Cheng Qian, Matej Cief +4
1LMU Munich & Munich Center for Machine Learning · 2UIUC · 3Amazon +1
Tool-augmented LLM agents tend to call tools indiscriminately, even when the model can answer directly. Each unnecessary call wastes API fees and latency, yet no existing benchmark systematically studies when a tool call is actually needed. We propose When2Tool, a benchmark of 18 environments (15 single-hop, 3 multi-hop) spanning three categories of tool necessity -- computational scale, knowledge boundaries, and execution reliability -- each with controlled difficulty levels that create a clear decision boundary between tool-necessary and tool-unnecessary tasks. We evaluate two families of training-free baselines: Prompt-only (varying the prompt to discourage unnecessary calls) and Reason-then-Act (requiring the model to reason about tool necessity before acting). Both provide limited control: Prompt-only suppresses necessary calls alongside unnecessary ones, and Reason-then-Act still incurs a disproportionate accuracy cost on hard tasks. To understand why these baselines fail, we probe the models' hidden states and find that tool necessity is linearly decodable from the pre-generation representation with AUROC 0.89--0.96 across six models, substantially exceeding the model's own verbalized reasoning. This reveals that models already know when tools are needed, but fail to act on this knowledge during generation. Building on this finding, we propose Probe&Prefill, which uses a lightweight linear probe to read the hidden-state signal and prefills the model's response with a steering sentence. Across all models tested, Probe&Prefill reduces tool calls by 38% with only 1.7% accuracy loss, while the best baseline at comparable accuracy only reduces 6% of tool calls, or achieves a similar tool call reduction but incurs a 5× higher accuracy loss. Our code is available at https://github.com/Trustworthy-ML-Lab/when2tool