The hidden states of large language models (LLMs) are known to capture rich information relating to model knowledge and behavior that can be hard to extract from examination of input and output alone. As LLM-based systems increasingly interface with the external world, one area of concern is detecting incorrect or improper use of tools. Motivated by this, we study the effectiveness of using linear probes to detect incorrect tool-calls, measuring probe efficacy across 18 tool-calling LLMs evaluated on the Berkeley Function Calling Leaderboard. Overall, we find that probing is an effective means to catch a range of different tool-calling errors, including errors arising from using an argument that has the wrong value but the correct type, which might not be recorded by standard logging frameworks. Important factors in success include model size, probing layer, and model post-training type. We also show that probes are capable of generalizing to novel types of errors, which is critical in real world deployments.
Figures & tables
Figure 1: (Left) AUROC of tool-call error detection (y-axis) across hidden activation layer depth (x-axis) for smaller models ( ∼ 8B parameters) and larger models (70B parameters). Solid line: mean. Shaded area: ±1σ . (Right) Tool-calling accuracy vs hallucination detection test AUROC (averaged across layers). Dot size corresponds to model size.
Figure 2: Label space transfer experiment. Left : Test AUROC for training on split A and evaluating on split B. Right : Test AUROC for training on split B and evaluating on split A.
Appendix figures & tables10 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 3: Probe tool-calling error detection performance for all models (colored by model size).
Model
Size
Python Tools
Java Tools
JavaScript Tools
Python Tools
(B)
(Single)
(Single)
(Single)
(Multi)
gemma-3-1b-it
1
24.00
5
4
0.00
Phi-4-mini-instruct
3.8
84.75
54
62
88.50
Falcon3-7B-Instruct
7
93.25
58
42
89.00
Llama-3.1-8B-Instruct
8
50.50
57
60
54.50
granite-3.1-8b
8
90.50
57
56
91.50
Appendix
Table 1: Tool-call accuracy (%) by model and evaluation setting. Models are sorted by size. Single denotes single tool use; Multi denotes selecting and correctly employing one tool from multiple candidates. Provider prefixes are omitted from model names for brevity. Models with high error rates ( ≥20% of all evaluation samples) are excluded from probe-based analysis and have their names grayed out.
Model
Size
Correct
Incor.
Missing
Unnec.
H. Tool/
Type
Syntax
(B)
Value
Param.
Tool
Param.
Error
Error
gemma-3-1b-it
1
13.73
3.74
22.00
0.26
6.27
1.86
52.14
Phi-4-mini-instruct
3.8
80.13
5.07
0.40
0.40
0.53
3.33
10.13
Falcon3-7B-Instruct
7
84.00
9.46
0.80
0.94
0.27
4.14
0.40
Llama-3.1-8B-Instruct
8
53.07
7.20
0.00
0.27
0.80
37.07
1.60
granite-3.1-8b
8
84.00
6.13
0.66
0.67
0.40
5.20
2.93
Appendix
Table 2: Tool-call error breakdown by model (in %). Models are sorted by size. Correct indicates fully correct tool-calls; all other columns indicate error categories. Provider prefixes are omitted from model names for brevity. Models with high error rates ( ≥20% overall) are excluded from probe-based analysis and have their names grayed out.
Model
Post-Training
Falcon3-7B-Instruct
Instruction Tuned
granite-3.1-8b
Instruction Tuned
granite-3.2-8b
Instruction Tuned
Ministral-8B-2410
Instruction Tuned
Qwen3-8B
Instruction Tuned
Hermes-2-Pro-Llama-3
Tool Finetuned
Appendix
Table 3: Classification of ∼ 8B parameter models by post-training regime. Tool finetuned models have been explicitly trained on function calling or agent action datasets. Provider prefixes are omitted for brevity.
Figure 4: Test AUROC (left) and Normalized AUPR (right) of tool-call error detection across recording layer depth for instruction tuned vs tool finetuned models (all ∼8 B parameter models). Solid line: mean. Shaded area: ±1σ .
Partition
Split
Error Categories
A/B
A
Incorrect Value, Missing Parameter, Syntax Error
B
Unnecessary Tool-Call, Hallucinated Tool/Param., Type Error
Table 4: Composition of label space transfer splits. Each split contains three of the six error categories; the correct category is always treated as the negative class. Within each partition, splits are designed to approximately balance the aggregate positive rate across the two subsets for most models.
Figure 5: Label space transfer experiment. Top row: metrics for C to D. Bottom row: metrics for D to C. Solid line: mean. Shaded area: ±1σ .
Figure 6: Normalized AUPR of tool-call error detection (y-axis) across hidden activation layer depth (x-axis) for smaller models ( ∼ 8B parameters) and larger models (70B parameters). Solid line: mean. Shaded area: ±1σ .
Figure 7: Normalized AUPR for the A/B label space transfer experiment. Left: normalized AUPR for A to B. Right: normalized AUPR for B to A. Solid line: mean. Shaded area: ±1σ .
Figure 8: FPR@90%TPR plotted against normalized probing layer depth. Lower is better. Solid line: mean. Shaded area: ±1σ .
Large language model agents derive much of their capability from tool use. Existing research on tool use has largely focused on selecting the right tool and orchestrating the order of calls. However, correctly filling the parameters of a tool call is equally critical for successful execution and has received far less attention. In domains such as cloud networking, even frontier models correctly complete fewer than half of tool calls. Inspired by recent analyses showing that LLM hidden states encode rich information about model predictions, we discover that while the model generates a parameter value, its hidden state contains a strong correctness signal: a simple linear probe can accurately predict whether the value will be correct. Based on this observation, we propose a unified probe-guided framework with two complementary approaches: probe-filtered bootstrapped training (PBT), which uses the probe to filter reliable self-generated calls for fine-tuning, and probe-guided reranking (PGR), which uses the probe to select better candidates during inference. To support systematic evaluation, we release ParamBench, a benchmark built from real cloud-network APIs that categorizes every instance into five difficulty levels according to parameter nesting depth, cross-parameter dependencies, and the reasoning required to derive values from earlier calls. Extensive experiments across 5 open models on ParamBench and 6 external benchmarks demonstrate that our method substantially improves parameter generation, raising the average exact match from 19.7% to 59.6%.
Agentic LLMs exhibit three consequential tool-use failures: invalid arguments (validity), unnecessary calls (over-calling), and omitted calls when tools are needed (missing). We find that a small, failure-specific set of MLP neurons could distinguish such failures with linearly separable decision boundaries. Building on this observation, we introduce PRISMS (Probing Representations In Support of Monitoring and Steering), a closed-loop framework that shares a failure-specific neuron basis between sparse detection and activation steering. PRISMS selects contribution-critical MLP neurons and fits an L1-regularized detector on their activations. Across six models from the Qwen3, Llama, and Gemma families, over-calling and missing are detected at the pre-generation prompt boundary with ROC-AUC 0.90-1.00, while validity is detected from the generated tool-call span with ROC-AUC 0.86-0.90. These results are achieved with highly sparse readouts: only 1-2 MLP neurons for missing, 2-16 for over-calling, and approximately 128 for validity. These sparse detectors match or outperform dense residual-stream baselines using 23-627 times fewer features. The shared neuron basis also supports bidirectional control over tool-calling behavior, suppressing unnecessary calls and eliciting omitted ones. PRISMS therefore gates intervention on predicted failure risk to mitigate the collateral effects of unconditional steering. Across all six models, PRISMS reduces pooled over-calling rate by 80% (from 0.131 to 0.026) while increasing tool-required accuracy by 14.2 percentage points (from 0.689 to 0.831). PRISMS thus provides lightweight failure detection and selective intervention across model families.
Linear probes detect corrupted context in language models with near-perfect accuracy, yet this does not translate into reliable failure prediction. The result is a dissociation with direct implications for deployment monitoring. Across multi-hop arithmetic chains, probes that detect corruption turn out to be uninformative about final answer correctness; models forced into structured confidence formats collapse to two values with indistinguishable error rates; and probe persistence across hops fails to separate correct from incorrect outcomes, refuting our pre-registered "persistence beats peak" hypothesis. This pattern of knowing but not saying generalises across model families including reasoning models. As a real-time monitor, probe-based interventions are sharply model and error-type dependent: branch-and-pick is net-positive across models and uniquely non-breaking on Llama-3.1-8B (4 rescued, 0 broken), while reprompt and replace-prior break correct traces at roughly the rate they rescue wrong ones. Probe-based monitoring is a necessary complement to verbalised confidence, but no single intervention dominates, and the deployable answer is model-aware, error-type-aware routing.