The Calls are Coming from Inside the Model: Investigating Probe-based Detection of Tool-Calling Errors in LLMs

Eric Yeats; Brendan Kennedy; Loc Truong; John Buckheit; Jung Lee; Jesse Friedbaum; John Emanuello; Henry Kvinge

cs.LGAug 27, 2026

The Calls are Coming from Inside the Model: Investigating Probe-based Detection of Tool-Calling Errors in LLMs

Authors: Eric Yeats, Brendan Kennedy, Loc Truong, John Buckheit, Jung Lee, Jesse Friedbaum, John Emanuello, Henry Kvinge

Organizations: Pacific Northwest National Laboratory · National Security Agency

Abstract

The hidden states of large language models (LLMs) are known to capture rich information relating to model knowledge and behavior that can be hard to extract from examination of input and output alone. As LLM-based systems increasingly interface with the external world, one area of concern is detecting incorrect or improper use of tools. Motivated by this, we study the effectiveness of using linear probes to detect incorrect tool-calls, measuring probe efficacy across 18 tool-calling LLMs evaluated on the Berkeley Function Calling Leaderboard. Overall, we find that probing is an effective means to catch a range of different tool-calling errors, including errors arising from using an argument that has the wrong value but the correct type, which might not be recorded by standard logging frameworks. Important factors in success include model size, probing layer, and model post-training type. We also show that probes are capable of generalizing to novel types of errors, which is critical in real world deployments.

Figures & tables

Panel of Figure 1 — Figure 1: (Left) AUROC of tool-call error detection (y-axis) across hidden activation layer depth (x-axis) for smaller models ( $\sim$ 8B parameters) and larger models (70B parameters). Solid line: mean. Shaded area: $\pm 1\sigma$ . (Right) Tool-calling accuracy vs hallucination detection test AUROC (averaged across layers). Dot size corresponds to model size.

Panel of Figure 2 — Figure 2: Label space transfer experiment. Left : Test AUROC for training on split A and evaluating on split B. Right : Test AUROC for training on split B and evaluating on split A.

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 3: Probe tool-calling error detection performance for all models (colored by model size).

Model	Size	Python Tools	Java Tools	JavaScript Tools	Python Tools
	(B)	(Single)	(Single)	(Single)	(Multi)
gemma-3-1b-it	1	24.00	5	4	0.00
Phi-4-mini-instruct	3.8	84.75	54	62	88.50
Falcon3-7B-Instruct	7	93.25	58	42	89.00
Llama-3.1-8B-Instruct	8	50.50	57	60	54.50
granite-3.1-8b	8	90.50	57	56	91.50

Appendix

Table 1: Tool-call accuracy (%) by model and evaluation setting. Models are sorted by size. Single denotes single tool use; Multi denotes selecting and correctly employing one tool from multiple candidates. Provider prefixes are omitted from model names for brevity. Models with high error rates (

\geq 20\%

of all evaluation samples) are excluded from probe-based analysis and have their names grayed out.

Model	Size	Correct	Incor.	Missing	Unnec.	H. Tool/	Type	Syntax
	(B)		Value	Param.	Tool	Param.	Error	Error
gemma-3-1b-it	1	13.73	3.74	22.00	0.26	6.27	1.86	52.14
Phi-4-mini-instruct	3.8	80.13	5.07	0.40	0.40	0.53	3.33	10.13
Falcon3-7B-Instruct	7	84.00	9.46	0.80	0.94	0.27	4.14	0.40
Llama-3.1-8B-Instruct	8	53.07	7.20	0.00	0.27	0.80	37.07	1.60
granite-3.1-8b	8	84.00	6.13	0.66	0.67	0.40	5.20	2.93

Appendix

Table 2: Tool-call error breakdown by model (in %). Models are sorted by size. Correct indicates fully correct tool-calls; all other columns indicate error categories. Provider prefixes are omitted from model names for brevity. Models with high error rates (

\geq 20\%

overall) are excluded from probe-based analysis and have their names grayed out.

Model	Post-Training
Falcon3-7B-Instruct	Instruction Tuned
granite-3.1-8b	Instruction Tuned
granite-3.2-8b	Instruction Tuned
Ministral-8B-2410	Instruction Tuned
Qwen3-8B	Instruction Tuned
Hermes-2-Pro-Llama-3	Tool Finetuned

Appendix

Table 3: Classification of

\sim

8B parameter models by post-training regime. Tool finetuned models have been explicitly trained on function calling or agent action datasets. Provider prefixes are omitted for brevity.

Figure 4: Test AUROC (left) and Normalized AUPR (right) of tool-call error detection across recording layer depth for instruction tuned vs tool finetuned models (all $\sim 8$ B parameter models). Solid line: mean. Shaded area: $\pm 1\sigma$ .

Partition	Split	Error Categories
A/B	A	Incorrect Value, Missing Parameter, Syntax Error
	B	Unnecessary Tool-Call, Hallucinated Tool/Param., Type Error
C/D	C	Missing Parameter, Type Error, Syntax Error
	D	Incorrect Value, Unnecessary Tool-Call, Hallucinated Tool/Param.

Appendix

Table 4: Composition of label space transfer splits. Each split contains three of the six error categories; the correct category is always treated as the negative class. Within each partition, splits are designed to approximately balance the aggregate positive rate across the two subsets for most models.

Panel of Figure 5 — Figure 5: Label space transfer experiment. Top row: metrics for C to D. Bottom row: metrics for D to C. Solid line: mean. Shaded area: $\pm 1\sigma$ .

Figure 6: Normalized AUPR of tool-call error detection (y-axis) across hidden activation layer depth (x-axis) for smaller models ( $\sim$ 8B parameters) and larger models (70B parameters). Solid line: mean. Shaded area: $\pm 1\sigma$ .

Panel of Figure 7 — Figure 7: Normalized AUPR for the A/B label space transfer experiment. Left: normalized AUPR for A to B. Right: normalized AUPR for B to A. Solid line: mean. Shaded area: $\pm 1\sigma$ .

Figure 8: FPR@90%TPR plotted against normalized probing layer depth. Lower is better. Solid line: mean. Shaded area: $\pm 1\sigma$ .

Language Model Error Detection

Language model error detection identifies unreliable answers, reasoning steps, and actions so systems can flag, verify, or repair them.

See all 91 papers

Getting the Parameters Right: A Difficulty-Graded Benchmark and Probe-Guided Training for LLM Tool Calls
Aug 4, 2026Guoyao Yu, Xiaoqing Sun, Ziqi Huang +13Tool-Augmented Language Model Agents AI Agent Benchmarks
A Few Neurons Reveal When LLMs Misuse Tools: Sparse Detection and Selective Steering for Reliable Tool Use
Jul 31, 2026Yutong Ke, Ming Yin, Chongwen Zhao +1LLM Interpretability LLM Tool Use
The Knowing-Saying Gap: When Probes See Errors that Confidence Misses
Jul 21, 2026Jyotin Goel, Ipshita Bandyopadhyay, Justin ShenkLanguage Model Error Detection Confidence Estimation in Language Models

Aug 4, 2026J/K move · Enter open · S save

Getting the Parameters Right: A Difficulty-Graded Benchmark and Probe-Guided Training for LLM Tool Calls

Guoyao Yu, Xiaoqing Sun, Ziqi Huang, Shaojing Fan+12

Tool-Augmented Language Model Agents AI Agent Benchmarks LLM Tool Use LLM Agent Training Tool Use

Open paper AskPDF arXiv

Large language model agents derive much of their capability from tool use. Existing research on tool use has largely focused on selecting the right tool and orchestrating the order of calls. However, correctly filling the parameters of a tool call is equally critical for successful execution and has received far less attention. In domains such as cloud networking, even frontier models correctly complete fewer than half of tool calls. Inspired by recent analyses showing that LLM hidden states encode rich information about model predictions, we discover that while the model generates a parameter value, its hidden state contains a strong correctness signal: a simple linear probe can accurately predict whether the value will be correct. Based on this observation, we propose a unified probe-guided framework with two complementary approaches: probe-filtered bootstrapped training (PBT), which uses the probe to filter reliable self-generated calls for fine-tuning, and probe-guided reranking (PGR), which uses the probe to select better candidates during inference. To support systematic evaluation, we release ParamBench, a benchmark built from real cloud-network APIs that categorizes every instance into five difficulty levels according to parameter nesting depth, cross-parameter dependencies, and the reasoning required to derive values from earlier calls. Extensive experiments across 5 open models on ParamBench and 6 external benchmarks demonstrate that our method substantially improves parameter generation, raising the average exact match from 19.7% to 59.6%.

Abstract

Language Model Error Detection

Explore similar work

Getting the Parameters Right: A Difficulty-Graded Benchmark and Probe-Guided Training for LLM Tool Calls

A Few Neurons Reveal When LLMs Misuse Tools: Sparse Detection and Selective Steering for Reliable Tool Use

The Knowing-Saying Gap: When Probes See Errors that Confidence Misses