OverAct: Measuring and Mitigating Proactive Over-Authorization in LLM Tool-Calling Agents

Taolin Zhang; Jiuheng Wan; Hanyu Wang; Tingyuan Hu; Chengyu Wang

cs.CROct 1, 2026

OverAct: Measuring and Mitigating Proactive Over-Authorization in LLM Tool-Calling Agents

Authors: Taolin Zhang, Jiuheng Wan, Hanyu Wang, Tingyuan Hu, Chengyu Wang

Organizations: Hefei University of Technology · East China Normal University · Alibaba Cloud Computing

Abstract

LLM agents with tool-calling capabilities can access external services and private user data, but they may retrieve more information than a user's request explicitly requires. We study this behavior in structured tool-calling agents and term it proactive over-authorization. This setting differs from filesystem-level coding agents because the main risk is unnecessary access to private data. We introduce OverAct, a controlled benchmark spanning eight privacy-sensitive domains with deterministic, judge-free scoring, together with an interpretive decision-theoretic framework that yields three testable predictions. Across seven models from four families, all models significantly exceed authorized scope. Request specificity is the strongest predictor of severity, over-authorization grows sublinearly with tool-pool size, and decoding temperature has little effect. These patterns are consistent with a cost-asymmetry account, suggesting that over-authorization arises more from structural decision tendencies than from decoding randomness. We also propose SelfAudit, a zero-shot inference-time method that generates request-grounded justifications and filters unjustified calls before execution. Ablation shows that explicit filtering is the main driver of scope reduction. SelfAudit reduces privacy-oriented excess by 43% without oracle knowledge.

Figures & tables

Figure 1: A decision-theoretic view. The agent assigns each tool a relevance estimate

\theta_{i}

; when omission is effectively costlier than commission, the decision threshold

\tau

becomes low and marginally relevant tools are selected.

Model	Baseline SIR	95% CI
Qwen3.6-Flash	1.50	[1.45, 1.55]
Qwen3.6-Plus	1.65	[1.59, 1.71]
Qwen3.7-Max	1.74	[1.67, 1.80]
DeepSeek-v3	1.18	[1.13, 1.23]
DeepSeek-v4-Pro	2.39	[2.31, 2.47]
GLM-5.2	1.96	[1.89, 2.03]

Table 1: Baseline SIR across all models. One-sided Wilcoxon signed-rank test (

H_{1}

: SIR

>1

p<0.001

Figure 2: SIR by model and request specificity. All models show near-perfect compliance for precise requests but substantial inflation for vague requests.

Figure 3: SIR as a function of tool-pool size. Over-authorization grows sublinearly from 4 to 24 tools, then plateaus at 36–48, consistent with P3.

Intervention	SIR	TCR	Oracle?
Permission List	0.96	0.95	Yes
Explicit Scope	1.42	0.89	No
Minimization	1.18	0.86	No
Intent Echo	1.74	0.96	No
Clarification	1.09	0.74	No

Table 2: Prompt-level intervention comparison, aggregated over all seven models and all episodes. Permission List is the only intervention that both reduces SIR and preserves TCR. It requires exact oracle knowledge and is best understood as a ceiling on scope control via external permissioning.

Figure 4: Intervention trade-offs between SIR reduction and task completion. Permission lists achieve substantial scope reduction while preserving TCR.

Figure 5: Permission list granularity. Excess calls increase approximately proportionally with the number of extra permitted tools, while TCR remains stable.

Figure 6: The SelfAudit pipeline. The agent justifies each planned tool call against the user’s words, then filters unjustified calls before execution.

Condition	SIR	TCR	PVS
Baseline	2.28	0.94	2.26
Justify-Only	2.42 $\uparrow$	0.96	2.46
Filter-Only	1.74	0.91	1.43
Full SelfAudit	1.68	0.89	1.29

Table 3: SelfAudit ablation on moderate and vague episodes. Justify-Only increases scope, Filter-Only captures most of the gain, and the full pipeline yields modest further reduction at the cost of lower TCR. Values are pooled identically to the aggregate results in Table 10 (Appendix F.2 ).

Model Pair	Jaccard $J$
Qwen3.6-Plus / Qwen3.7-Max	0.812
Qwen3.6-Plus / DeepSeek-v4-Pro	0.697
Qwen3.6-Plus / GLM-5.2	0.798
DeepSeek-v4-Pro / GLM-5.2	0.811
DeepSeek-v4-Pro / Kimi-K2.7-Code	0.748
GLM-5.2 / Kimi-K2.7-Code	0.819

Table 4: Pairwise Jaccard similarity on excess tool sets. Representative pairs are shown; the mean is computed over all 21 model pairs. Agreement is highest within model families but remains substantial across families.

Specificity	$n$	Unwanted	Neutral	Wanted
Precise	30	93.3%	6.7%	0.0%
Moderate	105	79.0%	13.3%	7.6%
Vague	205	63.9%	20.5%	15.6%
Overall	340	71.2%	17.0%	11.8%

Table 5: Human judgments on excess tool calls.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Domain	Mean SIR	Mean PVS
Smart Home	2.37	2.18
Social Media	1.93	1.62
Banking	1.86	2.03
Healthcare	1.53	1.00
Travel Booking	1.83	1.76
HR System	1.52	1.43

Appendix

Table 6: Per-domain SIR and PVS (baseline, all models averaged). PVS reflects both scope inflation and the density of PII categories accessed by excess tools.

Model	Condition	SIR	TCR	PVS
Qwen3.6-Flash	Baseline	1.50	0.95	0.94
	Permission List	0.99	0.99	0.00
	Explicit Scope	1.40	0.94	0.80
	Minimization	1.31	0.92	0.67
	Intent Echo	1.48	0.96	0.90
	Clarification	1.10	0.83	0.47

Appendix

Table 7: Full results for the Qwen family across intervention conditions.

Model	Condition	SIR	TCR	PVS
DeepSeek-v3	Baseline	1.18	0.80	0.60
	Permission List	0.83	0.83	0.00
	Explicit Scope	0.98	0.76	0.35
	Minimization	0.90	0.75	0.22
	Intent Echo	1.65	0.96	1.17
	Clarification	0.73	0.62	0.19

Appendix

Table 8: Full results for the DeepSeek, GLM, and Kimi families across intervention conditions.

Pool Size	Mean SIR	SD	$p$ vs. prev.
10	3.60	3.03	–
15	4.21	3.81	0.020
20	4.17	4.18	0.209

Appendix

Table 9: Ecological validation: SIR with extended schemas (

n{=}2{,}534

completed runs, two models, moderate and vague episodes). Saturation replicates at 15–20 tools with complex schemas.

Condition	SIR	TCR	PVS
Baseline	2.28 $\pm$ 0.04	0.94 $\pm$ 0.01	2.26 $\pm$ 0.07
SelfAudit	1.68 $\pm$ 0.03	0.89 $\pm$ 0.01	1.29 $\pm$ 0.06
$\Delta$	$-$ 26%	$-$ 5%	$-$ 43%

Appendix

Table 10: SelfAudit aggregate results (moderate and vague episodes, four high-inflation models; higher baseline reflects the subset and model selection). Mann-Whitney

p<0.001

, Cohen’s

d=0.36

Model	$\Delta$ SIR	$\Delta$ TCR	$\Delta$ PVS
DeepSeek-v4-Pro	$-$ 41%	$-$ 7%	$-$ 57%
GLM-5.2	$-$ 26%	$-$ 3%	$-$ 40%
Qwen3.6-Plus	$-$ 22%	$-$ 5%	$-$ 38%
Kimi-K2.7-Code	$-$ 18%	$-$ 3%	$-$ 35%
Qwen3.7-Max	$-$ 17%	$-$ 3%	$-$ 30%
Qwen3.6-Flash	$-$ 5%	$-$ 2%	$-$ 10%

Appendix

Table 11: Per-model SelfAudit effectiveness across all seven models. † DeepSeek-v3’s positive

\Delta

SIR and

\Delta

PVS reflect increased tool calling from a very low baseline; task completion rises alongside privacy exposure.

Explore similar work

ToolPrivacyBench: Benchmarking Purpose-Bound Privacy in Tool-Using LLM Agents
Jun 26, 2026Shijing Hu, Liang Liu, Zhu Meng +1Language-Model Agents Privacy
Beijing University of Posts and Telecommunications, Beijing, 100876, China · Beijing Academy of Blockchain and Edge Computing, Beijing, 100085, China
PrivacyPeek: Auditing What LLM-Based Agents Acquire, Not Just What They Say
May 29, 2026Mingxuan Zhang, Jiahui Han, Dadi Guo +5Privacy Attacker Large Language Model
Shanghai Artificial Intelligence Laboratory · Southeast University
When Lower Privileges Suffice: Investigating Over-Privileged Tool Selection in LLM Agents
Jun 18, 2026Kaiyue Yang, Yuyan Bu, Jingwei Yi +5Controlling Tool Use Large Language Model Agents
Institute of Information Engineering, Chinese Academy of Sciences · Beijing Academy of Artificial Intelligence · The Chinese University of Hong Kong +2

Jun 26, 2026, cs.CRJ/K move · Enter open · S save

ToolPrivacyBench: Benchmarking Purpose-Bound Privacy in Tool-Using LLM Agents

Shijing Hu, Liang Liu, Zhu Meng, Zhicheng Zhao

Beijing University of Posts and Telecommunications, Beijing, 100876, China · Beijing Academy of Blockchain and Edge Computing, Beijing, 100085, China

Language-Model Agents Privacy Information-Flow Control

Open paper AskPDF arXiv

Large language models (LLMs) have increasingly moved from standalone text generation systems to agents that invoke external tools, access environments, and execute multi-step tasks. However, conventional function-calling benchmarks mainly evaluate task completion and API correctness, while privacy evaluation benchmarks typically focus on final responses or privacy judgments. Neither perspective captures purpose-bound information flow across an executed multi-tool trajectory. Motivated by this limitation in current agent evaluation, ToolPrivacyBench audits whether task-private atoms are routed only to authorized tools and downstream sinks, thereby evaluating both task completion and privacy over-disclosure during tool use. The benchmark contains 2,150 cases, including 1,150 fully synthetic privacy-sensitive business workflows and 1,000 cases adapted from existing multi-tool and function-calling benchmarks. Each case is represented by a policy knowledge base. After an agent executes against mock business backends, the evaluator compares recorded tool arguments and backend audit logs with this policy knowledge base. The evaluation covers nine widely used agents to characterize purpose-bound privacy over-disclosure. The results show that successful tool execution does not imply appropriate privacy disclosure: an agent may complete a task while transmitting unnecessary private information through intermediate tool calls. ToolPrivacyBench therefore formalizes a need-to-know disclosure boundary, under which each tool should receive only the information necessary for its stated purpose, and uses trajectory-level auditing to identify privacy over-disclosure in multi-tool workflows.

Abstract

Explore similar work

ToolPrivacyBench: Benchmarking Purpose-Bound Privacy in Tool-Using LLM Agents

PrivacyPeek: Auditing What LLM-Based Agents Acquire, Not Just What They Say

When Lower Privileges Suffice: Investigating Over-Privileged Tool Selection in LLM Agents