This paper shifts the object of role engineering from human users to large language model (LLM) agents, and proposes PRISM to address the over-authorization that arises when an agent obtains full tool privileges through the Model Context Protocol (MCP). PRISM formulates whether a tool is necessary for a task as an intervenable causal proposition, and applies structured interventions in a sandbox to learn a counterfactual necessity model. It then compresses trajectories into discrete latent variables through an information bottleneck, and determines the number of roles from the knee of the rate-distortion curve. Finally, it formalizes pruning as a bilevel Stackelberg game, in which a generative flow network samples diverse near-optimal solutions. On three public benchmarks, AgentDojo, τ-bench and ToolEmu, PRISM compresses the number of authorized tools per role by 78.3%, attains a task success rate of 78.2%, reaches a counterfactual necessity accuracy of 88.7%, and lowers the blast radius to 15.9%.
Figures & tables
Figure 1: Overview of PRISM.
Figure 2: Role count from the rate-distortion knee.
Figure 3: Bilevel pruning game with diverse policy sampling.
Benchmark
#Tasks
#Tools
Avg. Steps
High Risk (%)
AgentDojo
76
63
5.6
28.6
τ -bench
165
29
7.5
41.4
ToolEmu
84
68
6.5
48.5
Overall
325
160
6.8
39.4
Table 1: Statistics of the three public benchmarks used in our evaluation. High Risk (%) is the proportion of tools that belong to the high-risk set H .
Method
Acc 1/2 (%)
Acc sparse (%)
Prec. (%)
Rec. (%)
AUROC
#Interv.
Time (min)
Frequency
66.8
55.4
61.5
75.8
0.712
0
4.3
LLM Prompting
72.1
64.9
68.5
79.2
0.768
0
9.8
Static Reachability
76.4
66.2
89.1
67.5
0.759
0
5.1
Shapley Sampling
83.5
75.3
81.3
87.9
0.874
6,144
83.2
PRISM (random)
85.2
76.8
83.5
90.3
0.903
1,024
14.4
PRISM
88.7
81.4
87.6
92.5
0.941
512
7.6
Table 2: Counterfactual necessity prediction, split by mask density. Acc 1/2 is accuracy on masks that remove one or two tools, Acc sparse is accuracy on sparse masks whose retention rate lies between 20% and 40%, and #Interv. is the number of sandbox interventions consumed. Time covers necessity modeling only.
Method
TSR (%)
#Roles
Tools/Role
Priv. Red. (%)
BR (%)
Hops
Time (min)
FullAccess
79.4
1.0
45.7
0.0
100.0
7.9
0.0
RoleMiner
64.2
18.3
21.4
53.2
58.6
5.4
9.2
GNRM
67.8
15.6
19.0
58.4
61.7
5.9
11.6
ToolEmu
70.9
n/a
17.4
61.9
44.9
4.3
18.7
GuardAgent
75.8
n/a
16.1
64.8
39.2
3.8
16.4
CaMeL
77.1
n/a
15.3
66.5
36.5
3.1
15.1
Table 3: Least-privilege quality of different methods. TSR is task success rate, Priv. Red. is the privilege reduction rate relative to FullAccess, BR is the blast radius defined in Equation (6), and Hops is the average number of tool hops an attacker needs to reach that radius. #Roles is averaged over 5 repetitions and is therefore not an integer. Time is end-to-end wall-clock time.
Variant
TSR (%)
#Roles
BR (%)
CNA (%)
Diversity
Time (min)
PRISM
78.2
6.7
15.9
88.7
0.68
12.9
w/o Causal Intervention
69.5
10.8
31.2
66.8
0.57
9.6
w/o Active Sampling
76.9
7.2
20.1
85.2
0.66
19.7
w/o Information Bottleneck
77.6
13.1
22.8
88.7
0.62
11.5
w/o Stackelberg Layer
77.8
6.7
27.3
88.7
0.65
10.9
w/o GFlowNet Sampling
77.4
6.7
18.6
88.7
0.19
11.2
Table 4: Ablation study of PRISM. TSR is task success rate, BR is blast radius, CNA is counterfactual necessity accuracy, and Diversity is the mean pairwise Jaccard distance among the candidate schemes delivered for one role.
Agentic LLMs exhibit three consequential tool-use failures: invalid arguments (validity), unnecessary calls (over-calling), and omitted calls when tools are needed (missing). We find that a small, failure-specific set of MLP neurons could distinguish such failures with linearly separable decision boundaries. Building on this observation, we introduce PRISMS (Probing Representations In Support of Monitoring and Steering), a closed-loop framework that shares a failure-specific neuron basis between sparse detection and activation steering. PRISMS selects contribution-critical MLP neurons and fits an L1-regularized detector on their activations. Across six models from the Qwen3, Llama, and Gemma families, over-calling and missing are detected at the pre-generation prompt boundary with ROC-AUC 0.90-1.00, while validity is detected from the generated tool-call span with ROC-AUC 0.86-0.90. These results are achieved with highly sparse readouts: only 1-2 MLP neurons for missing, 2-16 for over-calling, and approximately 128 for validity. These sparse detectors match or outperform dense residual-stream baselines using 23-627 times fewer features. The shared neuron basis also supports bidirectional control over tool-calling behavior, suppressing unnecessary calls and eliciting omitted ones. PRISMS therefore gates intervention on predicted failure risk to mitigate the collateral effects of unconditional steering. Across all six models, PRISMS reduces pooled over-calling rate by 80% (from 0.131 to 0.026) while increasing tool-required accuracy by 14.2 percentage points (from 0.689 to 0.831). PRISMS thus provides lightweight failure detection and selective intervention across model families.
As LLM agents increasingly select tools autonomously, their choices among tools with different privileges become safety-relevant. However, prior tool-selection studies focus on safety-agnostic metadata preferences, leaving privilege-sensitive choices underexplored. To address this gap, we study over-privileged tool selection, in which an agent selects or escalates to a higher-privilege tool despite a sufficient lower-privilege alternative. We introduce ToolPrivBench to evaluate whether agents choose higher-privilege tools despite sufficient lower-privilege alternatives, measuring both initial selection and escalation after transient tool failures. Across eight domains and five recurring risk patterns, we find that over-privileged tool selection is common among mainstream LLM agents and is further amplified by transient failures. We further find that general safety alignment does not reliably transfer to least-privilege tool choice, while prompt-level controls provide only limited mitigation under transient failures. We therefore introduce a privilege-aware post-training defense that teaches agents to prefer sufficient lower-privilege tools and escalate only when necessary. Our mitigation experiments show that this defense substantially reduces unnecessary high-privilege tool use while preserving general capabilities.
Kaiyue Yang, Yuyan Bu, Jingwei Yi +5
Institute of Information Engineering, Chinese Academy of Sciences · Beijing Academy of Artificial Intelligence · The Chinese University of Hong Kong +2
LLM agents with tool-calling capabilities can access external services and private user data, but they may retrieve more information than a user's request explicitly requires. We study this behavior in structured tool-calling agents and term it proactive over-authorization. This setting differs from filesystem-level coding agents because the main risk is unnecessary access to private data. We introduce OverAct, a controlled benchmark spanning eight privacy-sensitive domains with deterministic, judge-free scoring, together with an interpretive decision-theoretic framework that yields three testable predictions. Across seven models from four families, all models significantly exceed authorized scope. Request specificity is the strongest predictor of severity, over-authorization grows sublinearly with tool-pool size, and decoding temperature has little effect. These patterns are consistent with a cost-asymmetry account, suggesting that over-authorization arises more from structural decision tendencies than from decoding randomness. We also propose SelfAudit, a zero-shot inference-time method that generates request-grounded justifications and filters unjustified calls before execution. Ablation shows that explicit filtering is the main driver of scope reduction. SelfAudit reduces privacy-oriented excess by 43% without oracle knowledge.
Taolin Zhang, Jiuheng Wan, Hanyu Wang +2
Hefei University of Technology · East China Normal University · Alibaba Cloud Computing