This paper shifts the object of role engineering from human users to large language model (LLM) agents, and proposes PRISM to address the over-authorization that arises when an agent obtains full tool privileges through the Model Context Protocol (MCP). PRISM formulates whether a tool is necessary for a task as an intervenable causal proposition, and applies structured interventions in a sandbox to learn a counterfactual necessity model. It then compresses trajectories into discrete latent variables through an information bottleneck, and determines the number of roles from the knee of the rate-distortion curve. Finally, it formalizes pruning as a bilevel Stackelberg game, in which a generative flow network samples diverse near-optimal solutions. On three public benchmarks, AgentDojo, τ-bench and ToolEmu, PRISM compresses the number of authorized tools per role by 78.3%, attains a task success rate of 78.2%, reaches a counterfactual necessity accuracy of 88.7%, and lowers the blast radius to 15.9%.
Figures & tables
Figure 1: Overview of PRISM.
Figure 2: Role count from the rate-distortion knee.
Figure 3: Bilevel pruning game with diverse policy sampling.
Benchmark
#Tasks
#Tools
Avg. Steps
High Risk (%)
AgentDojo
76
63
5.6
28.6
τ -bench
165
29
7.5
41.4
ToolEmu
84
68
6.5
48.5
Overall
325
160
6.8
39.4
Table 1: Statistics of the three public benchmarks used in our evaluation. High Risk (%) is the proportion of tools that belong to the high-risk set H .
Method
Acc 1/2 (%)
Acc sparse (%)
Prec. (%)
Rec. (%)
AUROC
#Interv.
Time (min)
Frequency
66.8
55.4
61.5
75.8
0.712
0
4.3
LLM Prompting
72.1
64.9
68.5
79.2
0.768
0
9.8
Static Reachability
76.4
66.2
89.1
67.5
0.759
0
5.1
Shapley Sampling
83.5
75.3
81.3
87.9
0.874
6,144
83.2
PRISM (random)
85.2
76.8
83.5
90.3
0.903
1,024
14.4
PRISM
88.7
81.4
87.6
92.5
0.941
512
7.6
Table 2: Counterfactual necessity prediction, split by mask density. Acc 1/2 is accuracy on masks that remove one or two tools, Acc sparse is accuracy on sparse masks whose retention rate lies between 20% and 40%, and #Interv. is the number of sandbox interventions consumed. Time covers necessity modeling only.
Method
TSR (%)
#Roles
Tools/Role
Priv. Red. (%)
BR (%)
Hops
Time (min)
FullAccess
79.4
1.0
45.7
0.0
100.0
7.9
0.0
RoleMiner
64.2
18.3
21.4
53.2
58.6
5.4
9.2
GNRM
67.8
15.6
19.0
58.4
61.7
5.9
11.6
ToolEmu
70.9
n/a
17.4
61.9
44.9
4.3
18.7
GuardAgent
75.8
n/a
16.1
64.8
39.2
3.8
16.4
CaMeL
77.1
n/a
15.3
66.5
36.5
3.1
15.1
Table 3: Least-privilege quality of different methods. TSR is task success rate, Priv. Red. is the privilege reduction rate relative to FullAccess, BR is the blast radius defined in Equation (6), and Hops is the average number of tool hops an attacker needs to reach that radius. #Roles is averaged over 5 repetitions and is therefore not an integer. Time is end-to-end wall-clock time.
Variant
TSR (%)
#Roles
BR (%)
CNA (%)
Diversity
Time (min)
PRISM
78.2
6.7
15.9
88.7
0.68
12.9
w/o Causal Intervention
69.5
10.8
31.2
66.8
0.57
9.6
w/o Active Sampling
76.9
7.2
20.1
85.2
0.66
19.7
w/o Information Bottleneck
77.6
13.1
22.8
88.7
0.62
11.5
w/o Stackelberg Layer
77.8
6.7
27.3
88.7
0.65
10.9
w/o GFlowNet Sampling
77.4
6.7
18.6
88.7
0.19
11.2
Table 4: Ablation study of PRISM. TSR is task success rate, BR is blast radius, CNA is counterfactual necessity accuracy, and Diversity is the mean pairwise Jaccard distance among the candidate schemes delivered for one role.
Institute of Information Engineering, Chinese Academy of Sciences · Beijing Academy of Artificial Intelligence · The Chinese University of Hong Kong +2