Least-Privilege Auditing of Agent Tool Calls with Intent Graphs and Reasoning-Guided Escalation
Abstract
LLM agents that call tools are exposed to indirect prompt injection: instructions hidden in tool outputs can steer an agent into calls the user never asked for. We present a runtime audit that derives an executable intent graph from the user's objective and the deployment's tool catalog with one LLM call per task, and checks every tool call against it with a deterministic loop that makes no LLM call (median 5.7 microseconds per call). Two further mechanisms recover utility. Lookups of the user's own state bypass the audit, because the harm of an attack lies in its write or outward step. The calls that remain flagged are escalated to a reasoning guardian, in the spirit of tool-integrated reasoning models such as START: a reasoning model that sees only the objective, the plan, the call history without tool outputs, and the flagged call, and can verify facts with a restricted Python-expression tool. A current agent showed no baseline attack success, so end-to-end validation remains open.
Figures & tables
| Policy | ASR (first) [95% CI] | ASR (harmful) [95% CI] | Benign calls flagged | Benign tasks flagged [95% CI] | Benign tasks blocked |
|---|---|---|---|---|---|
| No defense | 0.457 [0.376, 0.532] | 0.457 [0.376, 0.532] | – | – | – |
| Tool allowlist | 0.102 [0.068, 0.138] | 0.096 [0.059, 0.135] | 0.101 | 0.162 [0.096, 0.237] | 0.162 |
| Scope-only | 0.055 [0.032, 0.080] | 0.043 [0.021, 0.068] | 0.173 | 0.329 [0.237, 0.425] | 0.162 |
| Ordering-only | 0.081 [0.050, 0.114] | 0.073 [0.040, 0.110] | 0.150 | 0.285 [0.202, 0.377] | 0.285 |
| Full Phase 2 | 0.033 [0.017, 0.050] | 0.020 [0.007, 0.036] | 0.250 | 0.404 [0.307, 0.504] | 0.311 |
| Allowlist + read bypass | 0.178 [0.132, 0.229] | 0.096 [0.059, 0.135] | 0.034 | 0.088 [0.035, 0.145] | 0.088 |
| System | Evaluation | ASR [95% CI] | Benign utility [95% CI] | |
|---|---|---|---|---|
| No defense | end-to-end | 0.457 [0.376, 0.532] | 0.671 [0.566, 0.776] | |
| Spotlighting (delimiting) | end-to-end | 0.402 [0.324, 0.471] | 0.697 [0.592, 0.803] | |
| Repeat user prompt | end-to-end | 0.254 [0.189, 0.314] | 0.829 [0.737, 0.908] | |
| PI detector (classifier) | end-to-end | 0.079 [0.048, 0.114] | 0.421 [0.316, 0.526] | |
| Tool filter (LLM) | end-to-end | 0.065 [0.040, 0.093] | 0.697 [0.592, 0.803] | |
| Tool allowlist (ours) | replay | 0.096 [0.059, 0.135] | 0.601 [0.491, 0.706] | 0.601 [0.491, 0.706] |
| Configuration | ASR [95% CI] | Benign calls flagged | Benign tasks flagged [95% CI] | Benign tasks blocked |
|---|---|---|---|---|
| Audit only | 0.041 [0.016, 0.073] | 0.265 [0.163, 0.375] | 0.408 [0.303, 0.526] | 0.329 |
| + guardian (independent overrides) | 0.041 [0.016, 0.073] | 0.182 [0.109, 0.263] | 0.368 [0.263, 0.474] | 0.289 |
| + guardian (state updated) | 0.041 [0.016, 0.073] | 0.175 [0.103, 0.256] | 0.342 [0.237, 0.461] | 0.263 |
| Configuration | ASR (harmful) [95% CI] | Benign calls flagged | Benign tasks flagged [95% CI] | Benign tasks blocked |
|---|---|---|---|---|
| Full design | 0.030 [0.006, 0.064] | 0.265 | 0.408 [0.303, 0.526] | 0.329 |
| + read bypass | 0.030 [0.006, 0.064] | 0.149 | 0.329 [0.224, 0.434] | 0.250 |
| + guardian | 0.030 [0.006, 0.064] | 0.109 | 0.276 [0.171, 0.382] | 0.197 |
| Scope-only design | 0.053 [0.022, 0.089] | 0.175 | 0.329 [0.224, 0.434] | 0.158 |
| + read bypass | 0.053 [0.022, 0.089] | 0.102 | 0.276 [0.184, 0.382] | 0.092 |
| + guardian | 0.053 [0.022, 0.089] | 0.076 | 0.211 [0.118, 0.303] | 0.079 |
| Detector | Splice | Drift | Hijack |
|---|---|---|---|
| Phase 2 verdict | 0.053 [0.047, 0.059] | 0.050 [0.043, 0.057] | 0.762 [0.718, 0.801] |
| Flag persistence (10) | 0.058 [0.052, 0.065] | 0.058 [0.053, 0.065] | 0.120 [0.111, 0.128] |
| Online per-session | 0.119 [0.100, 0.136] | 0.149 [0.131, 0.166] | 0.114 [0.085, 0.142] |
| Online per-session | 0.119 [0.100, 0.136] | 0.150 [0.131, 0.167] | 0.114 [0.086, 0.141] |
| Cross-session | 0.242 [0.199, 0.288] | 0.128 [0.088, 0.161] | 0.072 [0.037, 0.112] |
| Cross-session | 0.195 [0.157, 0.235] | 0.091 [0.060, 0.121] | 0.121 [0.078, 0.164] |
AI Agent Auditing
AI agent auditing examines agents’ actions, tool use, evidence, and outcomes to assess safety, compliance, reliability, and accountability.
See all 312 papers