ClawSentry: A Progressive Multi-Tier Security Monitor for Safeguarding Autonomous LLM Agents

Kai Wang; Zeming Wei; BiaoJie Zeng; Chang Jin; An Wang; Xiaokun Luan; Zhixiao Lin; Jingjing Qu

cs.CRAug 21, 2026

ClawSentry: A Progressive Multi-Tier Security Monitor for Safeguarding Autonomous LLM Agents

Authors: Kai Wang, Zeming Wei, BiaoJie Zeng, Chang Jin, An Wang, Xiaokun Luan, Zhixiao Lin, Jingjing Qu,

Organizations: Shanghai Artificial Intelligence Laboratory

Abstract

As large language model (LLM) agents move from conversation to executing code, reading local files, and orchestrating external tools, a single agent hijacked by a malicious third-party skill can cause data exfiltration, privilege escalation, or cascading compromise. We argue that agentic risk is progressive: it can enter at four loci of the agent control loop--skill admission, invocation-time intent, execution-time effect, and post-action consequence--while a denied dangerous objective can reappear across surface forms, tools, or turns; existing safeguards are typically local to one lifecycle boundary or one call. Guided by this threat model, we present ClawSentry, an open-source, framework-agnostic security supervision gateway for agent runtimes. Before a skill package is ever executed, First-use Skill Package Review (FSPR) audits it under a deterministic evidence floor, escalating unresolved cases to bounded read-only agentic review (locus A). At runtime, a three-tier progressive decision engine--a deterministic L1 layer, a rule-anchored L2 semantic reviewer, and a read-only L3 evidence-seeking agent--spends contextual review only on the residual ambiguity, while a session-level anti-bypass mechanism recognizes tool-switching and rephrased retries (loci B--C); a post-action path feeds high-severity evidence non-retroactively into later review (locus D). An Agent Harness Protocol (AHP) abstraction applies one policy across Codex, Claude Code, Kimi CLI, and Gemini CLI without modifying agent internals. On SkillInject with Codex/GPT-5.4, contextual ASR falls from 39.55% to 2.61% while contextual TSR moves only from 83.78% to 83.05%. Across five Work Agents on the full SkillsSafety benchmark, ClawSentry confines ASR to 9.09--15.03% from 33.5--49.7% unprotected, and aggregate TSR on clean skills remains 98.7%.

Figures & tables

Figure 1: ClawSentry aligns one gateway with four lifecycle loci. FSPR gates A; effect-normalized L1–L3 mediation covers B–C; retry memory catches capability-equivalent bypasses; and asynchronous D analysis informs later actions. AHP applies one policy across four frameworks; bottom badges report selective review and safety–utility.

System	A	B/C	Anti-	Frw.-	Agent
	(skill)	(gate)	bypass	agn.	bench.
LlamaGuard Inan et al. (2023)	✗	✗	✗	✓	✗
Spotlighting Hines et al. (2024)	✗	B only	✗	✓	✗
Rebuff Protect AI (2023)	✗	B only	✗	✓	✗
IsolateGPT Wu et al. (2025)	✗	C only	✗	✗	✗
DRIFT Li et al. (2025)	✗	✓	✗	✗	✓

Table 1: Coverage of representative prior agent defenses across risk loci, as reported in their respective papers. Columns: skill-admission review (A), runtime progressive gating (B/C), cross-attempt anti-bypass, framework-agnostic, and end-to-end agent benchmark. Among the systems surveyed here, ClawSentry is the only one reporting support across all axes; this is not an exhaustive comparison of the field.

Figure 2: The unprotected attack surface is structured rather than uniform. Per-domain SkillsSafety ASR for five Work Agent configurations, as published with the benchmark Jin et al. (2026) : RD1 contextual trust and RD2 agent permission are severe in every Work Agent family, while RD3 (execution and runtime) is markedly lower; a single aggregate rate would average across regimes that no individual domain exhibits, which is why the evaluation breaks results down by domain. Domain definitions are in Appendix B .

Figure 3: Observation is not recognition. After a destructive effect on a credential path is denied, the same capability and target can re-enter through rewritten arguments to the same tool, a sequence distributed across turns, or a different tool surface. A per-call defense may observe every resulting invocation at its boundary while still evaluating each in isolation. ClawSentry instead normalizes the current call by capability and target and compares it with session-scoped denied-effect memory. High-confidence equivalents follow the configured policy; uncertain cross-tool candidates are routed to L3 (§ 6.4 ).

Constraint	Mechanism	Evidence
Bounded exposure: cost and latency forbid reviewing every event	L1 $\rightarrow$ L2 $\rightarrow$ L3 selective-escalation funnel	13.8% of L1 volume; L3 alone 1.37% (§ 7.7 )
No authority to act: the reviewer is itself an attack surface and must never become an actuator	Read-only toolkits for FSPR and L3; analyzers supply evidence, the gateway holds policy authority	§ 6.1 , § 6.2
Extensible priors: reviewer competence must not be capped by one model’s priors	Versioned YAML attack-pattern rules; L3 review skills as YAML manifests validated against the global read-only whitelist	§ 6.2 , Appendix A
Supervision must not be bound to one model family	Reviewer substitution across providers	ASR 11.11–17.24% over five providers (§ 7.6 )

Table 2: Constraints on using frontier security capability defensively, the ClawSentry mechanism each produces, and where the paper establishes it.

Figure 4: AHP places one policy engine behind four heterogeneous event streams. Each adapter normalizes its framework’s pre-action hooks, tool executions, and post-action results into a CanonicalEvent , keeping what does not generalize in an isolated framework_meta field, so the safety core sees only canonical events. Pre-action events (loci A–C) travel over a synchronous local UDS that blocks execution until the gateway returns Allow, Defer, or Block; post-action events (locus D) stream asynchronously over HTTP webhooks for non-blocking audit. Adding a framework costs an adapter and nothing in the gateway.

Dimension	Evaluation Focus	High-Risk Example
D1: Tool-Type Danger	Intrinsic capability of the invoked tool.	A shell or execution tool operating on a system resource.
D2: Target Sensitivity	Security role of the normalized target path or resource.	Accessing /etc/shadow , .ssh/ , or .env .
D3: Command-Pattern Danger	Destructive or high-impact structure within a shell/exec command.	Recursive deletion, privilege modification, or an encoded eval .
D4: Session Accumulation	Recent high-risk decisions in the current session.	Several high-risk actions preceding a staged archive-and-egress attempt.
D5: Agent Trust Risk	Trust level supplied by gateway-owned agent/workspace context.	An untrusted or missing-trust execution context.
D6: Injection Signal	Heuristic instruction-manipulation evidence, including content origin.	An external document containing hidden override instructions or zero-width controls.

Table 3: The six detector dimensions used by the runtime risk snapshot. These dimensions are distinct from the benchmark’s RD1–RD6 attack domains.

Figure 5: The decision path a canonical event traverses; most events close at L1 and only the residual escalates. Normalization to an effect envelope precedes every other step, so one capability presented through different surfaces is adjudicated as one effect. The follow-up guard is consulted before ordinary tier selection, and its recognizer activates a gateway-configured review route rather than choosing an outcome. L1 closes clear cases through deterministic short circuits without a provider call; L2 may raise L1 risk freely but lowering it requires separately validated effect-bound clearance, and a hard finding remains in force regardless; L3 inspects bounded evidence through a toolkit with no execute, write, or network authority, its per-skill allowlist intersected with the gateway’s global read-only whitelist. Throughout, analyzers supply evidence and the gateway holds policy authority.

Figure 6: The security–utility asymmetry the gateway is built to produce. (a) SkillsSafety ASR for five Work Agents, unprotected (open) versus protected (filled), with exact values at both endpoints; full results are in Table 5 , denominators in Appendix B . (b) Clean-skill task success on 58 clean tasks per Work Agent: raw TSR is 100% for all four configurations and protected TSR remains at 98.2–100%, an aggregate of 98.7%. The horizontal scale in (b) is magnified.

Work Agent	Cond.	Obv. ASR $\downarrow$	Ctx. ASR $\downarrow$	Ctx. TSR $\uparrow$
	Raw	18.75	39.55	83.78
Codex/GPT-5.4	CS	4.47	2.61	83.05
	Raw	29.38	39.86	87.01
Codex/GPT-5.5	CS	4.09	8.40	84.75
	Raw	31.10	53.91	81.94
CC/GLM-5.1	CS	4.49	14.29	76.27

Table 4: SkillInject results (%); CS denotes ClawSentry, CC Claude Code, and light-blue rows mark protected conditions. Bold marks the best protected value per column. The 180-case obvious subset has no legitimate-task objective; the 139-case contextual subset reports both ASR and TSR.

Work Agent	ASR $\downarrow$		TSR $\uparrow$
	Raw	+CS	Raw	+CS
Codex/GPT-5.4	44.5	12.50	38.7	25.66
Codex/GPT-5.5	41.8	15.03	42.6	30.07
CC/GLM-5.1	33.5	12.40	37.4	23.14
CC/MiniMax-2.7	34.2	9.09	20.6	15.58
Kimi CLI/K2.5	49.7	10.20	31.0	16.33

Table 5: SkillsSafety aggregate results (%); CC denotes Claude Code. Raw columns are the published benchmark baselines Jin et al. (2026) . Both TSR columns are measured with the poisoned package present; § 7.4 decomposes that operating point.

Figure 7: Fixed-denominator task-success diagnostic on the 60 sampled Codex/GPT-5.4 SkillsSafety cases; the dashed line is the no-skill baseline. Conditions A–C contain no ClawSentry component and isolate the attack-side effect: a clean skill adds 6.7 points over that baseline, whereas poisoning removes 18.3 points before enforcement is introduced. D is the protected hard-block operating point, a further 6.7 points below C. Bar interiors give full-task-success counts.

Figure 8: The same four conditions broken down across the six risk domains, ten cases each. The aggregate pattern is not carried by a single domain: a clean skill raises the pass count above the no-skill baseline in RD2, RD5, and RD6, whereas poisoning lowers it relative to the clean skill in all six domains. RD1 shows the sharpest contrast, 8/10 with a clean skill versus 3/10 once the same class of skill is poisoned, with ClawSentry disabled in both. The enforcement margin from C to D is likewise concentrated rather than uniform. Per-domain counts are tabulated in Appendix C .

Figure 9: FSPR discrimination on 48 toxic and 53 clean packages. (a) Toxic-package recall for two diagnostic package-review backends. (b) Clean-package acceptance under the same evaluation. Labels report exact counts and percentages; orange segments denote missed toxic packages or false-blocked clean packages.

Study / condition	ASR $\downarrow$	TSR $\uparrow$
FSPR response to a flagged package (CC/MiniMax-M3)
Hard block (default)	8.47	18.64
Warning	24.56	21.05
Runtime feedback	10.53	17.54
Defense layer (CC/MiniMax-M3)
ClawSentry	8.62	18.97

Table 6: Controlled comparisons (%). Each block is an independent study on its own 60-case SkillsSafety subset with Work Agent, reviewer ( gemini-3.5-flash ), and remaining settings fixed, so values compare within a block, not across. Bold marks each block’s lowest ASR; Figure 10 plots each block against its TSR operating point.

Figure 10: Controlled security comparisons on fixed 60-case SkillsSafety subsets. (a) FSPR response policy: hard block attains the lowest ASR of the three responses, 2.06 points below runtime feedback and 16.09 points below a document-level warning. (b) Matched defense baseline: ClawSentry reduces ASR by 48.52 points relative to Guardrails. (c) Component ablation: removing FSPR or anti-bypass raises ASR by 11.00 and 6.57 points, respectively. Gray annotations report the corresponding TSR operating points.

Figure 11: Reviewer substitution and review volume, with Work Agent, policy, and sample fixed. (a) Five reviewers share a compact protected ASR–TSR region. (b) The aggregate progressive funnel across 300 provider–case executions: of 8,310 L1 events, 1,036 reach L2 and 114 reach L3. (c) Total Gateway review stays at 12.2–15.5% of L1 event volume across reviewers, compared with a 13.8% aggregate.

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 12: Resolved L1 score parameters for the high-security benchmark preset.

Figure 13: An excerpt of a customizable YAML attack pattern used by the L2 RuleBasedAnalyzer .

Figure 14: Schematic operator-defined L3 review skill aligned with the runtime manifest schema. A skill specializes triggering, evidence inspection, and review criteria while remaining within the gateway’s global read-only tool boundary.

Raw Work Agent	RD1	RD2	RD3	RD4	RD5	RD6	Overall ASR	TSR
Codex / GPT-5.4	64.0	52.0	30.8	51.9	46.2	23.1	44.5	38.7
Codex / GPT-5.5	64.0	48.0	11.5	48.1	53.8	26.9	41.8	42.6
Claude Code / GLM-5.1	36.0	44.0	19.2	44.4	38.5	19.2	33.5	37.4
Claude Code / MiniMax-2.7	48.0	48.0	15.4	37.0	38.5	19.2	34.2	20.6
Kimi CLI / Kimi-K2.5	76.0	72.0	26.9	55.6	50.0	19.2	49.7	31.0

Appendix

Table 7: Per-domain SkillsSafety raw baselines (%), as published with the benchmark Jin et al. (2026) . RD columns and overall are ASR (lower is better); TSR is benign task completion (higher is better).

Protected Work Agent	N	AS	TS	ASR $\downarrow$	TSR $\uparrow$
Codex / GPT-5.4	152	19	39	12.50	25.66
Codex / GPT-5.5	153	23	46	15.03	30.07
Claude Code / GLM-5.1	121	15	28	12.40	23.14
Claude Code / MiniMax-2.7	154	14	24	9.09	15.58
Kimi CLI / Kimi-K2.5	147	15	24	10.20	16.33

Appendix

Table 8: Counts behind the five protected SkillsSafety aggregate results (%). N is the evaluable-case denominator after applying the common exclusion rule in § 7.1 . All protected rows use gemini-3.5-flash for the independent Gateway and FSPR review paths.

Risk domain	Cases	A: no skill	B: clean	C: poisoned	D: +CS
RD1: contextual trust / prompt manipulation	10	9/10	8/10	3/10	5/10
RD2: agent permission / authorization	10	2/10	4/10	2/10	2/10
RD3: execution / runtime / protocol	10	4/10	4/10	3/10	0/10
RD4: data boundary / output exfiltration	10	5/10	5/10	4/10	4/10
RD5: memory / recovery / audit	10	4/10	5/10	4/10	2/10
RD6: knowledge model / supply chain / operations	10	2/10	4/10	3/10	2/10

Appendix

Table 9: Per-domain full-task-success counts for the fixed-denominator four-condition comparison. A–C carry no ClawSentry component; D applies the hard-block policy. The security benefit of D is evaluated by ASR elsewhere; TSR here characterizes the poisoned-package utility operating point.

Reviewer model	N	AS	TS	ASR $\downarrow$	TSR $\uparrow$
gemini-3.5-flash	55	7	14	12.73	25.45
minimax-m2.7	54	6	15	11.11	27.78
deepseek-v4-pro	58	10	13	17.24	22.41
minimax-m3	57	7	14	12.28	24.56
glm-5.1	58	10	10	17.24	17.24

Appendix

Table 10: System-level reviewer substitution on 60 attempted SkillsSafety cases (%). N is the evaluable-case denominator. The reviewer model supplies safety analysis only; the Work Agent is Codex/GPT-5.4 in every row.

Reviewer model	L1	L2	L3	L2+L3	Gateway/L1	FSPR	Anti-bypass
gemini-3.5-flash	1,646	233	22	255	15.5%	127	112
minimax-m2.7	1,591	210	16	226	14.2%	134	77
deepseek-v4-pro	1,661	182	28	210	12.6%	165	86
minimax-m3	1,669	220	26	246	14.7%	124	105
glm-5.1	1,743	191	22	213	12.2%	139	103
Total	8,310	1,036	114	1,150	13.8%	689	483

Appendix

Table 11: Observed request and trigger volume across five reviewer substitutions and 300 provider–case executions. L3 denotes Gateway L3 only; FSPR remains a separate first-use package review.

AI Agent Security

Artificial intelligence agent security studies attacks and safeguards for tool-using, persistent, and collaborative systems across digital and physical settings.

See all 510 papers

Jun 8, 2026

SecureClaw: Clawing Back Control of LLM Agents

Tool-using large language model (LLM) agents face two distinct security failures: unauthorized external actions and exposure of sensitive plaintext inside the runtime before any final output check can intervene. Existing defenses usually protect one boundary, either the planner/runtime or the point where external actions are executed, and therefore do not by themselves secure both surfaces. We present SecureClaw, a dual-boundary architecture that enforces authorization in a trusted executor and keeps raw protected data outside the LLM runtime. Sensitive reads pass through a trusted gateway that replaces raw values with reference IDs. In the evaluated deployment, the gateway also provides short planning summaries that deliberately release selected information, a process called declassification. Writes that change external state follow a PREVIEW

\rightarrow

COMMIT protocol in which only a trusted executor may execute the exact request authorized by policy. The runtime can still plan over summaries and reference IDs, but cannot directly resolve the IDs into secrets or perform side effects. Across the AgentDojo, AgentLeak, and Agent Security Bench (ASB) benchmarks, SecureClaw is the only defense we evaluate in a common harness that simultaneously retains usable task utility and achieves 0% attack success rate (ASR) on ASB, 0.64% ASR on AgentDojo, and 3.23% overall leak on AgentLeak's attacked parity lane, which measures final-output and internal-relay leakage. Code is available at https://github.com/ponyyuhan/SecureClaw.

Yuhan Ma, Stefan Schmid

TU Berlin

Data Leakage LLM Agent Security Privacy Leakage in Language Models Tool Access Control for LLM Agents Privacy Protection for LLM Agents

Ask PDF

Jun 10, 2026

Runtime Skill Audit: Targeted Runtime Probing for Agent Skill Security

Agent skills let LLM agents reuse instructions, resources, tools, and workflows, but they also create a new place for malicious behavior to hide. A skill may look benign in its documentation or code while becoming harmful only when it is invoked with particular user requests, local assets, persistent state, or multi-step tool interactions. This makes purely static vetting brittle. We present Runtime Skill Audit (RSA), a dynamic analysis method that audits skills by asking what the skill-mediated agent actually does under targeted runtime conditions. Instead of testing every skill with the same generic tasks, RSA profiles risk-relevant interfaces, prepares the execution context needed to exercise them, and assigns security labels from the resulting trace evidence. We instantiate RSA on OpenClaw and evaluate it on 100 skills against representative static baselines. RSA achieves 90.0% accuracy with an 88.0% true positive rate and an 8.0% false positive rate, improving accuracy by 13.0 percentage points over the best static baseline. Under self-evolving attacks, static detectors collapse after one or two rounds, while RSA continues to detect 19--20 out of 20 malicious skills across rounds.

Tu Lan, Chaowei Xiao

Johns Hopkins University

AI Agent Auditing LLM Agent Security Agent Skill Security Auditing

Ask PDF

Apr 1, 2026

ClawSafety: "Safe" LLMs, Unsafe Agents

Personal AI agents like OpenClaw run with elevated privileges on users' local machines, where a single successful prompt injection can leak credentials, redirect financial transactions, or destroy files. This threat goes well beyond conventional text-level jailbreaks, yet existing safety evaluations fall short: most test models in isolated chat settings, rely on synthetic environments, and do not account for how the agent framework itself shapes safety outcomes. We introduce CLAWSAFETY, a benchmark of 120 adversarial test scenarios organized along three dimensions (harm domain, attack vector, and harmful action type) and grounded in realistic, high-privilege professional workspaces spanning software engineering, finance, healthcare, law, and DevOps. Each test case embeds adversarial content in one of three channels the agent encounters during normal work: workspace skill files, emails from trusted senders, and web pages. We evaluate five frontier LLMs as agent backbones, running 2,520 sandboxed trials across all configurations. Attack success rates (ASR) range from 40% to 75% across models and vary sharply by injection vector, with skill instructions (highest trust) consistently more dangerous than email or web content. Action-trace analysis reveals that the strongest model maintains hard boundaries against credential forwarding and destructive actions, while weaker models permit both. Cross-scaffold experiments on three agent frameworks further demonstrate that safety is not determined by the backbone model alone but depends on the full deployment stack, calling for safety evaluation that treats model and framework as joint variables. Code and data will be available at: https://weibowen555.github.io/ClawSafety/.

Bowen Wei, Yunbei Zhang, Jinhao Pan +5

George Mason University · Tulane University · Oak Ridge National Laboratory +1

AI Agent Security AI Agent Safety Prompt Injection Attacks on AI Agents AI Agent Safety Benchmarks

Ask PDF

Abstract

AI Agent Security

Explore similar work

SecureClaw: Clawing Back Control of LLM Agents

Runtime Skill Audit: Targeted Runtime Probing for Agent Skill Security

ClawSafety: "Safe" LLMs, Unsafe Agents