Threat-Preserving Representation Sensitivity in Agent-Security Benchmarks

Neeraj Karamchandani; Piyush Nagasubramaniam; Xinhong Xie; Sencun Zhu; Dinghao Wu

cs.CROct 2, 2026

Threat-Preserving Representation Sensitivity in Agent-Security Benchmarks

Authors: Neeraj Karamchandani, Piyush Nagasubramaniam, Xinhong Xie, Sencun Zhu, Dinghao Wu

Organizations: The Pennsylvania State University University Park, PA, USA

Abstract

Security benchmarks for LLM-based agents often report the attack success rate (ASR) as a measure of model robustness and use these scores to compare different models and defense mechanisms, assuming that they describe the security of the agent. In this paper, we explore whether it also influences the benchmark's measurement. To measure the effect of the benchmark representation, we introduce threat-preserving representation sensitivity (TPRS), which measures how much the ASR changes when we change the agent-visible representation while holding the underlying task, harmful action, security policy, ground truth, environment, and the evaluation criteria fixed. On Agent Security Bench (ASB), replacing threat-related tool names with threat-neutral names raises the committed attack success rate by 11.67 percentage points on GPT-5-mini and by 13.21 points on Claude Haiku 4.5. On MCPTox, replacing the original neutral tool name with an explicit threat-related name lowers the ASR by 11.00 percentage points on GPT-5-mini and 4.11 points on Claude Haiku 4.5. On AgentDojo, adding threat-related wording to the attack-relevant tool changes ASR by only 0.50 percentage points on GPT-4o-mini, yet the benign utility falls by 5.36 points on tasks requiring that tool. We ran an experiment on MCPTox where we observed that a threat-neutral name matched on token count, length, and casing reproduces most of the shift produced by the threat-explicit name (8.54 of 11.00 points on GPT-5-mini). The results show that a security score measured under one representation may fail to generalize across threat-preserving representations of the same security problem. Robustness claims should therefore be supported by performance across a controlled set of threat-preserving representations rather than relying on a single representation-dependent score.

Figures & tables

Fig. 1: Overview of threat-preserving representation sensitivity (TPRS). (a) An agent-visible representation is transformed while security-relevant benchmark components remain fixed. (b) TPRS is the resulting change in attack success rate, $\Delta_{T}(B,m)=\operatorname{ASR}(T(B),m)-\operatorname{ASR}(B,m)$ . The transformed representation appears in different locations in ASB, MCPTox, and AgentDojo.

Benchmark	Attack setup	What we change	What remains fixed
ASB	A distinctly named malicious tool is presented to the agent.	Threat-related tool names or descriptions are replaced with more neutral wording.	Attack goal, task, environment, and evaluator.
MCPTox	A malicious instruction is placed in the description of a neutrally named tool.	The neutral tool name is replaced with threat-explicit or matched neutral names.	Poisoned description, requested action, task, and evaluator.
AgentDojo	Injected tool output tries to make the agent misuse an ordinary tool.	An ordinary tool name and description is replaced with a threat-explicit wording.	Injected content, task, initial environment, and security scorer.

TABLE I: Threat-preserving transformations across the three benchmarks.

Arm	Transformation	Type
A0	Original	Baseline
NULL2	Orthographic rename	Orthographic
A1	Neutralize tool names	Cue-altering
A1b	Neutralize descriptions	Cue-altering
A3	Neutralize names & descriptions	Cue-altering
A2	Make task specification concrete	Diagnostic

TABLE II: ASB representation arms.

Arm	Transformation	Type
N0	Original	Baseline
O1/O2/NULL2	Case/separator changes	Orthographic
S1/S2	Neutral alternative names	Semantic-neutral
S3	Neutral name matched to N1	Semantic-neutral
C1	Conservative threat wording	Cue-altering
N1	Explicit threat-category name	Cue-altering

TABLE III: MCPTox representation arms.

Arm	Transformation	Type
D0	Original	Baseline
Dnull	Orthographic rename	Orthographic
Dval	Threat wording on required tool	Cue-altering
Dirrelevant	Threat wording on unrelated tool	Cue-altering

TABLE IV: AgentDojo representation arms.

Fig. 2: Representation sensitivity across the three benchmarks. Points show $\Delta$ ASR in percentage points relative to each benchmark’s original representation (committed ASR for ASB); whiskers show 95% cluster-bootstrap confidence intervals. Marker shapes distinguish models, while gray, black, and red denote orthographic, semantic-neutral, and cue-altering transformations, respectively. Each benchmark is analyzed independently; the common axis is used only to visualize effect direction and magnitude.

	GPT-5-mini		Haiku 4.5
Arm	Committed	Native	Committed	Native
NULL2 (orthographic)	$-1.67$ $[-4.44,+0.74]$	$-0.93$ $[-3.33,+1.67]$	$-0.74$ $[-3.46,+1.66]$	$+0.37$ $[-2.03,+2.71]$
A1b (descriptions)	$+0.00$ $[-2.96,+2.96]$	$+3.15$ $[-0.19,+7.22]$	$-1.16$ $[-4.60,+1.94]$	$\mathbf{+3.28}$ $[+0.14,+8.59]$
A1 (names)	$\mathbf{+11.67}$ $[+4.44,+19.07]$	$\mathbf{+14.07}$ $[+5.56,+23.70]$	$\mathbf{+13.21}$ $[+6.33,+21.10]$	$\mathbf{+13.05}$ $[+6.26,+21.03]$
A3 (names + descriptions)	$\mathbf{+13.15}$ $[+5.56,+21.11]$	$\mathbf{+17.22}$ $[+7.59,+27.96]$	$\mathbf{+14.85}$ $[+8.31,+22.49]$	$\mathbf{+17.14}$ $[+9.41,+25.90]$

TABLE V: ASB representation effects. Values are percentage-point changes in ASR relative to the original benchmark (A0), with 95% cluster-bootstrap confidence intervals.

Failure treatment	GPT-5-mini	Haiku 4.5
Drop failed runs	$+11.67$ $[+4.44,+19.07]$	$+13.21$ $[+6.33,+21.10]$
Failure = not committed	$+11.67$ $[+4.44,+19.07]$	$+12.04$ $[+5.74,+19.07]$
Failure = committed	$+11.67$ $[+4.44,+19.07]$	$+13.89$ $[+6.85,+21.85]$

TABLE VI: Robustness of the ASB A1 committed ASR effect to alternative treatments of execution failures. Values are percentage-point changes relative to A0 with 95% cluster-bootstrap confidence intervals.

Arm	Compared with	GPT-5-mini	Haiku 4.5
O1 (orthographic)	N0	$-0.99$ $[-4.27,+2.30]$	$+0.33$ $[-1.64,+2.30]$
NULL2 (orthographic) †	N0	$-3.12$ $[-6.40,+0.16]$	$+1.02$ $[-0.66,+2.85]$
O2 (orthographic)	N0	$+1.31$ $[-2.30,+4.93]$	$-0.99$ $[-3.12,+0.99]$
S1 (neutral)	N0	$-2.96$ $[-7.22,+1.31]$	$-1.48$ $[-4.60,+1.48]$
S2 (neutral)	N0	$-2.30$ $[-6.73,+1.97]$	$-1.64$ $[-4.93,+1.31]$
S3 (matched neutral)	N0	$\mathbf{-8.54}$ $[-12.48,-4.76]$	$-0.16$ $[-3.94,+3.61]$

TABLE VII: MCPTox representation effects. Values are percentage-point changes in ASR with 95% cluster-bootstrap confidence intervals over cases. N0 is the original benchmark. Negative values indicate lower attack success.

Security: $\Delta$ ASR
Outcome / intervention	Effect	95% CI
Dnull: orthographic	$-0.88$	$[-3.02,+1.25]$
Dval: required tool	$\mathbf{-0.50}$	$\mathbf{[-3.65,+2.51]}$
Dirrelevant: unrelated tool	$+0.88$	$[-1.00,+2.76]$
Benign utility under Dval
Requires renamed tool	$-5.36$	$[-10.00,-0.71]$

TABLE VIII: AgentDojo effects on security and benign utility. Values are percentage-point changes relative to D0 with 95% cluster-bootstrap confidence intervals.

AI Agent Security Benchmarks

AI agent security benchmarks measure how tool-using and autonomous systems handle attacks, privacy risks, authorization, and harmful actions.

See all 309 papers

May 17, 2026

Trust No Tool: Evaluating and Defending LLM Agents under Untrusted Tool Feedback

Tool-using LLM agents increasingly rely on external tools to make consequential decisions, yet most existing agent-security benchmarks and defenses implicitly assume that tool feedback is trustworthy once a tool has been selected. We study a different failure mode, cognitive poisoning, in which a malicious tool behaves plausibly during exploration, accumulates trust through benign-looking feedback, and becomes harmful only when hidden state conditions align with the final executable action. To study this setting, we construct TRUST-Bench, a task-conditioned benchmark of 1,970 hidden-trigger tool-compromise episodes with matched safe controls, introduce an asymmetric penalty metric, GuardedJoint, to better reflect real deployment risk, and present VISTA-Guard, a backbone-agnostic framework for final-action risk scoring. The core idea is to abstract multi-step tool interaction into structured environment variables that encode trust-formation dynamics and then score the risk of the final executable action from this trajectory-conditioned representation. Experiments show that prompt-centric heuristics, scalarized features, and zero-shot judges fail in this regime, whereas trajectory-aware final-action scoring yields strong in-domain discrimination and remains effective under balanced out-of-distribution transfer. Under GuardedJoint, VISTA-Guard reaches

84.2

in-domain and

56.9

on balanced out-of-distribution evaluation, while methods that optimize only one side of the safety--utility tradeoff collapse to zero. These findings support a broader view of agent security in black-box tool ecosystems: the decisive defense target is not local prompt text or tool descriptors alone, but the way trust is formed across the interaction trajectory and committed through the final action.

Lecheng Yan, Ruizhe Li, Xicheng Han +5

Southern University of Science and Technology · University of Science and Technology of China · Zhejiang University +2

AI Agent Reliability LLM Agent Evaluation LLM Agent Security AI Agent Security Benchmarks AI Agent Safety Benchmarks

Ask PDF

Sep 4, 2026

TIER: Threat Implicitness Benchmark for Evaluating LLM Safety Behaviors

Current LLM safety benchmarks largely rely on binary metrics, overlooking how models respond to harmful prompts with varying threat implicitness. We introduce TIER, a Threat Implicitness Benchmark for behavioral safety evaluation of LLMs. TIER covers four risk domains and four threat levels, from explicit harmful requests to sophisticated jailbreaks. Responses are assessed using a six-label behavior scale and two independent LLM judges. Experiments on six open-weight LLMs show that safety behaviors evolve gradually across threat levels rather than shifting directly from refusal to compliance. Contextual prompts yield the most diverse behaviors, while jailbreaks reveal the largest robustness gaps. Furthermore, models with similar Attack Success Rates can exhibit distinct response distributions, highlighting the need for behavior-aware LLM safety evaluation.

Thu-Hien Trinh-Thi, Hai-Yen Vong, Thanh-Ha Ung-Dung +1

Faculty of Information Technology, University of Science, Vietnam National University, Ho Chi Minh City, Vietnam

LLM Safety Evaluation LLM Safety Benchmarks Adversarial Robustness of Language Models

Ask PDF

Jun 16, 2026

SafeClawBench: Separating Semantic, Audit-Evidence, and Sandbox Harm in Tool-Using LLM Agents

Tool-using language-model agents introduce security failures that go beyond unsafe text: they can disclose protected objects, write persistent memory, send messages, modify databases, or trigger harmful code and tool effects. Existing evaluations often collapse these stages into a single attack success rate, making it difficult to tell whether a model merely agreed with an attacker or actually produced observable harm. We introduce SafeClawBench, a staged benchmark for tool-using agent security with 600 controlled adversarial tasks across six attack families: direct and indirect prompt injection, tool-return injection, memory poisoning, memory extraction, and ambiguity-driven unsafe inference. SafeClawBench reports three separate endpoints: semantic attack acceptance, audit-visible harm evidence, and sandbox-observed tool/state harm. Evaluating five agent endpoints under four prompt-level policies, we find that these endpoints capture different failure modes. Without additional prompt protection, semantic failure rates vary widely across models, from 9.0% to 44.2%. Audited harm evidence is narrower than semantic failure, and under a separate executable protocol some matched task identities produce sandbox harm despite passing the Semantic Core call: in a 12,000-row matched analysis, 291 of 347 observed sandbox harms occur in rows that pass the semantic check. Prompt policies change endpoint outcomes, but their effects depend on both model and protocol. SafeClawBench provides a reproducible framework for comparing agent models and prompt-policy conditions without conflating textual compliance, evidence-supported harm, and executable state changes. The open-source dataset is available at https://huggingface.co/datasets/sairights/safeclawbench.

Yuchuan Tian, Mengyu Zheng, Haocheng Mei +5

Peking University · Beijing Jiaotong University · SUIBE +2

LLM Safety Benchmarks LLM Agent Security AI Agent Security Benchmarks Prompt Injection Attacks Prompt Injection Attacks on LLMs Prompt Injection Attacks on AI Agents

Ask PDF

Abstract

AI Agent Security Benchmarks

Explore similar work

Trust No Tool: Evaluating and Defending LLM Agents under Untrusted Tool Feedback

TIER: Threat Implicitness Benchmark for Evaluating LLM Safety Behaviors

SafeClawBench: Separating Semantic, Audit-Evidence, and Sandbox Harm in Tool-Using LLM Agents