Threat-Preserving Representation Sensitivity in Agent-Security Benchmarks

Neeraj Karamchandani; Piyush Nagasubramaniam; Xinhong Xie; Sencun Zhu; Dinghao Wu

cs.CROct 2, 2026

Threat-Preserving Representation Sensitivity in Agent-Security Benchmarks

Authors: Neeraj Karamchandani, Piyush Nagasubramaniam, Xinhong Xie, Sencun Zhu, Dinghao Wu

Organizations: The Pennsylvania State University University Park, PA, USA

Abstract

Security benchmarks for LLM-based agents often report the attack success rate (ASR) as a measure of model robustness and use these scores to compare different models and defense mechanisms, assuming that they describe the security of the agent. In this paper, we explore whether it also influences the benchmark's measurement. To measure the effect of the benchmark representation, we introduce threat-preserving representation sensitivity (TPRS), which measures how much the ASR changes when we change the agent-visible representation while holding the underlying task, harmful action, security policy, ground truth, environment, and the evaluation criteria fixed. On Agent Security Bench (ASB), replacing threat-related tool names with threat-neutral names raises the committed attack success rate by 11.67 percentage points on GPT-5-mini and by 13.21 points on Claude Haiku 4.5. On MCPTox, replacing the original neutral tool name with an explicit threat-related name lowers the ASR by 11.00 percentage points on GPT-5-mini and 4.11 points on Claude Haiku 4.5. On AgentDojo, adding threat-related wording to the attack-relevant tool changes ASR by only 0.50 percentage points on GPT-4o-mini, yet the benign utility falls by 5.36 points on tasks requiring that tool. We ran an experiment on MCPTox where we observed that a threat-neutral name matched on token count, length, and casing reproduces most of the shift produced by the threat-explicit name (8.54 of 11.00 points on GPT-5-mini). The results show that a security score measured under one representation may fail to generalize across threat-preserving representations of the same security problem. Robustness claims should therefore be supported by performance across a controlled set of threat-preserving representations rather than relying on a single representation-dependent score.

Figures & tables

Fig. 1: Overview of threat-preserving representation sensitivity (TPRS). (a) An agent-visible representation is transformed while security-relevant benchmark components remain fixed. (b) TPRS is the resulting change in attack success rate, $\Delta_{T}(B,m)=\operatorname{ASR}(T(B),m)-\operatorname{ASR}(B,m)$ . The transformed representation appears in different locations in ASB, MCPTox, and AgentDojo.

Benchmark	Attack setup	What we change	What remains fixed
ASB	A distinctly named malicious tool is presented to the agent.	Threat-related tool names or descriptions are replaced with more neutral wording.	Attack goal, task, environment, and evaluator.
MCPTox	A malicious instruction is placed in the description of a neutrally named tool.	The neutral tool name is replaced with threat-explicit or matched neutral names.	Poisoned description, requested action, task, and evaluator.
AgentDojo	Injected tool output tries to make the agent misuse an ordinary tool.	An ordinary tool name and description is replaced with a threat-explicit wording.	Injected content, task, initial environment, and security scorer.

TABLE I: Threat-preserving transformations across the three benchmarks.

Arm	Transformation	Type
A0	Original	Baseline
NULL2	Orthographic rename	Orthographic
A1	Neutralize tool names	Cue-altering
A1b	Neutralize descriptions	Cue-altering
A3	Neutralize names & descriptions	Cue-altering
A2	Make task specification concrete	Diagnostic

TABLE II: ASB representation arms.

Arm	Transformation	Type
N0	Original	Baseline
O1/O2/NULL2	Case/separator changes	Orthographic
S1/S2	Neutral alternative names	Semantic-neutral
S3	Neutral name matched to N1	Semantic-neutral
C1	Conservative threat wording	Cue-altering
N1	Explicit threat-category name	Cue-altering

TABLE III: MCPTox representation arms.

Arm	Transformation	Type
D0	Original	Baseline
Dnull	Orthographic rename	Orthographic
Dval	Threat wording on required tool	Cue-altering
Dirrelevant	Threat wording on unrelated tool	Cue-altering

TABLE IV: AgentDojo representation arms.

Fig. 2: Representation sensitivity across the three benchmarks. Points show $\Delta$ ASR in percentage points relative to each benchmark’s original representation (committed ASR for ASB); whiskers show 95% cluster-bootstrap confidence intervals. Marker shapes distinguish models, while gray, black, and red denote orthographic, semantic-neutral, and cue-altering transformations, respectively. Each benchmark is analyzed independently; the common axis is used only to visualize effect direction and magnitude.

	GPT-5-mini		Haiku 4.5
Arm	Committed	Native	Committed	Native
NULL2 (orthographic)	$-1.67$ $[-4.44,+0.74]$	$-0.93$ $[-3.33,+1.67]$	$-0.74$ $[-3.46,+1.66]$	$+0.37$ $[-2.03,+2.71]$
A1b (descriptions)	$+0.00$ $[-2.96,+2.96]$	$+3.15$ $[-0.19,+7.22]$	$-1.16$ $[-4.60,+1.94]$	$\mathbf{+3.28}$ $[+0.14,+8.59]$
A1 (names)	$\mathbf{+11.67}$ $[+4.44,+19.07]$	$\mathbf{+14.07}$ $[+5.56,+23.70]$	$\mathbf{+13.21}$ $[+6.33,+21.10]$	$\mathbf{+13.05}$ $[+6.26,+21.03]$
A3 (names + descriptions)	$\mathbf{+13.15}$ $[+5.56,+21.11]$	$\mathbf{+17.22}$ $[+7.59,+27.96]$	$\mathbf{+14.85}$ $[+8.31,+22.49]$	$\mathbf{+17.14}$ $[+9.41,+25.90]$

TABLE V: ASB representation effects. Values are percentage-point changes in ASR relative to the original benchmark (A0), with 95% cluster-bootstrap confidence intervals.

Failure treatment	GPT-5-mini	Haiku 4.5
Drop failed runs	$+11.67$ $[+4.44,+19.07]$	$+13.21$ $[+6.33,+21.10]$
Failure = not committed	$+11.67$ $[+4.44,+19.07]$	$+12.04$ $[+5.74,+19.07]$
Failure = committed	$+11.67$ $[+4.44,+19.07]$	$+13.89$ $[+6.85,+21.85]$

TABLE VI: Robustness of the ASB A1 committed ASR effect to alternative treatments of execution failures. Values are percentage-point changes relative to A0 with 95% cluster-bootstrap confidence intervals.

Arm	Compared with	GPT-5-mini	Haiku 4.5
O1 (orthographic)	N0	$-0.99$ $[-4.27,+2.30]$	$+0.33$ $[-1.64,+2.30]$
NULL2 (orthographic) †	N0	$-3.12$ $[-6.40,+0.16]$	$+1.02$ $[-0.66,+2.85]$
O2 (orthographic)	N0	$+1.31$ $[-2.30,+4.93]$	$-0.99$ $[-3.12,+0.99]$
S1 (neutral)	N0	$-2.96$ $[-7.22,+1.31]$	$-1.48$ $[-4.60,+1.48]$
S2 (neutral)	N0	$-2.30$ $[-6.73,+1.97]$	$-1.64$ $[-4.93,+1.31]$
S3 (matched neutral)	N0	$\mathbf{-8.54}$ $[-12.48,-4.76]$	$-0.16$ $[-3.94,+3.61]$

TABLE VII: MCPTox representation effects. Values are percentage-point changes in ASR with 95% cluster-bootstrap confidence intervals over cases. N0 is the original benchmark. Negative values indicate lower attack success.

Security: $\Delta$ ASR
Outcome / intervention	Effect	95% CI
Dnull: orthographic	$-0.88$	$[-3.02,+1.25]$
Dval: required tool	$\mathbf{-0.50}$	$\mathbf{[-3.65,+2.51]}$
Dirrelevant: unrelated tool	$+0.88$	$[-1.00,+2.76]$
Benign utility under Dval
Requires renamed tool	$-5.36$	$[-10.00,-0.71]$

TABLE VIII: AgentDojo effects on security and benign utility. Values are percentage-point changes relative to D0 with 95% cluster-bootstrap confidence intervals.

AI Agent Security Benchmarks

AI agent security benchmarks measure how tool-using and autonomous systems handle attacks, privacy risks, authorization, and harmful actions.

See all 309 papers

Trust No Tool: Evaluating and Defending LLM Agents under Untrusted Tool Feedback
May 17, 2026Lecheng Yan, Ruizhe Li, Xicheng Han +5AI Agent Reliability LLM Agent Evaluation
Southern University of Science and Technology · University of Science and Technology of China · Zhejiang University +2
TIER: Threat Implicitness Benchmark for Evaluating LLM Safety Behaviors
Sep 4, 2026Thu-Hien Trinh-Thi, Hai-Yen Vong, Thanh-Ha Ung-Dung +1LLM Safety Evaluation LLM Safety Benchmarks
Faculty of Information Technology, University of Science, Vietnam National University, Ho Chi Minh City, Vietnam
SafeClawBench: Separating Semantic, Audit-Evidence, and Sandbox Harm in Tool-Using LLM Agents
Jun 16, 2026Yuchuan Tian, Mengyu Zheng, Haocheng Mei +5LLM Safety Benchmarks LLM Agent Security
Peking University · Beijing Jiaotong University · SUIBE +2

May 17, 2026J/K move · Enter open · S save

Trust No Tool: Evaluating and Defending LLM Agents under Untrusted Tool Feedback

Lecheng Yan, Ruizhe Li, Xicheng Han, Wenxi Li+4

Southern University of Science and Technology · University of Science and Technology of China · Zhejiang University+2

AI Agent Reliability LLM Agent Evaluation LLM Agent Security AI Agent Security Benchmarks AI Agent Safety Benchmarks

Open paper AskPDF arXiv

Tool-using LLM agents increasingly rely on external tools to make consequential decisions, yet most existing agent-security benchmarks and defenses implicitly assume that tool feedback is trustworthy once a tool has been selected. We study a different failure mode, cognitive poisoning, in which a malicious tool behaves plausibly during exploration, accumulates trust through benign-looking feedback, and becomes harmful only when hidden state conditions align with the final executable action. To study this setting, we construct TRUST-Bench, a task-conditioned benchmark of 1,970 hidden-trigger tool-compromise episodes with matched safe controls, introduce an asymmetric penalty metric, GuardedJoint, to better reflect real deployment risk, and present VISTA-Guard, a backbone-agnostic framework for final-action risk scoring. The core idea is to abstract multi-step tool interaction into structured environment variables that encode trust-formation dynamics and then score the risk of the final executable action from this trajectory-conditioned representation. Experiments show that prompt-centric heuristics, scalarized features, and zero-shot judges fail in this regime, whereas trajectory-aware final-action scoring yields strong in-domain discrimination and remains effective under balanced out-of-distribution transfer. Under GuardedJoint, VISTA-Guard reaches

84.2

in-domain and

56.9

on balanced out-of-distribution evaluation, while methods that optimize only one side of the safety--utility tradeoff collapse to zero. These findings support a broader view of agent security in black-box tool ecosystems: the decisive defense target is not local prompt text or tool descriptors alone, but the way trust is formed across the interaction trajectory and committed through the final action.

Abstract

AI Agent Security Benchmarks

Explore similar work

Trust No Tool: Evaluating and Defending LLM Agents under Untrusted Tool Feedback

TIER: Threat Implicitness Benchmark for Evaluating LLM Safety Behaviors

SafeClawBench: Separating Semantic, Audit-Evidence, and Sandbox Harm in Tool-Using LLM Agents