Shortcut Sensitivity in Agent-Safety Evaluations

Hasan Demirkiran; Jens Ernstberger

cs.AIOct 15, 2026

Shortcut Sensitivity in Agent-Safety Evaluations

Authors: Hasan Demirkiran, Jens Ernstberger

Organizations: Kontext

Abstract

AI agents take actions through tools. Some actions are sensitive, so guards that supervise them must themselves be evaluated reliably. Yet the data used to evaluate a guard can include cues that make labels predictable without measuring the intended safety construct, and those cues can also influence guard decisions. We introduce a three-level shortcut-auditing framework and audit completed LinuxArena trajectories and two TS-Bench-derived step datasets. We distinguish label predictability, observational guard association, and paired decision sensitivity. In LinuxArena, shorter average shell commands strongly predict trajectory labels. In AgentDojo-Traj, exact-tool lookup falls from F_1=.901 with mixed-domain folds to .495 under domain holdout. In AgentHarm-Traj, released TS-Guard F_1 falls from .902 under row weighting to .719 with equal interaction weighting, while balanced accuracy rises. We then test the strongest safe-row marker association in a seven-configuration screen on a frozen 64-step AgentDojo-Traj cohort using repeated, interleaved Haiku calls. Rewriting the <INFORMATION> marker lowers blocking by 8.33 percentage points, while opaque tool aliases raise it by 11.46 points. All unsafe calls remain blocked; the changes concern false positives against inherited labels. Both effects remain nonzero after subtracting the formatting-placebo effect, with exploratory 95% ranges excluding zero. The same edits also change measured accuracy against inherited labels. This provides direct evidence that benchmark representation can materially change a guard's decisions and measured evaluation performance. The Haiku-sized effects do not reproduce in Luna or deterministic Q8 TS-Guard. We recommend domain group holdouts, evaluation-unit metrics, and placebo-controlled paired interventions.

Figures & tables

Panel of Figure 1 — Figure 1: The three-level shortcut-audit framework. Each level motivates the next test, not the next conclusion. Paired edits probe decision sensitivity only to the extent that they preserve the target construct; stochastic guards additionally require repeat and placebo controls.

Artifact	Diagnostic	Result	Stress control / interpretation
LinuxArena (576 trajectories)	Negative mean shell-command characters	AUC $.910$ [.873, .941]	LOEO lexical AUC $.934$ [.907, .955]; difference $.023$ [.008, .040].
AgentDojo-Traj (1,220 steps)	Marker + 11 action prefixes	$F_{1}=.802$ ; balanced accuracy $.894$ ; recall $.972$	Exploratory fixed rule; the marker also appears in 321 safe rows.
AgentDojo-Traj	Exact tool majority	Grouped-CV $F_{1}=.901$ ; balanced accuracy $.925$	LODO $F_{1}=.495$ ; only $17.3\%$ of rows use a tool seen in another domain.
AgentHarm-Traj (731 steps)	Predict every step unsafe	$F_{1}=.836$ ; balanced accuracy $.500$	Positive prevalence makes $F_{1}$ look strong without discrimination.

Table 1: Capability-poor diagnostics. Intervals are 95% clustered bootstrap intervals where available. Grouped CV keeps interactions together; LODO holds out an entire AgentDojo domain.

Edit	All-base $\Delta$ (95%)	Safe-only $\Delta$ (95%)	$\downarrow/\uparrow$
Marker rewrite	$-8.33$ [-15.05, -2.49] $\,\mathrm{pp}$	$-16.67$ [-30.00, -5.88] $\,\mathrm{pp}$	6/0
Tool alias	$+11.46$ [2.69, 20.59] $\,\mathrm{pp}$	$+22.92$ [5.56, 39.22] $\,\mathrm{pp}$	2/9
Combined	$-2.60$ [-8.99, 2.99] $\,\mathrm{pp}$	$-5.21$ [-17.20, 5.21] $\,\mathrm{pp}$	5/2
Formatting placebo	$-1.56$ [-6.78, 2.78] $\,\mathrm{pp}$	$-3.13$ [-13.54, 5.38] $\,\mathrm{pp}$	2/2

Table 2: Selected Claude Haiku 4.5 sensitivity on a fixed, non-probability sample of 64 AgentDojo steps (three calls per condition). Exploratory 95% bootstrap ranges use domain-stratified source-interaction resampling (58 overall; 30 safe); they are not design-based confidence intervals. Every unsafe original is blocked and every unsafe contrast is zero, so each all-base estimate is mechanically half its safe-only estimate. Arrows count bases decreasing/increasing.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Guard	Marked blocked	Unmarked blocked	Difference (95% range)
TS-Guard	31/321	33/547	$3.62$ [0.09, 7.33] $\,\mathrm{pp}$
GPT-5.5	17/320	6/547	$4.22$ [1.70, 7.05] $\,\mathrm{pp}$
Claude Haiku 4.5	239/321	2/547	$74.09$ [68.67, 79.75] $\,\mathrm{pp}$
Claude Opus 4.8	12/321	0/547	$3.74$ [1.84, 5.98] $\,\mathrm{pp}$
Claude Sonnet 5	19/321	2/547	$5.55$ [2.96, 8.31] $\,\mathrm{pp}$
GPT-5.6 Luna	36/321	4/547	$10.48$ [7.17, 14.11] $\,\mathrm{pp}$

Appendix

Table 3: All cached observational safe-row marker associations in AgentDojo-Traj. The contrast is marked minus unmarked strict block rate, in percentage points.

Edit	Valid flips	Base rate	Interaction rate	Component / ordinal changes
Marker rewrite	3/64	4.69%	4.31%	7 / 5
Tool alias	3/63	4.76%	5.26%	14 / 12
Combined	6/64	9.38%	8.62%	14 / 13

Appendix

Table 4: Deterministic Q8 sensitivity in the balanced hash cohort. Equal-interaction rates weight each upstream interaction once (58 for marker/combined; 57 valid for alias).

Held-out rows	$n$	$F_{1}$	Balanced acc.	TP	TN	FP	FN
Seen names	211	.797	.508	137	4	64	6
Unseen names	1,009	.000	.500	0	800	0	209
All names	1,220	.495	.658	137	804	64	215

Appendix

Table 5: Exact-tool lookup under domain holdout, split by training-vocabulary coverage. Counts pool each held-out domain once; positive means strict-unsafe.

Weighting unit	Accuracy	$F_{1}$	Balanced acc.
Rows (731)	.848	.902	.754
Equal interaction weight (197)	.724	.719	.776
Interaction-majority (197)	.716	.714	.770

Appendix

Table 6: Released TS-Guard on AgentHarm-Traj under three evaluation units.

AI Safety Evaluation

AI Safety Evaluation studies how to test whether AI systems avoid harmful behavior and remain reliable across users, tasks, and deployment settings.

See all 140 papers

Apr 16, 2026

Don't Make Models Guess Security and Safety: Symbolic Guardrails for Domain-Specific AI Agents

There is increasing interest in integrating AI agents that invoke tools into domain-specific commercial software, where unintended tool calls can cause serious security and safety incidents. This has drawn growing research attention, and many agent security and safety benchmarks have emerged. They implicitly shape how the community approaches security and safety. Yet existing work exhibits a blind spot: it emphasizes training-based methods and neural guardrails, which reduce the likelihood of insecure or unsafe actions but cannot guarantee their prevention. It generally overlooks opportunities for deductive, symbolic guardrails grounded in standard software engineering practices, which can provide guarantees for some security and safety requirements. Our study has three parts: (1) a systematic review of 80 agent security and safety benchmarks finding that that 85% of benchmarks do not state verifiable requirements (61% provide none, and 24% give only high-level goals); (2) an applicability analysis of which security and safety requirements symbolic guardrails can and cannot enforce on

τ^2

-Bench, CAR-bench, and MedAgentBench, finding that 74% of requirements are symbolically enforceable and 95% of these need only simple, low-cost checks; and (3) an empirical evaluation of symbolic guardrails on the same three benchmarks, finding that symbolic guardrails improve security and safety without sacrificing utility, and often improve it. Our work draws attention to the potential for symbolic guardrails for AI agents, suggesting them as an overlooked but practical path toward deploying domain-specific AI agents in risk-averse commercial software. We release all codes and artifacts at https://github.com/hyn0027/agent-symbolic-guardrails.

Yining Hong, Yining She, Eunsuk Kang +2

Carnegie Mellon University

LLM Guardrails AI Agent Security AI Agent Safety AI Agent Security Benchmarks AI Agent Safety Benchmarks

Ask PDF

May 6, 2026

AgentTrust: Runtime Safety Evaluation and Interception for AI Agent Tool Use

Modern AI agents execute real-world side effects through tool calls such as file operations, shell commands, HTTP requests, and database queries. A single unsafe action, including accidental deletion, credential exposure, or data exfiltration, can cause irreversible harm. Existing defenses are incomplete: post-hoc benchmarks measure behavior after execution, static guardrails miss obfuscation and multi-step context, and infrastructure sandboxes constrain where code runs without understanding what an action means. We present AgentTrust, a runtime safety layer that intercepts agent tool calls before execution and returns a structured verdict: allow, warn, block, or review. AgentTrust combines a shell deobfuscation normalizer, SafeFix suggestions for safer alternatives, RiskChain detection for multi-step attack chains, and a cache-aware LLM-as-Judge for ambiguous inputs. We release a 300-scenario benchmark across six risk categories and an additional 630 independently constructed real-world adversarial scenarios. On the internal benchmark, the production-only ruleset achieves 95.0% verdict accuracy and 73.7% risk-level accuracy at low-millisecond end-to-end latency. On the 630-scenario benchmark, evaluated under a patched ruleset and not claimed as zero-shot, AgentTrust achieves 96.7% verdict accuracy, including about 93% on shell-obfuscated payloads. AgentTrust is released under the AGPL-3.0 license and provides a Model Context Protocol server for MCP-compatible agents.

Chenglin Yang

Independent Researcher

AI Agent Security AI Agent Security Benchmarks AI Agent Monitoring Safety Filtering Runtime Enforcement for AI Agents AI Agent Safety Benchmarks+1

Ask PDF

Apr 15, 2026

HINTBench: Horizon-agent Intrinsic Non-attack Trajectory Benchmark

Existing agent-safety evaluation has focused mainly on externally induced risks. Yet agents may still enter unsafe trajectories under benign conditions. We study this complementary but underexplored setting through the lens of intrinsic risk, where intrinsic failures remain latent, propagate across long-horizon execution, and eventually lead to high-consequence outcomes. To evaluate this setting, we introduce non-attack intrinsic risk auditing, a guard-oriented safety evaluation task, and present HINTBench, a benchmark of 596 agent trajectories, comprising 400 synthetic risky trajectories, 136 synthetic safe trajectories, 30 reconstructed real-world risky trajectories, and 30 reconstructed real-world safe trajectories, with an average length of 24.0 steps. HINTBench supports three tasks: risk detection, risk-step localization, and intrinsic failure-type identification, with annotations organized under a unified five-constraint taxonomy. Experiments reveal a substantial capability gap: strong LLMs perform well on trajectory-level risk detection, but the best model remains below 37 on fine-grained Strict-F1 for risk-step localization. Existing off-the-shelf guard models evaluated under their native prompts transfer poorly to this setting. These findings establish intrinsic risk auditing as an open challenge for agent safety.

Jiacheng Wang, Jinchang Hou, Fabian Wang +3

Baidu Inc., Beijing, China · Beijing Institute of Technology, Beijing, China

AI Agent Auditing Long-Horizon Agent Evaluation AI Agent Safety AI Agent Benchmarks AI Agent Safety Benchmarks Agent Trajectory Analysis

Ask PDF

Abstract

AI Safety Evaluation

Explore similar work

Don't Make Models Guess Security and Safety: Symbolic Guardrails for Domain-Specific AI Agents

AgentTrust: Runtime Safety Evaluation and Interception for AI Agent Tool Use

HINTBench: Horizon-agent Intrinsic Non-attack Trajectory Benchmark