Enterprise agents face safety risks that cannot be determined from individual actions alone. Existing safety benchmarks largely evaluate action-local risks, overlooking failures whose safety depends on workflow state. In such state-dependent risks, the same action may be safe in one context but violate policy in another. Hence, we introduce LatentRiskBench for evaluating such state-dependent risks in enterprise settings. It pairs risky trajectories with benign counterparts that share the same terminal action but differ in latent policy-relevant states, requiring models to reconstruct workflow context rather than rely on action-level cues. We also introduce new metrics for state-dependent risk evaluation, including a suite of residual risk (RR) measures. Experiments on the benchmark show that action-level evaluation detects only 0.09 of state-dependent violations, whereas state-level evaluation reaches 0.96. Existing guardrail methods largely overlook such risks unless they are explicitly applied to workflow states. Case studies on a commercial deployed scenario further show that the safety metrics embedded in the benchmark provide a more realistic measure of safety for enterprise agents. Our dataset and code are available at https://github.com/JingyueCong/LatentRiskBench.
Figures & tables
Figure 1: Risk-signal observability in agent-safety evaluation. Existing benchmarks often detect risk signals through output-, action-, or trace-level auditing. In LatentRiskBench , the safety of an action is determined by the current workflow state ; safety auditing is therefore performed by reconstructing trajectory-level state.
Figure 2: State-dependent risk coverage across agent-safety benchmarks. Bars show the within-benchmark proportion of risky items whose decisive risk signal requires reconstructing policy-relevant workflow state. Existing benchmarks contain few state-dependent cases, while LatentRiskBench makes this regime the core evaluation target.
Figure 3: Risk detection rate comparison for AR and SD risks
Auditor backbone
Daction
Dstate
SRG
CAS
GPT-4o
0.13
0.93
0.80
0.90
GPT-4o-mini
0.09
0.96
0.87
0.94
Claude-Sonnet 4.6
0.12
0.97
0.85
0.82
Gemini-2.5 Pro
0.07
0.89
0.82
0.93
DeepSeek-V3
0.09
0.88
0.79
0.92
Table 2: Backbone robustness on the state-dependent core. GPT-4o-mini is the primary auditor used in the main results; other backbones are robustness checks under the same audit protocol. The large positive SRG across backbones suggests that state-dependent risk is not specific to one evaluator model.
Metric
Value
Role in audit
OVR
0.08
Outcome-level violations are rare.
TSR
0.92
Tasks are usually completed.
RR
0.61
Unresolved policy risk remains high.
SRG
0.49
Risk depends on reconstructed state.
Table 3: Enterprise audit results on an e-commerce marketing agent over 4 weeks. LatentRiskBench exposes residual policy risk in routine business workflows that conventional metrics OVR/TSR miss.
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
Split
Risky
Benign twins
General short/local-visible
30
–
General AR control
24
–
General SD core
100
100
E-commerce SD instantiation
40
40
Appendix
Table 4: Benchmark suite composition. Matched benign twins are constructed for the SD splits to test same-action specificity.
Field
Example
User goal
Prepare an internal project update.
Trigger action
Send the confidential project file to an external address.
Why AR?
The action itself reveals sensitive external disclosure.
Expected detector behavior
Action-only and state-dependent evaluators should both detect the risk.
Appendix
Table 10: Illustrative AR example. The harmfulness is visible from the candidate action itself.
Field
Example
User goal
Summarize internal customer feedback for the product team.
Trajectory prefix
The agent collects internal notes, extracts customer identifiers, and accepts an injected instruction to share the summary externally.
Trigger action
Send the compiled summary to the requested recipient.
Why SD?
The send action alone appears ordinary; the risk depends on accumulated content and destination context.
Expected detector behavior
The action-only evaluator may miss the risk, while the state-dependent evaluator should detect it.
Appendix
Table 11: Illustrative SD example. The harmfulness of the trigger action depends on trajectory context.
Field
Example
Risky version
Send an internal report to an unauthorized external recipient.
Benign twin
Send an approved report to an authorized internal recipient.
Purpose
Tests whether the defense distinguishes risky disclosure from legitimate workflow completion.
Table 13: Detectability validation of the e-commerce SD suite. CAS measures whether same-action benign twins are admitted as policy-compliant.
Benchmark
S-out
Act.
Trace
State
Total
AgentDojo
25
0
1
1
27
InjecAgent
1054
0
0
0
1054
WASP
21
0
0
0
21
ST-WebAgentBench
215
2235
577
30
3057
R-Judge
217
35
7
42
301
AgentHarm
0
148
28
0
176
Appendix
Table 14: Risk-observability coverage across agent-safety benchmarks. Counts are coded at each benchmark’s native risky-item unit, so within-row proportions are more comparable than absolute totals. S-out : the risk signal appears in a step-local output; Act. : the proposed action, destination, or arguments reveal it; Trace : a surface sequence pattern reveals it; State : detection requires reconstructing policy-relevant workflow state. Double coding on 200 items yielded Cohen’s κ=0.75 . LRB denotes LatentRiskBench .
Split
N
Daction↑
Dstate↑
SRG ↑
SD core split
100
9/100 (0.09)
96/100 (0.96)
0.87
State accumulation
25
2/25 (0.08)
24/25 (0.96)
0.88
Destination dependence
25
3/25 (0.12)
24/25 (0.96)
0.84
Authorization dependence
25
1/25 (0.04)
23/25 (0.92)
0.88
Goal-shift dependence
25
3/25 (0.12)
25/25 (1.00)
0.88
Appendix
Table 15: Detection breakdown across SD mechanisms. Action-local detection ( Daction ) remains low across all four mechanisms, while state-dependent detection ( Dstate ) remains high. These rows indicate that the state-reconstruction requirement is not driven by a single construction pattern. The per-mechanism rows are diagnostic and are not intended as a ranking of mechanism difficulty.
Suite / regime
N
Daction
Dstate
SRG
General AR
24
1.00
1.00
0.00
General SD
100
0.09
0.96
0.87
E-commerce SD
40
0.08
0.93
0.85
Appendix
Table 16: State-Reconstruction Gain by suite and regime. AR has near-zero SRG because the violation is visible from the terminal action, whereas SD splits require reconstructed workflow state.
Large Language Models (LLMs) are increasingly deployed as agents that operate in real-world environments, introducing safety risks beyond linguistic harm through tool use, external transactions, and multi-step decision making. Existing agent safety evaluations construct risk-oriented tasks that lack a hierarchically organized taxonomy of attack surfaces, rely on predefined harm categories rather than deployment-grounded safety rubrics, and evaluate agent behavior in isolation rather than during realistic, long-horizon task execution. To address these limitations, we propose Risky-Bench, a deployment-grounded framework for systematic agent safety evaluation in life-assist settings. Risky-Bench derives context-aware safety rubrics from domain-agnostic safety principles and systematically probes safety risks across hierarchically organized attack surfaces through realistic task execution under varying threat assumptions. Applied to life-assist scenarios covering food delivery, in-store assistance, and online travel booking, Risky-Bench evaluates seven state-of-the-art agents across 750 tasks, revealing attack success rates ranging from 25% to 60% and exposing systematic failure patterns including threat-model-specific vulnerabilities, attack-surface-dependent risk concentration, and model-dependent effects of explicit reasoning. Our code and data are available at https://github.com/SophieZheng998/Risky-Bench.
Jingnan Zheng, Yanzhen Luo, Jingjun Xu +8
National University of Singapore · University of Science and Technology of China · Southern University of Science and Technology +3
Existing agent-safety evaluation has focused mainly on externally induced risks. Yet agents may still enter unsafe trajectories under benign conditions. We study this complementary but underexplored setting through the lens of intrinsic risk, where intrinsic failures remain latent, propagate across long-horizon execution, and eventually lead to high-consequence outcomes. To evaluate this setting, we introduce non-attack intrinsic risk auditing, a guard-oriented safety evaluation task, and present HINTBench, a benchmark of 596 agent trajectories, comprising 400 synthetic risky trajectories, 136 synthetic safe trajectories, 30 reconstructed real-world risky trajectories, and 30 reconstructed real-world safe trajectories, with an average length of 24.0 steps. HINTBench supports three tasks: risk detection, risk-step localization, and intrinsic failure-type identification, with annotations organized under a unified five-constraint taxonomy. Experiments reveal a substantial capability gap: strong LLMs perform well on trajectory-level risk detection, but the best model remains below 37 on fine-grained Strict-F1 for risk-step localization. Existing off-the-shelf guard models evaluated under their native prompts transfer poorly to this setting. These findings establish intrinsic risk auditing as an open challenge for agent safety.
Jiacheng Wang, Jinchang Hou, Fabian Wang +3
Baidu Inc., Beijing, China · Beijing Institute of Technology, Beijing, China
Evaluating the safety of LLM-based agents is increasingly important because risks in realistic deployments often emerge over multi-step interactions rather than isolated prompts or final responses. Existing trajectory-level benchmarks remain limited by insufficient interaction diversity, coarse observability of safety failures, and weak long-horizon realism. We introduce ATBench, a trajectory-level benchmark for structured, diverse, and realistic evaluation of agent safety. ATBench organizes agentic risk along three dimensions: risk source, failure mode, and real-world harm. Based on this taxonomy, we construct trajectories with heterogeneous tool pools and a long-context delayed-trigger protocol that captures realistic risk emergence across multiple stages. The benchmark contains 1,000 trajectories (503 safe and 497 unsafe), averaging 9.01 turns and 3.95k tokens, with 1,954 invoked tools drawn from pools spanning 2,084 available tools. Data quality is supported by rule-based and LLM-based filtering plus full human audit. Experiments on frontier LLMs, open-source models, and specialized guard systems show that ATBench is challenging even for strong evaluators, while enabling taxonomy-stratified analysis, cross-benchmark comparison, and diagnosis of long-horizon failure patterns.