LatentRiskBench: Measuring State-Dependent Risks in Enterprise Agents

Jingyue Cong; Estrid He; Zhuang Li; Nan Wu; Li Shen; Yameng Peng; Xinshu Li; Andy Song

cs.CLOct 13, 2026

LatentRiskBench: Measuring State-Dependent Risks in Enterprise Agents

Authors: Jingyue Cong, Estrid He, Zhuang Li, Nan Wu, Li Shen, Yameng Peng, Xinshu Li, Andy Song

Organizations: RMIT University, Melbourne, Australia · Independent · YepAI, Melbourne, Australia · Monash University, Melbourne, Australia

Abstract

Enterprise agents face safety risks that cannot be determined from individual actions alone. Existing safety benchmarks largely evaluate action-local risks, overlooking failures whose safety depends on workflow state. In such state-dependent risks, the same action may be safe in one context but violate policy in another. Hence, we introduce LatentRiskBench for evaluating such state-dependent risks in enterprise settings. It pairs risky trajectories with benign counterparts that share the same terminal action but differ in latent policy-relevant states, requiring models to reconstruct workflow context rather than rely on action-level cues. We also introduce new metrics for state-dependent risk evaluation, including a suite of residual risk (RR) measures. Experiments on the benchmark show that action-level evaluation detects only 0.09 of state-dependent violations, whereas state-level evaluation reaches 0.96. Existing guardrail methods largely overlook such risks unless they are explicitly applied to workflow states. Case studies on a commercial deployed scenario further show that the safety metrics embedded in the benchmark provide a more realistic measure of safety for enterprise agents. Our dataset and code are available at https://github.com/JingyueCong/LatentRiskBench.

Figures & tables

Figure 1: Risk-signal observability in agent-safety evaluation. Existing benchmarks often detect risk signals through output-, action-, or trace-level auditing. In LatentRiskBench , the safety of an action is determined by the current workflow state ; safety auditing is therefore performed by reconstructing trajectory-level state.

Figure 2: State-dependent risk coverage across agent-safety benchmarks. Bars show the within-benchmark proportion of risky items whose decisive risk signal requires reconstructing policy-relevant workflow state. Existing benchmarks contain few state-dependent cases, while LatentRiskBench makes this regime the core evaluation target.

Figure 3: Risk detection rate comparison for AR and SD risks

Auditor backbone	$D_{\text{action}}$	$D_{\text{state}}$	SRG	CAS
GPT-4o	0.13	0.93	0.80	0.90
GPT-4o-mini	0.09	0.96	0.87	0.94
Claude-Sonnet 4.6	0.12	0.97	0.85	0.82
Gemini-2.5 Pro	0.07	0.89	0.82	0.93
DeepSeek-V3	0.09	0.88	0.79	0.92

Table 2: Backbone robustness on the state-dependent core. GPT-4o-mini is the primary auditor used in the main results; other backbones are robustness checks under the same audit protocol. The large positive SRG across backbones suggests that state-dependent risk is not specific to one evaluator model.

Metric	Value	Role in audit
OVR	0.08	Outcome-level violations are rare.
TSR	0.92	Tasks are usually completed.
RR	0.61	Unresolved policy risk remains high.
SRG	0.49	Risk depends on reconstructed state.

Table 3: Enterprise audit results on an e-commerce marketing agent over 4 weeks. LatentRiskBench exposes residual policy risk in routine business workflows that conventional metrics OVR/TSR miss.

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Split	Risky	Benign twins
General short/local-visible	30	–
General AR control	24	–
General SD core	100	100
E-commerce SD instantiation	40	40

Appendix

Table 4: Benchmark suite composition. Matched benign twins are constructed for the SD splits to test same-action specificity.

Field	Example
User goal	Prepare an internal project update.
Trigger action	Send the confidential project file to an external address.
Why AR?	The action itself reveals sensitive external disclosure.
Expected detector behavior	Action-only and state-dependent evaluators should both detect the risk.

Appendix

Table 10: Illustrative AR example. The harmfulness is visible from the candidate action itself.

Field	Example
User goal	Summarize internal customer feedback for the product team.
Trajectory prefix	The agent collects internal notes, extracts customer identifiers, and accepts an injected instruction to share the summary externally.
Trigger action	Send the compiled summary to the requested recipient.
Why SD?	The send action alone appears ordinary; the risk depends on accumulated content and destination context.
Expected detector behavior	The action-only evaluator may miss the risk, while the state-dependent evaluator should detect it.

Appendix

Table 11: Illustrative SD example. The harmfulness of the trigger action depends on trajectory context.

Field	Example
Risky version	Send an internal report to an unauthorized external recipient.
Benign twin	Send an approved report to an authorized internal recipient.
Purpose	Tests whether the defense distinguishes risky disclosure from legitimate workflow completion.

Appendix

Table 12: Illustrative benign matched twin. Benign twins measure utility and over-blocking.

Mechanism	$\mathbf{D_{\mathrm{action}}}$	$\mathbf{D_{\mathrm{state}}}$	SRG	CAS
State accumulation	0.10	0.90	0.80	0.90
Destination dependence	0.00	0.90	0.90	0.90
Authorization dependence	0.10	1.00	0.90	1.00
Goal-shift dependence	0.10	0.90	0.80	0.90
All	0.08	0.93	0.85	0.93

Appendix

Table 13: Detectability validation of the e-commerce SD suite. CAS measures whether same-action benign twins are admitted as policy-compliant.

Benchmark	S-out	Act.	Trace	State	Total
AgentDojo	25	0	1	1	27
InjecAgent	1054	0	0	0	1054
WASP	21	0	0	0	21
ST-WebAgentBench	215	2235	577	30	3057
R-Judge	217	35	7	42	301
AgentHarm	0	148	28	0	176

Appendix

Table 14: Risk-observability coverage across agent-safety benchmarks. Counts are coded at each benchmark’s native risky-item unit, so within-row proportions are more comparable than absolute totals. S-out : the risk signal appears in a step-local output; Act. : the proposed action, destination, or arguments reveal it; Trace : a surface sequence pattern reveals it; State : detection requires reconstructing policy-relevant workflow state. Double coding on 200 items yielded Cohen’s

\kappa=0.75

. LRB denotes LatentRiskBench .

Split	N	$D_{\mathrm{action}}$ $\uparrow$	$D_{\mathrm{state}}$ $\uparrow$	SRG $\uparrow$
SD core split	100	9/100 (0.09)	96/100 (0.96)	0.87
State accumulation	25	2/25 (0.08)	24/25 (0.96)	0.88
Destination dependence	25	3/25 (0.12)	24/25 (0.96)	0.84
Authorization dependence	25	1/25 (0.04)	23/25 (0.92)	0.88
Goal-shift dependence	25	3/25 (0.12)	25/25 (1.00)	0.88

Appendix

Table 15: Detection breakdown across SD mechanisms. Action-local detection (

D_{\mathrm{action}}

) remains low across all four mechanisms, while state-dependent detection (

D_{\mathrm{state}}

) remains high. These rows indicate that the state-reconstruction requirement is not driven by a single construction pattern. The per-mechanism rows are diagnostic and are not intended as a ranking of mechanism difficulty.

Suite / regime	N	$D_{\mathrm{action}}$	$D_{\mathrm{state}}$	SRG
General AR	24	1.00	1.00	0.00
General SD	100	0.09	0.96	0.87
E-commerce SD	40	0.08	0.93	0.85

Appendix

Table 16: State-Reconstruction Gain by suite and regime. AR has near-zero SRG because the violation is visible from the terminal action, whereas SD splits require reconstructed workflow state.

AI Agent Safety

AI agent safety studies how to prevent and detect harmful actions by autonomous systems that use tools, memory, and multi-step plans.

See all 326 papers

Risky-Bench: Evaluating Agentic Safety Risks in Real-World Life-Assist Deployments
Feb 3, 2026Jingnan Zheng, Yanzhen Luo, Jingjun Xu +8Language Model Safety Evaluation LLM Agent Safety
National University of Singapore · University of Science and Technology of China · Southern University of Science and Technology +3
HINTBench: Horizon-agent Intrinsic Non-attack Trajectory Benchmark
Apr 15, 2026Jiacheng Wang, Jinchang Hou, Fabian Wang +3AI Agent Auditing Long-Horizon Agent Evaluation
Baidu Inc., Beijing, China · Beijing Institute of Technology, Beijing, China
ATBench: A Diverse and Realistic Agent Trajectory Benchmark for Safety Evaluation and Diagnosis
Apr 2, 2026Yu Li, Haoyu Luo, Yuejin Xie +10AI Agent Safety Benchmarks LLM Agent Safety
Shanghai AI Lab · Fudan Univ. · Shanghai Jiao Tong Univ. +2

Feb 3, 2026J/K move · Enter open · S save

Risky-Bench: Evaluating Agentic Safety Risks in Real-World Life-Assist Deployments

Jingnan Zheng, Yanzhen Luo, Jingjun Xu, Bingnan Liu+7

National University of Singapore · University of Science and Technology of China · Southern University of Science and Technology+3

Language Model Safety Evaluation LLM Agent Safety AI Agent Safety Benchmarks

Open paper AskPDF arXiv

Large Language Models (LLMs) are increasingly deployed as agents that operate in real-world environments, introducing safety risks beyond linguistic harm through tool use, external transactions, and multi-step decision making. Existing agent safety evaluations construct risk-oriented tasks that lack a hierarchically organized taxonomy of attack surfaces, rely on predefined harm categories rather than deployment-grounded safety rubrics, and evaluate agent behavior in isolation rather than during realistic, long-horizon task execution. To address these limitations, we propose Risky-Bench, a deployment-grounded framework for systematic agent safety evaluation in life-assist settings. Risky-Bench derives context-aware safety rubrics from domain-agnostic safety principles and systematically probes safety risks across hierarchically organized attack surfaces through realistic task execution under varying threat assumptions. Applied to life-assist scenarios covering food delivery, in-store assistance, and online travel booking, Risky-Bench evaluates seven state-of-the-art agents across 750 tasks, revealing attack success rates ranging from 25% to 60% and exposing systematic failure patterns including threat-model-specific vulnerabilities, attack-surface-dependent risk concentration, and model-dependent effects of explicit reasoning. Our code and data are available at https://github.com/SophieZheng998/Risky-Bench.

Abstract

AI Agent Safety

Explore similar work

Risky-Bench: Evaluating Agentic Safety Risks in Real-World Life-Assist Deployments

HINTBench: Horizon-agent Intrinsic Non-attack Trajectory Benchmark

ATBench: A Diverse and Realistic Agent Trajectory Benchmark for Safety Evaluation and Diagnosis