GuardianBench: A Same-Scene Instruction-Contrastive Benchmark for Latent Contextual Risk in Embodied AI

Zhesheng Zhang; Jiahao Lu; Wei Liu; Cong Pan; Jianhua Yang; Yixiang Chen; Hongyuan Yu; Mengqi Zhang

cs.AIAug 22, 2026

GuardianBench: A Same-Scene Instruction-Contrastive Benchmark for Latent Contextual Risk in Embodied AI

Authors: Zhesheng Zhang, Jiahao Lu, Wei Liu, Cong Pan, Jianhua Yang, Yixiang Chen, Hongyuan Yu, Mengqi Zhang,

Organizations: Shandong University · National University of Singapore · Nanjing University of Aeronautics and Astronautics · Institute of Automation, Chinese Academy of Sciences ·

Abstract

In embodied AI, safety risk can be latent: a benign instruction and a safe scene become hazardous only when composed. Prior work has advanced embodied safety by varying visual contexts or evaluating execution-time dynamics, but the complementary axis of fixing the scene and varying only the instruction remains underexplored. We introduce GuardianBench, an instruction-contrastive benchmark grounded in international safety standards that isolates this latent contextual risk through 3,024 instruction-scene examples organized as same-scene Safe/Unsafe contrastive pairs across various hazard categories. Benchmarking state-of-the-art vision-language models (VLMs) reveals instruction-insensitive verdicts: models disproportionately approve both instructions under a given scene; across the primary models, average pair accuracy is only 24.1%. Our systematic rationale audit localizes the dominant failure: models fail to bind the instruction-relevant cues that differentiate safe from unsafe compositions. As a post-training case study, Verdict Log-Odds Supervision (VLOS), a lightweight verdict-level objective, substantially improves performance on open-weight backbones. Together, our latent contextual risk task formulation, standards-grounded contrastive benchmark construction, pair-level and rationale-level failure diagnosis, and benchmark-enabled verdict calibration establish GuardianBench as a controlled evaluation suite for exposing and improving safety reasoning over instruction-scene compositions under latent contextual risk.

Figures & tables

Figure 1: Contrastive illustration of latent risk. Under the same safe scene, changing only the instruction flips the correct verdict: the windowsill action is unsafe, while the coffee-table action is safe.

Validity control	Evidence in GuardianBench
Standards-grounded coverage	Four safety standards (ISO 12100, NFPA 1, CXC 1-1969, GHS) organized into $93$ source-grounded Hazard Origins and $14$ expert-verified Hazard Categories.
Image-cue shortcut control	Each scene is paired with $(i_{\mathrm{haz}},i_{\mathrm{safe}})$ ; both instructions and the shared scene are benign in isolation, balanced $1{:}1$ .
Label separation by risk threshold	Hazardous candidates with $S{\cdot}L<\tau_{\mathrm{crit}}{=}9$ are discarded; Safe labels come only from $i_{\mathrm{safe}}$ , never from low-risk hazardous cases.

Table 1: Validity controls behind GuardianBench construction.

Figure 2: Hierarchical composition of GuardianBench . Inner ring: share of hazardous instructions drawn from each of four foundational safety standards. Outer ring: share across the

14

expert-verified Hazard Categories nested within those standards; each category is defined by a documented set of source-grounded Hazard Origins (

93

total,

4

–

9

per category).

Figure 3: Severity–Likelihood Latent Risk Matrix. Cells at or above the threshold

\tau_{\mathrm{crit}}{=}9

are labeled Unsafe ; the hatched region below is discarded. Annotated integers report per-cell sample counts in the

N{=}1{,}512

hazardous subset.

Closed-Source Models
Model	Acc.	H-m.	Saf.	Utl.	Pair	Flip
Gemini-3-Pro-Preview †	65.9	55.1	39.3	92.5	35.1	38.4
Gemini-3-Flash-Preview †	69.8	65.2	51.9	87.7	43.5	47.3
Claude-Opus-4.5	63.2	47.7	31.9	94.4	28.7	31.1
Claude-Sonnet-4.5	58.2	32.6	19.6	96.8	18.3	20.3
Claude-Haiku-4.5	56.3	31.0	18.6	94.0	16.6	20.6

Table 2: Main results on GuardianBench . Acc is computed over

3{,}024

rows; Saf. and Utl. over

1{,}512

Unsafe/Safe rows respectively; Pair and Flip over

1{,}512

same-scene pairs. Pairs partition into correct flips (Pair), wrong flips (Flip

-

Pair), and no flips (

100-

Flip). † marks Google/Gemini-family models overlapping with or closely related to the generation stack; these rows are reported for completeness but excluded from primary ranking and aggregate claims, and only non- † cells are eligible for bolding. Bold marks the best non- † entry within each model-family block.

Figure 4: Instruction-insensitive verdicts visualized. Each bar represents the

1{,}512

same-scene pairs for one model, partitioned into four mutually exclusive outcomes. The dominance of Only-S (orange) across most models reveals instruction-insensitive verdicts: the model approves both instructions under the same scene, so it is correct only for the safe counterpart and misses the hazardous one.

Error pool	N	CUE	RUL	VRI	RES
All errors	19136	76.1	21.6	1.3	1.0
12 primary	14772	78.9	19.5	0.8	0.8
Missed-hazard	16028	78.2	19.3	1.3	1.1
12 primary	12477	81.3	17.0	0.7	0.9
Over-warning	3108	65.3	33.5	1.2	0.1
12 primary	2295	65.4	33.1	1.4	0.1

Table 3: Pooled error-stage decomposition by error direction. Rows sum to

100\%

. 12 primary excludes the judge (GPT-5.2, reasoning effort xhigh ) and three † Google/Gemini-family models; per-model decompositions are in Appendix Tables E3 and E4 .

Backbone: Qwen2.5-VL-7B-Instruct
Method	Acc.	H-m.	Saf.	Utl.	Pair
Base	53.6	53.5	50.8	56.4	27.7
CAI	48.8	48.5	52.8	44.9	22.4
Rationale-SFT	62.9	41.7	99.3	26.4	26.1
Label-only SFT	91.7	91.6	88.1	95.4	83.5
Standard GRPO	87.1	85.5	99.0	75.2	74.3

Table 4: Safety alignment results on the in-domain GuardianBench 606-example test split. VLOS uses

\alpha{=}0.05

; all rows share the same

2{,}418

-example training split. Constitutional AI (CAI) is inference-only (no parameter updates). SFT baselines are Qwen-only and train for

3

epochs. Bold marks the best column entry within each backbone block. Details in Appendices F.4 and F.2 .

Appendix figures & tables13 assets

Supplementary material from the paper’s appendix.

Appendix

You are an AI safety analyst. Your task is to analyze the user’s instruction and the visual context to determine if the action is safe.

You must output your reasoning as a three-stage structured rationale, followed by a final safety assessment. Your response must strictly follow this format:

1. [Perception] ...

2. [Knowledge] ...

3. [Prediction] ...

[Safety: <assessment>]

The assessment must be one of: Safe, Unsafe. Do not add any other text or explanations.

Appendix

Table A2: System prompt used for safety assessment in GuardianBench evaluation. The prompt enforces a structured-rationale format and is used as the shared response format for evaluation and post-training.

Benchmark	Scale	Primary safety slice	Evaluation interface	Controlled construction axis	Safety signal / associated intervention
MSSBench-Embodied ( Zhou et al. 2025 )	760 embodied pairs	Situational safety conditioned on visual context	Static image + household task instruction	A query/instruction is evaluated under safe vs. unsafe visual situations	Situational-safety judgment and reasoning diagnostics
ASIMOV-2.0 ( Jindal et al. 2025 )	319 text + 287 video + 164 image-text cases	Physical danger perception and operational constraint adherence	Text, video, and image-text physical-safety cases	Real-world injury narratives and embodiment-specific operational constraints	Human-labeled safety questions; constraint-violation evaluation; post-training / thinking analysis
EARBench ( Zhu et al. 2024 )	28 scenes / 2,636 samples	Physical risk awareness in task planning	Textual or visual observations $\rightarrow$ high-level plans	Risk-prone scenarios generated from safety guidelines across deployment domains	Task Risk Rate / Task Effectiveness Rate via LLM-based assessment; prompting-based mitigation
SafeAgentBench ( Yin et al. 2025 )	750 tasks	Safety-aware embodied task planning	Interactive simulator with low-level controller	Hazardous and safe tasks across common risk types and task abstractions	Execution-based and semantic evaluation; safety-prompting analysis
IS-Bench ( Lu et al. 2026 )	161 scenarios / 388 risks	Interactive safety and temporal mitigation order	High-fidelity household simulator	Dynamic hazards and process-order constraints during task execution	Process-oriented checks of whether mitigation actions occur before/after risk-prone steps; safety-aware chain-of-thought (CoT) analysis
GuardianBench (ours)	1,512 scenes / 3,024 rows; Real50 check	Pre-execution assessment of latent contextual risk	Single visual observation + instruction	Same-scene contrastive instructions with unimodal-safe filtering ( $H_{T}{=}H_{V}{=}0$ )	$5{\times}5$ Severity–Likelihood Latent Risk Matrix; Pair Accuracy; CUE/RUL/VRI error audit; VLOS verdict calibration

Appendix

Table B1: Construction-level positioning of embodied and physical-safety benchmarks. The table summarizes what each benchmark is designed to isolate. “Scale” reports the evaluation unit used in the corresponding work and is not directly comparable across rows, because benchmarks differ in whether they count images, task templates, interactive scenarios, or risk instances.

Field	Value	Img.
Room
Room	bathroom	13
Room	bedroom	8
Room	dining_room	10
Room	kitchen	11
Room	living_room	8

Appendix

Table C1: Real50 distribution over rooms and source safety standards. Image-level counts are over the

50

unique photos; each photo contributes one Safe and one Unsafe instruction as described in the text.

Model	Acc	Saf.	Utl.	Pair
Qwen3-VL-235B-Thinking	84	88	80	68
Claude-Opus-4.5	83	66	100	66
Gemini-3-Flash-Preview	82	76	88	66
Claude-Haiku-4.5	79	66	92	60
Qwen2.5-VL-72B-Instruct	77	66	88	56
Mistral-Large-2512 (675B)	76	76	76	54

Appendix

Table C2: Real50 16-model leaderboard, sorted by overall accuracy. Acc , Saf. , and Utl. are accuracy over

100

50

Unsafe, and

50

Safe instructions; Pair is the percentage of

50

pairs with both verdicts correct. All models completed evaluation with zero failed samples.

Model	Main	Real50	$\Delta$ Acc	Main-Saf.	Real50-Saf.	$\Delta$ Saf.
Gemini-3-Pro-Preview	65.9	71	+5.1	39.3	84	+44.7
Gemini-3-Flash-Preview	69.8	82	+12.2	51.9	76	+24.1
Claude-Opus-4.5	63.2	83	+19.8	31.9	66	+34.1
Claude-Sonnet-4.5	58.2	72	+13.8	19.6	46	+26.4
Claude-Haiku-4.5	56.3	79	+22.7	18.6	66	+47.4
GPT-5.2	56.8	63	+6.2	16.8	36	+19.2

Appendix

Table C3: Per-model accuracy on the main benchmark evaluation and on Real50 .

\Delta\textsc{Acc}

is Real50

-

main;

\Delta\textsc{Saf.}

is the same on Safety (Unsafe-instruction recall). Real50 yields higher absolute accuracies than the main benchmark, with the largest difference on Safety.

Closed-Source Models
Model	Pair (%)	Only-S (%)	Only-U (%)	BothW (%)	Flip (%)
Gemini-3-Pro-Preview †	35.1	57.4	4.2	3.3	38.4
Gemini-3-Flash-Preview †	43.5	44.2	8.5	3.8	47.3
Claude-Opus-4.5	28.7	65.7	3.2	2.4	31.1
Claude-Sonnet-4.5	18.3	78.4	1.3	2.0	20.3
Claude-Haiku-4.5	16.6	77.4	2.0	4.0	20.6

Appendix

Table D1: Pair-level outcome decomposition over

1{,}512

same-scene pairs. Pair : both verdicts correct (Pair Accuracy, same metric as in Table 2 ). Only-S : only the Safe instruction is correct. Only-U : only the Unsafe instruction is correct. BothW : both verdicts wrong (wrong-direction flip). Flip : fraction of pairs where the model issues different verdicts, regardless of correctness.

Boolean state	Stage	Interpretation
$\neg\,\texttt{cue\_match}$	CUE	Critical-cue mismatch
cue_match $\wedge$ $\neg\,\texttt{rule\_match}$	RUL	Rule/mechanism mismatch
cue_match $\wedge$ rule_match $\wedge$ $\neg\,\texttt{decision\_consistent}$	VRI	Verdict–rationale inconsistency
cue_match $\wedge$ rule_match $\wedge$ decision_consistent	RES	Audit-unexplained residual

Appendix

Table E1: Deterministic mapping from judge booleans to error stages. The source scripts use the internal labels PERCEPTION, KNOWLEDGE, CALIBRATION, and OTHER; the paper reports the corresponding CUE, RUL, VRI, and RES labels.

Error pool	$N$	Inconsistent	Consistent
All errors	19,136	6.9%	93.1%
Missed-hazard	16,028	5.3%	94.7%
Over-warning	3,108	15.3%	84.7%

Appendix

Table E2: Marginal verdict–rationale consistency over all error rows, independent of the first-unmet stage mapping.

Model	Errors	CUE%	RUL%	VRI%	RES%	Unc%
Claude-Opus-4.5	1030	71.4	25.1	1.8	1.7	1.7
Claude-Sonnet-4.5	1216	82.2	16.2	1.0	0.6	1.3
Claude-Haiku-4.5	1231	86.4	13.1	0.4	0.2	1.1
Gemini-3-Pro-Preview †	918	57.7	34.5	4.1	3.6	0.9
Gemini-3-Flash-Preview †	727	75.2	21.2	1.0	2.6	1.7
GPT-5.2	1258	60.7	32.6	5.8	0.9	0.0

Appendix

Table E3: Per-model decomposition for missed-hazard errors ( Unsafe

\to

Safe ) with judge-uncertainty rate ( Unc% ). Rows sum to

100\%

across CUE+RUL+VRI+RES.

Model	Errors	CUE%	RUL%	Unc%
Claude-Opus-4.5	84	65.5	34.5	3.6
Claude-Sonnet-4.5	49	59.2	40.8	2.0
Claude-Haiku-4.5	90	61.1	38.9	3.3
Gemini-3-Pro-Preview †	114	79.8	20.2	0.9
Gemini-3-Flash-Preview †	186	67.2	32.8	1.6
GPT-5.2	47	44.7	55.3	0.0

Appendix

Table E4: Per-model decomposition for over-warning errors ( Safe

\to

Unsafe ) with judge-uncertainty rate ( Unc% ). Rows sum to

100\%

across CUE+RUL+VRI+RES.

Hazard category	Errors	CUE%	RUL%	VRI%	RES%
Chemicals	3890	71.6	26.8	0.9	0.7
Microbiology	3426	76.5	22.8	0.5	0.2
Combustibles	1954	74.5	20.6	2.8	2.0
Storage	1795	78.6	19.2	0.9	1.3
Obstruction	1572	81.9	15.3	1.1	1.8
Substances	1125	76.3	20.4	1.9	1.5

Appendix

Table E5: Pooled error-stage decomposition by hazard category (all

14

unified Hazard Categories, sorted by pooled error count). Rows sum to

100\%

across CUE+RUL+VRI+RES.

Configuration	Acc.	H-m.	Saf.	Utl.	Pair
GRPO only	$87.6_{\pm 2.0}$	$86.3_{\pm 2.7}$	$97.7_{\pm 1.5}$	$77.5_{\pm 5.3}$	$75.4_{\pm 3.8}$
VLOS	$\mathbf{95.2_{\pm 0.5}}$	$\mathbf{95.1_{\pm 0.6}}$	$97.3_{\pm 0.5}$	$\mathbf{93.1_{\pm 1.4}}$	$\mathbf{90.3_{\pm 1.1}}$
Unsafe-only VLOS	$86.1_{\pm 1.3}$	$84.9_{\pm 1.8}$	$96.4_{\pm 0.7}$	$75.9_{\pm 3.0}$	$72.8_{\pm 2.5}$
Safe-only VLOS	$84.0_{\pm 2.6}$	$81.6_{\pm 3.8}$	$\mathbf{97.9_{\pm 0.8}}$	$70.1_{\pm 5.8}$	$68.4_{\pm 5.6}$

Appendix

Table F1: VLOS ablation on Qwen2.5-VL-7B-Instruct over three seeds

\{42,1234,2887\}

. Pair accuracy is computed over the

303

same-scene Safe/Unsafe pairs. Unsafe-only and Safe-only apply the verdict-level BCE term to only one side of the label space. All entries are mean

{}_{\pm\text{std}}

. Bold marks the best column entry.

Configuration	Acc.	H-m.	Saf.	Utl.	Pair
GRPO ( $\alpha{=}0$ )	87.1	85.5	99.0	75.2	74.3
VLOS ( $\alpha{=}0.05$ )	95.4	95.4	96.7	94.1	90.8
VLOS ( $\alpha{=}0.10$ )	94.1	93.9	98.0	90.1	88.1
VLOS ( $\alpha{=}0.20$ )	94.7	94.6	97.4	92.1	89.4
VLOS ( $\alpha{=}0.50$ )	94.4	94.3	96.7	92.1	89.1

Appendix

Table F2: VLOS

\alpha

-sensitivity sweep on the 606-example test set (single seed

{=}2887

\alpha{=}0

recovers standard GRPO; the same train/test split, optimization hyperparameters, and decoding configuration as Table F1 are used. Bold marks the best column entry across the sweep.

AI Safety Evaluation

AI Safety Evaluation studies how to test whether AI systems avoid harmful behavior and remain reliable across users, tasks, and deployment settings.

See all 135 papers

Mar 15, 2026

HomeGuard: VLM-based Embodied Safeguard for Identifying Contextual Risk in Household Task

Vision-Language Models (VLMs) empower embodied agents to execute complex instructions, yet they remain vulnerable to contextual safety risks where benign commands become hazardous due to subtle environmental states. Existing safeguards often prove inadequate. Rule-based methods lack scalability in object-dense scenes, whereas model-based approaches relying on prompt engineering suffer from unfocused perception, resulting in missed risks or hallucinations. To address this, we propose an architecture-agnostic safeguard featuring Context-Guided Chain-of-Thought (CG-CoT). This mechanism decomposes risk assessment into active perception that sequentially anchors attention to interaction targets and relevant spatial neighborhoods, followed by semantic judgment based on this visual evidence. We support this approach with a curated grounding dataset and a two-stage training strategy utilizing Reinforcement Fine-Tuning (RFT) with process rewards to enforce precise intermediate grounding. Experiments demonstrate that our model HomeGuard significantly enhances safety, improving risk match rates by over 30% compared to base models while reducing oversafety. Beyond hazard detection, the generated visual anchors serve as actionable spatial constraints for downstream planners, facilitating explicit collision avoidance and safety trajectory generation. Code and data are released under https://github.com/AI45Lab/HomeGuard

Xiaoya Lu, Yijin Zhou, Zeren Chen +6

Shanghai AI Laboratory · Shanghai Jiao Tong University · Beihang University +1

AI Agent Safety VLMs for Robotics Visual Grounding

Ask PDF

Jul 16, 2026

SafeRelBench: A Spatial-Relation-Aware Benchmark for Process-Level Safety in VLM-Driven Embodied Agents

Vision-language models (VLMs) are increasingly used as the reasoning backbone of embodied agents, enabling robots to interpret visual scenes, follow language instructions, and plan multi-step actions. In household environments, however, safety depends not only on recognizing objects, but also on how actions change the physical scene over time. Existing embodied safety evaluations largely focus on static risk recognition, unsafe instruction refusal, or final-state task completion. As a result, process-level safety failures induced by spatial relations such as support, containment, and proximity remain insufficiently studied. To address this gap, we introduce SAFERELBENCH, a spatial-relation-aware safety benchmark with 507 executable evaluation samples, including 248 spatial-relation samples and 259 non-spatial control samples. Using SAFERELBENCH to evaluate seven open- and closed-source VLM-driven embodied agents, we find a substantial gap between task success and process-level safety compliance: models often complete the requested task while violating process-level safety constraints. Unlike prior benchmarks, SAFERELBENCH explicitly tests whether agents satisfy safety conditions before risk-prone actions, making spatial relations a core dimension in embodied safety assessment. More broadly, our results show that safe embodied intelligence requires not only stronger perception and planning, but also reliable reasoning about how object relations shape risk during interaction.

Huaigang Yang, Ya Li, Min Ren +3

Beijing University of Posts and Telecommunications, Beijing, China · State Key Laboratory of General Artificial Intelligence, BIGAI, Beijing, China

Embodied Spatial Reasoning Robot Safety AI Agent Safety Benchmarks Embodied AI VLMs for Robotics

Ask PDF

Jul 1, 2026

Adversarial Pragmatics for AI Safety Evaluation: A Benchmark for Instruction Conflict, Embedded Commands, and Policy Ambiguity

Safety evaluations for language models increasingly depend on judgments about ambiguous natural-language behaviour: whether a model has followed an instruction, refused appropriately, complied with a policy, resisted an embedded command, or misreported progress in an agentic task. Existing benchmarks often compress these distinctions into pass/fail labels, obscuring whether failures arise from capability limits, policy ambiguity, instruction conflict, scaffold failure, or unstable evaluator judgments. This paper introduces adversarial pragmatics as a benchmark and annotation protocol for evaluating model behaviour under instruction conflict, embedded commands, quotation, scope ambiguity, deixis, indirect speech acts, and multi-turn agent transcripts. The contribution is empirical and methodological: a linguistically controlled taxonomy, an 18-item seed benchmark with validator-enforced metadata, a 54-row local seed pilot, an expert-evaluation protocol distinguishing task success, policy compliance, safety risk, refusal outcome, and evaluator confidence, and metrics for judge validity, diagnostic ambiguity, and taxonomy drift. The benchmark treats labels as inference licenses: it tests whether safety-relevant categories project across paraphrase, wrapper, model, and judge condition. In the pilot, a rubric-aided LLM judge graded its own outputs with expected-behaviour fields visible and still missed the safety-relevant minority classes.

Brett Reynolds

Humber Polytechnic & University of Toronto

LLM Safety Benchmarks Language Model Safety Evaluation Pragmatic Reasoning in Language Models

Ask PDF

Abstract

AI Safety Evaluation

Explore similar work

HomeGuard: VLM-based Embodied Safeguard for Identifying Contextual Risk in Household Task

SafeRelBench: A Spatial-Relation-Aware Benchmark for Process-Level Safety in VLM-Driven Embodied Agents

Adversarial Pragmatics for AI Safety Evaluation: A Benchmark for Instruction Conflict, Embedded Commands, and Policy Ambiguity