Safety for Whom? Benchmarking Understanding Beyond Refusal in Egocentric Multimodal Assistants

Yewon Han; Jisu Jeon; Suhyun Choi; Juan Yeo; Minsoo Jo; Gihoon Kim; Hyunggyu Lim; Taesup Kim

cs.AIOct 13, 2026

Safety for Whom? Benchmarking Understanding Beyond Refusal in Egocentric Multimodal Assistants

Authors: Yewon Han, Jisu Jeon, Suhyun Choi, Juan Yeo, Minsoo Jo, Gihoon Kim, Hyunggyu Lim, Taesup Kim

Organizations: Graduate School of Data Science, Seoul National University

Abstract

Egocentric multimodal assistants see through the user's eyes and guide what the user does next, so their outputs directly shape physical actions, including those that affect bystanders. Yet existing safety evaluations do not distinguish whom a response would harm, and score only whether a model refuses. Pooling all harms into one refusal rate hides two failures: a model may protect one target but not another, and it may refuse without understanding the risk, or understand it and comply anyway. We present STOU, a benchmark of 2,902 egocentric video-audio and video-text queries whose potential harm targets the user (Self), co-present individuals (The Others), or society at large (Others). Our evaluation decouples safety understanding from refusal behavior: beyond refusal, we score whether the model recognizes the user's intent, the presence and target of harm, and its downstream consequences. Across 10 open and closed source omnimodal and multimodal LLMs, we find that refusal varies systematically with the harm target, and that models often recognize a harm yet still assist with it. These results show that refusal rates alone cannot certify an assistant as safe, and that safety alignment must account for who would be harmed. Project page: https://onedayoneyeah.github.io/safety-for-whom

Figures & tables

Figure 1: Safety-critical scenarios for egocentric multimodal assistants. The model takes an egocentric video and audio query from the user as input, and responds in text or audio. We introduce four harm targets — Self , The Others , Others , and Unspecified —based on whether the target is identifiable or visible in the video and who may be harmed. We evaluate how well models defend against harmful queries and understand their potential harm across these targets.

Benchmark	View	Modality			Query	Evaluation		Domain	Video	Size	Length
		V	A	T		Target	Understanding
Video-SafetyBench	Exo				Adversarial			General	Synthetic	2.3k	10 s
Omni-SafetyBench	Exo				Adversarial			General	Synthetic	23.3k	—
iSafetyBench	Exo				Diagnostic QA			Industrial	Real	1.1k	4–8 s
EgoSafetyBench	Ego				–			Home / Factory	Rendered	1.2k	5 s
EgoSafe-Bench	Ego				Diagnostic QA			Security	Real	12k	$\sim$ 10 s

Table 1: Comparison with existing video safety benchmarks. STOU uniquely evaluates egocentric multimodal assistants with egocentric video and harmful user queries in audio, while incorporating potential harm targets in real-world interactions as a critical evaluation axis for comprehensive safety assessment. Beyond refusal behavior, STOU evaluates assistants’ understanding of query harmfulness. V, A, and T denote video, audio, and text input modalities, respectively.

Figure 2: Composition of STOU by harm target and harm type . By combining two harm dimensions, STOU comprises 2,902 samples spanning four harm targets (left) and six harm types (right), representing diverse real-world scenarios.

Figure 3: Harm-directed harmful text query generation pipeline. We first sample a scenario by harm target and harm type, and pair it with a video description. We then set a target phrase for the chosen harm target. For example, when the target is The Others , a visible person is taken from the video description as the target phrase; if no other person appears, another video is used. The scenario, video description, and target phrase are given to an LLM to generate a harmful text query. For human-written queries, annotators view the video and write the query using the same logic.

Figure 4: Safety understanding evaluation framework. Given a harmful query, the model identifies the user’s intended action and associated harm, including mechanism, target, and consequence. An LLM judge compares them with gold annotations to score both. The query’s harmfulness is considered “understood” when both are judged correct.

Figure 5: Human–LLM agreement in safety understanding across different harm thresholds

\tau

, measured by balanced accuracy.

\tau=3

achieves the highest balanced accuracy.

Panel of Table 2 — Table 2: Unsafe behavior of models across harm targets under two settings. Unsafe response rate (UR, %; lower is better) on STOU for audio-to-audio ( $\rightarrow$ ) and audio-to-text ( $\rightarrow$ ), with 2,902 samples per setting. For each model and setting, the highest and second-highest URs are shown in bold and underlined , respectively. “–” indicates an unsupported output modality.

Figure 6: Unsafe response rate across speaker voices. The *-marked voice is the benchmark default. Paralinguistic attributes can signal user vulnerability, yet models do not reliably adapt safety behavior to these cues. For self-directed harm, child-speaker inputs show URs comparable to or even higher than adult-speaker inputs, despite children potentially needing greater protection.

Figure 7: Safety performance by refusal and understanding in the audio-to-text setting. We evaluate safety by refusal behavior and safety understanding; bold marks each model’s dominant failure mode. Open-source models often answer unsafely without understanding the query harmfulness, while Gemini does so despite understanding it. Muse Spark, by contrast, often refuses harmful queries without understanding their harm. This two-axis view reveals distinct safety failure modes.

Figure 8: Qualitative analysis of four safety evaluation regimes : Ideal Guard , Reflexive Refusal , Alignment Gap , and Comprehension Gap . Our two-axis safety evaluation framework enables a more nuanced characterization of diverse safety-alignment failure modes.

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 9: Four stage data construction pipeline for STOU : (1) Video clip sampling, (2) Scenario construction and harm-directed text query generation, (3) Human verification and revision and (4) Speech Synthesis and multimodal construction.

Seed dataset	Clip length	# Clips	# Queries
Ego4D ( Grauman et al., 2022 )	5 s	609	2,012
AEA ( Lv et al., 2024 )	3–10 s	139	486
Nymeria+ ( DeTone et al., 2026 )	3–10 s	124	404
Total	–	872	2,902

Appendix

Table 4: Sources of video clips and harmful queries in STOU

Panel of Figure 10 — Figure 10: Examples from STOU across four harm targets and six harm types, each comprising egocentric video frames and a user query.

Figure 11: Unsafe response rate by harm type in audio-to-text settings. Bars show the percentage-point difference from each model’s overall unsafe response rate (UR); the parenthesized value next to each model name denotes its overall UR.

Model	Input	Harm Target				Action	Harm	Overall
Model	Input	S	T	O	U	Action	Harm	Overall
Open-source Models
MiniCPM-o 2.6	T $\rightarrow$ T	28.1	57.1	50.9	36.3	60.0	68.2	44.9
MiniCPM-o 2.6	A $\rightarrow$ T	25.0	50.3	46.0	34.2	56.5	64.5	40.3 ( $-$ 4.6)
MiniCPM-o 4.5	T $\rightarrow$ T	32.7	66.0	61.3	46.3	74.5	67.4	53.3
MiniCPM-o 4.5	A $\rightarrow$ T	30.8	61.3	60.2	47.1	70.7	67.2	51.1 ( $-$ 2.2)

Appendix

Table 5: Safety understanding by query modality (%). Each model receives the same video with the user query as text (T

\rightarrow

T) or as synthesized speech (A

\rightarrow

T, as in Table 3 ); the probe output is text in both cases (A: audio, T: text). Understood: intended action identified and harm score

\geq 3

(out of 5); Action / Harm : share meeting each condition. In each row, the harm target with the lowest understanding rate is shown in bold and the second lowest is underlined ; parentheses give the change in Overall from the text to the speech query (points). S: SELF , T: THE_OTHERS , O: OTHERS , U: UNSPECIFIED . † Baichuan 1.5 excludes 198 (text) and 98 (speech) samples with an empty probe response (at most 5 per modality are excluded for any other model).

Voice	Age	Sex	Engine	0	1	2	3	UR (%)
Jelly	child	F	Azure	60	26	200	4	70.3 [65.2, 75.5]
Ava	adult	F	Azure	62	30	194	4	68.3 [63.1, 73.4]
marin ∗	adult	F	OpenAI	53	28	206	3	72.1 [66.9, 77.2]
Kevin	child	M	Polly	55	34	196	5	69.3 [64.1, 74.5]
Andrew	adult	M	Azure	59	22	205	4	72.1 [66.9, 76.9]

Appendix

Table 7: Unsafe response rate (UR, %) of Qwen2.5-Omni-7B by speaker voice on

n{=}290

paired STOU items. Columns

0

–

3

count the responses with each refusal-judge label (Appendix B.7.1 ): refusal (

0

), safety awareness (

1

), unsafe answer (

2

), and non-safety failure (

3

). UR is the share labeled

2

3

. Brackets:

95\%

paired-bootstrap CI (

B{=}10{,}000

). ∗ marin is the benchmark’s canonical voice (adult female).

Contrast	$\Delta$ UR (pp)	$p$	Holm
Age (child $-$ adult)
Female lane: Jelly $-$ mean(Ava, marin)	+0.2 [ $-$ 2.4, +2.8]	–	–
Male lane: Kevin $-$ Andrew	$-$ 2.8 [ $-$ 6.6, +1.0]	–	–
Sex (male $-$ female)
Pooled: mean(male, 2 voices) $-$ mean(female, 3 voices)	+0.5 [ $-$ 1.6, +2.6]	–	–
Voice pairs

Appendix

Table 8: Speaker contrasts in UR (percentage points,

A-B

; positive means

A

is more permissive). Brackets: paired-bootstrap

95\%

CI; ∗ : CI excludes

0

p

: exact McNemar test on discordant items; Holm: adjusted over all

10

voice pairs. Age is contrasted within each sex lane; the pooled sex contrast averages voices within group.

Category	$n$	Jelly	Ava	marin ∗	Kevin	Andrew
ALL	290	70.3	68.3	72.1	69.3	72.1
Harm target
SELF	72	80.6	79.2	79.2	80.6	80.6
OTHERS	76	69.7	71.1	71.1	69.7	73.7
THE_OTHERS	94	58.5	56.4	63.8	58.5	62.8
UNSPECIFIED	48	79.2	70.8	79.2	72.9	75.0

Appendix

Table 9: UR (%) by harm target and harm type for each voice;

n

is the number of items in the category, so one item is worth

100/n

pp. ∗ marin is the benchmark’s canonical voice (adult female).

$\tau$	Understood (%)	TPR (%)	TNR (%)	BA (%)
1	76.4 ( $+$ 10.2)	99.3	68.4	83.9 [76.4, 90.2]
2	69.8 ( $+$ 3.6)	95.3	80.3	87.8 [81.8, 93.0]
3	62.7 ( $-$ 3.6)	89.3	89.5	89.4 [83.3, 94.4]
4	56.9 ( $-$ 9.3)	83.2	94.7	89.0 [83.8, 93.5]
5	53.3 ( $-$ 12.9)	78.5	96.1	87.3 [81.3, 92.4]

Appendix

Table 10: Harm-threshold selection against human judgments on 225 query–response pairs. Understood is the judge understanding rate, with the difference from the overall human rate (66.2%) shown in parentheses. Balanced accuracy (BA) is the primary selection criterion; 95% bootstrap CIs are shown in brackets. Best BA is in bold and second best is underlined .

Axis	Dimension	Agreement	F1
Refusal	Unsafe (label $\geq 2$ )	93.8 [90.7, 96.9]	95.1 [92.2, 97.6]
	4-class †	88.2 [82.6, 93.2]	75.4 [58.3, 86.3]
Understanding	Action	80.0 [73.8, 85.8]	86.8 [81.9, 90.9]
	Harm: affected party	96.4 [93.8, 98.7]	97.3 [94.9, 99.1]
	Harm: mechanism	88.4 [82.2, 93.3]	91.5 [86.5, 95.3]
	Harm: consequence	92.4 [87.1, 96.9]	93.6 [88.7, 97.3]

Appendix

Table 11: Agreement between the LLM judge (GPT-4.1) and human annotators for each evaluation dimension on 225 query–response pairs. The ground truth is the majority vote of three annotators. Values are %, with 95% bootstrap confidence intervals in brackets.

AI Safety Evaluation

AI Safety Evaluation studies how to test whether AI systems avoid harmful behavior and remain reliable across users, tasks, and deployment settings.

See all 143 papers

The Refusal--Compliance Tradeoff: A Large-Scale Safety Behavior Audit of Large Language Models
May 6, 2026Alif Al Hasan, Sumon BiswasLLM Safety Benchmarks Language Model Safety Evaluation
Department of Computer and Data Sciences Case Western Reserve University Cleveland, OH, USA
Safety for Whom? Boundary-Aware Self-Distillation for Controlled LLM Safety Refusal
Sep 3, 2026Alejo López-Ávila, Iker García-Ferrero, Jezabel Garcia +2LLM Safety Alignment Refusal Behavior in Language Models
Multiverse Computing
Behavior Beyond Text: Evaluating Refusal Transfer Failure in LLM Agents
Oct 20, 2026Ekagra Gupta, Raj Saha, Aniekan-Abasi Inyang +1LLM Agent Safety Refusal Behavior in Language Models
Harvard University · Clarkson University · Carleton University

May 6, 2026J/K move · Enter open · S save

The Refusal--Compliance Tradeoff: A Large-Scale Safety Behavior Audit of Large Language Models

Alif Al Hasan, Sumon Biswas

Department of Computer and Data Sciences Case Western Reserve University Cleveland, OH, USA

LLM Safety Benchmarks Language Model Safety Evaluation LLM Refusal Behavior LLM Safety Over-Refusal Demographic Bias in Language Models+1

Open paper AskPDF arXiv

Refusal rates are a poor proxy for LLM safety, i.e., a model may over-refuse benign prompts while still complying with harmful ones. We audit both failure modes across 21 open-weight LLMs on four safety benchmarks (OR-Bench, XSTest, ToxiGen, BOLD), using a composition adjustment to isolate model sensitivity from dataset toxicity confounds. We report three findings. First, models adopt fundamentally different calibration strategies: conservative ecosystems such as Llama suppress unsafe outputs at the cost of elevated over-refusals, while permissive ecosystems such as DeepSeek and Qwen preserve helpfulness but tolerate higher harmful compliance. Second, demographic protection is unequal: models over-protect prominent racial and religious groups, frequently refusing even benign prompts about them, while providing substantially weaker protection against disability-targeted attacks. Third, refusal and compliance tendencies are stable within model families across generations and scales, suggesting that post-training objectives shape safety behavior more than architecture. Our results call for joint, demographically-aware, and multi-judge safety evaluation.

Abstract

AI Safety Evaluation

Explore similar work

The Refusal--Compliance Tradeoff: A Large-Scale Safety Behavior Audit of Large Language Models

Safety for Whom? Boundary-Aware Self-Distillation for Controlled LLM Safety Refusal

Behavior Beyond Text: Evaluating Refusal Transfer Failure in LLM Agents