Organizations: Gaoling School of Artificial Intelligence, Renmin University of China · University of Chinese Academy of Sciences · Beijing University of Posts and Telecommunications
The rapid evolution of embodied agents has accelerated the deployment of household robots in real-world environments. However, unlike structured industrial settings, household spaces introduce unpredictable safety risks, where system limitations such as perception latency and lack of common sense knowledge can lead to dangerous errors. Current safety evaluations, often restricted to static images, text, or general hazards, fail to adequately benchmark dynamic unsafe action detection in these specific contexts. To bridge this gap, we introduce HomeSafe-Bench, a challenging benchmark designed to evaluate Vision-Language Models (VLMs) on unsafe action detection in household scenarios. HomeSafe-Bench is contrusted via a hybrid pipeline combining physical simulation with advanced video generation and features 438 diverse cases across six functional areas with fine-grained multidimensional annotations. Beyond benchmarking, we propose Hierarchical Dual-Brain Guard for Household Safety (HD-Guard), a hierarchical streaming architecture for real-time safety monitoring. HD-Guard coordinates a lightweight FastBrain for continuous high-frequency screening with an asynchronous large-scale SlowBrain for deep multimodal reasoning, effectively balancing inference efficiency with detection accuracy. Evaluations demonstrate that HD-Guard achieves a superior trade-off between latency and performance, while our analysis identifies critical bottlenecks in current VLM-based safety detection.
Figures & tables
Figure 1: HomeSafe-Bench construction pipeline. From LLM-generated hazard causes across six household locations, we create video descriptions, generate videos combining physical simulation with video generation models, and produce multi-dimensional annotations with quality checks.
Figure 2: HomeSafe-Bench Statistics. Left: Hierarchical taxonomy of household risks, spanning danger categories and severity levels. Right: Distribution analysis. Legend: Danger Categories (C1–C4), Severity (L1–L4), Reasoning Difficulty (D1–D3).
Figure 3: Temporal phase scoring and key frame definitions. The timeline is divided into five phases based on four annotated key frames. Detections in the Optimal window earn maximum scores (100) to reward early intervention, whereas delayed detections are strictly penalized.
Method
Hazard Detection
Effective Warning
Phase Distribution Breakdown (%)
Overall WSS ↑
HDR ↑
EWP ↑
Premature ↓
Optimal ↑
Suboptimal ↑
Irrelevant ↓
Missed ↓
Closed-Source VLMs
GPT-5.1
75.11
43.77
37.90
13.93
9.82
9.13
29.22
21.12
Claude-Opus-4.1
93.61
25.12
66.89
14.84
5.48
3.20
9.59
18.38
Open-Source VLMs
InternVL3.5-8B
97.03
38.35
53.88
22.15
9.82
5.48
8.68
28.42
Table 1: Main results on the HomeSafe-Bench benchmark. Best and second-best scores are highlighted in bold and underlined respectively.
Figure 5: Severity and Latency Evaluation. Left: Distribution of severity assessments across VLMs. indicates underestimation (dangerous), indicates correct predictions , and indicates overestimation (conservative). Right: Efficiency-Quality Trade-off. HD-Guard achieves a superior balance between low latency and high safety scores. The gray dashed line shows the Pareto frontier.
Figure 6: Fine-grained error analysis stratified by reasoning difficulty. We compare HD-Guard with baselines across three difficulty levels.
Figure 7: Ablation Study on Sampling Frequency. Left: Trade-off between Safety Score and Latency. Right: Warning state distribution.