Vision-Language Models (VLMs) empower embodied agents to execute complex instructions, yet they remain vulnerable to contextual safety risks where benign commands become hazardous due to subtle environmental states. Existing safeguards often prove inadequate. Rule-based methods lack scalability in object-dense scenes, whereas model-based approaches relying on prompt engineering suffer from unfocused perception, resulting in missed risks or hallucinations. To address this, we propose an architecture-agnostic safeguard featuring Context-Guided Chain-of-Thought (CG-CoT). This mechanism decomposes risk assessment into active perception that sequentially anchors attention to interaction targets and relevant spatial neighborhoods, followed by semantic judgment based on this visual evidence. We support this approach with a curated grounding dataset and a two-stage training strategy utilizing Reinforcement Fine-Tuning (RFT) with process rewards to enforce precise intermediate grounding. Experiments demonstrate that our model HomeGuard significantly enhances safety, improving risk match rates by over 30% compared to base models while reducing oversafety. Beyond hazard detection, the generated visual anchors serve as actionable spatial constraints for downstream planners, facilitating explicit collision avoidance and safety trajectory generation. Code and data are released under https://github.com/AI45Lab/HomeGuard
Figures & tables
Figure 1: Identifying implicit contextual risks via Context-Guided Chain-of-Thought.
Figure 2: The two-stage training pipeline and a visualization of the sequential reasoning process for detecting risk identification in household tasks.
Figure 3: (a) The four-stage data collection pipeline for generating the HomeSafe dataset. (b) Distribution statistics showing the diversity of scene types and risk categories.
Model
RIR ↑
RMR ↑
T-IoU ↑
C-IoU ↑
OR ↓
Open-Sourced VLMs
Qwen3-VL-4B-Thinking [ 43 ]
66.67
33.14
0.5902
0.2212
29.31
Qwen3-VL-8B-Thinking [ 43 ]
67.58
33.20
0.5732
0.2694
32.62
RoboBrain2.5-8B [ 38 ]
56.15
24.00
0.4961
0.1245
41.18
Qwen3-VL-235B-Thinking [ 43 ]
77.17
45.08
0.6964
0.4124
34.69
Proprietary Models
Table 1: Performance comparison on HomeSafe-Bench. The best results are highlighted in bold , and the second-best results are underlined . Note that for Oversafety, lower is better ( ↓ ), while for others, higher is better ( ↑ ).
Figure 4: Comparison of inference efficiency and safety performance, where the x-axis is plotted on a logarithmic scale.
Model
EARBench
MSSBench
PaSBench
SafeAgentBench
RIR ↑
RMR ↑
RIR ↑
OR ↓
RIR ↑
RMR ↑
RIR ↑
OR ↓
Open-Sourced VLMs
Qwen3-VL-4B-Thinking [ 43 ]
82.67
52.35
51.31
23.68
70.31
36.72
63.67
36.46
Qwen3-VL-8B-Thinking [ 43 ]
86.06
57.25
52.63
27.11
69.53
37.50
63.49
43.68
RoboBrain2.5-8B [ 38 ]
88.70
60.26
59.21
22.37
74.21
35.94
70.37
47.65
Qwen3-VL-235B-A22B-Thinking [ 43 ]
89.83
67.80
62.67
22.37
72.66
45.31
68.9
33.21
Table 2: Performance comparison on four public risk identification benchmarks.
Model
RIR ↑
RMR ↑
T-IoU ↑
C-IoU ↑
OR ↓
Qwen3-VL-4B-Thinking
66.67
33.14
0.5902
0.2212
29.31
w/o constraint iou reward
85.49
50.20
0.6764
0.2604
22.43
w/o target iou reward
90.00
56.27
0.5642
0.4239
37.03
w/o all iou reward
90.98
50.98
0.5956
0.4052
42.35
direct GT bbox
84.71
51.96
0.7936
0.6242
22.79
direct CoT
70.07
43.50
0.6488
0.3302
23.53
Table 3: Ablation study based on Qwen3-VL-4B-Thinking in HomeSafe-Bench. “direct bbox”, “direct CoT” and “direct principle” refer to directly incorporating the GT bounding boxes of the target and constraint areas, the four-step CG-COT reasoning process, and safety principles into the prompts, respectively, without fine-tuning.
Figure 5: An application case of HomeGuard facilitating safe trajectory generation. The underlying motion trajectories are generated by RoboBrain2.5-8B [ 38 ] .
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
ID
Title
Description
1
Flammable Items Near Heat
Ensure flammable materials and heat sources are kept far apart.
2
Unattended Heat
Do not leave the room for long periods while heat sources (e.g., stoves, heaters, candles) are turned on.
3
Material Safety (Microwave/Oven)
Ensure only suitable materials (non-metal for microwaves, oven-safe containers for ovens) are placed inside devices.
4
Power Off Before Operation
Turn off and unplug appliances before hazardous operation such as moving, cleaning, maintenance, or repair.
5
Water & Electricity Separation
Ensure there are no electrical appliances or components nearby before using water or pouring liquids. Spilling water, beverages, or other liquids onto powered-on electronics can cause electrical short circuits, device damage, electric shock, or fire.
6
Unsanitary Food Surfaces
Keep surfaces that may come into contact with food (e.g., cutting boards, utensils, countertops) clean.
Appendix
Table 4: Extracted Safety Principles in HomeSafe.
Figure 6: Examples from HomeSafe across seven risk categories.
Figure 7: An example presents the image editing process in HomeSafe.
Hyperparameter
Value (4B)
Value (8B)
Base Model
Qwen3-VL-4B-Thinking-sft
Qwen3-VL-8B-Thinking-sft
Learning Rate
5×10−7
1×10−6
LR Scheduler
Cosine (Warmup ratio 0.1)
Cosine (Warmup ratio 0.1)
Optimizer
AdamW
AdamW
Global Batch Size
16 (8 × 1 × 2)
16 (8 × 1 × 2)
Number of Generations
16
16
Appendix
Table 5: Hyperparameter settings for Reinforcement Fine-tuning.
Figure 8: The comparison of base model (Qwen3-VL-4B-Thinking) and HomeGuard-4B in unsafe and safe scenario.
Figure 9: An application case of HomeGuard facilitating safe planning generation.
Figure 10: An application case of HomeGuard facilitating safe trajectory generation.
Gaoling School of Artificial Intelligence, Renmin University of China · University of Chinese Academy of Sciences · Beijing University of Posts and Telecommunications