PlanGuard: A Guardrail for Multi-Step Plan Safety in Embodied Agents
Authors: Junchi Chen, Changtao Miao, Yuxiao Xiang, Zhenchao Jin, Haojie Yuan, Qi Chu, Tao Gong, He Liu, Bo Zhang, Jiansheng Cai, Zhe Li, Nenghai Yu
Organizations: Ant Digital Technologies, Ant Group · Anhui Province Key Laboratory of Digital Security · The University of Hong Kong · Individual Researcher
Embodied task planners may produce multi-step plans whose subtask dependencies and interactions with the environment create physical risks during execution. Yet existing safeguards overlook such compositional risks, as general-purpose guardrails focus on semantic harm and embodied safety detectors assess subtasks in isolation. To address this gap, we introduce PlanGuard, the first pre-execution detector that evaluates the physical safety of a complete multi-step plan in its current environment. For training and evaluation, we construct a Multi-Step Plan Safety (MSP-Safe) dataset through paired task construction, plan generation using diverse planners, and safety annotation by three judges. Task-oriented SFT on MSP-Safe establishes fundamental plan-safety assessment capabilities, yet a substantial gap remains between compact models suitable for real-time deployment and stronger but costlier large models. Accordingly, we propose Strong-Teacher Adaptive Compensation for On-Policy Distillation (STAC-OPD), which provides compact models with adaptive strong-teacher supervision along their on-policy trajectories. It combines token-level distribution transfer from a fine-tuned strong teacher with probability-routed sequence-level compensation, retaining student-generated targets when the student favors the reference safety decision and using teacher-reconstructed targets otherwise. Across all test subsets, PlanGuard-2B achieves average 87.15% ACC and 87.21% F1, demonstrating effective whole-plan physical-risk detection at compact model scale. Code and dataset will be publicly released.
Figures & tables
Figure 1: Complete multi-step plan safety detection comparison. (1) General-purpose guardrails lack physical-interaction understanding, while (2) single-subtask guardrails cannot model cross-step dependencies. In contrast, (3) PlanGuard jointly analyzes the visual environment and complete multi-step plan to identify the potential fire risk before execution, demonstrating the necessity of complete multi-step plan analysis for robust embodied safety detection.
Figure 2: Pipeline of PlanGuard. (a)-(c) MSP-Safe Construction: Producing complete multi-step plans with structured Result-Reason safety supervision. (d) PlanGuard Training: Task-oriented SFT first establishes task-specific plan-safety capabilities. STAC-OPD then transfers fine-grained knowledge from a strong teacher through on-policy distillation and probability-routed hard compensation. Probability Routing retains student-generated targets when the reference safety decision is already preferred and switches to teacher-reconstructed targets otherwise. Hard Compensation learns from these routed complete-response targets, consolidating stable student behavior while providing explicit sequence-level supervision for difficult cases after soft distillation.
Figure 3: (a) Distribution of the 13,692 training samples across planner families and planners. (b) Distribution of plan-level safety decisions.
Figure 4: Performance-efficiency gap under Task-Oriented SFT. (a) Average F1 across SFT epochs. The compact models converge below the 27B model. (b) Full-response and result-only inference latency under the same setting. Compact models and result-only generation substantially reduce latency, motivating knowledge transfer from the SFT-trained 27B model.
Model
MSP-Safe-Test
In-Domain
Scene-Heldout
Hazard-Heldout
Planner-Heldout
Avg
ACC ↑
F1 ↑
ACC ↑
F1 ↑
ACC ↑
F1 ↑
ACC ↑
F1 ↑
ACC ↑
F1 ↑
Qwen3.5-27B
82.60
81.68
80.08
78.70
82.20
81.02
80.60
79.92
81.37
80.33
Qwen3.5-Plus
83.00
82.55
82.11
82.04
81.80
81.39
78.80
79.92
81.43
81.48
GPT-5.5
80.80
78.38
80.49
77.25
80.00
78.54
77.80
74.25
79.77
77.10
Claude-Opus-4.6
79.72
80.54
79.67
80.39
78.20
80.43
80.52
81.45
79.53
80.70
Table 1: Performance comparison on MSP-Safe-Test. ACC (%) and F1 (%) are reported (%) for PlanGuard and representative baselines (Bold: Best, Underline: Second-Best).
Training Stage
In-Domain
Scene-Heldout
Hazard-Heldout
Planner-Heldout
Avg
ACC ↑
F1 ↑
ACC ↑
F1 ↑
ACC ↑
F1 ↑
ACC ↑
F1 ↑
ACC ↑
F1 ↑
Base
58.52
67.50
56.73
66.67
55.11
67.16
59.64
56.21
57.50
64.38
+ Task-Oriented SFT
80.80
79.92
79.27
78.57
77.80
78.69
79.60
78.66
79.37
78.96
+ Strong-Teacher OPD
83.80
83.77
80.49
80.57
78.40
79.62
82.00
81.93
81.17
81.47
+ Hard Compensation
84.80
84.55
81.71
81.56
79.60
80.53
82.60
82.35
82.18
82.25
Table 2: Training-stage ablation of PlanGuard-0.8B. Each “+” denotes the cumulative addition of the corresponding stage. ACC (%) and F1 (%) are reported. (Bold: Best, Underline: Second-Best)
Model
Stepwise
Full
ACC ↑
F1 ↑
ACC ↑
F1 ↑
EMBGuard-2B
49.00
55.65
55.00
54.55
PlanGuard-0.8B
58.00
59.62
60.00
66.10
PlanGuard-2B
56.00
57.69
67.00
72.73
Table 3: Comparison of Stepwise and Full safety detection on the auxiliary multi-step interaction diagnostic set. Stepwise aggregates subtask predictions using OR, while Full evaluates the complete plan in a single pass. ACC (%) and F1 (%) are reported. (Bold: Best, Underline: Second-Best).
MLLM-powered embodied agents deployed in real-world environments encounter physical hazards. However, existing approaches lack explicit mechanisms for identifying hazards and reasoning about action-conditioned risks, leading agents to either miss risky interactions or over-identify risks. To address this, we propose EMBGuard, the first MLLM-based safety guardrail for embodied agents designed to decouple physical risk reasoning from agent policy. By evaluating a (visual observation, action) pair, EMBGuard identifies hazardous configurations and provides natural language explanations of potential risks. Alongside EMBGuard, we contribute EMBHazard, a training dataset of 15.1K action-conditioned pairs, and EMBGuardTest, a benchmark of 329 manually curated real-world scenarios spanning seven physical risk categories. Through compositional variation of hazards and actions, we generate diverse risky and benign scenarios that agents may encounter during planning. Despite its compact size (2B, 4B), EMBGuard achieves performance competitive with proprietary MLLMs (e.g., GPT-5.1, Gemini-2.5-Pro) while significantly reducing the false-positive rates that hinder real-time deployment. We make the code, data, and models publicly available at https://github.com/dongwxxkchoi/EMBGuard
Dongwook Choi, Taeyoon Kwon, Bogyung Jeong +6
Department of Artificial Intelligence, Yonsei University · Department of Biomedical Engineering and the Department of Intelligent Precision Healthcare Convergence, Sungkyunkwan University · Independent Researcher
Embodied Task Planning with large language models faces safety challenges in real-world environments, where partial observability and physical constraints must be respected. Existing benchmarks often overlook these critical factors, limiting their ability to evaluate both feasibility and safety. We introduce SPOC, a benchmark for safety-aware embodied task planning, which integrates strict partial observability, physical constraints, step-by-step planning, and goal-condition-based evaluation. Covering diverse household hazards such as fire, fluid, injury, object damage, and pollution, SPOC enables rigorous assessment through both state and constraint-based online metrics. Experiments with state-of-the-art LLMs reveal that current models struggle to ensure safety-aware planning, particularly under implicit constraints. Code and dataset are available at https://github.com/khm159/SPOC
Hyungmin Kim, Hobeom Jeon, Dohyung Kim +2
ETRI School, University of Science and Technology, South Korean · Social Robotics Laboratory, Electronics and Telecommunication Research Institute, South Korea
Large language models are increasingly used as planners for robotic systems, yet how safely they plan remains an open question. To evaluate safe planning systematically, we introduce DESPITE, a benchmark of 12,279 tasks spanning physical and normative dangers with fully deterministic validation. Across 23 models, even near-perfect planning ability does not ensure safety: the best-planning model fails to produce a valid plan on only 0.4% of tasks but produces dangerous plans on 28.3%. Among 18 open-source models from 3B to 671B parameters, planning ability improves substantially with scale (0.4-99.3%) while safety awareness remains relatively flat (38-57%). We identify a multiplicative relationship between these two capacities, showing that larger models complete more tasks safely primarily through improved planning, not through better danger avoidance. Three proprietary reasoning models reach notably higher safety awareness (71-81%), while non-reasoning proprietary models and open-source reasoning models remain below 57%. As planning ability approaches saturation for frontier models, improving safety awareness becomes a central challenge for deploying language-model planners in robotic systems.
Tao Zhang, Kaixian Qu, Zhibin Li +4
ETH Zurich, Zurich, Switzerland. · University College London, London, United Kingdom. · Stanford University, Stanford, California, United States. +2