PlanGuard: A Guardrail for Multi-Step Plan Safety in Embodied Agents

Junchi Chen; Changtao Miao; Yuxiao Xiang; Zhenchao Jin; Haojie Yuan; Qi Chu; Tao Gong; He Liu

cs.AISep 26, 2026

PlanGuard: A Guardrail for Multi-Step Plan Safety in Embodied Agents

Authors: Junchi Chen, Changtao Miao, Yuxiao Xiang, Zhenchao Jin, Haojie Yuan, Qi Chu, Tao Gong, He Liu,

Organizations: Ant Digital Technologies, Ant Group · Anhui Province Key Laboratory of Digital Security · The University of Hong Kong · Individual Researcher

Abstract

Embodied task planners may produce multi-step plans whose subtask dependencies and interactions with the environment create physical risks during execution. Yet existing safeguards overlook such compositional risks, as general-purpose guardrails focus on semantic harm and embodied safety detectors assess subtasks in isolation. To address this gap, we introduce PlanGuard, the first pre-execution detector that evaluates the physical safety of a complete multi-step plan in its current environment. For training and evaluation, we construct a Multi-Step Plan Safety (MSP-Safe) dataset through paired task construction, plan generation using diverse planners, and safety annotation by three judges. Task-oriented SFT on MSP-Safe establishes fundamental plan-safety assessment capabilities, yet a substantial gap remains between compact models suitable for real-time deployment and stronger but costlier large models. Accordingly, we propose Strong-Teacher Adaptive Compensation for On-Policy Distillation (STAC-OPD), which provides compact models with adaptive strong-teacher supervision along their on-policy trajectories. It combines token-level distribution transfer from a fine-tuned strong teacher with probability-routed sequence-level compensation, retaining student-generated targets when the student favors the reference safety decision and using teacher-reconstructed targets otherwise. Across all test subsets, PlanGuard-2B achieves average 87.15% ACC and 87.21% F1, demonstrating effective whole-plan physical-risk detection at compact model scale. Code and dataset will be publicly released.

Figures & tables

Figure 1: Complete multi-step plan safety detection comparison. (1) General-purpose guardrails lack physical-interaction understanding, while (2) single-subtask guardrails cannot model cross-step dependencies. In contrast, (3) PlanGuard jointly analyzes the visual environment and complete multi-step plan to identify the potential fire risk before execution, demonstrating the necessity of complete multi-step plan analysis for robust embodied safety detection.

Figure 2: Pipeline of PlanGuard. (a)-(c) MSP-Safe Construction: Producing complete multi-step plans with structured Result-Reason safety supervision. (d) PlanGuard Training: Task-oriented SFT first establishes task-specific plan-safety capabilities. STAC-OPD then transfers fine-grained knowledge from a strong teacher through on-policy distillation and probability-routed hard compensation. Probability Routing retains student-generated targets when the reference safety decision is already preferred and switches to teacher-reconstructed targets otherwise. Hard Compensation learns from these routed complete-response targets, consolidating stable student behavior while providing explicit sequence-level supervision for difficult cases after soft distillation.

Figure 3: (a) Distribution of the 13,692 training samples across planner families and planners. (b) Distribution of plan-level safety decisions.

Figure 4: Performance-efficiency gap under Task-Oriented SFT. (a) Average F1 across SFT epochs. The compact models converge below the 27B model. (b) Full-response and result-only inference latency under the same setting. Compact models and result-only generation substantially reduce latency, motivating knowledge transfer from the SFT-trained 27B model.

Model	MSP-Safe-Test
	In-Domain		Scene-Heldout		Hazard-Heldout		Planner-Heldout		Avg
	ACC $\uparrow$	F1 $\uparrow$	ACC $\uparrow$	F1 $\uparrow$	ACC $\uparrow$	F1 $\uparrow$	ACC $\uparrow$	F1 $\uparrow$	ACC $\uparrow$	F1 $\uparrow$
Qwen3.5-27B	82.60	81.68	80.08	78.70	82.20	81.02	80.60	79.92	81.37	80.33
Qwen3.5-Plus	83.00	82.55	82.11	82.04	81.80	81.39	78.80	79.92	81.43	81.48
GPT-5.5	80.80	78.38	80.49	77.25	80.00	78.54	77.80	74.25	79.77	77.10
Claude-Opus-4.6	79.72	80.54	79.67	80.39	78.20	80.43	80.52	81.45	79.53	80.70

Table 1: Performance comparison on MSP-Safe-Test. ACC (%) and F1 (%) are reported (%) for PlanGuard and representative baselines (Bold: Best, Underline: Second-Best).

Training Stage	In-Domain		Scene-Heldout		Hazard-Heldout		Planner-Heldout		Avg
Training Stage	ACC $\uparrow$	F1 $\uparrow$	ACC $\uparrow$	F1 $\uparrow$	ACC $\uparrow$	F1 $\uparrow$	ACC $\uparrow$	F1 $\uparrow$	ACC $\uparrow$	F1 $\uparrow$
Base	58.52	67.50	56.73	66.67	55.11	67.16	59.64	56.21	57.50	64.38
+ Task-Oriented SFT	80.80	79.92	79.27	78.57	77.80	78.69	79.60	78.66	79.37	78.96
+ Strong-Teacher OPD	83.80	83.77	80.49	80.57	78.40	79.62	82.00	81.93	81.17	81.47
+ Hard Compensation	84.80	84.55	81.71	81.56	79.60	80.53	82.60	82.35	82.18	82.25

Table 2: Training-stage ablation of PlanGuard-0.8B. Each “+” denotes the cumulative addition of the corresponding stage. ACC (%) and F1 (%) are reported. (Bold: Best, Underline: Second-Best)

Model	Stepwise		Full
Model	ACC $\uparrow$	F1 $\uparrow$	ACC $\uparrow$	F1 $\uparrow$
EMBGuard-2B	49.00	55.65	55.00	54.55
PlanGuard-0.8B	58.00	59.62	60.00	66.10
PlanGuard-2B	56.00	57.69	67.00	72.73

Table 3: Comparison of Stepwise and Full safety detection on the auxiliary multi-step interaction diagnostic set. Stepwise aggregates subtask predictions using OR, while Full evaluates the complete plan in a single pass. ACC (%) and F1 (%) are reported. (Bold: Best, Underline: Second-Best).

AI Agent Safety

AI agent safety studies how to prevent and detect harmful actions by autonomous systems that use tools, memory, and multi-step plans.

See all 326 papers

EMBGuard: Constructing Hazard-Aware Guardrails for Safe Planning in Embodied Agents
May 29, 2026Dongwook Choi, Taeyoon Kwon, Bogyung Jeong +6LLM Guardrails AI Agent Safety
Department of Artificial Intelligence, Yonsei University · Department of Biomedical Engineering and the Department of Intelligent Precision Healthcare Convergence, Sungkyunkwan University · Independent Researcher
SPOC: Safety-Aware Planning Under Partial Observability And Physical Constraints
Feb 25, 2026Hyungmin Kim, Hobeom Jeon, Dohyung Kim +2Robot Task Planning Large Language Model-Based Robot Planning
ETRI School, University of Science and Technology, South Korean · Social Robotics Laboratory, Electronics and Telecommunication Research Institute, South Korea
Using large language models for embodied planning introduces systematic safety risks
Apr 20, 2026Tao Zhang, Kaixian Qu, Zhibin Li +4Large Language Model-Based Robot Planning Robot Safety
ETH Zurich, Zurich, Switzerland. · University College London, London, United Kingdom. · Stanford University, Stanford, California, United States. +2

May 29, 2026J/K move · Enter open · S save

EMBGuard: Constructing Hazard-Aware Guardrails for Safe Planning in Embodied Agents

Dongwook Choi, Taeyoon Kwon, Bogyung Jeong, Minju Kim+5

Department of Artificial Intelligence, Yonsei University · Department of Biomedical Engineering and the Department of Intelligent Precision Healthcare Convergence, Sungkyunkwan University · Independent Researcher

LLM Guardrails AI Agent Safety Robot Safety AI Agent Safety Benchmarks Embodied AI

Open paper AskPDF arXiv

MLLM-powered embodied agents deployed in real-world environments encounter physical hazards. However, existing approaches lack explicit mechanisms for identifying hazards and reasoning about action-conditioned risks, leading agents to either miss risky interactions or over-identify risks. To address this, we propose EMBGuard, the first MLLM-based safety guardrail for embodied agents designed to decouple physical risk reasoning from agent policy. By evaluating a (visual observation, action) pair, EMBGuard identifies hazardous configurations and provides natural language explanations of potential risks. Alongside EMBGuard, we contribute EMBHazard, a training dataset of 15.1K action-conditioned pairs, and EMBGuardTest, a benchmark of 329 manually curated real-world scenarios spanning seven physical risk categories. Through compositional variation of hazards and actions, we generate diverse risky and benign scenarios that agents may encounter during planning. Despite its compact size (2B, 4B), EMBGuard achieves performance competitive with proprietary MLLMs (e.g., GPT-5.1, Gemini-2.5-Pro) while significantly reducing the false-positive rates that hinder real-time deployment. We make the code, data, and models publicly available at https://github.com/dongwxxkchoi/EMBGuard

Abstract

AI Agent Safety

Explore similar work

EMBGuard: Constructing Hazard-Aware Guardrails for Safe Planning in Embodied Agents

SPOC: Safety-Aware Planning Under Partial Observability And Physical Constraints

Using large language models for embodied planning introduces systematic safety risks