Large Language Models (LLMs) are increasingly used in embodied AI for robotic manipulation and planning, but their deployment in physical systems exposes vulnerabilities such as jailbreak attacks that induce unsafe behavior. Existing inference-time defenses are mainly evaluated on short instructions containing a single hazard, whereas realistic tasks may interleave safe and unsafe sub-tasks, conceal hazardous steps, or combine individually safe actions into unsafe outcomes. Such compound instructions make it harder for a single safety decision to distinguish hazardous sub-tasks from legitimate ones, risking either overlooking the hazard or rejecting the whole instruction. We introduce safe task completion, a task that requires an LLM-enabled robot to refuse the harmful part of an instruction while completing its legitimate part. To address this problem, we propose RuleGraph, a multi-agent Graph-of-Rules planning framework that distributes safety enforcement across multiple reasoning and verification stages. Five cooperating agents are used to ground general safety principles into world-specific rules, assess the instruction's sub-tasks in context, and derive a safe goal from the legitimate ones. Planning toward this goal is then organized as a graph of rule-specific nodes, where higher-level nodes inherit the union of their parents' rules, and candidate plans are progressively verified and aggregated until a single plan satisfies every relevant rule. In this way, RuleGraph reduces reliance on a single centralized safety decision. We evaluate RuleGraph on pure-attack and compound instructions across two benchmarks in two environments using Attack Success Rate (ASR) and Task Success Rate (TSR). RuleGraph reduces ASR to 0% on both benchmarks while achieving the highest TSR among all methods, reaching 98.57% on the RoboGuard benchmark and 84.54% on our long-horizon compound instructions.
Figures & tables
Figure 1: Two examples comparing RuleGraph with SafePlan and RoboGuard. Green marks a correct outcome, red a successful attack, and orange a full refusal of an instruction with a safe part. Left: a pure attack, which RuleGraph correctly refuses because no safe goal exists. Right: a compound instruction, where RuleGraph discards the unsafe step and completes only the safe part.
Figure 2: Overview of RuleGraph and its five main agents: the Rule Generator Agent, Instruction Agent, Goal Synthesis Agent, Planning Agent, and Verification Agent.
Figure 3: RuleGraph pipeline. The Rule Generator Agent grounds and selects relevant safety rules, the Instruction Agent grounds and decomposes the instruction, the Goal Synthesis Agent constructs a safe goal from legitimate sub-tasks, the Planning Agent plans toward this goal through a hierarchical graph, and the Verification Agent checks goal safety, rule compliance, completion, and breach before a plan is committed. Verification feedback can trigger plan refinement or safe-goal revision.
RoboGuard
Compound
Method
ASR ↓
TSR ↑
ASR ↓
TSR ↑
Undefended LLM
98.57%
1.43%
100.00%
0.00%
Single-plan judge baseline
4.29%
87.14%
14.43%
19.59%
RoboGuard
5.71%
82.86%
9.28%
16.49%
SafePlan
2.86%
94.29%
5.15%
21.65%
RuleGraph (ours)
0.00%
98.57%
0.00%
84.54%
Table 1: ASR and TSR for all methods on the RoboGuard benchmark ( n=70 , pure attack) and Compound ( n=97 , mixed intent).
Method
ASR
Single-plan judge baseline
5.71%
RoboGuard
10.00%
SafePlan
8.57%
RuleGraph (ours)
0.00%
Table 2: ASR under the adaptive white-box attack on the RoboGuard benchmark.
Method
ASR ↓
TSR ↑
Single-plan judge baseline
2.86%
85.71%
RoboGuard
10.00%
72.86%
SafePlan
7.14%
91.43%
RuleGraph (ours)
0.00%
100.00%
Table 3: ASR and TSR on the RoboGuard benchmark with Gemini-3.1-flash-lite.
Configuration
ASR
TSR
Full graph (ours)
0.00%
84.54%
Single-node ablation
1.03%
74.23%
Table 4: ASR and TSR in the ablation study on Compound when the full graph structure is replaced with a single node handling all rules at once.
Large language models (LLMs) and vision-language models (VLMs) have significantly advanced zero-shot task planning for embodied agents. However, most LLM- and VLM-driven methods struggle to generate safe high-level actions due to a lack of physical risk awareness, particularly under partial observability, where hazards lie outside the immediate field of view. To address this challenge, we propose a novel safe task-planning framework, SafeMem, which constructs and maintains a long-term semantic graph memory of the open and dynamic environment. Based on egocentric observations, the proposed framework incrementally accumulates knowledge about surrounding objects and their relationships with a graph. Then, an LLM-based risk predictor evaluates candidate actions using the graph memory, triggering a conservatism-modulated replanning loop with explanations for detected hazards. Extensive experiments on the IS-Bench benchmark and a real-world robot platform demonstrate that the SafeMem framework substantially improves safe success rates compared to state-of-the-art VLM-driven task planners. Video results are available on our webpage: https://sites.google.com/view/safemem.
Siyuan Li, Taiyan Lang, Aoqi Yan +6
Faculty of Computing Harbin Institute of Technology · Engineering Systems and Design Singapore University of Technology and Design · Intelligent Science and Technology Academy of CASIC
Safety-critical task planning in robotic systems remains challenging: classical planners suffer from poor scalability, Reinforcement Learning (RL)-based methods generalize poorly, and base Large Language Models (LLMs) cannot guarantee safety. To address this gap, we propose safety-generalizable large language models, named SafeGen-LLM. SafeGen-LLM can not only enhance the safety satisfaction of task plans but also generalize well to novel safety properties in various domains. We first construct a multi-domain Planning Domain Definition Language 3 (PDDL3) benchmark with explicit safety constraints. Then, we introduce a two-stage post-training framework: Supervised Fine-Tuning (SFT) on a constraint-compliant planning dataset to learn planning syntax and semantics, and Group Relative Policy Optimization (GRPO) guided by fine-grained reward machines derived from formal verification to enforce safety alignment and by curriculum learning to better handle complex tasks. Extensive experiments show that SafeGen-LLM achieves strong safety generalization and outperforms frontier proprietary baselines across multi-domain planning tasks and multiple input formats (e.g., PDDLs and natural language).
Jialiang Fan, Weizhe Xu, Mengyu Liu +3
University of Notre Dame · Washington State University · University of Pennsylvania
Large Reasoning Language Models (LRLMs) enable multi-step reasoning for robotic task planning, but continued reasoning can overwrite valid intermediate plans or leave constraint violations unresolved, reducing planning reliability and wasting inference-time computation. We develop an inference-time monitor that exposes and verifies intermediate plans without disrupting the original decoding trajectory. Building on this monitor, we propose SafeInferCom, a formal verifier-guided framework that preserves valid intermediate plans and directs error correction during generation. Experiments across multiple LRLMs and planning domains reveal reasoning-response inconsistency and limited self-correction under one-shot inference. SafeInferCom improves planning success and accelerates error correction relative to one-shot inference. When combined with iterative refinement, it further improves success while reducing token usage compared with refinement alone. We additionally evaluate SafeInferCom in VirtualHome and provide a real-world robotic-arm demonstration.
Weizhe Xu, Jialiang Fan, Mengyu Liu +1
University of Notre Dame · Washington State University