MaCoPlanner: LLM-Assisted Manual-Compiled Task Planning with Proactive Safety Verification for Robotic Industrial Panel Operation
Authors: Guipeng Xin, Jiahe Xua, Mohammad Deghat, Chenhui Wan, Jie Liu, Youmin Hu, Zhongxu Hu
Organizations: School of Mechanical Science and Engineering, Huazhong University of Science and Technology, Wuhan, China · School of Mechanical and Manufacturing Engineering, University of New South Wales, Sydney, Australia · School of Civil and Hydraulic Engineering, Huazhong University of Science and Technology, Wuhan, China
Robotic industrial panel operation requires not only accurate control localization but also compliance with operating procedures, safety rules, and device-state constraints distributed across heterogeneous manuals. This study presents MaCoPlanner, a task-planning framework built on knowledge compiled from equipment manuals that converts equipment manuals into a typed intermediate representation, retrieves task- and state-relevant evidence, and uses it to support plan generation. Before actuation, candidate plans are symbolically rolled out and checked against procedural and state-transition constraints; detected violations are localized and returned for targeted repair, while unresolved plans are rejected. A separate execution interface grounds verified symbolic actions to physical controls and updates the device state. Under an independent evaluation oracle, MaCoPlanner achieves a final violation rate of 2.7%, and 26.3% of the runs in the repair analysis are rejected after exhausting the refinement budget. Compared with Raw-Manual, task success increases from 62.8% to 84.4% on Level-2 tasks and from 25.9% to 43.2% on Level-3 tasks. Experiments on a controller-panel simulator without an attached industrial load further demonstrate integrated execution feasibility under representative interaction conditions, without claiming industrial deployment readiness.
Figures & tables
Figure 1: Core challenge and planning-paradigm comparison for robotic industrial panel operation. The left part highlights hidden procedural, state-dependent, and safety constraints. The right part contrasts classical symbolic planning, direct VLM planning, and the proposed planning paradigm based on compiled equipment-manual knowledge.
Figure 2: Planning workflow of MaCoPlanner . (a) Manual knowledge compilation converts heterogeneous manuals into a structured IR that preserves procedural, safety, parameter, and relation knowledge. (b) Evidence-conditioned initial planning generates an initial task plan from the user query, device state, and structured evidence. (c) The proactive safety verification mechanism evaluates and iteratively refines the generated plan under procedural and state-dependent constraints before physical execution until a verified symbolic plan is obtained or the refinement budget is exhausted, in which case the task is rejected.
Figure 3: Query-decomposition-driven retrieval and evidence gating pipeline.
Figure 4: Illustration of manual-guided grounding and device memory. Multimodal perception and geometric parameter extraction are used to align symbolic controls with physical instances and support state-consistent execution.
Figure 5: Real-robot experimental platform for industrial panel operation.
Figure 6: Retrieval performance under cumulative retrieval ablations defined relative to Full MaCoPlanner .
Figure 7: Transfer from retrieval quality to downstream task-planning performance under proactive safety verification. The compact labels Dense, Decomp., Typed, and Typed+G. denote w/o Decomposition, Typed Indexing, and Evidence Gating , w/o Typed Indexing and Evidence Gating , w/o Evidence Gating , and Full MaCoPlanner , respectively.
Figure 9: Controlled cumulative retrieval and system-component ablations in MaCoPlanner . Colored markers and adjacent labels report exact success rates across the three task levels; the right panel reports the independent violation rate.
Figure 10: Initial and post-refinement outcomes for methods using post-generation verification or refinement. The dumbbell plot shows the change in task success; the adjacent panels report final violation rates and average refinement steps.
Figure 11: Representative case showing the positions and roles of key modules in the end-to-end task-planning and proactive-verification pipeline.
Figure 12: Backbone sensitivity on the 1/4 benchmark subset, including two pre-specified lower-capacity planners. Colored markers and adjacent labels report exact success rates across the three task levels; the right panel reports the violation rate.
Figure 13: Module-level decomposition of online runtime for typed retrieval, the initial planner/API call, LTL/FSM checking, and grounding/geometry.
Figure 14: Representative long-horizon execution sequence for switching the source to VFD, setting the speed, and starting the device.
Figure 15: Distribution of the 12 audited failed episodes and representative retrieval, grounding, and physical-execution failures.
Industrial panel operation is knowledge-intensive and safety-critical. Beyond control recognition and action generation, execution must satisfy constraints in operation manuals and safety regulations. While foundation-model-based planners show strong semantic capability, they typically lack computable, localizable, and reproducible mechanisms for violation detection and repair. To address this, we propose PanelShield, a verifiable closed-loop safety planning framework for manual-guided industrial panel operation. The framework generates parameterized action primitive sequences from task-relevant manual evidence and applies dual formal verification with LTL and a Safety FSM to enforce cross-step temporal correctness and local transition legality. When violations occur, it outputs a structured counterexample with the earliest violating step and cause, enabling targeted repair and re-verification. We build a multi-level long-horizon planning benchmark covering three representative industrial device panels, and evaluate the framework in simulation and real-world robotic experiments. Results show that PanelShield improves complex safety-constrained task performance over foundation-model-only planning baselines while reducing the violation rate to 2.7%, with 4.1 s total latency. Real-world experiments demonstrate end-toend feasibility. Overall, PanelShield offers a verifiable approach to robotic panel operation that balances flexibility, safety, and auditability.
Guipeng Xin, Jiahe Xu, Chenhui Wan +3
School of Mechanical Science and Engineering Huazhong University of Science and Technology Wuhan, China
Safety-critical task planning in robotic systems remains challenging: classical planners suffer from poor scalability, Reinforcement Learning (RL)-based methods generalize poorly, and base Large Language Models (LLMs) cannot guarantee safety. To address this gap, we propose safety-generalizable large language models, named SafeGen-LLM. SafeGen-LLM can not only enhance the safety satisfaction of task plans but also generalize well to novel safety properties in various domains. We first construct a multi-domain Planning Domain Definition Language 3 (PDDL3) benchmark with explicit safety constraints. Then, we introduce a two-stage post-training framework: Supervised Fine-Tuning (SFT) on a constraint-compliant planning dataset to learn planning syntax and semantics, and Group Relative Policy Optimization (GRPO) guided by fine-grained reward machines derived from formal verification to enforce safety alignment and by curriculum learning to better handle complex tasks. Extensive experiments show that SafeGen-LLM achieves strong safety generalization and outperforms frontier proprietary baselines across multi-domain planning tasks and multiple input formats (e.g., PDDLs and natural language).
Jialiang Fan, Weizhe Xu, Mengyu Liu +3
University of Notre Dame · Washington State University · University of Pennsylvania
Programmable logic controllers (PLCs) coordinate industrial equipment by reading sensor inputs and issuing control commands. Evaluating whether large language model (LLM)-generated PLC programs satisfy task requirements and safety constraints requires observing how their commands affect device and workpiece states. We introduce PLCWorld, a common closed-loop execution environment and benchmark that couples Structured Text (ST) execution with simulated plant responses and sensor feedback. Grounded in control relations identified in industrial PLC programs and engineering documentation, PLCWorld contains 100 synthetic tasks and 473 registered task-condition pairs across Motion Control and Material Handling, with difficulty defined by control-dependency scope. A common protocol reports Task Success and Safety Violation separately. Validation combines practitioner review, reference and alternative programs, targeted counterexamples, specification-evaluator alignment checks, and comparisons with independent ST runtimes. Reference and alternative programs satisfy their applicable cases, while all 542 targeted counterexamples activate their designated evaluator rules under at least one registered condition. Execution Gap relates submission-profile acceptance to subsequent task failure or observed Safety Violation. Across the constructed task groups, direct GPT-5.5 achieves 82.70% Task Success on Easy cases but 25.10% on Hard cases. Evaluations of six LLMs and four adapted generation-and-verification workflows further expose differences between completion, safety, and generation cost. Our code, simulation environment, benchmark tasks, and baseline implementations are publicly available at https://yunji0516.github.io/PLCWorld/.