MaCoPlanner: LLM-Assisted Manual-Compiled Task Planning with Proactive Safety Verification for Robotic Industrial Panel Operation
Authors: Guipeng Xin, Jiahe Xua, Mohammad Deghat, Chenhui Wan, Jie Liu, Youmin Hu, Zhongxu Hu
Organizations: School of Mechanical Science and Engineering, Huazhong University of Science and Technology, Wuhan, China · School of Mechanical and Manufacturing Engineering, University of New South Wales, Sydney, Australia · School of Civil and Hydraulic Engineering, Huazhong University of Science and Technology, Wuhan, China
Robotic industrial panel operation requires not only accurate control localization but also compliance with operating procedures, safety rules, and device-state constraints distributed across heterogeneous manuals. This study presents MaCoPlanner, a task-planning framework built on knowledge compiled from equipment manuals that converts equipment manuals into a typed intermediate representation, retrieves task- and state-relevant evidence, and uses it to support plan generation. Before actuation, candidate plans are symbolically rolled out and checked against procedural and state-transition constraints; detected violations are localized and returned for targeted repair, while unresolved plans are rejected. A separate execution interface grounds verified symbolic actions to physical controls and updates the device state. Under an independent evaluation oracle, MaCoPlanner achieves a final violation rate of 2.7%, and 26.3% of the runs in the repair analysis are rejected after exhausting the refinement budget. Compared with Raw-Manual, task success increases from 62.8% to 84.4% on Level-2 tasks and from 25.9% to 43.2% on Level-3 tasks. Experiments on a controller-panel simulator without an attached industrial load further demonstrate integrated execution feasibility under representative interaction conditions, without claiming industrial deployment readiness.
Figures & tables
Figure 1: Core challenge and planning-paradigm comparison for robotic industrial panel operation. The left part highlights hidden procedural, state-dependent, and safety constraints. The right part contrasts classical symbolic planning, direct VLM planning, and the proposed planning paradigm based on compiled equipment-manual knowledge.
Figure 2: Planning workflow of MaCoPlanner . (a) Manual knowledge compilation converts heterogeneous manuals into a structured IR that preserves procedural, safety, parameter, and relation knowledge. (b) Evidence-conditioned initial planning generates an initial task plan from the user query, device state, and structured evidence. (c) The proactive safety verification mechanism evaluates and iteratively refines the generated plan under procedural and state-dependent constraints before physical execution until a verified symbolic plan is obtained or the refinement budget is exhausted, in which case the task is rejected.
Figure 3: Query-decomposition-driven retrieval and evidence gating pipeline.
Figure 4: Illustration of manual-guided grounding and device memory. Multimodal perception and geometric parameter extraction are used to align symbolic controls with physical instances and support state-consistent execution.
Figure 5: Real-robot experimental platform for industrial panel operation.
Figure 6: Retrieval performance under cumulative retrieval ablations defined relative to Full MaCoPlanner .
Figure 7: Transfer from retrieval quality to downstream task-planning performance under proactive safety verification. The compact labels Dense, Decomp., Typed, and Typed+G. denote w/o Decomposition, Typed Indexing, and Evidence Gating , w/o Typed Indexing and Evidence Gating , w/o Evidence Gating , and Full MaCoPlanner , respectively.
Figure 9: Controlled cumulative retrieval and system-component ablations in MaCoPlanner . Colored markers and adjacent labels report exact success rates across the three task levels; the right panel reports the independent violation rate.
Figure 10: Initial and post-refinement outcomes for methods using post-generation verification or refinement. The dumbbell plot shows the change in task success; the adjacent panels report final violation rates and average refinement steps.
Figure 11: Representative case showing the positions and roles of key modules in the end-to-end task-planning and proactive-verification pipeline.
Figure 12: Backbone sensitivity on the 1/4 benchmark subset, including two pre-specified lower-capacity planners. Colored markers and adjacent labels report exact success rates across the three task levels; the right panel reports the violation rate.
Figure 13: Module-level decomposition of online runtime for typed retrieval, the initial planner/API call, LTL/FSM checking, and grounding/geometry.
Figure 14: Representative long-horizon execution sequence for switching the source to VFD, setting the speed, and starting the device.
Figure 15: Distribution of the 12 audited failed episodes and representative retrieval, grounding, and physical-execution failures.