Large Language Model-Based Robot Planning

Ask about this topic

Overview

This field studies how robots can use large language models to interpret instructions, reason about their surroundings, and plan tasks such as navigation, manipulation, and team coordination. Common approaches combine language or vision-language models with symbolic methods such as PDDL, behavior trees, scene graphs, and temporal logic, then use robot controllers to carry out the resulting plans. Systems also use visual feedback, memory, verification, and replanning to respond to changing environments and execution failures. A central challenge is making plans reliably feasible and safe in the physical world while keeping reasoning fast and affordable.

Latest papers

CardsList

235 papers48 papers in the last four weeks, up 78% on the four weeks before. 0.3% of all new papers.

  1. Humanoid Whole-Body Manipulation via Active Spatial Brain and Generalizable Action Cerebellum

    May 20, 2026Zhizhao Liang, Yi-Lin Wei, Xuhang Chen +6Humanoid Loco-ManipulationLarge Language Model-Based Robot Planning

    School of Computer Science and Engineering, Sun Yat-sen University, China · Peng Cheng Laboratory, China · Key Laboratory of Machine Intelligence and Advanced Computing, Ministry of Education, China

  2. The Yes-Man Syndrome: Benchmarking Abstention in Embodied Robotic Agents

    May 19, 2026Doguhan Yeke, Elif Su Temirel, Ananth Shreekumar +3VLM RobustnessLarge Language Model-Based Robot Planning

    Purdue University · Bilkent University · Work performed while an intern at Purdue University.

  3. Probing an Embodied LLM: When Higher Observation Fidelity Hurts Problem Solving

    May 19, 2026Oussama Zenkri, Oliver BrockLarge Language Model-Based Robot PlanningLLM Agent Evaluation

    Robotics and Biology Laboratory, Technische Universität Berlin, Germany · Science of Intelligence, Research Cluster of Excellence, Berlin, Germany · Robotics Institute Germany (RIG)

  4. NORM-Nav: Zero-Shot Mobile Robot Navigation with Natural Language Behavioral Constraints

    May 16, 2026Dongjie Huo, Junhui Wang, Chao Gao +3Language-Conditioned Robot PlanningTrajectory Generation

    College of Information Science and Technology, Beijing University of Chemical Technology · Institute for AI Industry Research (AIR), Tsinghua University · Institute of Systems Engineering and Collaborative Laboratory for Intelligent Science and Systems, Macau University of Science and Technology +1

  5. Propagating Unsafe Actions in LLM Controlled Multi-Robot Collaboration via Single Robot Compromise

    May 15, 2026Zhen Huang, Zhihuang Liu, Mengxuan Luo +2Large Language Model-Based Robot PlanningAdversarial Attacks on LLMs

    College of Computer Science and Technology, National University of Defense Technology

  6. Hybrid LLM-based Intelligent Framework for Robot Task Scheduling

    May 15, 2026Swayamjit Saha, Subhabrata Das, Haonan Duan +1Robotic ConstructionLarge Language Model-Based Robot Planning

    Department of Computer Science and Engineering, Mississippi State University, Starkville, MS, USA · 2Graduate School of Arts and Sciences, Columbia University, New York, NY, USA · 3Consumer and Community Banking, JPMorgan Chase, New York, NY, USA +2

  7. LMPath: Language-Mediated Priors and Path Generation for Aerial Exploration

    May 13, 2026Jonathan A. Diller, Fernando Cladera, Camillo J. Taylor +1Trajectory GenerationLarge Language Model-Based Robot Planning

    GRASP Laboratory, University of Pennsylvania

  8. Plan in Sandbox, Navigate in Open Worlds: Learning Physics-Grounded Abstracted Experience for Embodied Navigation

    May 11, 2026Zhixuan Shen, Jiawei Du, Ziyu Guo +5RL for RoboticsEmbodied Navigation

    School of Computing and Artificial Intelligence, Southwest Jiaotong University, China · Centre for Frontier AI Research, A*STAR, Singapore · School of Computer Science, University of Leeds, UK

  9. LASSA Architecture-Based Autonomous Fault-Tolerant Control of Unmanned Underwater Vehicles

    May 10, 2026Hong Chen, Zixiang Tang, Yuanbao Chen +1Language Model-Based ControlLarge Language Model-Based Robot Planning

    Wuhan Second Ship Design and Research Institute, Wuhan, 430060, China · School of Aeronautic Science and Engineering, Beihang University, Beijing, 100191, China

  10. NEXUS: Learning Safety Constraints and Action Models for Safe and Robust Embodied Planning

    May 10, 2026Tiehan Cui, Peipei Liu, Yanxu Mao +3Continual Learning for LLM AgentsLarge Language Model-Based Robot Planning

    School of Artificial Intelligence and Automation, Huazhong University of Science and Technology, China · School of Software, Henan University, China · Institute of Information Engineering, Chinese Academy of Sciences, China +2

  11. Hierarchical Prompting with Dual LLM Modules for Robotic Task and Motion Planning

    May 8, 2026Karolina Źróbek, Tessa Pulli, Paweł Gajewski +2Large Language Model-Based Robot PlanningLLM Prompting

    IBM, Krakow, Poland · TU Wien, Vienna, Austria · Jagiellonian University, Krakow, Poland +1

  12. Melding LLM and temporal logic for reliable human-swarm collaboration in complex scenarios

    May 8, 2026Junfeng Chen, Yuxiao Zhu, An Zhuo +6Swarm RoboticsLarge Language Model-Based Robot Planning

    School of Advanced Manufacturing and Robotics, Peking University, Beijing & 100871, China. · 2Division of Natural and Applied Sciences, Duke Kunshan University, Suzhou & 215316, China. · School of Automaton, Southeast University, Nanjing & 210096, China. +1

  13. CommandSwarm: Safety-Aware Natural Language-to-Behavior-Tree Generation for Robotic Swarms

    May 8, 2026Mohammed Majid, Amjad Yousef MajidMulti-Robot SystemsSwarm Robotics

  14. CoRAL: Contact-Rich Adaptive LLM-based Control for Robotic Manipulation

    May 4, 2026Berk Çiçek, Mert K. Er, Ozgur S. OguzLanguage Model-Based ControlLarge Language Model-Based Robot Planning

    LiRA Lab, Department of Computer Engineering, Bilkent University, Ankara, Türkiye

  15. TrajRAG: Retrieving Geometric-Semantic Experience for Zero-Shot Object Navigation

    May 3, 2026Yiyao Wang, Sixian Zhang, Keming Zhang +3Embodied NavigationLarge Language Model-Based Robot Planning

    State Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences, Beijing · University of Chinese Academy of Sciences, Beijing · Institute of Computing Technology, Chinese Academy of Sciences, Beijing

  16. Assistance Without Interruption: A Benchmark and LLM-based Framework for Non-Intrusive Human-Robot Assistance

    May 2, 2026Yuedi Zhang, Shuanghao Bai, Wanqi Zhou +4Large Language Model-Based Robot PlanningProactive Assistance

    Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University, Xi’an, China · School of Electrical Engineering, Xi’an University of Technology, Xi’an, China

  17. From Prompt to Physical Actuation: Holistic Threat Modeling of LLM-Enabled Robotic Systems

    Apr 29, 2026Neha Nagaraja, Hayretdin Bahsi, Carlo R. da CunhaLarge Language Model-Based Robot PlanningCybersecurity

    School of Informatics, Computing, and Cyber Systems, Northern Arizona University, Flagstaff, USA · Department of Software Science, Tallinn University of Technology, Tallinn, Estonia

  18. LLM-Flax : Generalizable Robotic Task Planning via Neuro-Symbolic Approaches with Large Language Models

    Apr 29, 2026Seongmin Kim, Daegyu LeeLarge Language Model-Based Robot PlanningSymbolic Planning

    Department of Physical AI Engineering, Jeonbuk National University(JBNU) · Faculty of Department of Advanced Defense Technology and Industry, Jeonbuk National University(JBNU)

  19. SafetyALFRED: Evaluating Safety-Conscious Planning of Multimodal Large Language Models

    Apr 21, 2026Josue Torres-Fonseca, Naihao Deng, Yinpei Dai +5Safety Evaluation of VLA ModelsLLM Safety Benchmarks

    University of Michigan · Boise State University

  20. Using large language models for embodied planning introduces systematic safety risks

    Apr 20, 2026Tao Zhang, Kaixian Qu, Zhibin Li +4Large Language Model-Based Robot PlanningRobot Safety

    ETH Zurich, Zurich, Switzerland. · University College London, London, United Kingdom. · Stanford University, Stanford, California, United States. +2

  21. Semantic Area Graph Reasoning for Multi-Robot Language-Guided Search

    Apr 17, 2026Ruiyang Wang, Hao-Lun Hsu, Jiwoo Kim +1Multi-Robot SystemsLarge Language Model-Based Robot Planning

  22. Foundation Models in Robotics: A Comprehensive Review of Methods, Models, Datasets, Challenges and Future Research Directions

    Apr 16, 2026Aggelos Psiris, Vasileios Argyriou, Evangelos K. Markakis +4Large Language Model-Based Robot PlanningRobot Foundation Models

    Department of Informatics and Telematics, Harokopio University of Athens, Athens, Greece · Department of Networks and Digital Media, Kingston University, London, United Kingdom · Department of Electrical and Computer Engineering, Hellenic Mediterranean University, Heraklion, Greece +4

  23. Automating Manual Tasks through Intuitive Robot Programming and Cognitive Robotics

    Apr 7, 2026Bijan Kavousian, Petar Tesic, Oliver Petrovic +1RoboticsHuman-Robot Interaction

    Laboratory for Machine Tools and Production Engineering (WZL) of RWTH Aachen University, Steinbachstraße 19, D-52074 Aachen

  24. Pre-Execution Safety Gate & Task Safety Contracts for LLM-Controlled Robot Systems

    Apr 7, 2026Ike Obi, Vishnunandan L. N. Venkatesh, Weizheng Wang +5Robot SafetyLLM Guardrails

  25. StretchBot: A Neuro-Symbolic Framework for Adaptive Guidance with Assistive Robots

    Apr 1, 2026Luca Vogelgesang, Ahmed Mehdi Soltani, Mohammadhossein Khojasteh +4Assistive RoboticsHuman-Robot Interaction

    Faculty of Science, Department of Computer Science and Robotics, University of Montpellier, 30 Pl. Eug`ene Bataillon, Montpellier, 34095, France. · Faculty of Science, Department of Computer Science, Vrije Universiteit Amsterdam, De Boelelaan 1105, Amsterdam, 1081 HV, Netherlands.

  26. Hybrid Framework for Robotic Manipulation: Integrating Reinforcement Learning and Large Language Models

    Mar 31, 2026Md Saad, Sajjad Hussain, Mohd SuhaibRobotic ManipulationLanguage-Conditioned Robot Manipulation

    Department of Mechanical Engineering, Jamia Millia Islamia, New Delhi, India · School of Architecture, Technology and Engineering, University of Brighton, Brighton, UK

  27. On-Demand Human Assistance for Task Continuation under Physical Action Failures in LLM-based Planning

    Mar 30, 2026Shoichi Hasegawa, Akira Taniguchi, Lotfi El Hafi +2Large Language Model-Based Robot PlanningRobot Failure Recovery

    Shoichi Hasegawa and Akira Taniguchi are with Ritsumeikan University; 2-150 Iwakura, Ibaraki, Osaka 567-8570, Japan. · Akira Taniguchi is with Kyushu Institute of Technology; 680-4 Kawadu, Iizuka, Fukuoka 820-0067, Japan. · Lotfi El Hafi is with Coarobo GK; 2-2-2 Hikaridai, Seika, Soura, Kyoto 619-0237, Japan. +4

  28. LLM-Enabled Low-Altitude UAV Natural Language Navigation via Signal Temporal Logic Specification Translation and Repair

    Mar 29, 2026Yuqi Ping, Huahao Ding, Tianhao Liang +6UAV NavigationRobot Navigation

    Guangdong Provincial Key Laboratory of Space-Aerial Networking and Intelligent Sensing, Harbin Institute of Technology, Shenzhen, China · Information Systems Technology and Design, Singapore University of Technology and Design, Singapore 487372 · School of Computer Science and Engineering, Central South University, Changsha, China +1

  29. Object Search in Partially-Known Environments via LLM-informed Model-based Planning and Prompt Selection

    Mar 25, 2026Abhishek Paudel, Abhish Khanal, Raihan I. Arnob +2Large Language Model-Based Robot PlanningModel-Based Planning

    Department of Computer Science at George Mason University, Fairfax, Virginia, USA.

  30. Task-Aware Positioning for Improvisational Tasks in Mobile Construction Robots via an AI Agent with Multi-LMM Modules

    Mar 24, 2026Seongju Jang, Francis Baek, SangHyun LeeRobotic ConstructionRobot Navigation

    Ph.D. Student, Department of Civil and Environmental Engineering, University of Michigan, 2350 Hayward St, Ann Arbor, MI 48109 · Assistant Professor, School of Civil and Environmental Engineering, Georgia Institute of Technology, North Avenue, Atlanta GA 30332 · Robert B. Harris Collegiate Professor, Department of Civil and Environmental Engineering, University of Michigan, 2350 Hayward St, Ann Arbor, MI 48109