cs.RONov 2, 2026

TAMPAgent: Agentic Belief-Space Planning for Mobile Manipulation

Authors: Utkarsh A. Mishra, Alperen Degirmenci, Yongxin Chen, Danfei Xu, Tucker Hermans, Caelan Garrett

Organizations: NVIDIA Research · Georgia Institute of Technology · University of Utah

Abstract

Open-world robotic mobile manipulation requires acting in physical environments with only partial information. While recent Large Vision-Language Model (LVLM) agents show strong open-world reasoning capabilities, existing architectures that rely on direct tool calling struggle with geometric uncertainty and thus are limited to fully-observable tabletop environments. In contrast, Task and Motion Planning (TAMP) systems can solve long-horizon tasks under partial observability but incur high computational costs and require closed worlds. We propose TAMPAgent, an agentic planning framework that unifies the reasoning of LVLMs with the structured constraint satisfaction of TAMP systems. We endow an LVLM agent with a belief-space planning model operating over an extensible API of perception, planning, and control tools. The agent formulates a partial task plan composed of parameterized observation and execution actions, which is subsequently completed using classical search algorithms. After extracting the plan's geometric constraints, the agent leverages its tool API to bind valid parameter values, satisfying feasibility requirements and producing an executable motion plan. Upon execution, the agent continuously updates its belief history and repeats this closed-loop process by interleaving observation and manipulation actions. Evaluated across complex household mobile manipulation tasks on Rainbow Y1 and Unitree G1 platforms, TAMPAgent significantly outperforms direct tool-calling agentic baselines, demonstrating that integrating TAMP constraint satisfaction into agentic planning loops is essential for unstructured, partially observable environments.

Figures & tables

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Imagine-TAMP: Imagination-Guided Task and Motion Planning in Partial Observability

    Sep 17, 2026Antareep Singha, Shivaram Kumar, Yoonwoo Kim +1Belief-Space PlanningActive Perception

  2. Deciding If, When, and Where to Replan for Robust Manipulation

    Aug 28, 2026Narendhiran Vijayakumar, Puru Ojha, Nav Singhal +2Large Language Model-Based Robot PlanningRobotic Manipulation

  3. Large-Language-Model-Guided State Estimation for Partially Observable Task and Motion Planning

    Mar 4, 2026Yoonwoo Kim, Raghav Arora, Roberto Martín-Martín +3Motion PlanningRobot State Estimation