Evaluating Long-Horizon Robot Planning under Uncertainty

Harsh Muriki; Gayathri Aravindan; Yuchen Zheng; Sonia Chernova

cs.ROOct 12, 2026

Evaluating Long-Horizon Robot Planning under Uncertainty

Authors: Harsh Muriki, Gayathri Aravindan, Yuchen Zheng, Sonia Chernova

Organizations: Georgia Institute of Technology, Atlanta, GA, USA

Abstract

Household robots must successfully navigate im-perfect circumstances, both in their internal models and the external world. We introduce a benchmark of seven long-horizon household tasks in Habitat that vary initial robot memory and environment configurations. We evaluate adaptations of ReAct, VLM-TAMP, and Tru-POMDP within a shared execution framework across 90 tasks-environment-memory combinations, totaling 1,350 episodes. Our evaluation measures task success, partial completion, and action efficiency, along with rule-based analysis of execution traces. ReAct achieves the highest overall task success (45.3%) and completion (71.7%), while having accurate initial memory does not consistently yield the highest performance for every planner. We identify recurring placement errors, precondition errors, and redundant actions through trace analysis. These findings highlight differences in how the evaluated planners use initial knowledge and respond to uncertainty.

Figures & tables

Scenario	Target	Substitute	Absent	Hidden	Distractor
BASE	✓
SUB		✓
ABS			✓
HID	✓			✓
DIS	✓				✓

TABLE I: Five environment configurations used to evaluate robustness.

Fig. 1: Overall task success, action-weighted SPL, task completion, and $\mathrm{TC\text{-}SPL}_{\mathrm{act}}$ across the three planning approaches.

Method	Accurate	Incomplete	Outdated
VLM-TAMP	0.34	0.39	0.41
ReAct	0.71	0.55	0.52
TRU-POMDP	0.39	0.32	0.47

TABLE II:

\mathrm{TC\text{-}SPL}_{\mathit{act}}

under different initial-memory conditions.

Fig. 2: $\Delta\mathrm{TC\text{-}SPL}_{act}$ from BASE across environment perturbations, averaged over all three memory conditions and reported in percentage points.

Fig. 3: Failure-finding rates across planning approaches, reported as the percentage of unsuccessful episodes exhibiting each failure; categories may overlap.

Fig. 4: Inefficiency-finding rates across planning approaches, reported as the percentage of all episodes exhibiting each inefficiency; categories may overlap.

Robot Task Planning

Robot task planning turns goals and observations into executable actions while coordinating skills, motion, uncertainty, and safety.

See all 242 papers

When Robots Do the Chores: A Benchmark and Agent for Long-Horizon Household Task Execution
May 14, 2026Zilin Zhu, Longteng Guo, Yanghong Mei +5Long-Horizon Planning Long-Horizon Agent Evaluation
Institute of Automation, Chinese Academy of Sciences, Beijing, China · Zhongguancun Academy, Beijing, China · University of Chinese Academy of Sciences, Beijing, China
TGPO: Trace-Guided Policy Optimization for Robot Task Planning via Verifiable Subgoal Generation
Feb 12, 2026Zhihong Liu, Yang Li, Renming Huang +3Robot Task Planning Large Language Model-Based Robot Planning
Shanghai Innovation Institute · Xi’an Jiao Tong University · Nankai University +1
Multi-Robot Learning-Informed Task Planning Under Uncertainty
Mar 20, 2026Abhish Khanal, Abhishek Paudel, Hung Pham +1Multi-Robot Coordination Decision-Making under Uncertainty
Department of Computer Science, George Mason University, Fairfax, VA, 22030, USA.

May 14, 2026J/K move · Enter open · S save

When Robots Do the Chores: A Benchmark and Agent for Long-Horizon Household Task Execution

Zilin Zhu, Longteng Guo, Yanghong Mei, Bowen Pang+4

Institute of Automation, Chinese Academy of Sciences, Beijing, China · Zhongguancun Academy, Beijing, China · University of Chinese Academy of Sciences, Beijing, China

Long-Horizon Planning Long-Horizon Agent Evaluation Robot Task Planning Embodied AI VLMs for Robotics Memory-Augmented Language Model Agents

Open paper AskPDF arXiv

Long-horizon household tasks demand robust high-level planning and sustained reasoning capabilities, which are largely overlooked by existing embodied AI benchmarks that emphasize short-horizon navigation or manipulation and rely on fixed task categories. We introduce LongAct, a benchmark designed to evaluate planning-level autonomy in long-horizon household tasks specified through free-form instructions. By abstracting away embodiment-specific low-level control, LongAct isolates high-level cognitive capabilities such as instruction understanding, dependency management, memory maintenance, and adaptive planning. We further propose HoloMind, a VLM-driven agent with a DAG-based long-horizon hierarchical planner, a Multimodal Spatial Memory for persistent world modeling, an Episodic Memory for experience reuse, and a global Critic for reflective supervision. Experiments with GPT-5 and Qwen3-VL models show that HoloMind substantially improves long-horizon performance while reducing reliance on model scale. Even top models achieve only 59% goal completion and 16% full-task success, underscoring the difficulty of LongAct and the need for stronger long-horizon planning in embodied agents.

Abstract

Robot Task Planning

Explore similar work

When Robots Do the Chores: A Benchmark and Agent for Long-Horizon Household Task Execution

TGPO: Trace-Guided Policy Optimization for Robot Task Planning via Verifiable Subgoal Generation

Multi-Robot Learning-Informed Task Planning Under Uncertainty