cs.RONov 8, 2026

CAFE: Closed-Loop Autonomous Task Formulation and Execution without Instance-Specific Instructions

Authors: Mingjia Chen, Xuewen Zhao, Philip Dames, Qi Mao, Fei Xie, Jun Chen

Organizations: School of Electrical and Automation Engineering, Nanjing Normal University, Nanjing, Jiangsu 210023, China · Department of Mechanical Engineering, Temple University, Philadelphia, PA 19122, USA

Abstract

Scene-driven robot autonomy requires selecting tasks from observations as well as executing them. We study scene-driven task formulation and execution under predefined cleaning, safety, and storage/organization categories, without instance-specific goals or action sequences. CAFE maintains online memory of objects, functional zones, and execution states. A rule-guided Trigger-LLM admits formulation from accumulated evidence; Draft, Complete, and Audit propose goals, resolve execution details, and check dependencies. Outcomes update memory for correction and subsequent discovery. Evaluation in 12 AI2-THOR scenes compares three backbones and task-complexity tiers using coverage, constraint, and execution metrics. Tri-LLM records higher GCR scores than the evaluated baselines across three backbones and task-complexity tiers. Controls examine completion, auditing, zones, feedback, triggering, and prompt examples. LimoCobot traces illustrate storage, cleanup, and recovery from a failed grasp. The study examines how shared scene state connects task discovery to execution and outcome-driven revision.

Figures & tables

Explore similar work

CardsList
  1. An LLM-Driven Closed-Loop Autonomous Learning Framework for Robots Facing Uncovered Tasks in Open Environments

    Apr 24, 2026Hong SuContinual Robot LearningRobot Skill Learning