RoboClaw: An Agentic Framework for Scalable Long-Horizon Robotic Tasks

Ruiying Li; Yunlang Zhou; YuYao Zhu; Kylin Chen; Jingyuan Wang; Sukai Wang; Kongtao Hu; Minhui Yu

cs.ROMar 12, 2026

RoboClaw: An Agentic Framework for Scalable Long-Horizon Robotic Tasks

Authors: Ruiying Li, Yunlang Zhou, YuYao Zhu, Kylin Chen, Jingyuan Wang, Sukai Wang, Kongtao Hu, Minhui Yu,

Organizations: AgiBot, China · National University of Singapore · Shanghai Jiao Tong University, Shanghai 200240, China · MoE Key Lab of Artificial Intelligence, AI Institute, SJTU

Abstract

Vision-Language-Action (VLA) systems have shown strong potential for language-driven robotic manipulation. However, scaling them to long-horizon tasks remains challenging. Existing pipelines typically separate data collection, policy learning, and deployment, resulting in heavy reliance on manual environment resets and brittle multi-policy execution. We present RoboClaw, an agentic robotics framework that unifies data collection, policy learning, and task execution under a single VLM-driven controller. At the policy level, RoboClaw introduces Entangled Action Pairs (EAP), which couple forward manipulation behaviors with inverse recovery actions to form self-resetting loops for autonomous data collection. This mechanism enables continuous on-policy data acquisition and iterative policy refinement with minimal human intervention. During deployment, the same agent performs high-level reasoning and dynamically orchestrates learned policy primitives to accomplish long-horizon tasks. By maintaining consistent contextual semantics across collection and execution, RoboClaw reduces mismatch between the two phases and improves multi-policy robustness. Experiments in real-world manipulation tasks demonstrate improved stability and scalability compared to conventional open-loop pipelines, while significantly reducing human effort throughout the robot lifecycle, achieving a 25% improvement in success rate over baseline methods on long-horizon tasks and reducing human time investment by 53.7%.

Figures & tables

Figure 1: RoboClaw workflow across the robot policy lifecycle. A robot developer specifies system configuration, MCP tools, and skills, while RoboClaw provides file-based memory, memory embeddings, search, and management. Data is collected through basic human demonstrations followed by online rollout with EAP self resetting, producing a VLA policy pool that is continuously updated via streaming data. Activated policies are then used to execute complex long-horizon tasks under high-level plans and contextual guidance.

Figure 2: RoboClaw system architecture. A Vision-Language-Model (VLM) acts as a meta-controller operating under an in-context learning paradigm. Multimodal observations are integrated with structured memory (role identity, task-level memory, and working memory) to form the decision context. Through chain-of-thought (CoT) reasoning, the agent generates high-level decisions and invokes tools through a unified MCP execution interface. The same agent core governs both data collection and policy deployment, ensuring consistent control semantics across the full system lifecycle.

Figure 3: RoboClaw Autonomous Data Collection Workflow. This diagram illustrates the process of the agent interacting with a user to initiate a data collection task for the robot (”place the primer into the drawer”). The agent autonomously processes visual observations using MCP tools, evaluates the initial state of the environment, and formulates a task plan. Subsequently, it continuously executes a forward-reverse operational loop (i.e., placing the item into the drawer and then taking it out) while monitoring for anomalies in real-time during execution, thereby continuously acquiring the robotic manipulation dataset.

Figure 4: Human effort comparison for data collection. (a) Relative human time required to collect the same amount of data. (b) Relative human intervention during rollout execution. All values are normalized with respect to our method (Ours = 1). (c) Success rate across iterations on the vanity table organization task. RoboClaw (Ours) significantly outperforms both end-to-end VLA baselines and the expected success rate computed as the product of four independent subtask success rates. The improvement comes from RoboClaw’s ability to monitor task progress and automatically invoke recovery policies when failures occur. Results are averaged over 20 trials.

General Settings		LoRA Settings
Precision	bfloat16	Rank ( $r$ )	16
Batch size	16	Alpha ( $\alpha$ )	16
Training steps	10k	Dropout	0.1
Warmup steps	100	Target modules	all-linear
Learning rate	$2.5\times 10^{-5}$	Inference steps	3
Gradient checkpointing	✓

Table 1: Training hyperparameters for

\pi_{0.5}

fine-tuning.

Task	Body Lotion	Primer	Lipstick	Tissue Wipe
Success Rate	36/50	38/50	43/50	39/50

Table 2: Success rates of inverse reset policies across four manipulation tasks.

Iteration	Body Lotion	Primer	Lipstick	Tissue Wipe
1	21/50	23/50	2/50	11/50
2	25/50	31/50	4/50	13/50
3	32/50	31/50	11/50	14/50
4	37/50	34/50	16/50	21/50
5	43/50	40/50	23/50	26/50

Table 3: Success rates of forward manipulation policies across rollout iterations.

Figure 5: Long-horizon task execution with agent orchestration. The same VLM-based agent plans over the vanity table tidying task and dynamically composes independent forward policy checkpoints (primer placement, lipstick insertion, lotion placement and tissue wipe), invoking re-planning when needed.

Long-Horizon Robotic Manipulation

Long-horizon robotic manipulation studies how robots plan, remember, and recover across extended sequences of physical tasks.

See all 413 papers

ROSClaw: A Hierarchical Semantic-Physical Framework for Heterogeneous Multi-Agent Collaboration
Apr 6, 2026Rongfeng Zhao, Xuanhao Zhang, Zhaochen Guo +4VLMs for Robotics Cross-Embodiment Robot Learning
Shanghai Research Institute for Intelligent Autonomous Systems, Shanghai 201210, China · National Key Laboratory of Autonomous Intelligent Unmanned Systems (Tongji University), Shanghai 201210, China · Frontiers Science Center for Intelligent Autonomous Systems, Shanghai 201210, China +1
LiLo-VLA: Compositional Long-Horizon Manipulation via Linked Object-Centric Policies
Feb 25, 2026Yue Yang, Shuo Cheng, Yu Fang +4Long-Horizon Robotic Manipulation Robotic Manipulation
University of North Carolina at Chapel Hill · Georgia Institute of Technology · Carnegie Mellon University
S $^2$ -VLA: State-Space Guided Vision-Language-Action Models for Long-Horizon Manipulation
Jun 26, 2026Zhipeng Xie, Zongyi Han, Xiangyi Wei +3Long-Horizon Robotic Manipulation Vision-Language-Action Models
School of Computer Science and Technology, East China Normal University, Shanghai 200062, China · 2State Key Laboratory of Submarine Geoscience, School of Automation and Intelligent Sensing, Shanghai Jiao Tong University, Shanghai 200240, China

Apr 6, 2026J/K move · Enter open · S save

ROSClaw: A Hierarchical Semantic-Physical Framework for Heterogeneous Multi-Agent Collaboration

Rongfeng Zhao, Xuanhao Zhang, Zhaochen Guo, Xiang Shao+3

Shanghai Research Institute for Intelligent Autonomous Systems, Shanghai 201210, China · National Key Laboratory of Autonomous Intelligent Unmanned Systems (Tongji University), Shanghai 201210, China · Frontiers Science Center for Intelligent Autonomous Systems, Shanghai 201210, China+1

VLMs for Robotics Cross-Embodiment Robot Learning Multi-Robot Coordination Long-Horizon Robotic Manipulation Continual Robot Learning

Open paper AskPDF arXiv

The integration of large language models (LLMs) with embodied agents has improved high-level reasoning capabilities; however, a critical gap remains between semantic understanding and physical execution. While vision-language-action (VLA) and vision-language-navigation (VLN) systems enable robots to perform manipulation and navigation tasks from natural language instructions, they still struggle with long-horizon sequential and temporally structured tasks. Existing frameworks typically adopt modular pipelines for data collection, skill training, and policy deployment, resulting in high costs in experimental validation and policy optimization. To address these limitations, we propose ROSClaw, an agent framework for heterogeneous robots that integrates policy learning and task execution within a unified vision-language model (VLM) controller. The framework leverages e-URDF representations of heterogeneous robots as physical constraints to construct a sim-to-real topological mapping, enabling real-time access to the physical states of both simulated and real-world agents. We further incorporate a data collection and state accumulation mechanism that stores robot states, multimodal observations, and execution trajectories during real-world execution, enabling subsequent iterative policy optimization. During deployment, a unified agent maintains semantic continuity between reasoning and execution, and dynamically assigns task-specific control to different agents, thereby improving robustness in multi-policy execution. By establishing an autonomous closed-loop framework, ROSClaw minimizes the reliance on robot-specific development workflows. The framework supports hardware-level validation, automated generation of SDK-level control programs, and tool-based execution, enabling rapid cross-platform transfer and continual improvement of robotic skills. Ours project page: https://www.rosclaw.io/.

Abstract

Long-Horizon Robotic Manipulation

Explore similar work

ROSClaw: A Hierarchical Semantic-Physical Framework for Heterogeneous Multi-Agent Collaboration

LiLo-VLA: Compositional Long-Horizon Manipulation via Linked Object-Centric Policies

S2^22-VLA: State-Space Guided Vision-Language-Action Models for Long-Horizon Manipulation

S $^2$ -VLA: State-Space Guided Vision-Language-Action Models for Long-Horizon Manipulation