Cracking Mobile Manipulation: From Failure Diagnosis to Agentic Robot System

Zixuan Wang; Jia-Ning Li; Yinzhou Tang; Yanzhao Yu; Biao Wu; Meng Fang; Xin Li; Yiwu Zhong

cs.ROOct 14, 2026

Cracking Mobile Manipulation: From Failure Diagnosis to Agentic Robot System

Authors: Zixuan Wang, Jia-Ning Li, Yinzhou Tang, Yanzhao Yu, Biao Wu, Meng Fang, Xin Li, Yiwu Zhong

Organizations: School of Intelligence Science and Technology, Peking University · State Key Laboratory of General Artificial Intelligence, Peking University · Alibaba DAMO Academy, Alibaba Group · Institute of Automation, Chinese Academy of Sciences · Tsinghua University · University of Technology Sydney · University of Liverpool

Abstract

Recent vision-language-action (VLA) models have shown strong performance on local robot manipulation, yet their success does not readily translate to long-horizon mobile manipulation. In our experiments, a pretrained subtask VLA achieves 80% success rate on atomic manipulation tasks but only 19% when these skills are composed into mobile manipulation. We argue that this gap stems from failures across the execution process rather than from local manipulation capability alone. To uncover these bottlenecks, we systematically diagnose failures in long-horizon mobile manipulation and identify three dominant regimes: task planning, navigation and base positioning, and local manipulation. Based on this diagnosis, we develop failure-guided agentic intervention that augments a fixed subtask VLA with demonstration-based planning guidance, adaptive visual memory, accessibility-aware base adjustment, and local end-effector adjustment. On RoboCasa365, these interventions progressively improve task success from 19% to 47%, with the finalized system reaching 49% on held-out episode seeds, compared with 18% for the baseline. The same design improves performance from 14% to 37% on BEHAVIOR-1K and from 5/20 to 13/20 on real-robot tasks. These results suggest that reliable mobile manipulation can be advanced by diagnosing and intervening on the current bottleneck, rather than uniformly scaling the underlying policy. Our code will be released at https://github.com/PKU-VaLuE-Lab/Cracking-Mobile-Manipulation.git.

Figures & tables

Figure 1: From the mobile manipulation gap to failure-guided intervention. (a) Strong subtask VLA does not translate to reliable long-horizon mobile manipulation. (b) Failure analysis identifies the dominant failures: task planning, navigation and base positioning, and manipulation. These observations motivate our agentic intervention, including: in-context guidance from video demonstrations, adaptive visual memory, accessibility checking and base adjustment, and end-effector adjustment.

Figure 2: Failure attribution of the agentic baseline. Among the 81 failed episodes, the primary failures are attributed to task planning (21.0%), navigation and base positioning (25.9%), and local manipulation (53.1%).

Figure 3: Overview of the failure-guided agentic system. The agentic planner leverages the skill library for task planning guidance, visual memory for relevant observation retrieval, and geometric skills for accessibility checking, base adjustment, and end-effector adjustment.

Figure 4: Progressive integration of failure-guided interventions. Each point adds one intervention group to the previous configuration.

Figure 5: Failure distribution during progressive integration. Counts are reported over the same 100 matched episodes.

Type	Method	Development (%)	Held-out (%)
VLA	$\pi_{0}$ ( Black et al., 2025b )	2.0	2.0
	$\pi_{0.5}$ ( Black et al., 2025a )	3.0	4.0
	GR00T N1.6 ( NVIDIA, 2025b )	7.0	8.0
	GR00T N1.5 ( NVIDIA, 2025a )	8.0	9.0
	RLDX-1 ( Kim et al., 2026a )	15.0	14.0
	Xiaomi Robotics 1 ( Xiaomi Robotics Team et al., 2026 )	44.0	45.0

Table 1: Performance of mobile manipulation tasks on the RoboCasa365 benchmark. We evaluate the 20 of 32 composite tasks that require mobile manipulation.

Appendix figures & tables18 assets

Supplementary material from the paper’s appendix.

Appendix

Setting	Value
VLM backbone	Qwen3-VL-4B-Instruct
Visual encoder	DINOv2 ViT-S/14
Action model	DiT-B
Image resolution	$224\times 224$
State dimension	16
Action dimension	12

Appendix

Table 3: Training settings of the subtask VLA. The same checkpoint is used by all agentic configurations unless otherwise stated.

Method	Batch Size	Training Steps
$\pi_{0}$	64	75,000
$\pi_{0.5}$	64	75,000
GR00T N1.5	128	120,000
GR00T N1.6	128	120,000
RLDX-1	192	250,000
Xiaomi Robotics 1	512	120,000

Appendix

Table 4: Training settings of the released end-to-end baselines. Batch sizes and training steps follow the corresponding RoboCasa365 leaderboard submissions.

Failure Category	Episodes	Unanimous
Task planning	7	6
Navigation and base positioning	8	7
Manipulation	15	13
Total	30	26

Appendix

Table 5: Agreement of independent failure attribution on a random subset of baseline failures.

Fine-grained failure category	Episodes	Unanimous
Placement target or receptacle assignment	4	3
Subtask ordering	3	2
Ineffective approach or poor base positioning	4	4
Collision, path blockage, or stalled navigation	3	2
Unsuitable base orientation	2	2
Incorrect object placement	5	4

Appendix

Table 6: Agreement of fine-grained failure attribution. Three reviewers independently annotate 30 sampled failed episodes without access to the system configuration or enabled skills.

Adaptive visual memory
Parameter	Value
Maximum candidate frames	256
Retrieved frames ( $k$ )	16
Recall model	Qwen3.8-27B
Input image resolution	$448\times 448$
Geometric skills

Appendix

Table 7: Implementation settings for adaptive visual memory and geometric skills.

Comparison	Success (%)	$\Delta$ (pp)	$n_{01}$	$n_{10}$	95% CI (pp)	McNemar $p$
Baseline $\rightarrow$ +Demo	19 $\rightarrow$ 25	+6	8	2	[0, 12]	0.109
+Demo $\rightarrow$ +Memory	25 $\rightarrow$ 29	+4	7	3	[-2, 10]	0.344
+Memory $\rightarrow$ +Base	29 $\rightarrow$ 40	+11	15	4	[3, 19]	0.019
+Base $\rightarrow$ +EEF	40 $\rightarrow$ 47	+7	10	3	[0, 14]	0.092
Baseline $\rightarrow$ Full	19 $\rightarrow$ 47	+28	30	2	[18, 38]	$<0.001$

Appendix

Table 8: Paired analysis of progressive integration. The first four rows compare consecutive configurations; the final row compares the baseline with the full system on the same development episodes.

\Delta

is reported in percentage points.

Added intervention	Recovered	Planning	Nav./Base	Manip.
Skill library	8	5	1	2
Adaptive visual memory	7	4	1	2
Base adjustment	15	0	11	4
End-effector adjustment	10	0	1	9

Appendix

Table 10: Primary failure types of episodes recovered after each intervention.

Baseline $\rightarrow$ Full	Success	Planning	Nav./Base	Manip.	Total
Success	17	0	1	1	19
Planning	9	1	1	6	17
Nav./Base	12	0	3	6	21
Manipulation	9	0	0	34	43
Total	47	1	5	47	100

Appendix

Table 11: Paired episode-level outcome transitions from the agentic baseline to the full system.

Figure 6: Fine-grained failure attribution for the full system. Counts are reported over the same 100 matched development episodes.

Task horizon	# Tasks	Baseline	Full system	$\Delta$ (pp)
2–4 subtasks	13	15/65 (23.1%)	35/65 (53.8%)	+30.7
5–15 subtasks	7	4/35 (11.4%)	12/35 (34.3%)	+22.9
Overall	20	19/100 (19.0%)	47/100 (47.0%)	+28.0

Appendix

Table 13: Performance across task horizons. Tasks are grouped by the number of annotated subtasks.

Method	Storage Box Transport	Table Service	Overall
$\pi_{0.5}$ (end-to-end VLA)	2/10	2/10	4/20
Agentic baseline	2/10	3/10	5/20
Full system	6/10	7/10	13/20

Appendix

Table 15: Real-robot evaluation. Each method is evaluated for 10 trials on each task.

Figure 7: Real-robot execution of the Table Service task. A representative successful rollout of the full system on Astribot-S1. The robot moves the prepared dish from the serving cart to the dining table, adds the topping, and places the wine glass beside the dish.

Configuration	Task Success Rate(%)	$\Delta$ from Full (pp)
Full system	47.0	–
$-$ Skill library	39.0	-8.0
$-$ Adaptive visual memory	41.0	-6.0
$-$ Base adjustment	35.0	-12.0
$-$ End-effector adjustment	40.0	-7.0

Appendix

Table 16: Leave-one-out ablation of intervention groups on the development episodes.

Memory Access	Uniform	Relevance-based
Every turn	40.0	45.0
Planner-triggered	44.0	47.0

Appendix

Table 18: Separating memory access and frame selection. All configurations use a 16-frame memory budget and the same 100 matched development episodes.

Figure 8: Multi-stage closed-loop execution. The high-level planner decomposes the task into a sequence of subtask-VLA calls, including object pickup and placement, navigation to the microwave, interaction with the microwave door, and pressing the start button.

Figure 9: Accessibility-aware base adjustment before manipulation. After navigating near the target, the planner checks whether the current base pose is suitable for manipulation and invokes approach_xy for a local base adjustment before continuing with the subtask VLA.

Figure 10: Coordinated navigation and manipulation in a long-horizon task. The planner uses the same subtask VLA to open the cabinet, pick and place the bread, transport the basket, navigate to the dining counter, and complete the final placement.

Figure 11: End-effector adjustment after a failed placement attempt. After the initial placement fails, the planner invokes a local end-effector adjustment and retries the placement with the same subtask VLA.

Explore similar work

MobileManiBench: Simplifying Model Verification for Mobile Manipulation
Feb 5, 2026Wenbo Wang, Fangyun Wei, QiXiu Li +5Mobile Manipulation Robotics Simulation
University of Sydney · 1Microsoft Research Asia · 3Tsinghua University
Make Your VLA More Robust Without More Data By Interleaving Motion Planning
May 31, 2026Dan BW Choe, Sundhar Vinodh Sangeetha, Samuel Coogan +1VLM Robustness Long-Horizon Robotic Manipulation
Georgia Institute of Technology, Atlanta, GA, USA
MobiAgent: Dual-Loop Recursive Policy Self-Improvement for Long-Horizon Mobile Manipulation
Oct 2, 2026Chenzhi Liu, Yue Zhang, Jiehong Lin +4Long-Horizon Robotic Manipulation Mobile Manipulation
The University of Hong Kong · Astribot · Southern University of Science and Technology

Feb 5, 2026J/K move · Enter open · S save

MobileManiBench: Simplifying Model Verification for Mobile Manipulation

Wenbo Wang, Fangyun Wei, QiXiu Li, Xi Chen+4

University of Sydney · 1Microsoft Research Asia · 3Tsinghua University

Mobile Manipulation Robotics Simulation Synthetic Data Generation for Robotics Vision-Language-Action Models Robot Manipulation Benchmarks

Open paper AskPDF arXiv

Vision-language-action models have advanced robotic manipulation but remain constrained by reliance on the large, teleoperation-collected datasets dominated by the static, tabletop scenes. We propose a simulation-first framework to verify VLA architectures before real-world deployment and introduce MobileManiBench, a large-scale benchmark for mobile-based robotic manipulation. Built on NVIDIA Isaac Sim and powered by reinforcement learning, our pipeline autonomously generates diverse manipulation trajectories with rich annotations (language instructions, multi-view RGB-depth-segmentation images, synchronized object/robot states and actions). MobileManiBench features 2 mobile platforms (parallel-gripper and dexterous-hand robots), 2 synchronized cameras (head and right wrist), 630 objects in 20 categories, 5 skills (open, close, pull, push, pick) with over 100 tasks performed in 100 realistic scenes, yielding 300K trajectories. This design enables controlled, scalable studies of robot embodiments, sensing modalities, and policy architectures, accelerating research on data efficiency and generalization. We benchmark representative VLA models and report insights into perception, reasoning, and control in complex simulated environments, with all code, datasets, and models publicly released.

Abstract

Explore similar work

MobileManiBench: Simplifying Model Verification for Mobile Manipulation

Make Your VLA More Robust Without More Data By Interleaving Motion Planning

MobiAgent: Dual-Loop Recursive Policy Self-Improvement for Long-Horizon Mobile Manipulation