Organizations: School of Intelligence Science and Technology, Peking University · State Key Laboratory of General Artificial Intelligence, Peking University · Alibaba DAMO Academy, Alibaba Group · Institute of Automation, Chinese Academy of Sciences · Tsinghua University · University of Technology Sydney · University of Liverpool
Recent vision-language-action (VLA) models have shown strong performance on local robot manipulation, yet their success does not readily translate to long-horizon mobile manipulation. In our experiments, a pretrained subtask VLA achieves 80% success rate on atomic manipulation tasks but only 19% when these skills are composed into mobile manipulation. We argue that this gap stems from failures across the execution process rather than from local manipulation capability alone. To uncover these bottlenecks, we systematically diagnose failures in long-horizon mobile manipulation and identify three dominant regimes: task planning, navigation and base positioning, and local manipulation. Based on this diagnosis, we develop failure-guided agentic intervention that augments a fixed subtask VLA with demonstration-based planning guidance, adaptive visual memory, accessibility-aware base adjustment, and local end-effector adjustment. On RoboCasa365, these interventions progressively improve task success from 19% to 47%, with the finalized system reaching 49% on held-out episode seeds, compared with 18% for the baseline. The same design improves performance from 14% to 37% on BEHAVIOR-1K and from 5/20 to 13/20 on real-robot tasks. These results suggest that reliable mobile manipulation can be advanced by diagnosing and intervening on the current bottleneck, rather than uniformly scaling the underlying policy. Our code will be released at https://github.com/PKU-VaLuE-Lab/Cracking-Mobile-Manipulation.git.
Figures & tables
Figure 1: From the mobile manipulation gap to failure-guided intervention. (a) Strong subtask VLA does not translate to reliable long-horizon mobile manipulation. (b) Failure analysis identifies the dominant failures: task planning, navigation and base positioning, and manipulation. These observations motivate our agentic intervention, including: in-context guidance from video demonstrations, adaptive visual memory, accessibility checking and base adjustment, and end-effector adjustment.
Figure 2: Failure attribution of the agentic baseline. Among the 81 failed episodes, the primary failures are attributed to task planning (21.0%), navigation and base positioning (25.9%), and local manipulation (53.1%).
Figure 3: Overview of the failure-guided agentic system. The agentic planner leverages the skill library for task planning guidance, visual memory for relevant observation retrieval, and geometric skills for accessibility checking, base adjustment, and end-effector adjustment.
Figure 4: Progressive integration of failure-guided interventions. Each point adds one intervention group to the previous configuration.
Figure 5: Failure distribution during progressive integration. Counts are reported over the same 100 matched episodes.
Type
Method
Development (%)
Held-out (%)
VLA
π0 ( Black et al., 2025b )
2.0
2.0
π0.5 ( Black et al., 2025a )
3.0
4.0
GR00T N1.6 ( NVIDIA, 2025b )
7.0
8.0
GR00T N1.5 ( NVIDIA, 2025a )
8.0
9.0
RLDX-1 ( Kim et al., 2026a )
15.0
14.0
Xiaomi Robotics 1 ( Xiaomi Robotics Team et al., 2026 )
44.0
45.0
Table 1: Performance of mobile manipulation tasks on the RoboCasa365 benchmark. We evaluate the 20 of 32 composite tasks that require mobile manipulation.
Appendix figures & tables18 assets
Supplementary material from the paper’s appendix.
Appendix
Setting
Value
VLM backbone
Qwen3-VL-4B-Instruct
Visual encoder
DINOv2 ViT-S/14
Action model
DiT-B
Image resolution
224×224
State dimension
16
Action dimension
12
Appendix
Table 3: Training settings of the subtask VLA. The same checkpoint is used by all agentic configurations unless otherwise stated.
Method
Batch Size
Training Steps
π0
64
75,000
π0.5
64
75,000
GR00T N1.5
128
120,000
GR00T N1.6
128
120,000
RLDX-1
192
250,000
Xiaomi Robotics 1
512
120,000
Appendix
Table 4: Training settings of the released end-to-end baselines. Batch sizes and training steps follow the corresponding RoboCasa365 leaderboard submissions.
Failure Category
Episodes
Unanimous
Task planning
7
6
Navigation and base positioning
8
7
Manipulation
15
13
Total
30
26
Appendix
Table 5: Agreement of independent failure attribution on a random subset of baseline failures.
Fine-grained failure category
Episodes
Unanimous
Placement target or receptacle assignment
4
3
Subtask ordering
3
2
Ineffective approach or poor base positioning
4
4
Collision, path blockage, or stalled navigation
3
2
Unsuitable base orientation
2
2
Incorrect object placement
5
4
Appendix
Table 6: Agreement of fine-grained failure attribution. Three reviewers independently annotate 30 sampled failed episodes without access to the system configuration or enabled skills.
Parameter
Value
Adaptive visual memory
Maximum candidate frames
256
Retrieved frames ( k )
16
Recall model
Qwen3.8-27B
Input image resolution
448×448
Geometric skills
Appendix
Table 7: Implementation settings for adaptive visual memory and geometric skills.
Comparison
Success (%)
Δ (pp)
n01
n10
95% CI (pp)
McNemar p
Baseline → +Demo
19 → 25
+6
8
2
[0, 12]
0.109
+Demo → +Memory
25 → 29
+4
7
3
[-2, 10]
0.344
+Memory → +Base
29 → 40
+11
15
4
[3, 19]
0.019
+Base → +EEF
40 → 47
+7
10
3
[0, 14]
0.092
Baseline → Full
19 → 47
+28
30
2
[18, 38]
<0.001
Appendix
Table 8: Paired analysis of progressive integration. The first four rows compare consecutive configurations; the final row compares the baseline with the full system on the same development episodes. Δ is reported in percentage points.
Added intervention
Recovered
Planning
Nav./Base
Manip.
Skill library
8
5
1
2
Adaptive visual memory
7
4
1
2
Base adjustment
15
0
11
4
End-effector adjustment
10
0
1
9
Appendix
Table 10: Primary failure types of episodes recovered after each intervention.
Baseline → Full
Success
Planning
Nav./Base
Manip.
Total
Success
17
0
1
1
19
Planning
9
1
1
6
17
Nav./Base
12
0
3
6
21
Manipulation
9
0
0
34
43
Total
47
1
5
47
100
Appendix
Table 11: Paired episode-level outcome transitions from the agentic baseline to the full system.
Figure 6: Fine-grained failure attribution for the full system. Counts are reported over the same 100 matched development episodes.
Task horizon
# Tasks
Baseline
Full system
Δ (pp)
2–4 subtasks
13
15/65 (23.1%)
35/65 (53.8%)
+30.7
5–15 subtasks
7
4/35 (11.4%)
12/35 (34.3%)
+22.9
Overall
20
19/100 (19.0%)
47/100 (47.0%)
+28.0
Appendix
Table 13: Performance across task horizons. Tasks are grouped by the number of annotated subtasks.
Method
Storage Box Transport
Table Service
Overall
π0.5 (end-to-end VLA)
2/10
2/10
4/20
Agentic baseline
2/10
3/10
5/20
Full system
6/10
7/10
13/20
Appendix
Table 15: Real-robot evaluation. Each method is evaluated for 10 trials on each task.
Figure 7: Real-robot execution of the Table Service task. A representative successful rollout of the full system on Astribot-S1. The robot moves the prepared dish from the serving cart to the dining table, adds the topping, and places the wine glass beside the dish.
Configuration
Task Success Rate(%)
Δ from Full (pp)
Full system
47.0
–
− Skill library
39.0
-8.0
− Adaptive visual memory
41.0
-6.0
− Base adjustment
35.0
-12.0
− End-effector adjustment
40.0
-7.0
Appendix
Table 16: Leave-one-out ablation of intervention groups on the development episodes.
Memory Access
Uniform
Relevance-based
Every turn
40.0
45.0
Planner-triggered
44.0
47.0
Appendix
Table 18: Separating memory access and frame selection. All configurations use a 16-frame memory budget and the same 100 matched development episodes.
Figure 8: Multi-stage closed-loop execution. The high-level planner decomposes the task into a sequence of subtask-VLA calls, including object pickup and placement, navigation to the microwave, interaction with the microwave door, and pressing the start button.
Figure 9: Accessibility-aware base adjustment before manipulation. After navigating near the target, the planner checks whether the current base pose is suitable for manipulation and invokes approach_xy for a local base adjustment before continuing with the subtask VLA.
Figure 10: Coordinated navigation and manipulation in a long-horizon task. The planner uses the same subtask VLA to open the cabinet, pick and place the bread, transport the basket, navigate to the dining counter, and complete the final placement.
Figure 11: End-effector adjustment after a failed placement attempt. After the initial placement fails, the planner invokes a local end-effector adjustment and retries the placement with the same subtask VLA.
Vision-language-action models have advanced robotic manipulation but remain constrained by reliance on the large, teleoperation-collected datasets dominated by the static, tabletop scenes. We propose a simulation-first framework to verify VLA architectures before real-world deployment and introduce MobileManiBench, a large-scale benchmark for mobile-based robotic manipulation. Built on NVIDIA Isaac Sim and powered by reinforcement learning, our pipeline autonomously generates diverse manipulation trajectories with rich annotations (language instructions, multi-view RGB-depth-segmentation images, synchronized object/robot states and actions). MobileManiBench features 2 mobile platforms (parallel-gripper and dexterous-hand robots), 2 synchronized cameras (head and right wrist), 630 objects in 20 categories, 5 skills (open, close, pull, push, pick) with over 100 tasks performed in 100 realistic scenes, yielding 300K trajectories. This design enables controlled, scalable studies of robot embodiments, sensing modalities, and policy architectures, accelerating research on data efficiency and generalization. We benchmark representative VLA models and report insights into perception, reasoning, and control in complex simulated environments, with all code, datasets, and models publicly released.
Wenbo Wang, Fangyun Wei, QiXiu Li +5
University of Sydney · 1Microsoft Research Asia · 3Tsinghua University
Vision-Language-Action (VLA) models have shown remarkable progress for mobile manipulation, but their performance on long-horizon tasks remains poor. These tasks are especially challenging because (1) progress toward high-level goals must be maintained across extended sequences of spatially distributed subtasks, and (2) early execution errors compound rapidly over the task horizon. These challenges persist despite finetuning on large human teleoperated mobile manipulation data, indicating that more data alone may not resolve the problem. To address these challenges, we propose MPVI: Motion Planner / VLA Interleaving, a framework that integrates model-based motion planning with VLAs to improve robustness without further training. The proposed integration enables localization and navigation to distant or occluded target objects through cluttered scenes using open-vocabulary object detection, frontier exploration and motion planning. However, such integration is non-trivial, requiring reliable switching between modules; we show one way forward via VLM-based completion checking with proprioceptive triggers. We evaluate our approach on the BEHAVIOR-1K benchmark and demonstrate 113% improvement in task progress over a top end-to-end VLA baseline. Additional details are available at the project page: https://mpvi.netlify.app/.
Dan BW Choe, Sundhar Vinodh Sangeetha, Samuel Coogan +1
Long-horizon mobile manipulation presents significant challenges due to compounding execution errors and capacity interference between locomotion and arm control. While recent Vision-Language-Action models excel at short-horizon tasks, they lack the hierarchical reasoning required for multi-stage objectives. Furthermore, existing hierarchical agents suffer from rigid sub-task mapping, inflexible replanning, and a lack of continuous learning. To address these limitations, we introduce MobiAgent, a dual-loop agentic framework that bridges robust deployment execution and recursive policy self-improvement. During deployment, the Inner Loop decouples high-level reasoning from low-level control through highly composable atomic skills. It employs Vision-Language models for receding-horizon planning and visual reflection, dynamically composing skills to ensure robust error recovery. These skills are executed by specialized flow-matching experts that share a unified VLM backbone, maximizing reusability while mitigating capacity interference. Concurrently, the Outer Loop drives automated lifelong learning by autonomously segmenting and verifying deployment rollouts, clustering them to discover atomic skills, and continuously fine-tuning the skill library without human annotations. Evaluations on RoboCasa, BEHAVIOR-1K, and real-world tasks demonstrate the effectiveness of MobiAgent. It outperforms π0.5-TA by 22.5 percentage points on BEHAVIOR-1K and enables robust recovery from execution failures. Through autonomous data recycling, success improves from 7.50% to 27.50% on RoboCasa and from 32.5% to 57.5% on Astribot S1.
Chenzhi Liu, Yue Zhang, Jiehong Lin +4
The University of Hong Kong · Astribot · Southern University of Science and Technology