Long-horizon mobile manipulation presents significant challenges due to compounding execution errors and capacity interference between locomotion and arm control. While recent Vision-Language-Action models excel at short-horizon tasks, they lack the hierarchical reasoning required for multi-stage objectives. Furthermore, existing hierarchical agents suffer from rigid sub-task mapping, inflexible replanning, and a lack of continuous learning. To address these limitations, we introduce MobiAgent, a dual-loop agentic framework that bridges robust deployment execution and recursive policy self-improvement. During deployment, the Inner Loop decouples high-level reasoning from low-level control through highly composable atomic skills. It employs Vision-Language models for receding-horizon planning and visual reflection, dynamically composing skills to ensure robust error recovery. These skills are executed by specialized flow-matching experts that share a unified VLM backbone, maximizing reusability while mitigating capacity interference. Concurrently, the Outer Loop drives automated lifelong learning by autonomously segmenting and verifying deployment rollouts, clustering them to discover atomic skills, and continuously fine-tuning the skill library without human annotations. Evaluations on RoboCasa, BEHAVIOR-1K, and real-world tasks demonstrate the effectiveness of MobiAgent. It outperforms π0.5-TA by 22.5 percentage points on BEHAVIOR-1K and enables robust recovery from execution failures. Through autonomous data recycling, success improves from 7.50% to 27.50% on RoboCasa and from 32.5% to 57.5% on Astribot S1.
Figures & tables
Figure 1: Overview of MobiAgent. MobiAgent solves a long-horizon picking_up_trash task by interleaving dynamic planning, skill-conditioned execution, and critic-guided recovery. The rollout trace illustrates a failed placement where the can drops outside the trash. In response, MobiAgent triggers replanning, guiding the robot to approach the displaced can, re-grasp it, and complete the task. This behavior is driven by our dual-loop framework (bottom left), which couples an online Inner Loop executing atomic skills with an offline Outer Loop that autonomously curates data for continuous policy evolution. The bottom right shows real-robot deployment on paper, bottle, and can disposal, as well as pouring blue particles into a cup.
Figure 2: Illustration of the MobiAgent framework. A dual-loop agentic system bridging robust execution and continuous policy evolution. (a) The Inner Loop ensures adaptive deployment via a Task Planner for dynamic skill composition, a Skill Executor utilizing reusable skill-specific policies, and a Reflection Critic that triggers flexible replanning for robust error recovery. (b) The Outer Loop drives automated self-improvement via a Data Curator, Skill Generator, and Skill Trainer, which autonomously extract, standardize, and fine-tune atomic skills from trajectory data.
Method
Baselines
MobiAgent
CaP-X [ 4 ]
π0.5 [ 36 ] (R0)
R1
R2
R3
R4
R5
R6
Success Rate (%)
5.00
7.50
18.75
21.25
27.50
25.00
27.50
26.25
Table 1: Success rates (%) on RoboCasa Composite-Seen [ 31 , 32 ] . R0 denotes the base π0.5 policy, while R1–R6 denote successive deployment-to-training rounds of MobiAgent.
Method
push-radio
dispose-trash
stack-storage
fetch-beer
Mean
π0.5 -TA [ 16 ]
60
40
60
10
42.5
MobiAgent (Ours)
70
60
70
60
65.0
Table 2: Success rates (%) on four BEHAVIOR-1K tasks. MobiAgent shares skill policies across tasks, whereas π0.5 -TA uses a separately trained model for each task.
Method / Round
trash-garbage
trash-bottle
trash-can
pour-blue
Mean
π0.5 [ 36 ]
10.0
20.0
10.0
0.0
10.0
MobiAgent R1 (Bootstrap)
40.0
50.0
20.0
20.0
32.5
MobiAgent R2
40.0
60.0
30.0
30.0
40.0
MobiAgent R3
50.0
60.0
70.0
50.0
57.5
Table 3: Success rates (%) on four real-world tasks. Both π0.5 and MobiAgent Round 1 use 100 demonstrations per task. Subsequent rounds refine MobiAgent using approximately 20 automatically curated deployment trajectories per task.
Figure 3: Representative real-robot evaluation scenes and the Astribot S1 platform.
Method
push-radio
dispose-trash
stack-storage
fetch-beer
Mean
Fixed task-specific sub-tasks (w/o Planner)
50
0
0
30
20.0
Single shared action head
40
40
50
40
42.5
w/o Reflection Critic
30
10
0
0
10.0
w/o Global Replanning
70
40
30
20
40.0
MobiAgent (Full)
70
60
70
60
65.0
Table 4: Component ablations on BEHAVIOR-1K. All entries report task success rates (%).
Figure 4: Examples of robust error recovery. MobiAgent overcomes execution failures through critic-driven global replanning: (a) generating new retrieval sub-tasks after an object is dropped, and (b) correcting the robot’s base pose before retrying the grasp.
Appendix figures & tables21 assets
Supplementary material from the paper’s appendix.
Appendix
Agent
Model
Task Planner
GPT-5.4
Reflection Critic
GPT-5.4
Data Curator (segmentation)
Gemini-3.1-Pro
Data Curator (verification)
GPT-5.4
Skill Generator (aggregation)
GPT-5.4
Skill Trainer
GPT-5.4
Appendix
Table S1: Default LLM/VLM models used in our MobiAgent.
Task ID
Task name
Instruction
task-0000
push-radio
“Turn on the radio on the coffee table.”
task-0001
dispose-trash
“Put the three cans of soda from the living room inside the trash can in the kitchen.”
task-0016
stack-storage
“Move the two storage containers from the living room to the garage. In the garage, place one container on the floor and stack the other container on top of it (either order is fine).”
task-0017
fetch-beer
“Bring two beer bottles from the refrigerator to the coffee table, then close the refrigerator.”
Appendix
Table S2: Task instructions and main-paper task names for the four evaluated BEHAVIOR-1K tasks.
Task
Subtask Sequence (verbatim)
push-radio
1. move to radio 2. pick up radio from coffee table 3. press radio 4. place radio on coffee table
dispose-trash
1. move to trash can 2. pick up trash can from floor 3. move to can of soda 4. pick up can of soda from floor 5. place can of soda in trash can 6. move to can of soda 7. pick up can of soda from floor 8. place can of soda in trash can 9. move to can of soda 10. pick up can of soda from floor 11. place can of soda in trash can 12. place trash can on floor
stack-storage
1. move to door 2. open door 3. move to storage container 4. pick up storage container from floor 5. move to floor 6. place storage container on floor 7. move to storage container 8. pick up storage container from floor 9. move to floor 10. place storage container on storage container
fetch-beer
1. move to refrigerator 2. open refrigerator 3. pick up beer bottles from refrigerator 4. move to coffee table 5. place beer bottles on coffee table 6. move to refrigerator 7. close refrigerator
Appendix
Table S3: Representative verbatim subtask sequences for the four BEHAVIOR-1K tasks.
Setting
Simulation
Precision
bfloat16
Batch size
480
Training steps
50,000
Warmup steps
1,000
Learning rate
1×10−4
Appendix
Table S4: Training hyperparameters for the per-skill π0.5 fine-tuning in simulation (BEHAVIOR-1K).
Figure S1: Real-robot rollouts on the Astribot S1, one task per row, with head-camera frames in temporal order (left to right) and the executed skill labelled under each frame. (a) trash-garbage : pick up the garbage and place it in the trash can. (b) trash-can : pick up the can and place it in the trash can. (c) trash-bottle : pick up the bottle and place it in the trash can. (d) pour-blue : pick up the bottle and pour the blue particles into the cup.
Task ID
Prompt
Subtask Sequence (verbatim)
trash-garbage
Pick up the garbage and place it in the trash can.
1. move to the garbage 2. pick up the garbage 3. move to the trash can while carrying the garbage 4. place the garbage in the trash can
trash-bottle
Pick up the bottle and place it in the trash can.
1. move to the bottle 2. pick up the bottle 3. move to the trash can while carrying the bottle 4. place the bottle in the trash can
trash-can
Pick up the can and place it in the trash can.
1. move to the can 2. pick up the can 3. move to the trash can while carrying the can 4. place the can in the trash can
pour-blue
Pick up the bottle containing blue particles and pour the blue particles into the cup.
1. move to the table 2. pick up the bottle containing blue particles 3. pour the blue particles into the cup 4. place the bottle on the table
Appendix
Table S6: Real-robot tasks and their corresponding subtask sequences. The prompt column is used identically as the Task Planner’s deployment prompt and as the per-skill training prompt. The full subtask sequence for each task is given verbatim.
Setting
Real robot (S1)
Precision
bfloat16
Batch size
240
Training steps
20,000
Warmup steps
100
Learning rate
5×10−5
Appendix
Table S7: Training hyperparameters for the per-skill π0.5 fine-tuning on the real robot (Astribot S1).
Figure S2: Representative views of the Unitree R1D home-clean experiment, illustrating door interaction, bottle collection and disposal, and table cleaning.
Figure S3: Data-processing wall-clock time vs. number of processed clips across all four tasks. At 800 clips, manual annotation requires ∼28.9 h compared to MobiAgent’s ∼0.82 h (a ∼35× speedup).
π0.5 -TA
MobiAgent (ours)
Task
Total Runtime
Action Exec.
API
Total Runtime
push-radio
60
81
98
179
dispose-trash
249
334
381
715
stack-storage
525
707
747
1454
fetch-beer
538
764
792
1556
Appendix
Table S8: Comparison of per-episode runtime (in seconds) for successful rollouts across the four BEHAVIOR-1K tasks. For MobiAgent, the total runtime is the sum of the action execution time (policy inference and physics stepping, excluding rendering) and the VLM API time. The π0.5 -TA baseline operates end-to-end without API calls.
Atomic Skill
Simulation SR (%)
Real SR (%)
move_to
100.00
100.00
pick_up
88.57
70.00
place_in
75.00
94.73
place_on
100.00
100.00
open
86.67
–
close
100.00
–
Appendix
Table S9: Per-skill success rates (%) of the Skill Executor in both simulation and real-world deployments, evaluated on instances successfully reached during execution. The “–” symbol denotes skills that are exclusive to one domain (e.g., open / close are simulation-only, whereas pour is real-world-only).
Figure S4: FM1: Reflection Critic completion-timing error on dispose-trash (head-camera frames, left to right). Having accepted the move_to while the robot was still too far, the subsequent pick_up reaches repeatedly but cannot grasp the out-of-range can, which stays on the floor until the retry budget is exhausted.
Figure S5: FM2: Bimanual-coordination failures across three tasks. (a) The two arms are poorly coordinated, knocking over the target. (b) The picking arm must flip the radio in-hand, frequently dropping it. (c) Only one gripper secures the large container, causing it to tilt and derailing subsequent stacking.
Figure S6: FM3: Inter-skill handoff pose mismatch on dispose-trash (head-camera frames, left to right). The pick_up grasps the can off the floor and succeeds, but ends with the arm raised and rotated away from the carried bin; the next place_in inherits this pose, cannot bring the can over the bin opening, and the placement fails.
Figure S7: FM4: Low-level perceptual limits on the real robot. (a) The policy misjudges the bin’s spatial configuration, dropping the bottle on the floor instead of inside. (b) The clear glass is hard to localise, causing the liquid to be poured onto the table beside the glass.
CaP-X [ 4 ]
MobiAgent (ours)
Task
Navigation SR
Task SR
Navigation SR
Task SR
push-radio
100
20
100
90
dispose-trash
90
80
100
100
Appendix
Table S10: Comparison with CaP-X [ 4 ] on the two tasks within its navigate-and-grasp repertoire ( 10 trials each, held-out test scenes, matched observations and embodiment). Navigation and task (including navigation) success are reported separately (%), following CaP-X’s protocol.
Task
RoboClaw subtask prompts (verbatim)
push-radio (2)
1. Move both arms clear, visually locate the radio on the coffee table in the living room, and position yourself so one gripper can safely reach the radio power control. 2. Use the most suitable gripper to press or toggle the radio’s power control on the coffee table until the radio turns on.
dispose-trash (3)
1. Pick up one soda can from the living room, move to the kitchen trash can, and drop the can into the trash. Finish with empty grippers and do not pick a second can in the same run. 2. Pick up one remaining soda can from the living room, move to the kitchen trash can, and drop the can into the trash. Finish with empty grippers and do not pick another can in the same run. 3. Pick up the last remaining soda can from the living room, move to the kitchen trash can, and drop the can into the trash. Finish with empty grippers.
stack-storage (2)
1. Pick up one storage box in the living room, carry or move it to the garage, and place it stably on the garage floor where it can serve as the base for stacking. 2. Pick up the remaining storage box in the living room, move it to the garage, align it above the first box already on the garage floor, and place it stably on top.
fetch-beer (5)
1. Open the fridge door, grasp one bottle of beer from inside the fridge, and take it out securely. 2. Place the beer bottle you are holding onto the coffee table in a stable upright position. 3. Return to the open fridge, grasp a second bottle of beer from inside the fridge, and take it out securely. 4. Place the second beer bottle onto the coffee table in a stable upright position near the first bottle. 5. Close the fridge door fully without disturbing the bottles on the coffee table.
Appendix
Table S11: RoboClaw’s planner decomposition of the four evaluated tasks (verbatim subtask prompts from its run logs). Each subtask is a coarse composite that already bundles several of MobiAgent’s atomic skills (e.g. move_to + pick_up + place_in ).
Execution Phase
RoboClaw
MobiAgent (ours)
Task Initialization
Generates the complete, ordered plan upfront.
Invokes reactive Task Planner for the immediate subtask (full sketch optional).
Normal Execution
Executes the fixed plan sequentially without deviation.
Re-queries the Task Planner at each step; continuously adapts to observations.
Error Recovery
Triggers conditional re-planning via an explicit replan action.
Inherently recovers by re-evaluating the current state at the next step.
Appendix
Table S12: Comparison of planning paradigms. RoboClaw employs a static, plan-then-execute approach with conditional recovery, whereas MobiAgent continuously re-plans at every step for dynamic adaptation.
RoboClaw
MobiAgent (ours)
Task
v0
v1
v2
Δ
v0
v1
v2
Δ
push-radio
3
2
3
1
4
4
4
0
dispose-trash
6
12
3
9
12
12
12
0
stack-storage
4
2
2
2
10
9
10
1
fetch-beer
5
4
4
1
7
7
10
3
Mean Δ
3.25
1.00
Appendix
Table S13: Instruction robustness: Number of subtasks emitted by each planner for the three semantically equivalent phrasings (v0, v1, v2) detailed in Table S14 . The spread Δ=max−min quantifies consistency across paraphrases. Both planners utilize the same GPT-5.4 backbone and starting frame to isolate the evaluation to the planning decomposition. paraphrase-robust planner yields a consistent subtask count (i.e., a near-zero Δ ).
Task
Variant
Instruction
push-radio
v0
Turn on the radio sitting on the coffee table in the living room.
v1
Switch on the radio that is on the living-room coffee table.
v2
Power up the radio resting on the coffee table in the living room.
dispose-trash
v0
There are three cans of soda in the living room. Pick up each can and throw it into the trash can in the kitchen, one at a time, until all three cans are in the kitchen trash can.
v1
Take the three soda cans from the living room and drop them, one by one, into the kitchen trash can.
v2
Clear all three soda cans out of the living room by putting each one into the kitchen bin, handling them one at a time.
Appendix
Table S14: Instruction paraphrases utilized in the robustness evaluation. For each of the four BEHAVIOR-1K tasks, the underlying goal remains fixed while the instruction is reworded into three semantically equivalent variants (v0, v1, v2). Both planners receive the exact same prompt string for a given variant.
Variant
RoboClaw
MobiAgent (ours)
push-radio ( task-0000 )
v0
1. Move to the living room coffee table and position yourself within reach of the radio while keeping a safe manipulation distance. 2. Visually align an arm and gripper with the radio’s power button or power switch so the control can be pressed or toggled cleanly without knocking the radio over. 3. Press or toggle the radio’s power control once to turn the radio on, then release contact without moving the radio significantly.
1. move to radio 2. pick up radio from coffee table 3. press radio 4. place radio on coffee table
v1
1. Find the radio on the living-room coffee table, identify its power button or switch, and move into a stable pre-press position with a clear approach to the control without activating it yet. 2. Use a careful precise motion to press or toggle the radio’s power control on the living-room coffee table and turn the radio on. Stop after the control has been actuated once clearly.
1. move to radio 2. pick up radio from living-room coffee table 3. press radio 4. place radio on living-room coffee table
v2
1. Approach the radio on the coffee table in the living room and position your gripper in front of its power control with a stable, collision-free pose ready to actuate it. 2. Actuate the radio’s power control on the coffee table to turn the radio on. Press the power button firmly once or rotate the power knob only as much as needed to switch the device from off to on, then release safely. 3. Hold position briefly, observe the radio after actuation, and make any minimal corrective re-press only if needed so the radio is left powered on.
1. move to radio 2. pick up radio from coffee table 3. press radio 4. place radio on coffee table
dispose-trash ( task-0001 )
v0
1. In the living room, locate one soda can, approach it safely, and grasp it securely with a stable hold for transport. 2. Carry the grasped soda can from the living room to the kitchen trash can, position it over the trash opening, and release it so the can lands inside the trash can. 3. Return attention to the living room, locate a second soda can that is not yet disposed, approach it safely, and grasp it securely with a stable hold for transport. 4. Carry the second grasped soda can from the living room to the kitchen trash can, position it over the trash opening, and release it so the can lands inside the trash can. 5. Locate the last remaining soda can in the living room, approach it safely, and grasp it securely for transport without disturbing the environment unnecessarily. 6. Carry the last grasped soda can to the kitchen trash can, align over the trash opening, and release it so the can lands fully inside the trash can.
1. move to trash can 2. pick up trash can from floor 3. move to can of soda 4. pick up can of soda from floor 5. place can of soda in trash can 6. move to can of soda 7. pick up can of soda from floor 8. place can of soda in trash can 9. move to can of soda 10. pick up can of soda from floor 11. place can of soda in trash can 12. place trash can on floor
Appendix
Table S15: Verbatim emitted plans for the instruction-robustness study. This table presents every subtask produced by each planner across the four BEHAVIOR-1K tasks and their three semantically equivalent phrasings (v0, v1, v2) detailed in Table S14 .
Recent vision-language-action (VLA) models have shown strong performance on local robot manipulation, yet their success does not readily translate to long-horizon mobile manipulation. In our experiments, a pretrained subtask VLA achieves 80% success rate on atomic manipulation tasks but only 19% when these skills are composed into mobile manipulation. We argue that this gap stems from failures across the execution process rather than from local manipulation capability alone. To uncover these bottlenecks, we systematically diagnose failures in long-horizon mobile manipulation and identify three dominant regimes: task planning, navigation and base positioning, and local manipulation. Based on this diagnosis, we develop failure-guided agentic intervention that augments a fixed subtask VLA with demonstration-based planning guidance, adaptive visual memory, accessibility-aware base adjustment, and local end-effector adjustment. On RoboCasa365, these interventions progressively improve task success from 19% to 47%, with the finalized system reaching 49% on held-out episode seeds, compared with 18% for the baseline. The same design improves performance from 14% to 37% on BEHAVIOR-1K and from 5/20 to 13/20 on real-robot tasks. These results suggest that reliable mobile manipulation can be advanced by diagnosing and intervening on the current bottleneck, rather than uniformly scaling the underlying policy. Our code will be released at https://github.com/PKU-VaLuE-Lab/Cracking-Mobile-Manipulation.git.
Zixuan Wang, Jia-Ning Li, Yinzhou Tang +5
School of Intelligence Science and Technology, Peking University · State Key Laboratory of General Artificial Intelligence, Peking University · Alibaba DAMO Academy, Alibaba Group +4
Recent vision-language-action (VLA) systems have demonstrated strong capabilities in embodied manipulation. However, most existing VLA policies rely on limited observation windows and end-to-end action prediction, which makes them brittle in long-horizon, memory-dependent tasks with partial observability, occlusions, and multi-stage dependencies. Such tasks require not only precise visuomotor control, but also persistent memory, adaptive task decomposition, and explicit recovery from execution failures. To address these limitations, we propose a dual-system framework for long-horizon embodied manipulation. Our framework explicitly separates high-level semantic reasoning from low-level motor execution. A high-level planner, implemented as a VLM-based agentic module, maintains structured task memory and performs goal decomposition, outcome verification, and error-driven correction. A low-level executor, instantiated as a VLA-based visuomotor controller, carries out each sub-task through diffusion-based action generation conditioned on geometry-preserving filtered observations. Together, the two systems form a closed loop between planning and execution, enabling memory-aware reasoning, adaptive replanning, and robust online recovery. Experiments on representative RMBench tasks show that the proposed framework substantially outperforms representative baselines, achieving a 32.4% average success rate compared with 9.8% for the strongest baseline. Ablation studies further confirm the importance of structured memory and closed-loop recovery for long-horizon manipulation.
Zhen Liu, Xinyu Ning, Zhe Hu +8
Beijing University of Posts and Telecommunications · InspireOmni AI · Tsinghua University
General-purpose robots must master long-horizon manipulation, defined as tasks involving multiple kinematic structure changes (e.g., attaching or detaching objects) in unstructured environments. While Vision-Language-Action (VLA) models offer the potential to master diverse atomic skills, they struggle with the combinatorial complexity of sequencing them and are prone to cascading failures due to environmental sensitivity. To address these challenges, we propose LiLo-VLA (Linked Local VLA), a modular framework capable of zero-shot generalization to novel long-horizon tasks without ever being trained on them. Our approach decouples transport from interaction: a Reaching Module handles global motion, while an Interaction Module employs an object-centric VLA to process isolated objects of interest, ensuring robustness against irrelevant visual features and invariance to spatial configurations. Crucially, this modularity facilitates robust failure recovery through dynamic replanning and skill reuse, effectively mitigating the cascading errors common in end-to-end approaches. We introduce a 21-task simulation benchmark consisting of two challenging suites: LIBERO-Long++ and Ultra-Long. In these simulations, LiLo-VLA achieves a 69% average success rate, outperforming Pi0.5 by 41% and OpenVLA-OFT by 67%. Furthermore, real-world evaluations across 8 long-horizon tasks demonstrate an average success rate of 85%. Project page: https://yy-gx.github.io/LiLo-VLA/.
Yue Yang, Shuo Cheng, Yu Fang +4
University of North Carolina at Chapel Hill · Georgia Institute of Technology · Carnegie Mellon University