In embodied manipulation, Vision Language Action (VLA) models can execute atomic skills, but long-horizon manipulation requires recombining these skills without demonstrations for every skill combination. The full instruction route treats skill chaining as long-horizon imitation. In a controlled study, full instruction training on limited skill combination sequences reaches 40.0% ordered success on covered combinations but only 7.7% on recombination, suggesting memorization of sequence templates rather than robust skill recombination. We introduce Hierarchical Subtask Execution (HSE), a diagnostic framework that exposes skill recombination through a shared modular interface over the planner, trigger, and policy, decomposing long-horizon failures into separable modes. We introduce RoboCombine, a benchmark for skill recombination without combined skill training trajectories, with metrics for ordered success and transition conversion across subtask boundaries. RoboCombine reveals that atomic competence alone does not ensure ordered execution. Failures concentrate at transitions where downstream subtasks launch from handoff states from preceding subtasks rather than canonical atomic starts. We identify this mismatch as the start state gap. To address it, we introduce Pose Robust Task Initialization (PRTI), a lightweight atomic task post-training strategy that broadens start state coverage beyond canonical starts. Under oracle timed HSE on RoboCombine, PRTI reaches 18.16% ordered success, compared with 4.21% under random perturbation and 9.47% under a robot-reset oracle, without combined skill training trajectories. These results show that combinatorial generalization requires handoff robust atomic skills that convert first stage progress into reliable downstream execution, not merely decomposition into known subtasks. Code: https://github.com/AutoLab-SAI-SJTU/RoboCombine.
Figures & tables
Figure 1: Finite composed trajectory training shows an A → B → C degradation as composition novelty increases.
Figure 2: RoboCombine task overview. Each row shows one composition task. The target subtask starts from the source terminal state; full instructions are listed in Tab. 4 .
Figure 3: Full instruction fine tuning and HSE make different recombination assumptions. For a command such as “open the drawer, then place the persimmon from the right plate into the drawer, then close the drawer,” Full Inst FT treats it as one long horizon instruction supervised by composed trajectories, which can transfer poorly to unseen skill combinations. HSE instead decomposes it into a subtask queue q , switches by a handoff cue g , and invokes the atomic policy π one subtask at a time, exposing order, timing, and execution failures.
Task
Motus [ 2 ]
FastWAM [ 35 ]
LingBot-VLA [ 29 ]
π0.5 [ 11 ]
Oracle
Ordered
Full Inst
Oracle
Ordered
Full Inst
Oracle
Ordered
Full Inst
Oracle
Ordered
Full Inst
Arrange Sign
10.0
5.0
0
0
0
0
0
0
0
0
0
0
Bedside
30.0
25.0
30.0
0
0
0
0
0
15.0
5.0
5.0
0
Breakfast Prep
75.0
80.0
20.0
10.0
5.0
5.0
0
0
0
10.0
0
5.0
Checkout Counter
30.0
15.0
15.0
0
0
5.0
0
5.0
0
5.0
0
0
Counter Cleanup
15.0
10.0
15.0
0
0
0
0
0
0
0
0
0
Table 1: Per-composition ordered success grouped by policy and execution mode. Each row is one RoboCombine composition and reports ordered success (%) over 20 rollouts. Under each policy, the three subcolumns correspond to Oracle-HSE , Ordered-HSE , and Full-Inst . Oracle-HSE denotes the oracle HSE track with ground-truth subtask order and handoff cue, and Ordered-HSE denotes the ordered HSE track with ground-truth subtask order and predicted handoff cue. Aggregate ordered success rates ( n=380 per policy variant) are reported in the final row.
Figure 4: RoboCombine pairwise handoff diagnostics under oracle timed HSE . We evaluate Motus on RoboCombine under the oracle HSE track, which supplies the ground-truth subtask order and handoff cue. Each cell corresponds to one ordered composition, with color and number indicating the metric value. The left matrix reports first-stage success for the source subtask, and the right matrix reports transition conversion, i.e., the success rate of the target subtask conditioned on successful completion of the source subtask.
Task
Base
Random
Reset
PRTI
Task
Base
Random
Reset
PRTI
Arrange Sign
5
0
10
5
Office Organize
15
0
5
20
Bedside
0
5
10
25
Reception Desk
5
0
10
15
Breakfast Prep
5
20
40
30
Stationery Sort
0
5
10
15
Checkout Counter
5
0
0
0
Store Display
0
0
0
20
Counter Cleanup
0
0
5
35
Tabletop Arrange
5
0
0
15
Desk Arrange
0
10
10
20
Workspace Setup
5
20
15
30
Table 2: Ordered success across compositions. Each task uses 20 episodes per method. All values are percentages. Only tasks with nonzero success under at least one matched HSE execution are shown. Summary metrics use all 19 tasks. Random uses unstructured perturbations of initial states as a control for generic state diversity. Reset restores only the robot’s canonical pose between subtasks, leaving the preceding subtask’s object and contact states unchanged.
Task Split
Task ID
FULL-INST
FULL-INST-FT
Order-HSE
Oracle-HSE
Oracle-HSE + PRTI
A: Collected Compositions
1-7
0.62
0.94
0.82
0.76
0.82
1-8-6
0.24
0.62
0.34
0.44
0.66
1-15-11-16
0.08
0.64
0.46
0.38
0.52
7-5-8-6-10
0.06
0.06
0.46
0.62
0.72
B: Overlap Held-Out
1-12
0.04
0.88
0.88
0.94
0.94
1-2
0.20
0.50
0.94
1.00
1.00
Table 3: Real-robot mean progress on compositional tasks. Each entry averages five physical rollouts using the task-length-specific ordered-milestone rubric in Sec. E.2 . Each task ID denotes an ordered sequence of atomic tasks; detailed definitions of the atomic task IDs are provided in Sec. E . Full-Inst-FT is trained with the collected Set A compositions. Oracle-HSE+PRTI differs from Oracle-HSE only by PRTI atomic post-training; neither HSE variant uses composed trajectories.
Figure 5: Real-robot execution trace on the five-step compositional task 4-1-6-2-10 . The top row shows representative frames from the rollout, and the blue curve plots the progress predictor score over time. The score rises within each stage and drops after a successful switch, illustrating hierarchical execution over the full task sequence.
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
ID
Scenario
Family
Source instruction s1
Target instruction s2
00
Arrange Sign
pick-place
Sort objects
Rotate QR code
01
Bedside
pick-place
Place fan on pad
Move pill bottle to pad
02
Breakfast Prep
pick-place
Place cup on coaster
Place bread in basket
03
Checkout Counter
pick-place
Place phone on stand
Rotate QR code
04
Counter Cleanup
pick-place
Place container on plate
Rotate QR code
05
Desk Arrange
pick-place
Move stapler to pad
Place phone on stand
Appendix
Table 4: Full RoboCombine task instructions. The main overview figure uses short labels for readability; this table gives the complete source and target instructions for each ordered composition.
Atomic Tasks
ID
Task Prompt
Episodes
ID
Task Prompt
Episodes
1
Open the drawer.
120
9
Press the green start button at the bottom.
40
2
Close the drawer.
120
10
Press the purple stop button at the top.
40
3
Put the orange persimmon from the center plate onto the left plate.
120
11
Place the Cube from the left plate into the drawer.
60
4
Put the orange persimmon from the center plate onto the right plate.
120
12
Place the Cube from the right plate into the drawer.
60
5
Place the persimmon from the left plate into the drawer.
120
13
Move the Cube from the center board to the left board.
60
Appendix
Table 6: Atomic and compositional task prompts used in our experiments. The episode count is reported for each atomic task.
Task ID
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
Success
10/10
9/10
8/10
7/10
5/10
8/10
6/10
7/10
10/10
8/10
6/10
5/10
9/10
7/10
3/10
2/10
Appendix
Table 7: Atomic-task success rates of the π0.5 model on the real robot.
Initiate/complete S1,…,S5 : 0.1/0.2, 0.3/0.4, 0.5/0.6, 0.7/0.8, and 0.9/1.0, respectively.
Appendix
Table 8: Ordered-milestone rubric for real-robot mean progress. For the five-stage tasks, each pair gives the score for initiating/completing the corresponding subtask.
Figure 6: Real-robot execution trace on the two-step compositional task 1-2 . The top row shows representative frames from the rollout, and the blue curve plots the progress predictor score over time. The red dashed line indicates the switching threshold, green vertical lines mark stage boundaries, and green dotted lines indicate the moments when the system switches to the next subtask. The score rises within each stage and drops after a successful switch, illustrating hierarchical execution over the full task sequence.
Figure 7: Representative real-robot executions of our framework on four compositional tasks. Each column corresponds to one task, and each row shows temporally sampled frames from the rollout. For each frame, the left panel shows the real-robot observation, while the right panel visualizes the progress predictor score over frames. The score curves provide an interpretable trace of subtask progress and switching behavior during execution, illustrating how our framework completes multi-step tasks by sequentially invoking atomic skills.
School of Artificial Intelligence, College of Intelligence and Computing, Tianjin University, China · School of Computer Science and Information Engineering, Hefei University of Technology, China