cs.ROOct 11, 2026

Exploring the Limits of Combinatorial Generalization in Embodied Manipulation

Authors: Xupeng Zhang, Yantai Yang, Chang Guo, Zhaokai Yin, Zhipeng Zhang

Organizations: AutoLab, School of Artificial Intelligence, Shanghai Jiao Tong University

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

In embodied manipulation, Vision Language Action (VLA) models can execute atomic skills, but long-horizon manipulation requires recombining these skills without demonstrations for every skill combination. The full instruction route treats skill chaining as long-horizon imitation. In a controlled study, full instruction training on limited skill combination sequences reaches 40.0% ordered success on covered combinations but only 7.7% on recombination, suggesting memorization of sequence templates rather than robust skill recombination. We introduce Hierarchical Subtask Execution (HSE), a diagnostic framework that exposes skill recombination through a shared modular interface over the planner, trigger, and policy, decomposing long-horizon failures into separable modes. We introduce RoboCombine, a benchmark for skill recombination without combined skill training trajectories, with metrics for ordered success and transition conversion across subtask boundaries. RoboCombine reveals that atomic competence alone does not ensure ordered execution. Failures concentrate at transitions where downstream subtasks launch from handoff states from preceding subtasks rather than canonical atomic starts. We identify this mismatch as the start state gap. To address it, we introduce Pose Robust Task Initialization (PRTI), a lightweight atomic task post-training strategy that broadens start state coverage beyond canonical starts. Under oracle timed HSE on RoboCombine, PRTI reaches 18.16% ordered success, compared with 4.21% under random perturbation and 9.47% under a robot-reset oracle, without combined skill training trajectories. These results show that combinatorial generalization requires handoff robust atomic skills that convert first stage progress into reliable downstream execution, not merely decomposition into known subtasks. Code: https://github.com/AutoLab-SAI-SJTU/RoboCombine.

Figures & tables

Figure 1: Finite composed trajectory training shows an A →\rightarrow B →\rightarrow C degradation as composition novelty increases.
Figure 2: RoboCombine task overview. Each row shows one composition task. The target subtask starts from the source terminal state; full instructions are listed in Tab. 4 .
Figure 3: Full instruction fine tuning and HSE make different recombination assumptions. For a command such as “open the drawer, then place the persimmon from the right plate into the drawer, then close the drawer,” Full Inst FT treats it as one long horizon instruction supervised by composed trajectories, which can transfer poorly to unseen skill combinations. HSE instead decomposes it into a subtask queue qq , switches by a handoff cue gg , and invokes the atomic policy π\pi one subtask at a time, exposing order, timing, and execution failures.
TaskMotus [ 2 ]FastWAM [ 35 ]LingBot-VLA [ 29 ]π0.5\pi_{0.5} [ 11 ]
 OracleOrderedFull InstOracleOrderedFull InstOracleOrderedFull InstOracleOrderedFull Inst
Arrange Sign10.05.00000000000
Bedside30.025.030.00000015.05.05.00
Breakfast Prep75.080.020.010.05.05.000010.005.0
Checkout Counter30.015.015.0005.005.005.000
Counter Cleanup15.010.015.0000000000
Table 1: Per-composition ordered success grouped by policy and execution mode. Each row is one RoboCombine composition and reports ordered success (%) over 20 rollouts. Under each policy, the three subcolumns correspond to Oracle-HSE , Ordered-HSE , and Full-Inst . Oracle-HSE denotes the oracle HSE track with ground-truth subtask order and handoff cue, and Ordered-HSE denotes the ordered HSE track with ground-truth subtask order and predicted handoff cue. Aggregate ordered success rates ( n=380n=380 per policy variant) are reported in the final row.
Figure 4: RoboCombine pairwise handoff diagnostics under oracle timed HSE . We evaluate Motus on RoboCombine under the oracle HSE track, which supplies the ground-truth subtask order and handoff cue. Each cell corresponds to one ordered composition, with color and number indicating the metric value. The left matrix reports first-stage success for the source subtask, and the right matrix reports transition conversion, i.e., the success rate of the target subtask conditioned on successful completion of the source subtask.
TaskBaseRandomResetPRTITaskBaseRandomResetPRTI
Arrange Sign50105Office Organize150520
Bedside051025Reception Desk501015
Breakfast Prep5204030Stationery Sort051015
Checkout Counter5000Store Display00020
Counter Cleanup00535Tabletop Arrange50015
Desk Arrange0101020Workspace Setup5201530
Table 2: Ordered success across compositions. Each task uses 20 episodes per method. All values are percentages. Only tasks with nonzero success under at least one matched HSE execution are shown. Summary metrics use all 19 tasks. Random uses unstructured perturbations of initial states as a control for generic state diversity. Reset restores only the robot’s canonical pose between subtasks, leaving the preceding subtask’s object and contact states unchanged.
Task SplitTask IDFULL-INSTFULL-INST-FTOrder-HSEOracle-HSEOracle-HSE + PRTI
A: Collected Compositions1-70.620.940.820.760.82
 1-8-60.240.620.340.440.66
 1-15-11-160.080.640.460.380.52
 7-5-8-6-100.060.060.460.620.72
B: Overlap Held-Out1-120.040.880.880.940.94
 1-20.200.500.941.001.00
Table 3: Real-robot mean progress on compositional tasks. Each entry averages five physical rollouts using the task-length-specific ordered-milestone rubric in Sec. E.2 . Each task ID denotes an ordered sequence of atomic tasks; detailed definitions of the atomic task IDs are provided in Sec. E . Full-Inst-FT is trained with the collected Set A compositions. Oracle-HSE+PRTI differs from Oracle-HSE only by PRTI atomic post-training; neither HSE variant uses composed trajectories.
Figure 5: Real-robot execution trace on the five-step compositional task 4-1-6-2-10 . The top row shows representative frames from the rollout, and the blue curve plots the progress predictor score over time. The score rises within each stage and drops after a successful switch, illustrating hierarchical execution over the full task sequence.
Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

IDScenarioFamilySource instruction s1s_{1}Target instruction s2s_{2}
00Arrange Signpick-placeSort objectsRotate QR code
01Bedsidepick-placePlace fan on padMove pill bottle to pad
02Breakfast Preppick-placePlace cup on coasterPlace bread in basket
03Checkout Counterpick-placePlace phone on standRotate QR code
04Counter Cleanuppick-placePlace container on plateRotate QR code
05Desk Arrangepick-placeMove stapler to padPlace phone on stand
Appendix
Table 4: Full RoboCombine task instructions. The main overview figure uses short labels for readability; this table gives the complete source and target instructions for each ordered composition.
Atomic Tasks
IDTask PromptEpisodesIDTask PromptEpisodes
1Open the drawer.1209Press the green start button at the bottom.40
2Close the drawer.12010Press the purple stop button at the top.40
3Put the orange persimmon from the center plate onto the left plate.12011Place the Cube from the left plate into the drawer.60
4Put the orange persimmon from the center plate onto the right plate.12012Place the Cube from the right plate into the drawer.60
5Place the persimmon from the left plate into the drawer.12013Move the Cube from the center board to the left board.60
Appendix
Table 6: Atomic and compositional task prompts used in our experiments. The episode count is reported for each atomic task.
Task ID12345678910111213141516
Success10/109/108/107/105/108/106/107/1010/108/106/105/109/107/103/102/10
Appendix
Table 7: Atomic-task success rates of the π0.5\pi_{0.5} model on the real robot.
Task lengthOrdered milestones and scores
2 stagesInitiate S1S_{1} : 0.2; complete S1S_{1} : 0.5; switch to S2S_{2} : 0.7; complete S2S_{2} : 1.0.
3 stagesInitiate S1S_{1} : 0.1; complete S1S_{1} : 0.3; switch to S2S_{2} : 0.4; complete S2S_{2} : 0.6; switch to S3S_{3} : 0.7; complete S3S_{3} : 1.0.
4 stagesInitiate S1S_{1} : 0.1; complete S1S_{1} : 0.2; switch to S2S_{2} : 0.3; complete S2S_{2} : 0.4; switch to S3S_{3} : 0.5; complete S3S_{3} : 0.7; switch to S4S_{4} : 0.8; complete S4S_{4} : 1.0.
5 stagesInitiate/complete S1,…,S5S_{1},\ldots,S_{5} : 0.1/0.2, 0.3/0.4, 0.5/0.6, 0.7/0.8, and 0.9/1.0, respectively.
Appendix
Table 8: Ordered-milestone rubric for real-robot mean progress. For the five-stage tasks, each pair gives the score for initiating/completing the corresponding subtask.
Figure 6: Real-robot execution trace on the two-step compositional task 1-2 . The top row shows representative frames from the rollout, and the blue curve plots the progress predictor score over time. The red dashed line indicates the switching threshold, green vertical lines mark stage boundaries, and green dotted lines indicate the moments when the system switches to the next subtask. The score rises within each stage and drops after a successful switch, illustrating hierarchical execution over the full task sequence.
Figure 7: Representative real-robot executions of our framework on four compositional tasks. Each column corresponds to one task, and each row shows temporally sampled frames from the rollout. For each frame, the left panel shows the real-robot observation, while the right panel visualizes the progress predictor score over frames. The score curves provide an interpretable trace of subtask progress and switching behavior during execution, illustrating how our framework completes multi-step tasks by sequentially invoking atomic skills.
More in this topic

Generalization in Robotic Manipulation

Generalization in robotic manipulation studies how robots transfer grasping and interaction skills to new objects, scenes, tasks, and embodiments.

See all 353 papers

Explore similar work

CardsList
Dec 20, 2025

Learning Semantic Atomic Skills for Multi-Task Robotic Manipulation

Scaling imitation learning to diverse multi-task robot manipulation remains challenging due to suboptimal demonstrations, behavioral multi-modality, and destructive interference across tasks. While skill-based methods offer a promising direction by decomposing behaviors into reusable abstractions, existing approaches often learn skills that are either biased toward linguistic structure or lack semantic alignment across tasks, limiting generalization. In this work, we propose AtomSkill, a novel framework that learns a semantically aligned Atomic Skill Space from demonstrations and enables robust long-horizon execution through keypose imagination. Our method introduces: (1) semantic contrastive skill alignment, which partitions demonstrations into variable-length atomic skills and employs a contrastive objective to jointly enforce semantic consistency and temporal coherence, yielding a compact and reusable skill library; and (2) action decoding with keypose imagining, where the policy predicts both a skill's terminal keypose and immediate actions, thereby supporting progress-aware skill transitions. During inference, an atomic skill diffusion sampler generates plausible skill sequences, while predicted keyposes autonomously trigger smooth skill chaining. Extensive experiments in simulation and real-world settings show that AtomSkill consistently outperforms state-of-the-art imitation learning and skill-based baselines. Project page: https://atom-skill.github.io.

Yihang Zhu, Weiqing Wang, Shijie Wu +2

ShanghaiTech University, Shanghai, China · InstAdapt

Long-Horizon Robotic ManipulationRobot Skill LearningGeneralization in Robotic ManipulationMulti-Task Robotic ManipulationRobotic ManipulationRobot Imitation Learning+1
AskPDF
May 2, 2026

Decompose and Recompose: Reasoning New Skills from Existing Abilities for Cross-Task Robotic Manipulation

Cross-task generalization is a core challenge in open-world robotic manipulation, and the key lies in extracting transferable manipulation knowledge from seen tasks. Recent in-context learning approaches leverage seen task demonstrations to generate actions for unseen tasks without parameter updates. However, existing methods provide only low-level continuous action sequences as context, failing to capture composable skill knowledge and causing models to degenerate into superficial trajectory imitation. We propose Decompose and Recompose, a skill reasoning framework using atomic skill-action pairs as intermediate representations. Our approach decomposes seen demonstrations into interpretable skill--action alignments, enabling the model to recompose these skills for unseen tasks through compositional reasoning. Specifically, we construct a task-adaptive dynamic demonstration library via visual-semantic retrieval combined with skill sequences from a planning agent, complemented by a coverage-aware static library to fill missing skill patterns. Together, these yield skill-comprehensive demonstrations that explicitly elicit compositional reasoning for skill composition and execution ordering. Experiments on the AGNOSTOS benchmark and real-world environments validate our method's zero-shot cross-task generalization capability.

Xitie Zhang, Aming Wu, Yahong Han

School of Artificial Intelligence, College of Intelligence and Computing, Tianjin University, China · School of Computer Science and Information Engineering, Hefei University of Technology, China

ICL for RoboticsZero-Shot Robotic ManipulationRobot Skill LearningGeneralization in Robotic ManipulationMulti-Task Robotic ManipulationRobotic Manipulation+2
AskPDF
Mar 8, 2026

AtomicVLA: Unlocking the Potential of Atomic Skill Learning in Robots

Recent advances in Visual-Language-Action (VLA) models have shown promising potential for robotic manipulation tasks. However, real-world robotic tasks often involve long-horizon, multi-step problem-solving and require generalization for continual skill acquisition, extending beyond single actions or skills. These challenges present significant barriers for existing VLA models, which use monolithic action decoders trained on aggregated data, resulting in poor scalability. To address these challenges, we propose AtomicVLA, a unified planning-and-execution framework that jointly generates task-level plans, atomic skill abstractions, and fine-grained actions. AtomicVLA constructs a scalable atomic skill library through a Skill-Guided Mixture-of-Experts (SG-MoE), where each expert specializes in mastering generic yet precise atomic skills. Furthermore, we introduce a flexible routing encoder that automatically assigns dedicated atomic experts to new skills, enabling continual learning. We validate our approach through extensive experiments. In simulation, AtomicVLA outperforms π0π_{0} by 2.4% on LIBERO, 10% on LIBERO-LONG, and outperforms π0π_{0} and π0.5π_{0.5} by 0.22 and 0.25 in average task length on CALVIN. Additionally, our AtomicVLA consistently surpasses baselines by 18.3% and 21% in real-world long-horizon tasks and continual learning. These results highlight the effectiveness of atomic skill abstraction and dynamic expert composition for long-horizon and lifelong robotic tasks. The project page is here.

Likui Zhang, Tao Tang, Zhihao Zhan +9

Sun Yat-sen University · Yinwang Intelligent Technology Co. Ltd. · Peng Cheng Laboratory

Robot Skill LearningVLMs for RoboticsLong-Horizon Robotic ManipulationContinual Robot LearningContinual Learning for VLA ModelsMixture of Experts
AskPDF