Generalist robot manipulation requires a single vision-language-action (VLA) policy to execute diverse skills across scenes and instructions. However, current multi-skill VLA policies often leave two aspects of an instruction insufficiently modeled: which primitive skill to execute and where/how to contact and move objects. We propose MAST-VLA (Multi-skill Adaptation with Skill-conditioned experts and Trajectory-affordance priors), a dual-conditioning framework for robot manipulation. MAST-VLA introduces skill-conditioned LoRA Mixture-of-Experts (LoRA-MoE) adapters in the LLM backbone, where skill-query tokens and routing biases guide low-rank expert specialization, and converts a predicted affordance heatmap into a patch-level trajectory-affordance spatial prior for the action head. On the CALVIN ABC→D long-horizon multi-skill cross-scene generalization setting, MAST-VLA achieves state-of-the-art performance, reaching an Avg. Len of 4.65 over five-subtask evaluation sequences. Real-robot multi-skill sequence experiments further validate its deployability under real perception and control noise.
Figures & tables
Figure 1: Overview of MAST-VLA. MAST-VLA introduces two complementary conditioning paths for multi-skill VLA policies: skill-conditioned LoRA-MoE adaptation in the LLM backbone and a trajectory-affordance spatial prior in the action head. The expanded LLM-block view shows FFN-side LoRA-MoE routing with skill-conditioned routing bias, while the action-head view shows how the predicted affordance heatmap is converted into a patch-level spatial prior for action generation.
Method
Tasks Completed in a Row (%)
Avg. Len ↑
1
2
3
4
5
OpenVLA [ 17 ]
91.3
77.8
62.0
52.1
43.5
3.27
UniVLA [ 6 ]
95.5
85.8
75.4
66.9
56.5
3.80
PointMapPolicy [ 15 ]
96.1
88.6
80.5
72.3
63.6
4.01
EL3DD [ 4 ]
96.6
90.0
82.4
74.2
66.2
4.09
OpenVLA-OFT [ 16 ]
96.3
89.1
82.4
75.8
66.5
4.10
Table 1: Long-horizon robotic manipulation evaluation on CALVIN ABC → D.
Figure 2: Representative real-robot evaluation tasks. Representative test-time primitive subtasks are shown for two tabletop scenes. The examples cover diverse interaction modes, including nesting, pick-and-place, pushing, stacking, and insertion.
Figure 3: Real-robot sequence-completion results. (a) Per-sequence Avg. Len across eight length-3 sequences in two tabletop scenes. (b) Success@1–3 comparison over consecutive primitive completion. (c) Overall Avg. Len comparison across all real-robot episodes.
Query
LoRA-MoE
Aff. Heatmap
Skill Cond.
Tasks Completed in a Row (%)
Avg. Len ↑
1
2
3
4
5
–
98.5
95.0
90.5
85.3
80.0
4.50
✓
matched
98.7
95.6
91.3
86.4
81.1
4.53
✓
✓
matched
98.8
96.3
92.8
88.0
83.1
4.59
✓
–
99.0
96.7
93.4
88.6
83.7
4.61
✓
✓
✓
matched
99.4
97.4
94.2
89.2
84.7
4.65
Table 2: Ablation on CALVIN ABC → D. Query: skill-query tokens. LoRA-MoE: skill-conditioned low-rank experts. Aff. Heatmap: trajectory-affordance spatial prior. Skill Cond.: matched uses the instruction-derived primitive skill condition; mismatched replaces it with a random non-matching condition at test time.
Figure 4: Skill-conditioned Top-2 LoRA-MoE routing signatures. Expert-usage shifts across primitive skills and token groups are measured in percentage points (pp) relative to average usage. Stars/circles denote the experts with the highest/second-highest average routing usage for each skill; they indicate learned routing preferences rather than fixed skill-to-expert assignments.
Figure 5: RGB observations and predicted affordance heatmaps are shown across representative manipulation skills.
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Skill ID
Primitive skill condition
Representative fine-grained commands
0
Rotation
Rotate red/blue/pink blocks left or right.
1
Sliding / pushing
Push red/blue/pink blocks left or right; push tabletop objects to target regions.
2
Articulated-object operation
Move the slider left or right; open or close the drawer; close articulated containers.
3
Lifting / picking
Lift red/blue/pink blocks to the table, slider, or drawer; pick tabletop objects.
4
Placing / storing
Place objects onto the slider, into the drawer/box, or in other target regions.
5
Stacking / nesting
Stack or unstack blocks/erasers; nest disposable cups in real-robot sequences.
Appendix
Table 3: Primitive skill taxonomy and skill-condition ids used by MAST-VLA.
Item
Setting
Base policy
VLA-Adapter-Pro initialized from prism-qwen25-extra-dinosiglip-224px-0_5b .
Language backbone
Qwen2.5-0.5B, hidden dimension 896.
Visual input
Static third-person RGB and wrist/secondary RGB, resized to 224×224 .
Visual tokens
256 patch tokens per image; 512 visual tokens for two-view input.
Action representation
Continuous action chunk with horizon H=8 ; each 7-D action contains 6-DoF end-effector motion and one gripper command.
Action query tokens
64 learnable action tokens.
Appendix
Table 4: Main implementation settings of MAST-VLA.
Item
Setting
Training data
CALVIN ABC training split.
Training schedule
50k MAST-VLA fine-tuning updates.
GPUs
8 GPUs with distributed data parallel training.
Batch size
16 per GPU, gradient accumulation 1.
Optimizer
AdamW with parameter groups.
LoRA learning rate
2×10−4 .
Appendix
Table 5: Training hyperparameters for the main MAST-VLA run.
Scene
Seq.
Primitive instruction sequence
Scene 1
1
Pick cup → Nest cup → Push box
Scene 1
2
Stack eraser → Push box → Pick cup
Scene 1
3
Pick marker → Place marker → Stack eraser
Scene 1
4
Push box → Pick marker → Place marker
Scene 2
1
Pick flower → Insert flower → Stack blocks
Scene 2
2
Stack blocks → Unstack blocks → Push mango
Appendix
Table 6: Real-robot length-3 task sequences used in evaluation.
Scene
Seq.
Method
Tasks Completed in a Row (%)
Avg. Len ↑
1
2
3
Scene 1
1
OpenVLA-OFT
90
70
15
1.75
π0
100
85
50
2.35
π0.5
100
90
65
2.55
VLA-Adapter-Pro
95
85
30
2.10
MAST-VLA
100
85
85
2.70
Appendix
Table 7: Per-sequence real-robot evaluation results. Each method has 20 trials per sequence and 160 trials in total. Success rates are percentages and Avg. Len is out of 3.
Current Vision-Language-Action (VLA) models predominantly rely on end-to-end fine-tuning. While effective, this paradigm compromises the inherent generalization capabilities of Vision-Language Models (VLMs) and incurs catastrophic forgetting. To address these limitations, we propose M2-VLA, which demonstrates that a generalized VLM is able to serve as a powerful backbone for robotic manipulation directly. However, it remains a key challenge to bridge the gap between the high-level semantic understanding of VLMs and the precise requirements of robotic control. To overcome this, we introduce the Mixture of Layers (MoL) strategy that selectively extracts task-critical information from dense semantic features. Furthermore, to facilitate efficient trajectory learning under constrained model capacity, we propose a Meta Skill Module (MSM) that integrates strong inductive biases. Extensive experiments in both simulated and real-world environments demonstrate the effectiveness of our approach. Furthermore, generalization and ablation studies validate the architecture's zero-shot capabilities and confirm the contribution of each key component. Our code and pre-trained models will be made publicly available.
Siyao Xiao, Yuhong Zhang, Zhifang Liu +9
Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen, China. · PengCheng Laboratory, Shenzhen, China. · College of Intelligent Robotics and Advanced Manufacturing, Fudan University, Shanghai, China. +1
While Vision-Language-Action (VLA) models pretrained on large-scale robot datasets provide a strong foundation for robot manipulation, their performance can degrade when adapted to new tasks with limited task-specific demonstrations. Retrieval offers a practical way to reuse existing demonstrations for data-efficient adaptation, but existing methods often rely on visual similarity, state-action representations, or task-level language matching. These approaches may overlook the hierarchical structure of long-horizon manipulation tasks, where complete task matches are rare but reusable skills are often abundant. To address this challenge, we propose Hierarchical Skill Retrieval (HSR), a retrieval framework for data-efficient VLA adaptation. Specifically, HSR first decomposes a target task into candidate skill sequences. It evaluates each plan based on both semantic plausibility and skill reliability estimated from the prior dataset. The selected decomposition is then used for hybrid retrieval. This combines subtask-level language retrieval with behavior-feature reranking to identify demonstrations that are both semantically relevant and compatible with the target task. Finally, we adapt the policy through a two-stage pretraining and finetuning pipeline, which separates general skill acquisition from task-specific adaptation. Experiments on the LIBERO benchmark and several real-world robot manipulation tasks show that HSR improves the average success rate by 10.3% and 21.3% over the strongest baseline, respectively. These results demonstrate the effectiveness of structured skill-level retrieval for data-efficient VLA adaptation. Videos and code are available at https://hoar012.github.io/HSR-Project.
Haoran Hao, Shahram Najam Syed, Jeff Schneider +1
Robotics Institute, Carnegie Mellon University, Pittsburgh, PA 15213.
Vision-Language-Action (VLA) models have shown strong performance on robotic manipulation, but they often struggle to generalize to unseen tasks, configurations, and long-horizon settings. A key challenge is that VLAs overfit to training scenes and fail to follow novel language instructions. Off-the-shelf vision-language models (VLMs) often provide stronger generalization, but cannot directly control robot actions. To combine the common sense of VLMs with VLA control, we propose Guided Trace VLA (GT-VLA), a steerable framework that accepts guidance from an external generalist VLM through trace-conditioned action generation. GT-VLA uses a generalist model to identify semantic guidance for the current skill, converts this guidance into a 2D visual trace, and conditions its action policy on the resulting trace-rendered observation. This design separates semantic target acquisition, trace generation, and low-level action execution, allowing high-level guidance to propagate to robot actions. GT-VLA uses a Mixture-of-Experts architecture with skill-specific trace and action modules for robust execution. We evaluate GT-VLA on LIBERO and a physical robot platform, showing improved generalization over recent VLA baselines in both settings. The code and additional supplemental materials are available on our project website at https://ivaniz.github.io/gt-vla/.
Ninghan Zhong, Jing-Chen Peng, Sriram Vishwanath
School of Electrical and Computer Engineering, Georgia Institute of Technology, Atlanta, GA, USA.