Vision-Language-Action (VLA) models have shown promising results in multi-task robotic manipulation by leveraging visual and language understanding for action generation. However, existing lightweight VLA models employ a single dense action expert to handle all tasks, which limits their multi-task capability under constrained computational budgets. To address this issue, we propose SmolVLA-MoE, a novel framework that introduces sparse Mixture-of-Experts (MoE) into the FFN layers of the action expert in VLA models. SmolVLA-MoE replaces dense FFN layers with multiple parallel expert FFNs and a token-level router, enabling each token to adaptively select the most suitable expert for action generation. We further introduce a shared expert with a tunable weight parameter to balance the inheritance of pre-trained capabilities and task-specific expert specialization. Through sparse activation, SmolVLA-MoE activates only a single expert FFN during inference, with a total activated parameter count of approximately 500M including the frozen VLM backbone, while benefiting from approximately 800M total stored parameters. We evaluate our method on four task suites of the LIBERO benchmark. SmolVLA-MoE achieves an average success rate of 84.25%, outperforming SmolVLA (82.75%) and OpenVLA-7B (76.5%), while maintaining comparable inference efficiency. Our code is publicly available at https://github.com/dudu435/smolvla-moe.
Figures & tables
Figure 1: Overview of the SmolVLA-MoE architecture. The frozen VLM backbone processes visual observations and language instructions to produce contextual representations. The action expert replaces dense FFN layers with MoE-FFN layers, each containing a shared expert Es (always activated, weighted by α ), N routed expert FFNs, a token-level router with top- K selection, and a scaling adapter. Through sparse activation, only K experts are selected per token during both training and inference.
Figure 2: Detailed structure of the MoE-FFN layer. The input token is processed by both the shared expert and the top- K routed experts selected by the router. The scaling adapter provides additional per-token adaptive scaling. The final output is a weighted combination of the shared expert output (scaled by α ) and the routed expert output (scaled by 1−α ).
Figure 3: Visualization of complete rollouts in the LIBERO simulation environment. Each row shows key frames of the execution trajectory for a multi-step manipulation task from start to completion.
Method
Params
Spatial
Object
Goal
LIBERO-10
Avg.
Diffusion Policy [ 5 ]
321M
78.3
92.5
68.3
50.5
72.4
Octo [ 22 ]
93M
78.9
85.7
84.6
51.1
75.1
OpenVLA [ 14 ]
7B
84.7
88.4
79.2
53.7
76.5
π0 (Paligemma) [ 2 ]
3B
87
63
89
48
71.8
π0 [ 2 ]
3.3B
90
86
95
73
86.0
SmolVLA [ 25 ]
500M
87
93
88
63
82.75
Table 1: Main Results on the LIBERO Benchmark (Success Rate %)
Figure 4: Per-suite success rate comparison across all methods on the LIBERO benchmark. SmolVLA-MoE (highlighted with black borders) achieves competitive or superior performance across all four suites with ∼ 500M total activated parameters ( ∼ 100M in the action expert).
Figure 5: Heatmap visualization of the Top-K × shared expert weight ( α ) ablation study across all four LIBERO suites. Best values in each suite are shown in bold. Top-1 routing with α=0.3 achieves the best overall performance.
Configuration
Goal
SmolVLA-MoE (w/o shared expert, α=0 )
88
SmolVLA-MoE (w/ shared expert, α=0.3 )
93
Table 3: Ablation Study — Shared Expert (Success Rate %)
Configuration
Goal
SmolVLA-MoE (w/o load balancing, λ=0 )
90
SmolVLA-MoE (w/ load balancing, λ=0.01 )
93
Table 4: Ablation Study — Load Balancing Loss (Success Rate %)
Current Vision-Language-Action (VLA) models predominantly rely on end-to-end fine-tuning. While effective, this paradigm compromises the inherent generalization capabilities of Vision-Language Models (VLMs) and incurs catastrophic forgetting. To address these limitations, we propose M2-VLA, which demonstrates that a generalized VLM is able to serve as a powerful backbone for robotic manipulation directly. However, it remains a key challenge to bridge the gap between the high-level semantic understanding of VLMs and the precise requirements of robotic control. To overcome this, we introduce the Mixture of Layers (MoL) strategy that selectively extracts task-critical information from dense semantic features. Furthermore, to facilitate efficient trajectory learning under constrained model capacity, we propose a Meta Skill Module (MSM) that integrates strong inductive biases. Extensive experiments in both simulated and real-world environments demonstrate the effectiveness of our approach. Furthermore, generalization and ablation studies validate the architecture's zero-shot capabilities and confirm the contribution of each key component. Our code and pre-trained models will be made publicly available.
Siyao Xiao, Yuhong Zhang, Zhifang Liu +9
Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen, China. · PengCheng Laboratory, Shenzhen, China. · College of Intelligent Robotics and Advanced Manufacturing, Fudan University, Shanghai, China. +1
In this study, we address the problem of language-guided robotic manipulation, where a robot is required to manipulate a wide range of objects based on visual observations and natural language instructions. This task is essential for service robots that operate in human environments, and requires safety, efficiency, and task-level generality. Although Vision-Language-Action models (VLAs) have demonstrated strong performance for this task, their deployment in resource-constrained environments remains challenging because of the computational cost of standard transformer backbones. To overcome this limitation, we propose AnolisVLA, a lightweight VLA that uses a deep state space model to process multimodal sequences efficiently. The model uses lightweight, fast sequential state modeling to process visual and textual inputs. This design enables efficient trajectory generation. We evaluated the proposed method in simulation and on two real robot platforms. Compared with SmolVLA at a comparable parameter scale, AnolisVLA achieved higher task success rates, faster inference, and comparable GPU memory usage. In mobile manipulation, it outperformed the larger, 3-billion-parameter π0.5 model by 21 percentage points in task success rate while achieving approximately three times faster inference.
Vision-Language-Action (VLA) models have shown remarkable success in robotic tasks like manipulation by fusing a language model's reasoning with a vision model's 3D understanding. However, their high computational cost remains a major obstacle for real-world applications that require real-time performance. We observe that the actions within a task have varying levels of importance: critical steps demand high precision, while less important ones can tolerate more variance. Leveraging this insight, we propose DySL-VLA, a novel framework that addresses computational cost by dynamically skipping VLA layers based on each action's importance. DySL-VLA categorizes its layers into two types: informative layers, which are consistently executed, and incremental layers, which can be selectively skipped. To intelligently skip layers without sacrificing accuracy, we invent a prior-post skipping guidance mechanism to determine when to initiate layer-skipping. We also propose a skip-aware two-stage knowledge distillation algorithm to efficiently train a standard VLA into a DySL-VLA. Our experiments indicate that DySL-VLA achieves 2.1% improvement in success length over Deer-VLA on the Calvin dataset, while simultaneously reducing trainable parameters by a factor of 85.7 and providing a 3.75x speedup relative to the RoboFlamingo baseline at iso-accuracy. Our code is available on https://github.com/PKU-SEC-Lab/DYSL_VLA.
Zebin Yang, Yijiahao Qi, Tong Xie +3
Institute for Artificial Intelligence, Peking University, Beijing, China · School of Integrated Circuits, Peking University, Beijing, China · School of Electronics Engineering and Computer Science, Peking University, Beijing, China +2