In this study, we address the problem of language-guided robotic manipulation, where a robot is required to manipulate a wide range of objects based on visual observations and natural language instructions. This task is essential for service robots that operate in human environments, and requires safety, efficiency, and task-level generality. Although Vision-Language-Action models (VLAs) have demonstrated strong performance for this task, their deployment in resource-constrained environments remains challenging because of the computational cost of standard transformer backbones. To overcome this limitation, we propose AnolisVLA, a lightweight VLA that uses a deep state space model to process multimodal sequences efficiently. The model uses lightweight, fast sequential state modeling to process visual and textual inputs. This design enables efficient trajectory generation. We evaluated the proposed method in simulation and on two real robot platforms. Compared with SmolVLA at a comparable parameter scale, AnolisVLA achieved higher task success rates, faster inference, and comparable GPU memory usage. In mobile manipulation, it outperformed the larger, 3-billion-parameter π0.5 model by 21 percentage points in task success rate while achieving approximately three times faster inference.
Figures & tables
Figure 1: Overview of AnolisVLA and real-world performance. (Left) Our deep SSM backbone processes language instructions and robot observations to generate trajectories, leveraging a two-stage training strategy with the acceleration loss to improve temporal consistency in action sequences. (Right) Real-world mobile manipulation experiment results. The horizontal and vertical axes represent inference speed and the average success rate, respectively.
Figure 2: Model architecture of AnolisVLA. Multimodal tokens, including proprioception, state delta, vision, and language, are concatenated and processed by a Mamba backbone, and the final token predicts an H -step action chunk. The two-stage training supervises action commands and their temporal differences. Here, s(t) , Δs(t) , xv(t) , and xl denote the state, state delta, visual observation, and language instruction at time step t , respectively. On the right-hand side, y^ and y denote the predicted future actions and ground truth, while Δy^ and Δy denote their temporal differences.
Method
Total Params
Easy
Medium
Hard
Very Hard
Avg.
π0.5 [ 1 ]
3.0 B
68.20
37.30
41.70
28.00
43.80
VLA-Adapter [ 29 ]
0.5 B
3.75
0.00
0.00
0.00
0.94
TinyVLA [ 30 ]
0.42 B
77.60
21.50
11.40
15.80
31.58
SmolVLA [ 23 ]
0.45 B
82.50
41.80
45.00
60.00
57.33
AnolisVLA
0.47 B
92.86
72.73
91.67
60.00
79.32
Table 1: Quantitative comparison between AnolisVLA and the baseline methods on the Meta-World benchmark. The best results are highlighted in bold , and the second-best results are underlined .
Figure 3: Qualitative results. We display the initial observation xv(0) and the chronological sequence of observations xv(t) during task execution. In each example, the robot executes the task based on the following instructions: (a) “Pick and place a puck onto a shelf.” (b) “Hammer a screw on the wall.” (c) “Push the puck to a goal.”
Model
Backbone
Acceleration loss
Success rate [%]
(i)
SmolLM (Transformer)
✓
76.00
(ii)
Mamba-370M (Mamba)
74.59
(iii)
Mamba-370M (Mamba)
✓
79.32
Table 2: Quantitative results of the ablation studies. The table presents the average success rates on the Meta-World benchmark. The best score is in bold.
Figure 4: (a) Robot platform and experimental environment used in the mobile manipulation experiments. We used the Human Support Robot (HSR) as the robot platform. (b) Everyday objects used in the experiments. We used objects with diverse appearances, shapes, and sizes to emulate common household manipulation scenarios. (c) ALOHA robot platform and experimental setup used in the tabletop manipulation experiments.
Method
Success rate [%]
Mobile move
Mobile pick
Mobile open
Mobile close
Mobile push
Avg.
π0.5 [ 1 ]
15
20
30
100
45
42
VLA-Adapter [ 29 ]
35
20
0
100
45
40
TinyVLA [ 30 ]
5
15
0
90
35
29
SmolVLA [ 23 ]
30
25
55
100
50
52
AnolisVLA
45
40
75
100
55
63
Table 3: Quantitative comparison between AnolisVLA and the baseline methods in mobile manipulation experiments. The best results are highlighted in bold , and the second-best results are underlined .
Method
Success rate [%]
Inference time ↓
VRAM ↓
Stack cup
Put in drawer
Avg.
[ms/chunk]
[GB]
π0.5 [ 1 ]
70
80
75
578
17.05
VLA-Adapter [ 29 ]
50
30
40
101
4.65
TinyVLA [ 30 ]
30
40
35
1290
2.38
SmolVLA [ 23 ]
40
70
55
309
1.56
AnolisVLA
60
80
70
216
1.64
Table 4: Quantitative comparison between AnolisVLA and the baseline methods on the ALOHA robot, including task success rates, inference time, and inference-time VRAM consumption. The ‘VRAM’ column reports the inference-time GPU memory consumption. The best results are highlighted in bold , and the second-best results are underlined .
Figure 5: Successful episodes of AnolisVLA in the real-world experiments. Instructions were as follows: (a) “Open the drawer.” (b) “Put the lemon into the cup.” (c) “Put the object in the drawer.” (d) “Stack the cups.”
Vision-Language-Action (VLA) models have recently emerged as a promising paradigm for robotic manipulation, in which reliable action prediction critically depends on accurately interpreting and integrating visual observations conditioned on language instructions. Although recent works have sought to enhance the visual capabilities of VLA models, most approaches treat the LLM backbone as a black box, providing limited insight into how visual information is grounded into action generation. Therefore, we perform a systematic analysis of multiple VLA models across different action-generation paradigms and observe that sensitivity to visual tokens progressively decreases in deeper layers during action generation. Motivated by this observation, we propose DeepVision-VLA, built on a Vision-Language Mixture-of-Transformers (VL-MoT) framework. This framework enables shared attention between the vision foundation model and the VLA backbone, injecting multi-level visual features from the vision expert into deeper layers of the VLA backbone to enhance visual representations for precise and complex manipulation. In addition, we introduce Action-Guided Visual Pruning (AGVP), which leverages shallow-layer attention to prune irrelevant visual tokens while preserving task-relevant ones, reinforcing critical visual cues for manipulation with minimal computational overhead. DeepVision-VLA outperforms prior state-of-the-art methods by 9.0% and 7.5% on simulated and real-world tasks, respectively, providing new insights for the design of visually enhanced VLA models.
Yulin Luo, Hao Chen, Zhuangzhe Wu +10
State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University · The Chinese University of Hong Kong · Simplexity Robotics
Current Vision-Language-Action (VLA) models predominantly rely on end-to-end fine-tuning. While effective, this paradigm compromises the inherent generalization capabilities of Vision-Language Models (VLMs) and incurs catastrophic forgetting. To address these limitations, we propose M2-VLA, which demonstrates that a generalized VLM is able to serve as a powerful backbone for robotic manipulation directly. However, it remains a key challenge to bridge the gap between the high-level semantic understanding of VLMs and the precise requirements of robotic control. To overcome this, we introduce the Mixture of Layers (MoL) strategy that selectively extracts task-critical information from dense semantic features. Furthermore, to facilitate efficient trajectory learning under constrained model capacity, we propose a Meta Skill Module (MSM) that integrates strong inductive biases. Extensive experiments in both simulated and real-world environments demonstrate the effectiveness of our approach. Furthermore, generalization and ablation studies validate the architecture's zero-shot capabilities and confirm the contribution of each key component. Our code and pre-trained models will be made publicly available.
Siyao Xiao, Yuhong Zhang, Zhifang Liu +9
Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen, China. · PengCheng Laboratory, Shenzhen, China. · College of Intelligent Robotics and Advanced Manufacturing, Fudan University, Shanghai, China. +1
Vision-Language-Action (VLA) models have emerged as a powerful paradigm for open-world robot manipulation, but their practical deployment is often constrained by cost: billion-scale VLM backbones and iterative diffusion/flow-based action heads incur high latency and compute, making real-time control expensive on commodity hardware. We present A1, a fully open-source and transparent VLA framework designed for low-cost, high-throughput inference without sacrificing manipulation success; Our approach leverages pretrained VLMs that provide implicit affordance priors for action generation. We release the full training stack (training code, data/data-processing pipeline, intermediate checkpoints, and evaluation scripts) to enable end-to-end reproducibility. Beyond optimizing the VLM alone, A1 targets the full inference pipeline by introducing a budget-aware adaptive inference scheme that jointly accelerates the backbone and the action head. Specifically, we monitor action consistency across intermediate VLM layers to trigger early termination, and propose Inter-Layer Truncated Flow Matching that warm-starts denoising across layers, enabling accurate actions with substantially fewer effective denoising iterations. Across simulation benchmarks (LIBERO, VLABench) and real robots (Franka, AgiBot), A1 achieves state-of-the-art success rates while significantly reducing inference cost (e.g., up to 72% lower per-episode latency for flow-matching inference and up to 76.6% backbone computation reduction with minor performance degradation). On RoboChallenge, A1 achieves an average success rate of 29.00%, outperforming baselines including pi0(28.33%), X-VLA (21.33%), and RDT-1B (15.00%).