DySL-VLA: Efficient Vision-Language-Action Model Inference via Dynamic-Static Layer-Skipping for Robot Manipulation
Authors: Zebin Yang, Yijiahao Qi, Tong Xie, Bo Yu, Shaoshan Liu, Meng Li
Organizations: Institute for Artificial Intelligence, Peking University, Beijing, China · School of Integrated Circuits, Peking University, Beijing, China · School of Electronics Engineering and Computer Science, Peking University, Beijing, China · Shenzhen Institute of Artificial Intelligence and Robotics for Society, Shenzhen, China · Beijing Advanced Innovation Center for Integrated Circuits, Beijing, China
Vision-Language-Action (VLA) models have shown remarkable success in robotic tasks like manipulation by fusing a language model's reasoning with a vision model's 3D understanding. However, their high computational cost remains a major obstacle for real-world applications that require real-time performance. We observe that the actions within a task have varying levels of importance: critical steps demand high precision, while less important ones can tolerate more variance. Leveraging this insight, we propose DySL-VLA, a novel framework that addresses computational cost by dynamically skipping VLA layers based on each action's importance. DySL-VLA categorizes its layers into two types: informative layers, which are consistently executed, and incremental layers, which can be selectively skipped. To intelligently skip layers without sacrificing accuracy, we invent a prior-post skipping guidance mechanism to determine when to initiate layer-skipping. We also propose a skip-aware two-stage knowledge distillation algorithm to efficiently train a standard VLA into a DySL-VLA. Our experiments indicate that DySL-VLA achieves 2.1% improvement in success length over Deer-VLA on the Calvin dataset, while simultaneously reducing trainable parameters by a factor of 85.7 and providing a 3.75x speedup relative to the RoboFlamingo baseline at iso-accuracy. Our code is available on https://github.com/PKU-SEC-Lab/DYSL_VLA.
Figures & tables
Figure 1. Different actions in robot manipulation have different importance. We show an example when the robot is performing task “Grasp the black cup and drop it into basket”. (a) shows the task completion rates when adding noise with different magnitudes to VLA model weights at different action steps. When adding noise at important action steps, the task completion rate drops faster as noise magnitude increases. We sample 50 times on each noise magnitude for each step range. We show the robot status at (b) step 25, (c) step 75, and (d) step 125 when using the origin VLA model.
Figure 2. VLA model architecture.
Method
Approach
Action Importance
Specialized
Training
Aware
Kernel Free
Modules
SparseVLM ( Zhang et al., 2024b )
Pruning
✗
✗
–
FastV ( Chen et al., 2024 )
Pruning
✗
✗
–
QAIL ( Park et al., 2024 )
Quantization
✗
✗
LLM Backbone
MoLe-VLA ( Zhang et al., 2025 )
Mixture-of-Layers
✗
✓
LLM Backbone
DeeR-VLA ( Yue et al., 2024 )
Early-exit
✓
✓
LLM Backbone
Table 1. Comparison between different methods.
Figure 3. The average cosine similarity between the output activations of different VLA layers for (a) RoboFlamingo-3B and (b) RoboFlamingo-9B. The similarity between the input and output activations of each layer and the model performance when skipping each VLA layer in a zero-shot manner for (c) RoboFlamingo-3B and (d) RoboFlamingo-9B.
Figure 4. (a) The ratio of different numbers of kept layers in VLA model inference when only using skipping controllers. The inference latency for different numbers of kept layers using (b) RoboFlamingo-3B in FP32 and (c) RoboFlamingo-9B in FP16.
Figure 5. The inference mode of (a) original VLA model, (b) using early exit with multiple action heads, (c) using early exit with adapters, (d) using traditional layer skipping methods, and (e) using dynamic-static layer skipping. The modules with light colour are not activated in current inference. We set the same legend for VLA layers in (a), (b), (c), (d), and dynamic layers in (e).
Figure 6. The proportion of action prediction steps in different continuity ranges and the accuracy loss when conducting layer skipping at the steps in different continuity ranges for (a) RoboFlamingo-3B and (b) RoboFlamingo-9B.
Figure 7. Prior-post skipping guidance. The modules with light colour are not activated in current inference.
Figure 8. Skipping-allow point changes in different steps. Here we use 4 dynamic layers between 2 static layers as an example, with 5 possible positions for skipping-allow point.
Figure 9. The (a) first stage and (b) second stage of skip-aware two-stage knowledge distillation method. The layers in red boxes are the selected layers in current training step.
Method
# Fine-tuned
# Fine-tuned
Training Cost
D → D
ABC → D
RTX 4090
Parameters
Steps
(GPU Hour)
Avg Length
Avg Length
Latency
HULC ( Mees et al., 2022a )
–
–
–
2.64
0.67
–
SPIL ( Zhou et al., 2024 )
–
–
–
2.67
1.71
–
RoboFlamingo 3B ( Li et al., 2023 )
–
–
–
2.92
2.85
51.0ms
DeeR-VLA 3B ( Yue et al., 2024 )
1.2B
9.2⋅104
112
2.83
2.82
19.3ms
Random Skip
–
–
–
0.38
0.45
22.6ms
Table 2. Evaluation on Calvin dataset.
Method
# Fine-tuned
Spatial
Object
Goal
Long
Average
A6000
Jetson Orin
Parameters
SR (%)
SR (%)
SR (%)
SR (%)
SR (%)
Latency
Latency
MDT (scratch) ( Reuss et al., 2024 )
–
78.5
87.5
73.5
64.8
76.1
–
–
π0 + FAST ( Black et al., 2024 )
–
96.4
96.8
88.6
60.2
85.5
–
–
OpenVLA-OFT 7B ( Kim et al., 2025 )
–
97.6
98.4
97.9
94.5
97.1
53.0ms
676ms
DeeR-VLA 7B ( Yue et al., 2024 )
7.1B
97.0
98.2
97.4
88.6
95.3
40.2ms
495ms
Random Skip
–
11.3
0.6
0.0
0.6
3.1
34.9ms
383ms
Table 3. Evaluation on LIBERO dataset.
Method
Avg Length
Avg Latency
RoboFlamingo 3B ( Li et al., 2023 )
2.82
51.0ms
Random Skip
0.38
22.6ms
DySL-VLA
2.89
13.6ms
w/o Post-skip Verification
2.79
13.4ms
w/o Pre-skip Prediction
2.42
20.1ms
w/o Skip-aware Two-stage Knowledge Distillation
2.82
74.0ms
Table 4. Individual influence of our methods (evaluated on Calvin D → D dataset).
Figure 10. Latency breakdown of LLM backbone.
Static Layer Ratio
10%
15%
20%
25%
30%
Average Length
2.81
2.84
2.89
2.88
2.89
Average Latency (ms)
12.6
13.1
13.6
14.7
15.8
Table 5. Ablation study on static layer ratio, evaluated on Calvin D → D dataset.
δl
1
2
3
4
5
Adaptive
SR
96.2
96.4
96.4
97.4
97.0
98.0
Table 6. Ablation study on δl (using LIBERO Spatial dataset).
k
1
3
5
7
9
Average Length
2.59
2.76
2.89
2.80
2.69
Table 7. Influence of the value of k in pre-skip prediction, evaluated on Calvin D → D dataset.
Vision-Language-Action (VLA) models have emerged as a powerful paradigm for open-world robot manipulation, but their practical deployment is often constrained by cost: billion-scale VLM backbones and iterative diffusion/flow-based action heads incur high latency and compute, making real-time control expensive on commodity hardware. We present A1, a fully open-source and transparent VLA framework designed for low-cost, high-throughput inference without sacrificing manipulation success; Our approach leverages pretrained VLMs that provide implicit affordance priors for action generation. We release the full training stack (training code, data/data-processing pipeline, intermediate checkpoints, and evaluation scripts) to enable end-to-end reproducibility. Beyond optimizing the VLM alone, A1 targets the full inference pipeline by introducing a budget-aware adaptive inference scheme that jointly accelerates the backbone and the action head. Specifically, we monitor action consistency across intermediate VLM layers to trigger early termination, and propose Inter-Layer Truncated Flow Matching that warm-starts denoising across layers, enabling accurate actions with substantially fewer effective denoising iterations. Across simulation benchmarks (LIBERO, VLABench) and real robots (Franka, AgiBot), A1 achieves state-of-the-art success rates while significantly reducing inference cost (e.g., up to 72% lower per-episode latency for flow-matching inference and up to 76.6% backbone computation reduction with minor performance degradation). On RoboChallenge, A1 achieves an average success rate of 29.00%, outperforming baselines including pi0(28.33%), X-VLA (21.33%), and RDT-1B (15.00%).
Vision-Language-Action (VLA) models pre-trained on massive video-robot datasets have revolutionized robotic manipulation, yet their multi-billion parameter architectures impose prohibitive computational burdens during downstream fine-tuning and real-time inference. In this work, we reveal a highly non-trivial architectural characteristic of these continuous control foundation policies (e.g., pi_0, GR00T-N1.5): despite being trained on diverse physical trajectories, they exhibit severe layer-wise representational redundancy. To exploit this, we introduce a structural compression pipeline that is entirely training-free, bypassing the need of existing methods to load full-scale models to learn optimized token reductions or dynamic layer selectors. Instead, using only a single forward pass via Centered Kernel Alignment to identify redundant layer features, we remove twin layers to permanently compress the model depth by up to 50% across both the VLM backbone and the continuous control policy head. Downstream fine-tuning of this streamlined architecture yields a dual acceleration benefit: a 40-50% reduction in training time and up to 30% faster real-time inference, while matching or exceeding full-scale base model performance. We comprehensively validate our method across three simulation benchmarks (LIBERO, RoboCasa, SimplerEnv) and 10 diverse real-world manipulation tasks across 4 unique robotic embodiments. These results prove that advanced VLAs require significantly fewer layers than previously assumed, offering a highly compute-efficient paradigm for scalable robot learning.
Gia-Binh Nguyen, Trong-Bao Ho, Thien-Loc Ha +18
Center for AI Research, VinUniversity · VinRobotics · University of Arkansas +10
Vision-Language-Action (VLA) models exhibit strong generalization for robotic manipulation, yet their high inference latency limits real time deployment. We identify two primary sources of temporal redundancy in existing VLA pipelines: repeated visual encoding of highly similar consecutive frames and multi step iterative sampling in diffusion based policies. To address this, we propose a system level acceleration strategy that reduces computation in both perception and action generation. On the perception side, we incrementally update only tokens corresponding to dynamic scene regions instead of re-encoding entire frames. On the policy side, we compress diffusion sampling into a compact 2-step schedule through efficiency oriented training while preserving action precision. Experiments on Libero, RobotWin, and Real Robot Platforms demonstrate over 2 times speedup while maintaining high performance, achieving up to 98% success rate on general manipulation benchmarks. Our codes will be released on Github.
Yuzhou Wu, Yuxin Zheng, Muchun Niu +6
1Tianji KernalMind co ltd · 2Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen, China · 3Shanghai Jiao Tong University, Shanghai, China +2