Fast-VLA: You Only Think Once for Robot Manipulation
Organizations: Huawei Technologies Co., Ltd · Huawei Central Research Institute
Abstract
Vision-language-action (VLA) policies commonly place a multi-billion-parameter model inside the feedback loop, repeatedly recomputing task context whenever they predict a new action chunk. Yet task understanding changes far more slowly than the robot state. We introduce Fast-VLA, which invokes a frozen 4B VLM once per episode to contextualize the instruction and initial views as fixed, multi-layer key-value task memory. During execution, a lightweight online actor fuses learned time- and arm-indexed queries with current visual and proprioceptive feedback to form action queries. These queries retrieve task context from the fixed memory and predict action chunks without re-running the foundation model. Fast-VLA achieves 92.46% and 92.40% success on RoboTwin 2.0 Clean and Randomized, and 95.95% and 79.11% on LIBERO and LIBERO-Plus evaluation. Against three continually conditioned baselines on the same hardware, it reduces RoboTwin amortized policy inference time by 10.9-17.1x and requires 1.22 GiB of post-prefill online resident memory. Task-memory swaps further show that the actor relies on episode-matched task memory. These results show that task context can be established once while closed-loop control remains responsive throughout execution. Project page: https://garfield-kh.github.io/FastVLA
Figures & tables
| Setting | RoboTwin 2.0 | LIBERO |
|---|---|---|
| Learning rate | ||
| Batch size | 256 | 32 |
| Training steps | 200,000 | 30,000 |
| Training compute (accelerator-hours) |
| Type | Method | Action PT | Pred./ exec. | ms/chunk | ms/step | GiB | Clean | Rand. | Mean |
| VLA | ( Black et al., 2025 ) | 50/50 | 72.38 | 1.580 | 6.26 | 65.92 ♭ | 58.40 ♭ | 62.16 | |
| ( Physical Intelligence et al., 2025 ) | 50/50 | 80.51 | 1.758 | 6.47 | 82.74 ♭ | 76.76 ♭ | 79.75 | ||
| X-VLA ( Zheng et al., 2025 ) | – | 30/30 | 211.03 | 7.452 | 3.29 | 72.90 ♭ | 72.80 ♭ | 72.85 | |
| LingBot-VLA ( Wu et al., 2026a ) | 50/25 | 740.35 | 30.919 | 7.99 | 86.50 | 85.34 | 85.92 | ||
| Qwen-VLA ( Wang et al., 2026 ) | – | – | – | – | 86.10 | 87.20 | 86.65 | ||
| LingBot-VLA 2.0 ( Wu et al., 2026b ) | 50/25 | 931.29 | 38.892 | 11.98 | 93.52 | 92.80 | 93.16 |
| Type | Method | Action PT | Pred./ exec. | ms/chunk | Amort. ms/step | GiB | LIBERO Std. | LIBERO Plus |
| VLA | ( Black et al., 2025 ) | 50/5 | 81.96 | 16.577 | 6.26 | 94.10 | 53.60 | |
| -FAST ( Pertsch et al., 2025 ) | 10/5 | 353.06 | 71.885 | 5.57 | 85.50 ♭ | 61.60 | ||
| OpenVLA-OFT ( Kim et al., 2025 ) | 8/8 | 107.00 | 13.685 | 14.41 | 97.10 ♭ | 69.60 | ||
| ( Physical Intelligence et al., 2025 ) | 10/5 | 86.35 | 17.466 | 6.47 | 96.90 | 85.70 | ||
| UniVLA ( Bu et al., 2025 ) | 12/1 | 241.96 | 241.960 | 14.15 | 95.50 ♭ | 43.90 | ||
| X-VLA ( Zheng et al., 2025 ) | – | 30/30 | 199.40 | 7.220 | 3.29 | 98.10 | 72.56 ∥ |
| Memory producer | Attention | Memory | Std. | Plus |
|---|---|---|---|---|
| PaliGemma v1 ( Beyer et al., 2024 ) | 18L, MQA | 9.8 MB | 95.35 | 73.15 |
| 18L, MQA | 9.8 MB | 94.90 | 71.48 | |
| Qwen2.5-VL-7B ( Bai et al., 2025 ) | 28L, GQA-4 | 31.8 MB | 95.35 | 75.55 |
| DM0.5 (default) | 34L, GQA-4 | 79.7 MB | 95.95 | 79.11 |
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
| Setting | RoboTwin 2.0 | LIBERO |
|---|---|---|
| Visual encoder | DINOv2 ViT-B/14 | DINOv2 ViT-B/14 |
| Actor image resolution | ||
| VLM prefill images | , 3 cameras | , 2 cameras |
| VLM image tokens | 768 | 512 |
| Decoder width / layers / heads | 256 / 4 / 8 | 256 / 4 / 8 |
| Cross-memory GQA KV heads | 8 | 8 |
| Item | Protocol |
|---|---|
| Execution precision | BF16 for the VLM; BF16 autocast with gradient computation disabled for the actor. |
| Batch size | |
| Warmup and timing | 3 warmup calls followed by 10 timed calls; report mean latency. |
| Device synchronization | Synchronized immediately before and after each timed call. |
| Preprocessing and transfer | Timing starts from raw uint8 HWC NumPy images and includes resizing, normalization, and host-to-device transfer. |
| Inference backend | PyTorch 2.7.1+cu128; no torch.compile or TensorRT. |
Vision-Language-Action Models
Vision-language-action models turn visual observations and natural-language instructions into actions for robots and other embodied agents.
See all 1,168 papers