Vision-Language-Action Models: A Life-Cycle Survey of Design, Training and Deployment
Organizations: Witsense AI · University of Huddersfield · Department of Computer Science, University of Huddersfield
Abstract
Vision-language-action (VLA) models adapt web-pretrained vision-language models to output robot actions, so that web knowledge carries over to physical control. In under 4 years the field has moved from RT-2, the first VLA, to open models that fine-tune on a single consumer GPU and to proprietary generalists that clean unseen homes. This survey reviews 88 works from December 2022 to October 2026, mainly on manipulation and navigation, plus humanoid control, world-action models and evaluation. We follow a VLA through its life cycle, from model design to training and deployment, along seven axes. Model design covers (a) inputs and memory, what the model observes; (b) architecture, how its pipeline is built; (c) action output, how it produces motor commands; and (d) intermediate predictions, what else it predicts, from reasoning steps to future observations. Training covers (e) pretraining data, the mix of robot, web, human and simulated data, and (f) post-training, adaptation to the target robot. Deployment covers (g) real-time execution, keeping pace with the robot. We analyse 22 models on these axes and report their training data, model size, inference speed and hardware. Three trends emerge: (i) training data has broadened, from about 10,000 hours of teleoperated robot data () to 23,000 community-collected episodes (SmolVLA) and 20,000 hours of egocentric human video (GR00T N1.7); (ii) model sizes have split: most recent models have 3-7 billion parameters, while a new group under one billion runs on consumer GPUs; and (iii) generalist VLAs are emerging that handle both manipulation and navigation, such as UniVLA and Qwen-VLA. We also compare VLAs with world-action models, which predict future observations as well as actions. We review evidence that some VLAs with near-perfect benchmark scores fail when objects or the camera move slightly, and open up further research questions.
Figures & tables
| Family | Representative works | Trade-off |
|---|---|---|
| Discrete bin tokens | RT-2 [ 1 ] , OpenVLA [ 36 ] | Reuses the language-model head unchanged; per-dimension binning struggles with high-frequency dexterous data and decodes slowly. |
| Compressed tokens | FAST [ 37 ] , SpatialVLA [ 38 ] | Compression with the discrete cosine transform (DCT) lets autoregressive VLAs match diffusion VLAs with up to faster training; adaptive grids re-discretise per robot. |
| Plain text numbers | VLA-0 [ 39 ] | No change to the VLM; reported to outperform several more complex designs on LIBERO [ 40 ] . |
| Latent actions | LAPA [ 41 ] , UniVLA [ 42 ] , VLA-JEPA [ 43 ] | Pretrain on action-free human or web video; a small decoder maps latents to real actions. |
| Diffusion head or expert | Octo [ 44 ] , CogACT [ 33 ] , RDT-1B [ 45 ] , DexVLA [ 46 ] , GR00T N1 [ 32 ] | Represents multimodal action distributions; requires iterative sampling. |
| Flow-matching expert | [ 17 ] , [ 31 ] , SmolVLA [ 47 ] , X-VLA [ 48 ] | Smooth high-rate chunks; the current default in frontier systems. |
| Model | Params | Reported inference speed | Hardware |
| RT-2 [ 1 ] | 55B (5B) | 1–3 Hz (5B: 5 Hz) | Multi-TPU cloud service, queried over the network |
| OpenVLA [ 36 ] | 7B | 6 Hz; 15 GB in bf16 | One RTX 4090 |
| TinyVLA [ 84 ] | 1B | 14 ms per action prediction (OpenVLA-7B: 292 ms) | One A6000 |
| [ 17 ] | 3.3B | 73 ms per chunk on board, 86 ms off board; control up to 50 Hz | One RTX 4090; off-board over Wi-Fi for the mobile robot |
| OpenVLA-OFT [ 49 ] | 7B | 71.4 Hz throughput, 0.11 s latency (LIBERO); real bimanual control at 25 Hz | One A100 |
| GR00T N1 [ 32 ] | 2.2B | 63.9 ms per 16-action chunk; VLM at 10 Hz, actions at 120 Hz | One L40 |
| Version | Vision-language backbone | Main changes |
|---|---|---|
| N1 [ 32 ] | Eagle-2 VLM [ 91 ] | Dual system: VLM plus a flow-matching diffusion-transformer (DiT) action head, 2.2B parameters; trained on real humanoid data, human video, simulation and generated “neural trajectories”. |
| N1.5 [ 89 ] | Eagle 2.5 [ 92 ] , frozen in pretraining and fine-tuning | Simplified multilayer-perceptron (MLP) adapter; FLARE future-latent alignment objective alongside flow matching, for learning from egocentric human video; DreamGen trajectories; language following on Fourier GR-1 humanoid tasks reported at 93.3% versus 46.6% for N1; new heads for single-arm end-effector control and AgiBot Genie-1. |
| N1.6 [ 90 ] | Internal Cosmos-Reason-2B variant [ 93 ] a ; native aspect ratio | DiT doubled to 32 layers; adapter removed and the top 4 VLM layers unfrozen; state-relative action chunks; several thousand added hours of teleoperation (YAM bimanual arms, AgiBot Genie-1, simulated Galaxea R1 Pro, Unitree G1 loco-manipulation), for 10k+ hours of robot data in total. |
| N1.7 [ 76 ] | Cosmos-Reason2-2B (Qwen3-VL architecture [ 94 ] ) | 3B parameters; relative end-effector action space shared by human and robot data; 20k hours of EgoScale [ 75 ] human video in pretraining; DiT reduced to 16 layers; state/action dimensions 29 132 and action horizon 16 40; dataset-mixture weighting; ONNX/TensorRT export; Apache-2.0 licence. Reported as comparable to N1.6 with better generalisation and language following. |
| Model | Released | Training data | Key idea |
|---|---|---|---|
| RT-1 [ 23 ] | Dec 2022 | 130k episodes, 700+ tasks, 13 robots of one type | Transformer over image and language tokens; data diversity beats volume. |
| PaLM-E [ 24 ] | Mar 2023 | Web VL data + task-and-motion-planning (TAMP) sim, Language-Table [ 95 ] , mobile manipulation | 562B embodied LM as planner; positive transfer from task mixtures. |
| Diffusion Policy [ 25 ] | Mar 2023 | Per-task demos (90–210 real) | Policy as conditional denoising over action sequences. |
| ACT [ 18 ] | Apr 2023 | 50 demos per task on ALOHA | Action chunking with a conditional variational autoencoder (CVAE) transformer; low-cost bimanual hardware. |
| RT-2 [ 1 ] | Jul 2023 | RT-1 data co-fine-tuned with web VQA | Coined “VLA”; actions as text tokens; emergent semantic reasoning. |
| Open X-Embodiment [ 26 ] | Oct 2023 | Dataset: 1M+ trajectories, 22 embodiments | Standard cross-lab format; RT-X shows positive transfer. |
| Model | Released | Training data | Key idea |
|---|---|---|---|
| ViNT [ 67 ] | Jun 2023 | 100+ h real trajectories, 8 platforms | Navigation foundation transformer; swappable goal encoders. |
| NoMaD [ 68 ] | Oct 2023 | GNM + SACSoN [ 105 ] (100+ h) | Goal masking unifies exploration and goal reaching. |
| NaVid [ 30 ] | Feb 2024 | 510k VLN-CE samples + 763k web data | Monocular RGB video only; history as memory. |
| Mobility VLA [ 21 ] | Jul 2024 | No policy training; demonstration-tour video | Long-context VLM finds the goal frame; topological graph executes. |
| LeLaN [ 69 ] | Oct 2024 | 130 h action-free video (robot, YouTube, walking) | VLM labelling of in-the-wild video for object navigation. |
| NaVILA [ 22 ] | Dec 2024 | R2R-CE, RxR-CE, touring videos, VQA | Mid-level language commands executed by RL locomotion. |
| Model | Released | Training data | Key idea |
|---|---|---|---|
| Humanoid-VLA [ 111 ] | Feb 2025 | Human motion datasets with text, egocentric video, self-generated question–answer pairs from motion | Language–motion pre-alignment, then video-conditioned fine-tuning over whole-body control. |
| LeVERB [ 110 ] | Jun 2025 | Synthetic: 154 trajectories 100 randomisations, 17.1 h of rendered rollouts plus 2.7 h language-only | Latent “verbs” from a vision-language policy drive an RL whole-body controller; 150-task benchmark. |
| SONIC [ 112 ] | Nov 2025 | 700 h of motion capture (100M+ frames), 21k GPU hours | Motion-tracking controller up to 42M parameters; one token space for VR teleoperation and VLAs. |
| SENTINEL [ 64 ] | Nov 2025 | Human motion tracked in simulation by a pretrained controller, with text | End-to-end language and proprioception to joint actions by flow matching; residual RL head; Unitree G1. |
| WholeBodyVLA [ 65 ] | Dec 2025 | Action-free egocentric human video (latent actions) plus AgiBot X2 teleoperation | Latent VLA plus a loco-manipulation RL policy; +21.3 points over OpenVLA-OFT. |
| WOLF-VLA [ 66 ] | Jun 2026 | 277 h of optimal-control trajectories, six locomotion task families | VLA trained on dynamically feasible trajectories with egocentric vision and language; RH5 humanoid. |
| Vision-language-action model (VLA) | World-action model (WAM) | |
|---|---|---|
| Pretrained prior | Vision-language model trained on image–text data | Video generation model trained on video |
| Training signal | Robot actions, often co-trained with VQA and web data [ 1 , 31 ] | Future frames or latents together with actions [ 119 , 120 ] |
| Predicts | Action chunk (sometimes text subtasks or reasoning [ 31 , 54 ] ) | Future observations and an action chunk; some also values [ 121 ] |
| Strengths | Language following, semantic generalisation, reasoning [ 1 ] | Physical dynamics, temporal consistency, use of action-free video [ 122 , 77 ] , test-time planning [ 121 ] |
| Weaknesses | Weak grounding in dynamics [ 118 ] ; relies on action-labelled data | Weaker semantics and reasoning [ 118 ] ; costly video backbones [ 123 ] |
| Examples | RT-2 [ 1 ] , OpenVLA [ 36 ] , [ 17 ] , [ 31 ] , GR00T N1 [ 32 ] | UniPi [ 122 ] , VPP [ 124 ] , UVA [ 119 ] , WorldVLA [ 120 ] , Genie Envisioner [ 125 ] , Cosmos Policy [ 121 ] , LeWAM [ 123 ] |
| Benchmark | Venue | What it tests |
|---|---|---|
| LIBERO [ 40 ] | NeurIPS 2023 (D&B) | Simulated Franka tasks in suites for spatial layout, objects, goals and long horizons; the most widely reported VLA benchmark. |
| CALVIN [ 86 ] | RA-L 2022 | Long chains of language-conditioned tasks across four simulated environments; generalisation from environments A–C to D. |
| SimplerEnv [ 114 ] | CoRL 2024 | Simulated replicas of the Google Robot and WidowX setups, built to correlate with real-world results. |
| RoboCasa [ 126 ] | RSS 2024 | 100 household kitchen tasks in large-scale simulation with generated assets. |
| VLABench [ 127 ] | ICCV 2025 | 100 task categories and over 2,000 objects testing world knowledge, implicit instructions and long-horizon reasoning. |
| RoboTwin 2.0 [ 78 ] | ICML 2026 | 50 dual-arm tasks on five embodiments with domain randomisation over clutter, lighting, background, table height and language. |
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
| Situation | Start from | Why |
|---|---|---|
| One consumer GPU and a low-cost arm | SmolVLA [ 47 ] | 0.45B parameters; trains on one GPU; pretrained on community data from low-cost SO-100 arms; asynchronous inference (Sections 3.5 and 3.7 ). |
| Fine-tuning an open VLA for one robot or task set | OpenVLA-OFT recipe [ 49 ] | Parallel decoding, action chunking, continuous actions and an L1 loss raised OpenVLA’s LIBERO average from 76.5% to 97.1% (Section 3.6 ). |
| Dexterous, bimanual or high-rate control | A flow-matching action expert, as in [ 17 ] | Smooth action chunks at up to 50 Hz; the current default in frontier systems (Table 1 ). |
| Simulation benchmark with few demonstrations | VLA-Adapter [ 50 ] or VLA-0 [ 39 ] | Strong LIBERO results with no robot pretraining, but LIBERO tests far less than real-world use (Section 9 ). |
| Humanoid | GR00T N1 line [ 32 ] , or a VLA over a learned whole-body controller | Open weights and humanoid training data; most humanoid VLAs rely on a separately trained whole-body controller (Section 6 ). |
| Real-robot navigation with on-board compute | Visual navigation foundation models (ViNT, NoMaD, OmniVLA) | Trained on real robot data, output waypoints and target edge deployment (Section 5 ). |
| Term | Meaning | Section |
|---|---|---|
| Model | ||
| Policy | The function that maps what the robot observes to the action it takes; a VLA is one kind of policy. | 2 |
| Vision-language model (VLM) | A model pretrained on web images and text that answers questions about images. VLAs start from one and add robot actions as an output. | 2 |
| Vision-language-action model (VLA) | A VLM adapted to output robot actions from camera images and a language instruction. | 2 |
| Backbone | The pretrained network a VLA is built on, usually the VLM. | 3.2 |
| Proprioception | The robot’s sensing of its own state, such as joint angles and gripper opening. | 2 |
Vision-Language-Action Models
Vision-language-action models turn visual observations and natural-language instructions into actions for robots and other embodied agents.
See all 1,234 papers