Vision-Language-Action Models
Also known as VLA
Overview
These models combine vision-language backbones with action generators that produce continuous motion or discrete action tokens, often in chunks and with proprioceptive input. They are commonly trained on robot demonstrations, with flow-matching and diffusion policies, reinforcement fine-tuning, and predictive world models used to improve control and planning. Research spans tabletop and bimanual manipulation, mobile and aerial navigation, autonomous driving, and humanoid control, evaluated on benchmarks such as LIBERO, CALVIN, RoboTwin, and driving simulators. A central challenge is making broad visual and language competence translate into reliable closed-loop behavior under new objects, viewpoints, embodiments, delays, and safety constraints; benchmark success can still conceal weak instruction grounding and sim-to-real failures.
Latest papers
1,154 papers249 papers in the last four weeks, up 144% on the four weeks before. 1.5% of all new papers.