Vision-Language-Action Models

Also known as VLA

These models combine vision-language backbones with action generators that produce continuous motion or discrete action tokens, often in chunks and with proprioceptive input. They are commonly trained on robot demonstrations, with flow-matching and diffusion policies, reinforcement fine-tuning, and predictive world models used to improve control and planning. Research spans tabletop and bimanual manipulation, mobile and aerial navigation, autonomous driving, and humanoid control, evaluated on benchmarks such as LIBERO, CALVIN, RoboTwin, and driving simulators. A central challenge is making broad visual and language competence translate into reliable closed-loop behavior under new objects, viewpoints, embodiments, delays, and safety constraints; benchmark success can still conceal weak instruction grounding and sim-to-real failures.

Latest papers

1,154 papers249 papers in the last four weeks, up 144% on the four weeks before. 1.5% of all new papers.

TrAct: Bridging Robot Control and Visual Prediction with Visual Tracks
Aug 25, 2026Zhi Cao, Howard Ji, Kevin Zhang +4Vision-Based Robot Control World Models for Robotics
University of Michigan · Stanford University
Hierarchical Skill Retrieval for Data-Efficient Adaptation of Vision-Language-Action Models
Aug 25, 2026Haoran Hao, Shahram Najam Syed, Jeff Schneider +1Data-Efficient Robot Learning Vision-Language-Action Models
Robotics Institute, Carnegie Mellon University, Pittsburgh, PA 15213.
Act with Intent: Distilling Behavior Intent for Vision-Language-Action Models
Aug 24, 2026Sangoh Lee, Sangwoo Mo, Wook-Shin HanVision-Language-Action Models Robotic Manipulation
GSAI, POSTECH · IME, POSTECH
ROS2SmolVLA: Enabling Small Vision-Language-Action Models for Integration into Industrial-Grade Lightweight Robots
Aug 24, 2026Nils Mandischer, Noah Böckmann, Ludwig Holl +1Robotics Industrial Robotics
University of Augsburg, Chair of Mechatronics, Am Technologiezentrum 8, 86159 Augsburg, Germany
Think Only When Needed: Prompt-Authority Control for Selective Slow-Path Intervention in Vision-Language-Action Manipulation
Aug 24, 2026Zhiruo Zhou, Zelin Li, Xiwen Chen +4Vision-Language-Action Models VLMs for Robotics
SIGS, Tsinghua University, China · Wuhan University of Technology, China · Shanghai Jiao Tong University, China +3
Pointing-VLA: Typed Spatial Grounding Interfaces for Vision-Language-Action Manipulation
Aug 24, 2026Xiwen Chen, Zelin Li, Zhiruo Zhou +3Vision-Language-Action Models Robotic Manipulation
SIGS, Tsinghua University, China · Shanghai Jiao Tong University, China · Wuhan University of Technology, China +2
InstructMove: A Text-Indispensable Benchmark for Instruction-Following Manipulation
Aug 24, 2026Mengao Zhao, Ziang Li, Chaodong Huang +15Robot Manipulation Benchmarks Language-Conditioned Robot Manipulation
Horizon Robotics · Southeast University · Huazhong University of Science and Technology +1
UniMem: Unifying Multimodal Memory and Control for Vision-Language-Action Models
Aug 24, 2026Lars Osterberg, Maggie Wang, Mac SchwagerMemory-Augmented VLMs Vision-Language-Action Models
Stanford University, Stanford, CA, USA
WorldToken: Time-First Sequence Modeling for Robotic Imitation Learning
Aug 23, 2026Chunkai Yang, Andong Yang, Di Huang +2Robot Policy Learning Long-Horizon Robotic Manipulation
School of Remote Sensing and Information Engineering, Wuhan University, Wuhan, China. · Department of Electronic Engineering, Tsinghua University, Beijing, China. · Institute for AI Industry Research, Tsinghua University, Beijing, China.
Robust Bimanual Vision-Language-Action Models via Embarrassingly Simple Modality Masking
Aug 23, 2026Dongzhou Cheng, Ziang Li, Yixiao Zhou +6Vision-Language-Action Models Robustness of VLA Models
Southeast University · Shanghai Innovation Institute · Wuhan University +4
Just Noticeable Difference Modeling for Token Compression in Vision-Language-Action Models
Aug 21, 2026Zhuoyuan Li, Rui Zhao, Jin Wang +5Vision-Language-Action Models Visual Token Pruning
Department of Electrical and Electronic Engineering, The Hong Kong Polytechnic University, Hong Kong · School of Computer Science and Engineering, Nanyang Technological University, Singapore 639798 · Bytedance Inc. +2
ForeTime-VLA: Causal Future-Token Distillation from a World Action Model for Conveyor-Belt Manipulation
Aug 21, 2026Siyuan Ma, Yutian Zhang, Boshi Zhang +4Visuomotor Policy Learning Vision-Language-Action Models
Tsinghua University, Beijing, China · Shanghai Artificial Intelligence Laboratory, Shanghai, China · Harbin Institute of Technology, Harbin, China +1
Logic-VLA: A Temporal Logic Conditioned Vision-Language-Action Model
Aug 20, 2026Celina Shiyu Wang, Yiqi Zhao, Junjie Ye +2Vision-Language-Action Models Reasoning for VLA Models
Thomas Lord Department of Computer Science, University of Southern California
EndoLIFT: Language-Disambiguated Latent-Conditioned Rectified Flow for Bidirectional Endoscopic Control
Aug 20, 2026Chi Kit Ng, Yidong Zhang, Lui Siu Hing +9Robotic Control Vision-Language-Action Models
The Chinese University of Hong Kong, Hong Kong SAR, China · The Sixth Affiliated Hospital, Sun Yat-sen University, Guangzhou, China
EXIMO: VLM Guided Exploration of VLA Policies
Aug 20, 2026Bhavya Sukhija, Oliver Groth, Mohit Shridhar +5Data-Efficient Robot Learning Vision-Language-Action Models
Google DeepMind
Fine-Tuning VLAs with Self-Demonstrated Generative Control for Multi-Task Manipulation
Aug 19, 2026Prachi Garg, Steve Xing, Prahit Yaugand +2Robot Learning Vision-Language-Action Models
University of Illinois Urbana-Champaign
Role-Conditioned Sub-Token Routing for Efficient Vision-Language-Action Policies
Aug 19, 2026Wei Jiang, Wei WangEfficient VLA Models Multimodal Token Compression
Futurewei Technologies San Jose, CA 95131, USA
LIBERO-VIFO: Benchmarking the Capability and Safety of Visual Cue Following in Vision-Language-Action Models
Aug 18, 2026Zhengyan Qian, Rui Yan, Alex Jinpeng Wang +1Vision-Language-Action Models Safety Evaluation of VLA Models
Geo-VLA: Geometry-Aware Vision-Language-Action Planning via Internalization of Map Semantics
Aug 18, 2026Ran Chen, Jiaxing Ren, Zhikun Zhang +3Vision-Language-Action Models Autonomous Driving Planning
University of Science and Technology Beijing Beijing, China
Reuse Before You Retrieve: Diagnosing Headroom and Complementarity for Test-Time Augmentation of Embodied Multimodal Policies
Aug 18, 2026Yuhwan Jeong, Kuk-Jin YoonTest-Time Adaptation of VLA Models Robot Policy Learning
KAIST, Visual Intelligence Lab.
EATR-Stereo: Embodiment-Aware Token Routing of Paired Stereo Evidence for Humanoid Vision-Language-Action Control
Aug 18, 2026Songwei Wu, Rui Zhao, Fan Yang +7Robotic Control Vision-Language-Action Models
Harbin Institute of Technology · Honor
Inference-Time Attention Steering for Vision-Language-Action Driving Models
Aug 17, 2026Darshan Nagendra Prasad, Lars Ullrich, Knut GraichenVision-Language-Action Models VLMs for Autonomous Driving
FAU Erlangen-Nürnberg, Germany
$τ_0$ -VLA: a Hierarchical Robot Foundation Model with World-Model-Guided Test-Time Computation
Aug 17, 2026Xiaowei Cai, Yunuo Cai, Bingao Chen +36Robot Policy Learning Long-Horizon Robotic Manipulation
Agibot Finch · Shanghai Innovation Institute · The Chinese University of Hong Kong
HAF: Adapting Generalist VLAs to Humanoid Whole-Body Loco-manipulation via Hierarchical Action Flow and Spectral Latent RL
Aug 17, 2026Langzhe Gu, Chengkai Hou, Meng Li +14Humanoid Loco-Manipulation Whole-Body Robotic Manipulation
State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University · Beijing Innovation Center of Humanoid Robotics · Nankai University +1
NebulaVLA: A Dual-Frequency Vision-Language-Action Model With Guide Action for Robotic Manipulation
Aug 17, 2026Cong Zhao, Shuai Tian, Xu Zhang +11Vision-Language-Action Models Efficient VLA Models
ZTE Corporation China
SparkVLA: Stop-Aware Hierarchical VLA with Adaptive Action Chunking for Long-Horizon Manipulation
Aug 17, 2026Xunyao Lei, Renjun Wu, Tianlin Huo +1Vision-Language-Action Models Long-Horizon Robotic Manipulation
Beijing Institute of Technology · Tsinghua University
US-VLA: An Ultrasound Vision-Language-Action Model for Embodied Abdomina
Aug 17, 2026Cheng Zhang, Xingzheng Wu, Guihao Yan +4Robotic Ultrasound Vision-Language-Action Models
Innovation School of Artificial Intelligence, Hefei University of Technology Hefei, China
GigaBrain-0.7: Scaling Embodied Foundation Models to Emergent Capabilities with a Three-System Architecture
Aug 16, 2026GigaBrain Team, Angen Ye, Axiang Sun +56Robot Foundation Models Vision-Language-Action Models
ViTaR: Visuo-Tactile Residual Adaptation for Foundation VLA Manipulation
Aug 16, 2026Yi Wang, Renjun Wu, Jinyan Liu +1Contact-Rich Robotic Manipulation Vision-Language-Action Models
Beijing Institute of Technology
Bit-Flip Attacks on Vision-Language-Action Models: Action-Decoding Architecture Shapes the Vulnerability
Aug 16, 2026Yudong Gao, Linghan Chen, Wenhan Wu +5Adversarial Attacks Adversarial Attacks on VLA Models
The Hong Kong University of Science and Technology · Adelaide University · Wuhan University +3

Aug 25, 2026J/K move · Enter open · S save

TrAct: Bridging Robot Control and Visual Prediction with Visual Tracks

Zhi Cao, Howard Ji, Kevin Zhang, Kuangzhi Ge+3

University of Michigan · Stanford University

Vision-Based Robot Control World Models for Robotics World Model Learning Vision-Language-Action Models

Open paper AskPDF arXiv

Robot actions are inherently embodiment-specific and only weakly aligned with image-space visual changes, limiting their effectiveness as conditioning signals for robot world models. In contrast, visual tracks provide an embodiment-agnostic representation of how task-relevant points move through a scene, offering dense image-space guidance for accurate and spatially precise future video prediction. Building on this observation, we propose TrAct, a world-model-based robot decision-making framework that uses visual tracks as an intermediate interface between control and prediction. TrAct consists of three components: a Vision-Language-Action-and-Track model (VLAT) that jointly predicts candidate actions and corresponding visual tracks from the current observation and language instruction; a track-conditioned world model (TWM) that predicts future visual outcomes conditioned on the proposed tracks; and a vision-language reward model (VLAC) that scores the predicted outcomes. At inference time, VLAT generates candidate action-track pairs, TWM rolls out their visual consequences, and VLAC selects the track whose predicted outcome best satisfies the instruction; the action paired with the selected track is then executed by the robot. Experiments on the proposed LIBERO-INTEGRAL benchmark and real-world Franka manipulation show that TrAct improves success rates from 27% to 55% in simulation and from 49% to 76% on real-world tasks compared with the strong VLA baseline

π_{0.5}

. Furthermore, TWM consistently improves video prediction quality over the action-conditioned world model (AWM). These results demonstrate that visual tracks provide an effective shared interface between robot control and visual prediction, enabling more accurate world modeling and stronger robot generalization.