Vision-Language-Action Models

Also known as VLA

These models combine vision-language backbones with action generators that produce continuous motion or discrete action tokens, often in chunks and with proprioceptive input. They are commonly trained on robot demonstrations, with flow-matching and diffusion policies, reinforcement fine-tuning, and predictive world models used to improve control and planning. Research spans tabletop and bimanual manipulation, mobile and aerial navigation, autonomous driving, and humanoid control, evaluated on benchmarks such as LIBERO, CALVIN, RoboTwin, and driving simulators. A central challenge is making broad visual and language competence translate into reliable closed-loop behavior under new objects, viewpoints, embodiments, delays, and safety constraints; benchmark success can still conceal weak instruction grounding and sim-to-real failures.

Latest papers 1,154

249 papers in the last four weeks, up 144% on the four weeks before. 1.5% of all new papers.

All topics

HiMoE-VLA: Hierarchical Mixture-of-Experts for Generalist Vision-Language-Action Policies
Dec 5, 2025Zhiying Du, Bei Liu, Yaobo Liang +7Visuomotor Policy Learning Vision-Language-Action Models
1Fudan University · 2Microsoft Research Asia · 3Xi’an Jiaotong University +1
Transport Discrepancy as a Reliability Signal for Vision-Language-Action Models
Dec 1, 2025Wanpeng Zhang, Ye Wang, Hao Luo +6Uncertainty Estimation for VLA Models Vision-Language-Action Models
1Peking University · 3BeingBeyond · 2Renmin University of China
XR-1: Towards Versatile Vision-Language-Action Models via Learning Unified Vision-Motion Representations
Nov 4, 2025Shichao Fan, Kun Wu, Zhengping Che +12Representation Learning Robot Skill Learning
Beijing Innovation Center of Humanoid Robotics, Beijing, China · State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University, Beijing, China · State Key Laboratory of Virtual Reality Technology and Systems, SCSE, Beihang University, Beijing, China +1
A Survey on Efficient Vision-Language-Action Models
Oct 27, 2025Zhaoshu Yu, Bo Wang, Pengpeng Zeng +7Model Compression Robotic Data Collection
School of Computer Science and Technology, Tongji University, China · School of Computing and Artificial Intelligence, Southwest Jiaotong University, China · School of Computer Science and Engineering, University of Electronic Science and Technology of China, China +1
DropVLA: An Action-Level Backdoor Attack on Vision-Language-Action Models
Oct 13, 2025Zonghuan Xu, Jiayu Li, Yunhan Zhao +3Adversarial Attacks on VLA Models Robotic Manipulation
Institute of Trustworthy Embodied AI, Fudan University, Shanghai, China. · Shanghai Key Laboratory of Multimodal Embodied AI, Shanghai, China. · City University of Hong Kong, Hong Kong SAR, China.
Reinforcement Fine-Tuning of Flow-Matching Policies for Vision-Language-Action Models
Oct 11, 2025Mingyang Lyu, Yinqian Sun, Erliang Lin +4Flow Matching Robotic Manipulation
Brain-inspired Cognitive AI Lab, Institute of Automation, Chinese Academy of Sciences, Beijing, China. · Beijing Institute of AI Safety and Governance, China. · University of Chinese Academy of Sciences (UCAS), Beijing, China. +3
Don't Run with Scissors: Pruning Breaks VLA Models but They Can Be Recovered
Oct 9, 2025Jason Jabbour, Dong-Ki Kim, Max Smith +6Vision-Language-Action Models Efficient VLA Model Inference
FieldAI · Harvard University
Verifier-free Test-Time Sampling for Vision-Language-Action Models
Oct 7, 2025Suhyeok Jang, Dongyoung Kim, Changyeon Kim +2Test-Time Scaling Vision-Language-Action Models
1KAIST · 3RLWRLD · 2Seoul National University
MM-Nav: Multi-View VLA Model for Robust Visual Navigation via Multi-Expert Learning
Oct 3, 2025Tianyu Xu, Jiawei Chen, Jiazhao Zhang +5Robot Navigation Vision-Language-Action Models
Peking University, Beijing, China · Galbot, Beijing, China · Shanghai Jiao Tong University, Shanghai, China +1
FailSafe: Reasoning and Recovery from Failures in Vision-Language-Action Models
Oct 2, 2025Zijun Lin, Jiafei Duan, Haoquan Fang +4Robot Failure Recovery Synthetic Data Generation for Robotics
1Nanyang Technological University · Centre for Frontier AI Research, A*STAR · 3Allen Institute for AI +1
Ask-to-Clarify: Resolving Instruction Ambiguity through Multi-turn Dialogue
Sep 18, 2025Xingyao Lin, Xinghao Zhu, Tianyi Lu +6Language-Conditioned Robot Manipulation Vision-Language-Action Models
College of Computer Science and Artificial Intelligence, Fudan University, Shanghai, China. · Shanghai Innovation Institute, Shanghai, China. · Mechanical Systems Control Lab, UC Berkeley, California, USA.
VOTE: Vision-Language-Action Optimization with Trajectory Ensemble Voting
Jul 7, 2025Juyi Lin, Amir Taherin, Arash Akbari +11Robotic Control Efficient VLA Model Inference
Northeastern University, Boston, USA · EmbodyX,San Mateo,USA
RoboFAC: A Comprehensive Framework for Robotic Failure Analysis and Correction
May 18, 2025Zewei Ye, Weifeng Lu, Minghao Ye +4Robot Failure Recovery Robustness of VLA Models
School of AI, Shanghai Jiao Tong University · Harbin Institute of Technology, Shenzhen
3D-MoE: Towards Spatial Intelligence with Mixture-of-Experts for 3D Reasoning and Action Generation
Jan 28, 2025Yueen Ma, Zenglin Xu, Irwin King3D Spatial Reasoning Mixture-of-Experts Inference
Department of Computer Science and Engineering, The Chinese University of Hong Kong, Hong Kong SAR

Dec 5, 2025J/K move · Enter open · S save

HiMoE-VLA: Hierarchical Mixture-of-Experts for Generalist Vision-Language-Action Policies

Zhiying Du, Bei Liu, Yaobo Liang, Yichao Shen+6

1Fudan University · 2Microsoft Research Asia · 3Xi’an Jiaotong University+1

Visuomotor Policy Learning Vision-Language-Action Models Mixture-of-Experts Models Negative Transfer

Open paper AskPDF arXiv

Generalist vision--language--action (VLA) policies are typically trained on heterogeneous mixtures of robot demonstrations spanning diverse embodiments, action spaces, and observation configurations. Modeling such heterogeneity with a shared dense action module can induce negative transfer, particularly when action spaces or visual observations differ across data sources. We address this issue with HiMoE-VLA, a VLA framework built around a Hierarchical Mixture-of-Experts (HiMoE) action module. HiMoE uses Action-Space MoE layers at the input/output boundaries to specialize computation for distinct action spaces, Heterogeneity-Balancing MoE layers in neighboring layers to provide balanced capacity for residual variation in observations, scenes, and embodiments, and dense Transformer blocks in the middle to integrate shared representations. Two auxiliary objectives further guide this hierarchy: a contrastive Action-Space Regularization objective for boundary specialization and a load-balancing objective for stable expert utilization. HiMoE-VLA reaches 3.98 on CALVIN, 98.0% on LIBERO, and 75.0% and 63.7% average success on real xArm7 and ALOHA tasks; under controlled heterogeneous co-training, it turns the negative transfer observed in strong baselines into positive transfer. The code and models are publicly available at https://github.com/ZhiyingDu/HiMoE-VLA.