VLMs for Robotics

VLM: Vision-Language Model

Follow this topic Ask about this topic

Momentum

92 papers in the last four weeks, up 608% on the four weeks before. 0.9% of all new papers.

Jul 13Week of Sep 28

Latest papers 392

All topics

Auditing Instruction-Trajectory Mismatches in Multimodal Robot Demonstrations
Aug 8, 2026Simon Holk, Ryosuke Takanami, Tatsuya Matsushima +4Robot Imitation Learning VLMs for Robotics
AI Robot Association (AIRoA), Tokyo, Japan
Capek 0.5: An Execution-Centric Vision-Language Model for Embodied Intelligence
Aug 7, 2026Ying Chen, Weizhen Li, Zhe Hu +7VLMs for Robotics Reasoning for VLA Models
XPENG ROBOTICS
AtlasVLA: Persistent World-Ego State Modeling for Vision-Language-Action Models
Aug 7, 2026Guiyu Zhao, Longteng Guo, Yanghong Mei +7Long-Horizon Robotic Manipulation Vision-Language-Action Models
Institute of Automation, Chinese Academy of Sciences, Beijing, China · University of Chinese Academy of Sciences, Beijing, China · Beijing Freedo Technology Co., Ltd. +1
Beyond Flat Policies: Hierarchical Post-Training for Embodied Agents in Robotic Manipulation
Aug 6, 2026He Kong, Zengjue Chen, Qi Wang +6Long-Horizon Robotic Manipulation Hierarchical RL
School of Artificial Intelligence, Jilin University · College of computer science and technology, Jilin University · Joy Future Academy, JD
In-Context VLA: Endowing Vision-Language-Action Models with Language via In-Context Post-Training and Agentic Tool Use
Aug 6, 2026Jiarui Yang, Wen Huang, Jiale Zhang +2ICL for Robotics Continual Learning for VLA Models
Nankai University · Tsinghua University · Swiss Federal Institute of Technology in Lausanne (EPFL)
Hijacking Robots with a Piece of Paper: A Systematic Study of Physical Prompt Injection in VLM-Controlled Robots
Aug 6, 2026S. M . Bhagya P. Samarakoon, M. A. Viraj J. Muthugala, W. K. R. Sachinthana +1VLM Robustness Adversarial Attacks on VLMs
Engineering Product Development Pillar · Singapore University of Technology and Design · Somapah Rd, Singapore 487372
Fast and Accurate: An Adaptive VLA Inference Framework through Environment-aware Model Selection
Aug 6, 2026Yuewei Sun, Lang Qin, Zechuan Tian +11Efficient VLA Model Inference VLMs for Robotics
ACSLab, Huawei Technologies · Southern University of Science and Technology
Explicit Language Memory for Long-Horizon Planning in Vision-Language-Action Models
Aug 5, 2026Houze Xu, Jizhong Li, Ziyi YeMemory-Augmented VLMs Vision-Language-Action Models
Fudan University
Suppression Sticks, Locality Is Fragile: A Closed-Loop Target-and-Control Audit of Task-Vector Negation in VLA Policies
Aug 5, 2026Shaoguang Wang, Weiyu Guo, Rushi Dai +3Robotic Policy Evaluation Task Arithmetic
The Hong Kong University of Science and Technology (Guangzhou) · AI2 Robotics · The Hong Kong University of Science and Technology
VLAff: Vision-Language-Affordance Model for Unified Actionable Affordances
Aug 5, 2026Jihoon Oh, Kento Kawaharazuka, Kei OkadaRobot Skill Learning Language-Conditioned Robot Manipulation
University of Tokyo, Japan
Track4Action: Distilling World-Centric 3D Tracker into Vision-Language-Action Policies
Aug 4, 2026Chenyi Wang, Xinkai Wang, Bokai Lin +4Visuomotor Policy Learning Vision-Language-Action Models
1Zhejiang University, China · 3Shanghai Innovation Institute, China · 2Shanghai Jiao Tong University, China +1
From Routes to Steps: Separating Semantic Progress from Local Execution in Vision-and-Language Navigation
Aug 4, 2026Xiangyun Huang, Xiangchen Wang, Runfeng Lin +5Vision-Language Navigation VLMs for Robotics
1Beihang University · 2Southern University of Science and Technology · 3Central South University +2
How Should Vision-Language-Action Models Use Proprioceptive State?
Aug 4, 2026Yiren Zhao, Ziyang Chen, Ziyang Rao +5Robotic Control Vision-Language-Action Models
1The Hong Kong University of Science and Technology (Guangzhou) · 3AI2 Robotics X-Lab · 2Multimedia Laboratory (MMLab), The Chinese University of Hong Kong
Grounded Semantic Re-Binding for Robust Instruction Generalization in Vision-Language-Action Models
Aug 3, 2026Zhaokai Yin, Zhipeng ZhangVision-Language Grounding Vision-Language-Action Models
1AutoLab, School of Artificial Intelligence, Shanghai Jiao Tong University · 2Research Lab, Anyverse Dynamics
ReTouch: Empowering Contact-Rich Dexterous Manipulation with Online-Refined Tactile Prediction
Aug 3, 2026Shiqi Zhang, Xin Zhang, Yedong Shen +9Contact-Rich Robotic Manipulation Robotic Manipulation
University of Science and Technology of China · The Chinese University of Hong Kong · iFLYTEK
Uncovering and Mitigating Positional Blind Spots in Vision-Language-Action Models
Aug 3, 2026Dongdong An, Pengjie Zhao, Yihao Huang +5Vision-Language-Action Models VLMs for Robotics
Shanghai Normal University Shanghai, China · East China Normal University Shanghai, China · Northwest A&F University Yangling, Shaanxi, China +1
ORCESTRA: VLM-driven Visual Robot programming in Mixed Reality
Aug 1, 2026Ivan Snegirev, Elizaveta Semenyakina, Mikhail Konenkov +3Digital Twins Language-Conditioned Robot Control
Intelligent Space Robotics Lab, Skolkovo Institute of Science and Technology, Moscow, Russian Federation · R&D Center, MWS, Moscow, Russian Federation
Latency-Tolerant Cloud-Edge Collaborative Vision-Language-Action Models via Emergent Representational Specialization
Aug 1, 2026Daojie Peng, Fulong Ma, Bingtao Wang +2Robotic Control Edge Computing
HKUST(GZ) · Shandong University · HKUST
SpatialAfford: Teaching Compact VLMs Where to Look and Where to Ground for Affordance
Aug 1, 2026Yufei Zhang, Chenlu Zhan, Donghui Sun +2Vision-Language Grounding VLMs for Robotics
Zhejiang University · Vivo Mobile Communication Co., Ltd
FibVLA: An Efficient Temporal Vision-Language-Action Model with Fibonacci Sampling
Jul 31, 2026Li Lin, Wujun Xu, Weiwei Meng +3Efficient VLA Model Inference Efficient VLA Models
Southeast University, Nanjing, Jiangsu, China · Nanyang Technological University, Singapore.
SAGP: Semantic Affordance-Guided Grasp Planning via Coarse-Zone VLM Reasoning
Jul 31, 2026Muhayy Ud Din, Irfan HussainRobotic Grasping VLMs for Robotics
KU Center for Autonomous Robotic Systems (KUCARS), Khalifa University, United Arab Emirates
D-VLC: Decentralized Vision-Language Collaboration for Heterogeneous Embodied Multi-Robot Systems in Unknown Environments
Jul 31, 2026Yuan Zhou, Ruitong Lin, Shen Wang +5Multi-Agent LLM Systems Multi-Robot Systems
Zhejiang University, Hangzhou, China · Differential Robotics, Hangzhou, China · Beihang University, Beijing, China
TEA-AgriVLN: Traversability Estimation Alarm for Agricultural Vision-and-Language Navigation
Jul 30, 2026Xiaobei Zhao, Xingqi Lyu, Xin Chen +1Embodied Navigation Vision-Language Navigation
China Agricultural University · China Agricultural University-Sichuan Advanced Agricultural & Industrial Institute
RedFlow: Redirect Failure into Action-level Corrections for Flow-matching VLA Policy
Jul 30, 2026Zhengyang Yan, Junhao Li, Fangqi Zhu +6Robot Policy Learning Policy Optimization
The Hong Kong University of Science and Technology
A Systems Engineering Framework for Vision-Language-Enabled UAV Triage and Disaster Response
Jul 30, 2026Swapnil Saha, Bhuvan Rajanasiriyur Jagadeesha, Karishma Patnaik +1Aerial Robotics Language Model-Based Control
University of Arkansas, Fayetteville, Arkansas, 72701, USA · University of Michigan-Dearborn, Dearborn, Michigan, 48128, USA
HumanCLAW: Can Vision-Language Models Act Through a Body?
Jul 29, 2026Li Siyao, Jiawei Gu, Shuai Liu +15VLM Evaluation Embodied AI
Meta · University of Washington · Nanyang Technological University +2
Route by Kinematics, Act by Observation: Kinematics-Supervised Expert Routing in MoE-Augmented VLA
Jul 29, 2026Tianhang Yang, Yanze Zheng, Junjie Wang +3Robot Policy Learning Expert Routing
Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen, China · Pengcheng Lab, Shenzhen, China
Speech2Grasp: Data-Efficient Transfer of Text-Conditioned Grasp Detection to Speech in Humanoid Robots
Jul 29, 2026Hung Nguyen, Kim Nhat Minh Nguyen, Van Duc Vu +6Robotic Grasping Language-Conditioned Robot Manipulation
University of California San Diego · VinMotion · Hanoi University of Science and Technology +2
A Causality-aware Infer-diagnose-refine Framework for Test-time Modality Adaptation in VLA Models
Jul 28, 2026Haoyu Zhang, Yuwei Wu, Jin Chen +6Robotic Manipulation Vision-Language-Action Models
1Beijing Institute of Technology · 2AInnovation Co. Ltd. · 3Beijing Innovation Center of Humanoid Robotics
Room-Mediated Co-occurrence for Zero-Shot Object-Centric Semantic Navigation via Frontier Scoring
Jul 28, 2026Adam Scicluna, Gavin Paul, Alen AlempijevicVLMs for Robotics Object-Goal Navigation
Robotics Institute, Faculty of Engineering and Information Technology, University of Technology Sydney (UTS), Sydney, Australia

Aug 8, 2026, cs.ROJ/K move · Enter open · S save

Auditing Instruction-Trajectory Mismatches in Multimodal Robot Demonstrations

Simon Holk, Ryosuke Takanami, Tatsuya Matsushima, Yusuke Iwasawa+3

AI Robot Association (AIRoA), Tokyo, Japan

Robot Imitation Learning VLMs for Robotics

Open paper AskPDF arXiv

Robot demonstration datasets used to train vision-language-action policies can contain a subtle but harmful failure mode: trajectories that are behaviorally correct but paired with the wrong language instruction. We study post-hoc auditing of these Instruction-Trajectory Mismatches (ITMs). Unlike failed rollouts, ITMs often look plausible, and can corrupt the language-behavior mapping learned by the policy. We propose Multimodal Probabilistic Fusion (MMPF), a training-free auditing framework that treats each modality as an expert, estimates a task-label distribution from local neighborhood agreement and global prototype similarity, and then fuses modalities with predictive-entropy weighting in a product of experts. Across LIBERO benchmarks with injected instruction mismatches and noisy real-robot data, MMPF achieves the strongest overall ITM detection and label correction accuracy. We also show that auditing improves most downstream policy learning in settings where language is needed to disambiguate the task. We demonstrate in real robot experiments that our method can achieve improved policy performance and show the trade-off of filtering demonstrations compared to relabeling.

Momentum

Related topics

Latest papers 392