VLMs for Robotics

VLM: Vision-Language Model

Follow this topic Ask about this topic

Momentum

92 papers in the last four weeks, up 608% on the four weeks before. 0.9% of all new papers.

Jul 13Week of Sep 28

Latest papers 392

All topics

UniLM-Nav: A Unified Framework for Zero-Shot Last-Mile Navigation
Jul 7, 2026Zhuofan Zhang, Tianxu Wang, Guoxi Zhang +6Mobile Manipulation Robot Navigation
1Tsinghua University · 2State Key Laboratory of General Artificial Intelligence, BIGAI · 3Harbin Institute of Technology +1
Training-Free Acceleration for Vision-Language-Action Models with Action Caching and Refinement
Jul 7, 2026Ryuji Oi, Hikari Otsuka, Kosuke Matsushima +4Efficient VLA Model Inference Vision-Language-Action Models
Institute of Science Tokyo
Optimal Transport Q-Learning for Flow Policy Steering and Acceleration
Jul 7, 2026Andreas Sochopoulos, Esmeralda S. Whitammer, Nikolaos Tsagkas +3Robot Policy Adaptation Q-Learning
University of Edinburgh · 2Honda Research Institute Europe
RoboTALES: Learning Reasoning-Guided Robot Policies via Task-Aligned Simulated Futures
Jul 7, 2026Hanan Gani, Tejal Kulkarni, Madhoolika Chodavarapu +2Robot Policy Learning Robot Skill Learning
University of California, San Diego, CA 92093, USA
Cortex: A Bidirectionally Aligned Embodied Agent Framework for Long-horizon Manipulation
Jul 6, 2026Jiaqi Peng, Xiqian Yu, Delin Feng +10Long-Horizon Robotic Manipulation Language-Conditioned Robot Manipulation
Tsinghua University · Shanghai AI Laboratory · Peking University +1
RoboVista: Evaluating Vision Language Models for Diverse Robot Applications
Jul 6, 2026Shuangyu Xie, Kaiyuan Chen, Ziyang Chen +8VLM Evaluation Embodied QA
University of California, Berkeley · Princeton University · Google DeepMind
Look Before You Leap: Distilling Tree Search into Action Evaluation for Frozen VLA Models
Jul 4, 2026Xinyi Xie, Zican Hu, Zhanyu Liu +7Value Function Estimation Test-Time Scaling
Nanjing University · Australian National University · Institute of Automation, Chinese Academy of Sciences +1
Seek to Segment: Active Perception for Panoramic Referring Segmentation
Jul 2, 2026Song Tang, Shuming Hu, Xincheng Shuai +2Active Perception Referring Image Segmentation
Fudan University, China
Learning to Move Before Learning to Do: Task-Agnostic pretraining for VLAs
Jul 2, 2026Junhao Shi, Siyin Wang, Xiaopeng Yu +3Self-Supervised Pre-Training Efficient VLA Models
1Fudan University · 2Shanghai Innovation Institute
LIME: Learning Intent-aware Camera Motion from Egocentric Video
Jul 2, 2026Boyang Sun, Jiajie Li, Yung-Hsu Yang +6Active Perception VLMs for Robotics
ETH Zurich · Microsoft · University of Bonn
VLAFlow: A Unified Training Framework for Vision-Language-Action Models via Co-training and Future Latent Alignment
Jul 2, 2026Guoyang Xia, Fengfa Li, Hongjin Ji +4Vision-Language-Action Models VLMs for Robotics
Li Auto Inc. · School of Artificial Intelligence, Beijing University of Posts and Telecommunications · The Chinese University of Hong Kong, Shenzhen
Overthink-Triggered Slowdown Attacks on LVLM-Based Robotic Systems
Jul 1, 2026Qiang Han, Jie Wu, Bo ChenAdversarial Attacks on VLMs Overthinking in Language Models
Department of Computer Science, Michigan Technological University
Where Am I? Semantic Map Grounding via Vision-Language Models for Multi-Modal Localization
Jul 1, 2026Suraj Borate, Aarav Shah, Madhu VadaliMap-Based Localization Robot Localization
IIT Gandhinagar, Gandhinagar, Gujarat, India
Adapting Generalist Robot Policies with Semantic Reinforcement Learning
Jun 30, 2026Jagdeep Singh Bhatia, Andrew Wagenmaker, William Chen +1Robot Policy Learning Robot Policy Adaptation
U.C. Berkeley
3D HAMSTER: Bridging Planning and Control in Hierarchical Vision Language Action Models through 3D Trajectory Guidance
Jun 30, 2026Dongyoon Hwang, Byungkun Lee, Dongjin Kim +7Robotic Control Vision-Language-Action Models
Kim Jaechul Graduate School of AI, KAIST, Seoul, Republic of Korea · Graduate School of Artificial Intelligence, POSTECH, Pohang, Republic of Korea · Holiday Robotics, Seoul, Republic of Korea +1
Agentic RAG-VLM: Affordance-Aware Retrieval-Augmented Generation with Self-Reflective Planning for Robotic Grasping
Jun 30, 2026Tao Chen, Lizheng Liu, Jiaxu Wang +4Robotic Grasping Long-Horizon Robotic Manipulation
Fudan University, Shanghai, China · Kean University
A Modular Vision-Language-Action Robotics Framework for Indoor Environments
Jun 30, 2026Anindya Jana, Snehasis Banerjee, Arup Sadhu +1Vision-Language Navigation VLMs for Robotics
Physical AI Research TCS Research, Tata Consultancy Services Kolkata, India
Vision-Language Procedural Reasoning for Context-Aware Reward Modeling of Robotic Endovascular Guidewire Navigation
Jun 29, 2026Wentong Tian, Jiyuan Zhao, Tianliang Yao +4RL for Robotics Robotic Endovascular Intervention
Department of Control Science and Engineering, College of Electronic and Information Engineering, and Shanghai Institute of Intelligent Science and Technology, Tongji University, Shanghai 200092, China. · Department of Electronic Engineering, Faculty of Engineering, The Chinese University of Hong Kong, Hong Kong SAR 999077, China. · Shanghai Operation Robot Co., Ltd., Shanghai 201318, China.
ViTL: Temporal Logic-Guided Zero-Shot Natural Language Navigation via Vision-Language Models
Jun 29, 2026Kaier Liang, Hengde Dai, Cristian-Ioan VasileLarge Language Model-Based Robot Planning Vision-Language Navigation
Lehigh University, PA, USA
Fast Enough to Act: Spatio-Temporal Visual Token Merging for Low-Latency Robotic VLMs and VLAs
Jun 28, 2026Junzhou Chen, Jindong Wang, Gang ZhouEfficient VLM Inference Vision-Language-Action Models
Department of Computer Science, William & Mary, Williamsburg, VA 23188, USA · Department of Data Science, William & Mary, Williamsburg, VA 23188, USA
Vision-Language Models for Deployable Social Robot Navigation: Bridging Semantic Reasoning and Low-Level Control
Jun 27, 2026Runji Cai, Toshihiko Yamasaki, Ling XiaoSafe Robot Navigation Social Robot Navigation
Graduate School of Information Science and Technology, Hokkaido University, Sapporo, 060-0814, Japan. · Graduate School of Information Science and Technology, The University of Tokyo, Tokyo, 113-8656, Japan.
Translation as a Bridging Action: Transferring Manipulation Skills from Humans to Robots
Jun 26, 2026Sijin Chen, Kaixuan Jiang, Haixin Shi +6Bimanual Robotic Manipulation Cross-Embodiment Robot Learning
1HKU-MMLab · 2ByteDance Seed
LocalNav: Distilling Frontier VLMs and Embodied RL for On-Device Object Goal Navigation
Jun 26, 2026Nicolas Baumann, Liam Boyle, Pu Deng +5Efficient VLM Inference VLM Distillation
Center for Project-Based Learning, 2Integrated Systems Laboratory, 3Computer Vision and Geometry ETH Zurich
SpikeVLA: Vision-Language-Action Models with Spiking Neural Networks
Jun 26, 2026Ruiqi Song, Dujun Nie, Siyu Teng +7Spiking Neural Networks Vision-Language-Action Models
College of Surveying and Geo-informatics, Tongji University, Shanghai, China · State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, China · OpenSpace Lab, China +6
Direct Action-Head Injection of A Grounded 3D Point Unlocks Spatial and Task Generalization
Jun 26, 2026Shiang-Feng Tsai, Jin-Cheng Jhang, Yen-Ling Tai +5Generalization in Robotic Manipulation Robotic Manipulation
National Tsing Hua University · National Yang Ming Chiao Tung University
E-TTS: A New Embodied Test-Time Scaling Framework for Robotic Manipulation
Jun 25, 2026Wen Ye, Peiyan Li, Tingyu Yuan +7Long-Horizon Robotic Manipulation Test-Time Scaling
New Laboratory of Pattern Recognition (NLPR), Institute of Automation, Chinese Academy of Sciences, Beijing, China · School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China · Foundation Model Research Center, Institute of Automation, Chinese Academy of Sciences, Beijing, China +1
SSI-Policy: Learning Structured Scene Interfaces for Vision-Language Robotic Manipulation
Jun 25, 2026Kaijun Wang, Zikai Ouyang, Xuping Wu +6Robot Skill Learning Language-Conditioned Robot Manipulation
Southern University of Science and Technology, Shenzhen 518055, China · Peng Cheng Laboratory, Shenzhen 518000, China · The University of Hong Kong, Hong Kong SAR, China +1
Event-Adaptive Motion Planning with Distilled Vision-Language Model in Safety-Critical Situations
Jun 24, 2026Zhenwei Huang, Changsheng You, Shuai Wang +3Safe Robot Navigation Model Predictive Control
Southern University of Science and Technology · Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences · Manifold Tech Limited
Toward Low-Latency Vision-Language Models with Doubly-Correct Predictions in Egocentric Visual Understanding
Jun 23, 2026Qitong Wang, Fan Du, Pranav Maneriker +2Efficient VLM Inference Egocentric Vision
Dolby Laboratories, Inc., 1275 Market St, San Francisco, California, 94103, USA · Department of Computer and Information Sciences at the University of Delaware, Newark, DE 19711, USA · Dolby Laboratories, Inc.
Flatness Preserves Instruction Following in Vision-Language-Action Models
Jun 22, 2026Haochen Zhang, Yonatan BiskSharpness-Aware Minimization Vision-Language-Action Models
Carnegie Mellon University

Jul 7, 2026, cs.ROJ/K move · Enter open · S save

UniLM-Nav: A Unified Framework for Zero-Shot Last-Mile Navigation

Zhuofan Zhang, Tianxu Wang, Guoxi Zhang, Yixiong Lin+5

1Tsinghua University · 2State Key Laboratory of General Artificial Intelligence, BIGAI · 3Harbin Institute of Technology+1

Mobile Manipulation Robot Navigation VLMs for Robotics

Open paper AskPDF arXiv

Mobile manipulation requires a robot to navigate to a target object or receptacle and then perform intended manipulation. However, reaching the vicinity of the target does not guarantee a manipulation-ready base pose, a problem known as last-mile navigation. Prior methods for last-mile navigation either rely on manual pose annotation or task-specific training, limiting their scalability to open-vocabulary settings with fine-grained spatial constraints. We propose UniLM-Nav, a unified framework for zero-shot open-vocabulary last-mile navigation. UniLM-Nav decomposes last-mile navigation into view selection, task-conditioned affordance grounding, and geometry-aware base-pose reasoning, all resolved with a shared multimodal large language model (MLLM) backend. Specifically, UniLM-Nav first selects a reference view that best captures the target object or receptacle from recently collected observations. It then grounds task-relevant affordance point in the selected view and lifts the result into the robot-centric coordinate frame. Finally, conditioned on the grounded affordance, task context, and robot geometry, it infers a manipulation-ready base pose for the robot. We evaluate UniLM-Nav on the OVMM benchmark, where it outperforms the previous state-of-the-art method, MoTo, by 3.13 percentage points. Analyses show that the components of our method are crucial to final performance, and that the choice of MLLM also has a substantial effect. We further deploy UniLM-Nav on a Unitree B2 quadruped robot with a 6-DoF Unitree Z1 manipulator, validating its applicability to real-world mobile manipulation tasks.

Momentum

Related topics

Latest papers 392