VLMs for Robotics

VLM: Vision-Language Model

Overview

This field studies how robots can use visual observations and language instructions to understand scenes, plan tasks, and act in the physical world. Common approaches adapt pretrained vision-language models with geometric or 3D grounding, maps and affordances, action heads, skill libraries, or feedback loops; some separate slow semantic planning from fast control. Researchers train and test these systems using robot demonstrations, human videos, simulation, and real-robot tasks, with benchmarks such as LIBERO, RLBench, and Habitat. A central challenge is turning plausible language-based reasoning into reliable, metrically grounded action despite occlusion, changing scenes, latency, and safety constraints.

Latest papers

CardsList

717 papers71 papers in the last four weeks, down 46% on the four weeks before. 0.5% of all new papers.

  1. Affordance-Aware Interactive Decision-Making and Execution for Ambiguous Instructions

    Feb 5, 2026Hengxuan Xu, Fengbo Lan, Zhixin Zhao +5RoboticsRobot Task Planning

  2. TIDAL: Temporally Interleaved Diffusion and Action Loop for High-Frequency VLA Control

    Jan 21, 2026Yuteng Sun, Haoran Wang, Ruofei Bai +4Robotic ControlDiffusion Policy

  3. Visual-Language-Guided Task Planning for Horticultural Robots

    Jan 17, 2026Jose Cuaran, Kendall Koe, Aditya Potnis +2Large Language Model-Based Robot PlanningVision-Language Grounding

  4. Learning to Feel the Future: DreamTacVLA for Contact-Rich Manipulation

    Dec 29, 2025Guo Ye, Zexi Zhang, Xu Zhao +4Contact-Rich Robotic ManipulationWorld Models

  5. Embodied Learning of Reward for Musculoskeletal Control with Vision Language Models

    Dec 28, 2025Saraswati Soedarmadji, Yunyue Wei, Chen Zhang +2Multimodal Reward ModelingLanguage Model-Based Control

  6. StereoVLA: Enhancing Vision-Language-Action Models with Stereo Vision

    Dec 26, 2025Shengliang Deng, Mi Yan, Yixin Zheng +7Visual Spatial ReasoningRobotic Manipulation

  7. Towards Spatial Trace with Reasoning in Vision-Language Models for Robotics

    Dec 15, 2025Enshen Zhou, Yibo Li, Jingkun An +123D Spatial ReasoningVLMs for Robotics

  8. Safe Learning for Contact-Rich Robot Tasks: A Survey from Classical Learning-Based Methods to Safe Foundation Models

    Dec 10, 2025Heng Zhang, Rui Dai, Gokhan Solak +4Safe Robot ManipulationContact-Rich Robotic Manipulation

  9. IndustryNav: Evaluating Spatial Reasoning and Safety of Embodied Agents in Dynamic Industrial Navigation

    Nov 21, 2025Yifan Li, Lichi Li, Anh Dao +15Visual Spatial ReasoningEmbodied Navigation

  10. METIS: Multi-Source Egocentric Training for Integrated Dexterous Vision-Language-Action Model

    Nov 21, 2025Yankai Fu, Ning Chen, Junkai Zhao +5Egocentric Video Learning for RoboticsRobotic Manipulation

  11. SlotVLA: Towards Modeling of Object-Relation Representations in Robotic Manipulation

    Nov 10, 2025Taisei Hanyu, Nhat Chung, Huy Le +10Representation LearningObject-Centric Representation Learning

  12. AERMANI-VLM: Structured Prompting and Reasoning for Aerial Manipulation with Vision Language Models

    Nov 3, 2025Sarthak Mishra, Rishabh Dev Yadav, Avirup Das +3Aerial RoboticsStructured Prompting

  13. UrbanVLA: A Vision-Language-Action Model for Urban Micromobility

    Oct 27, 2025Anqi Li, Zhiyong Wang, Jiazhao Zhang +5Robot NavigationSafe Robot Navigation

  14. Semantic World Models

    Oct 22, 2025Jacob Berg, Chuning Zhu, Yanda Bao +2World Models for RoboticsWorld Model-Based Planning

  15. OmniVIC: A Self-Improving Variable Impedance Controller with Vision-Language In-Context Learning for Safe Robotic Manipulation

    Oct 20, 2025Heng Zhang, Wei-Hsing Huang, Gokhan Solak +1Safe Robot ManipulationContact-Rich Robotic Manipulation

  16. VisCoP: Visual Probing for Video Domain Adaptation of Vision Language Models

    Oct 15, 2025Dominick Reilly, Manish Kumar Govind, Le Xue +1Domain AdaptationVLM Adaptation

  17. Verifier-free Test-Time Sampling for Vision-Language-Action Models

    Oct 7, 2025Suhyeok Jang, Dongyoung Kim, Changyeon Kim +2Test-Time ScalingVision-Language-Action Models

  18. ZeST: A VLM-based Zero-Shot Traversability Navigation for Unknown Environments

    Aug 26, 2025Shreya Gummadi, Mateus V. Gasparino, Gianluca Capezzuto +2Robot NavigationSafe Robot Navigation

  19. Foundation Model Driven Robotics: A Comprehensive Review

    Jul 14, 2025Muhammad Tayyab Khan, Ammar Waheed, Seung Ki MoonRobot Foundation ModelsRobotics

  20. VOTE: Vision-Language-Action Optimization with Trajectory Ensemble Voting

    Jul 7, 2025Juyi Lin, Amir Taherin, Arash Akbari +11Robotic ControlEfficient VLA Inference

  21. AntiGrounding: Executable Robot Trajectories as Visual Prompts for VLM-Guided Manipulation

    Jun 14, 2025Wenbo Li, Yiteng Chen, Wenhao Li +1Trajectory GenerationLanguage-Conditioned Robot Manipulation

  22. Gondola: Grounded Vision Language Planning for Robotic Manipulation

    Jun 12, 2025Shizhe Chen, Ricardo Garcia, Paul Pacaud +1Long-Horizon Robotic ManipulationRobot Task Planning

  23. Training-free Task-oriented Grasp Generation

    Feb 7, 2025Jiaming Wang, Jizhuo Chen, Diwen Liu +1Robotic GraspingVLMs for Robotics

  24. Image-based Geo-localization for Robotics: Are Black-box Vision-Language Models there yet?

    Jan 28, 2025Sania Waheed, Bruno Ferrarini, Michael Milford +2VLM RobustnessCross-View Geo-Localization

  25. Open-Vocabulary Object-Goal Navigation by Generalizing Semantic Mapping with Dense CLIP

    Jul 12, 2024Meng Wei, Chenyang Wan, Tai Wang +6Embodied NavigationVision-Language Navigation

  26. Bridging Language and Action: A Survey of Language-Conditioned Robot Manipulation

    Dec 17, 2023Xiangtong Yao, Hongkuan Zhou, Oier Mees +12Language-Conditioned Robot ManipulationVLMs for Robotics