Visual Grounding for Robotics

Follow this topic Ask about this topic

Latest papers

31 papers7 papers in the last four weeks, up 40% on the four weeks before. 0.0% of all new papers.

AffordJEV: Affordance Grounding by Confirming Candidate Regions
Nov 5, 2026Jincheng Li, Yutian Zhang, Jianyu Zhang +2Robotic Affordance Inference Visual Grounding for Robotics
Peking University
STARE: Scene Text-Aware multimodal Retrieval for Everyday objects
Nov 4, 2026Kento Tokura, Ryosuke Korekata, Daichi Yashima +4Multimodal IR Robotic Object Search
Keio University, Japan
Surrogate Uncertainty for Black-box Visual Grounding
Oct 29, 2026Hanbo Zhang, Tianrun Hu, Cedar Site Bai +2Uncertainty Quantification Visual Grounding
National University of Singapore · Shanghai Innovation Institute · Purdue University
EBV-CAN: Vision--Language Grounding for Contact-Aided Endoscope Navigation
Oct 22, 2026Chi Kit Ng, Shuchang Liu, Jinsong Lin +4Robotic Endoscopy Endoscopy
The Chinese University of Hong Kong. · National University of Singapore.
OMAR: Open-World Multi-Attribute Reasoning for Scene-Conditioned Shelf Organization
Oct 21, 2026Benedikt Kreis, Rohit Menon, Benno Wingender +3Robotic Manipulation Multimodal Spatial Reasoning
Humanoid Robots Lab, University of Bonn, Germany. · Center for Robotics, University of Bonn, Germany. · Lamarr Institute for ML and AI, Bonn, Germany. +1
ACE-Brain-0.6: Scaling Spatial Grounding for Physical Foundation Models
Oct 19, 2026Yinhe Wang, Tao Tao, Zexin He +11Multimodal Grounding Visual Grounding for Robotics
StopWatch: Goal Grounding for Object-goal Navigation
Oct 16, 2026Xinhao Liu, Yunsheng Tian, Yuheng Qiu +4Object-Goal Navigation Visual Grounding for Robotics
Amazon FAR · New York University · UC Berkeley +1
Spa-Bench: A Real-Robot Benchmark for Spatial Grounding in Vision-Language-Action Models
Oct 8, 2026Justin Tien-Smith, Matthew Pidden, Rolandos Alexandros PotamiasSpatial Grounding for VLA Models Visual Grounding for Robotics
Imperial College London
Instance-anchored interaction evidence: Grounding robot plans in human pointing and handling
Oct 8, 2026Xinliang Xiao, Bowen Yang, Wenjing Zhang +2Visual Grounding for Robotics Robot Task Planning
Nanjing University of Science and Technology, Nanjing, 210094, China
GroundingPI: A Grounding Foundation Model towards Physical Intelligence with Visual Primitives
Sep 30, 2026Qize Yu, Lianrui Fan, Boyu Chen +23Vision-Language Grounding VLMs for Robotics
XPeng Inc. · Peking University · The University of Hong Kong +5
Pro-Bench: Prompt-Robust Open-Vocabulary Visual Grounding Across Real-World Heterogeneous Environments
Sep 22, 2026Linus Nwankwo, Muslim Alaran, Christian Rauch +2Vision-Language Grounding Visual Grounding
Chair of Cyber-Physical Systems, Montanuniversität Leoben, Austria. · Department of Design Engineering and Mathematics, Middlesex University, London, UK.
MIGU: Multimodal Instruction Grounding under Uncertainty for Manipulation Planning
Sep 21, 2026Mingke Lu, Anxing Xiao, David HsuMultimodal Grounding Robot Task Planning
School of Computing and Smart Systems Institute, National University of Singapore, Singapore · Department of Electrical and Computer Engineering, University of California, Los Angeles, Los Angeles, CA, USA
Grounded Action Model: 3D Grounding as a Foundation for Robotics
Sep 20, 2026Gehao Zhang, Weikai Huang, Shailesh Shailesh +3Long-Horizon Robotic Manipulation Language-Conditioned Robot Manipulation
Northwestern University · University of Washington · National University of Singapore +1
AR-WAM: A Visual-Conditioned Agent-Ready World Action Model for Robotic Manipulation
Sep 20, 2026Yicheng Jiang, Zesen Gan, Xiaobo Wang +8Robotic Control Long-Horizon Robotic Manipulation
The Hong Kong University of Science and Technology · Shenzhen University of Advanced Technology · Sangfor Technologies Inc. +4
RefGuard: Identity-Aware Language-Guided Robot Manipulation via Joint Target-Anchor-Frame Grounding
Sep 5, 2026Lan Wei, Kangyi Lu, Yongchen Wang +5Language-Conditioned Robot Manipulation Vision-Language Grounding
Imperial College London, UK · RACE, United Kingdom Atomic Energy Authority, UK · Oxford Robotics Institute, University of Oxford, UK +1
GRAFT: Grounded and Efficient Online Reinforcement Adaptation for Fine-Grained Robot Manipulation
Aug 27, 2026Yibo Qiu, Haoliang Ye, Shu'ang Sun +3Visual Grounding for Robotics RL for Robotics
Suzhou Institute for Advanced Research, University of Science and Technology of China, Suzhou, Jiangsu, China, and also with the School of Biomedical Engineering, Division of Life Sciences and Medicine, University of Science and Technology of China, Hefei, Anhui, China
Pointing-VLA: Typed Spatial Grounding Interfaces for Vision-Language-Action Manipulation
Aug 24, 2026Xiwen Chen, Zelin Li, Zhiruo Zhou +3Spatial Grounding for VLA Models Vision-Language-Action Models
SIGS, Tsinghua University, China · Shanghai Jiao Tong University, China · Wuhan University of Technology, China +2
Artificial Foveated Perception for Mitigating Shortcut Learning in Robotic Foundation Models
Jul 12, 2026Xiatao Sun, Yuan Zhuang, Mateo Sanchez Lopez Negrete +9Robot Foundation Models Visual Grounding
Yale University · University of Connecticut · Peking University +2
GeoProp: Grounding Robot State in Vision for Generalist Manipulation
Jul 8, 2026Guoyang Zhao, Quanhao Qian, Gongjie Zhang +5Generalization in Robotic Manipulation Visuomotor Policy Learning
Tongji University · DAMO Academy, Alibaba Group · HuPan Lab +2
Where Am I? Semantic Map Grounding via Vision-Language Models for Multi-Modal Localization
Jul 1, 2026Suraj Borate, Aarav Shah, Madhu VadaliMap-Based Localization Robot Localization
IIT Gandhinagar, Gandhinagar, Gujarat, India
GROW $^2$ : Grounding Which and Where for Robot Tool Use
Jun 29, 2026Yuhong Deng, Yuyao Liu, David Hsu3D Visual Grounding Robotic Tool Use
National University of Singapore · Massachusetts Institute of Technology
SPARC: Reliable Spatial Annotations from Robot Demonstrations at Scale
Jun 11, 2026Nils Blank, Paul Mattes, Maximilian Xiling Li +53D Visual Grounding Visual Grounding for Robotics
Karlsruhe Institute of Technology · NVIDIA · Robotics Institute Germany
RGB-S: Image-Aligned Tactile Saliency for Robust Dexterous Manipulation
Jun 7, 2026Shengcheng Luo, Kefei Wu, Xiaoying Zhou +3Contact-Rich Robotic Manipulation Visuo-Tactile Representation Learning
ShanghaiTech University · Beijing Institute for General Artificial Intelligence
EmbodimentSemantic: A Spatial Scene-Graph Dataset and Benchmark for Vision-Language Models on Embodied Manipulation Trajectories
Jun 6, 2026Hassan Jaber, Refinath S N, Luca Cagliero +2Visually Grounded Reasoning Visual Spatial Reasoning
Politecnico di Torino, Italy · Huawei, Noah’s Ark Lab, United Kingdom · University College London, United Kingdom
Goal2Pixel: Grounding Goals to Pixels for Vision-Language Navigation
Jun 1, 2026Muyi Bao, Yuxin Cai, Hang Xu +7Embodied Navigation VLM Adaptation
Carnegie Mellon University · Nanyang Technological University
RobotEQ: Transitioning from Passive Intelligence to Active Intelligence in Embodied AI
May 7, 2026Kuofei Fang, Xinyi Che, Haomin Ouyang +12Human-Robot Interaction Embodied AI
State Key Laboratory of Autonomous Intelligent Unmanned Systems, Tongji University · Tsinghua University · The Chinese University of Hong Kong +1
HiVLA: A Visual-Grounded-Centric Hierarchical Embodied Manipulation System
Apr 15, 2026Tianshuo Yang, Guanyu Chen, Yutian Chen +8Hierarchical Robot Control Visual Grounding for Robotics
The University of Hong Kong · Shanghai AI Laboratory · Shanghai Jiao Tong University +1
Meanings and Measurements: Multi-Agent Probabilistic Grounding for Vision-Language Navigation
Mar 19, 2026Swagat Padhan, Lakshya Jain, Bhavya Minesh Shah +3Visual Grounding for Robotics Spatial Grounding for VLA Models
Arizona State University · Haverford College
SVLL: Staged Vision-Language Learning for Physically Grounded Embodied Task Planning
Mar 12, 2026Yuyuan Yang, Junkun Hong, Hongrong Wang +11Visual Grounding for Robotics Large Language Model-Based Robot Planning
FNii-Shenzhen, CUHKSZ · Shenzhen University · SAI, CUHKSZ +4
OpenFrontier: General Navigation with Visual-Language Grounded Frontiers
Mar 5, 2026Esteban Padilla-Cerdio, Boyang Sun, Marc Pollefeys +1Robot Navigation Vision-Language Navigation
1ETH Zurich · 2Microsoft Spatial AI Lab · University of Bonn

Nov 5, 2026J/K move · Enter open · S save

AffordJEV: Affordance Grounding by Confirming Candidate Regions

Jincheng Li, Yutian Zhang, Jianyu Zhang, Xinshun Wang+1

Peking University

Robotic Affordance Inference Visual Grounding for Robotics

Open paper AskPDF arXiv

Affordance grounding asks which visible part of an object performs a requested operation, a basic capability for embodied manipulation and human--robot interaction. Most existing methods follow a System Two paradigm: a multimodal language model first writes a chain of thought and then turns its conclusion into a box or points for a segmenter. This incurs considerable inference cost, gives the semantic decision little direct control over the extent of the final region, and makes mistakes hard to correct. Yet locating an affordance is essentially a perceptual judgment that need not be argued out in words. We therefore propose, in which frozen segmenters propose candidate regions and a System One model decides directly which of them perform the operation, generating no text and reading each decision from a single forward pass. Our contributions are twofold. (1) An inference framework that confirms candidate regions level by level, from object to part to sub-part, with evidence passed between the levels in both directions so that the levels correct one another. (2) A System One model trained for this framework: existing System One models are built for general-purpose decisions, judge one candidate per request and are ill-suited to the framework's region questions, so ours learns directly from the requests the framework itself produces and judges all candidates of a level at once. Trained only on InstructPart and evaluated with definition-aligned operation words, outperforms chain-of-thought methods in domain and in zero-shot transfer to UMD and IIT-AFF, is better overall than existing System One models run in the same framework, and needs fewer judge requests. To our knowledge, is the first approach to bring System One models to affordance grounding.