Vision-Language Grounding

Ask about this topic

Overview

Vision-language grounding studies how models connect language to specific visual evidence, enabling them to locate a described object, segment a region, track an entity through video, or act on an instruction. Common approaches align text with image regions using contrastive learning, detection, segmentation, and point or box prediction; newer multimodal models also use attention, structured scene representations, visual search, and step-by-step evidence grounding. Evaluation spans referring-expression benchmarks, high-resolution and video localization, document understanding, and embodied tasks such as navigation and robotic manipulation. A central challenge is that correct answers do not prove models used the image: language priors, ambiguity, small or changing targets, and long-horizon context can all undermine faithful and robust grounding.

Latest papers

CardsList

420 papers49 papers in the last four weeks, down 6% on the four weeks before. 0.3% of all new papers.

  1. Attention at Rest Stays at Rest: Breaking Visual Inertia to Mitigate Relation Hallucinations

    Apr 2, 2026Boyang Gong, Yu Zheng, Fanye Kong +2Emotional Hallucination in VLMsVLM Hallucination Mitigation

    Tsinghua University

  2. First Logit Boosting: Visual Grounding Method to Mitigate Object Hallucination in Large Vision-Language Models

    Apr 1, 2026Jiwoo Ha, Jongwoo Baek, Jinhyun SoVLM Hallucination MitigationObject Hallucination in VLMs

    DGIST EECS

  3. Q-Mask: Query-driven Causal Masks for Text Anchoring in OCR-Oriented Vision-Language Models

    Mar 31, 2026Longwei Xu, Feng Feng, Shaojie Zhang +7Optical Character RecognitionVision-Language Models

    MiLM Plus, Xiaomi Inc

  4. AMIGO: Agentic Multi-Image Grounding Oracle Benchmark

    Mar 30, 2026Min Wang, Ata MahjoubfarVLM EvaluationVisual Question Answering

    Target Corporation

  5. LIBERO-Para: A Diagnostic Benchmark and Metrics for Paraphrase Robustness in VLA Models

    Mar 30, 2026Chanyoung Kim, Minwoo Kim, Minseok Kang +2Vision-Language GroundingVision-Language-Action Models

    School of Computer Science and Engineering, Soongsil University, Seoul, Korea · Department of Artificial Intelligence, Chung-Ang University, Seoul, Korea

  6. MolmoPoint: Better Pointing for VLMs with Grounding Tokens

    Mar 30, 2026Christopher Clark, Yue Yang, Jae Sung Park +8Vision-Language GroundingLarge Vision-Language Models

    Allen Institute for AI · University of Washington

  7. SAGE: Sink-Aware Grounded Decoding for Multimodal Hallucination Mitigation

    Mar 29, 2026Tripti Shukla, Zsolt KiraVLM Hallucination MitigationVision-Language Grounding

    Georgia Institute of Technology

  8. Beyond Textual Knowledge-Leveraging Multimodal Knowledge Bases for Enhancing Vision-and-Language Navigation

    Mar 27, 2026Dongsheng Yang, Yinfeng Yu, Liejun WangVision-Language NavigationVision-Language Grounding

  9. Interpretable Zero-shot Referring Expression Comprehension with Query-driven Scene Graphs

    Mar 26, 2026Yike Wu, Necva Bolucu, Stephen Wan +3Referring Expression ComprehensionVision-Language Grounding

  10. OptiSAR-Net++: A Large-Scale Benchmark and Transformer-Free Framework for Cross-Domain Remote Sensing Visual Grounding

    Mar 25, 2026Xiaoyu Tang, Jun Dong, Jintao Cheng +1Remote SensingVision-Language Grounding

    School of Electronics and Information Engineering, and Xingzhi College, South China Normal University, Foshan 528225, China · School of Data Science and Engineering, and Xingzhi College, South China Normal University, Shanwei, 516600, China · Department of Electronic and Computer Engineering, Hong Kong University of Science and Technology, Hong Kong SAR, China +1

  11. Heuristic-inspired Reasoning Priors Facilitate Data-Efficient Referring Object Detection

    Mar 25, 2026Xu Zhang, Zhe Chen, Jing Zhang +1Vision-Language GroundingObject Detection

    The University of Sydney, Australia · La Trobe University, Australia · Wuhan University, China +1

  12. The Dual Mechanisms of Spatial Variable Binding in Vision-Language Models

    Mar 23, 2026Kelly Cui, Nikhil Prakash, Shoval Messica +4VLM InterpretabilityVision-Language Grounding

    MIT CSAIL · Northeastern University · Sony Playstation

  13. SteelDefectX: A Multi-Form Vision-Language Dataset and Benchmark for Steel Surface Defect Analysis

    Mar 23, 2026Shuxian Zhao, Jie Gui, Baosheng Yu +1VLM EvaluationVision-Language Grounding

    Southeast University · Purple Mountain Laboratories · Nanyang Technological University

  14. Getting to the Point: Pointing Improves LVLMs at Counting

    Mar 23, 2026Simone Alghisi, Massimo Rizzoli, Seyed Mahed Mousavi +1VLM ReasoningLarge Vision-Language Models

    Signals and Interactive Systems Lab, University of Trento, Italy

  15. Tinted Frames: Question Framing Blinds Vision-Language Models

    Mar 19, 2026Wan-Cyuan Fan, Jiayun Luo, Declan Kutscher +2Vision-Language ModelsVLM Robustness

    University of British Columbia · Vector Institute for AI · University of California, Berkeley

  16. MERGE: Guided Vision-Language Models for Multi-Actor Event Reasoning and Grounding in Human-Robot Interaction

    Mar 19, 2026Joerg Deigmoeller, Nakul Agarwal, Stephan Hasler +8Vision-Language GroundingHuman-Robot Interaction

    Honda Research Institute Europe, 63073 Offenbach, Germany · Honda Research Institute USA, San Jose, CA 95134, USA.

  17. Cognitive Mismatch in Multimodal Large Language Models for Discrete Symbol Understanding

    Mar 19, 2026Yinghui Li, Jiayi Kuang, Peng Xing +11Multimodal Large Language ModelsMultimodal Model Evaluation

    Tsinghua University, China · Sun Yat-sen University, China · Independent Researcher +3

  18. To See or To Please: Uncovering Visual Sycophancy and Split Beliefs in VLMs

    Mar 19, 2026Rui Hong, Shuxue QuanSelective Prediction for VLMsSycophancy in VLMs

    George Mason University, Fairfax, VA, USA · Independent Researcher

  19. EchoGen: Cycle-Consistent Learning for Unified Layout-Image Generation and Understanding

    Mar 18, 2026Kai Zou, Hongbo Liu, Dian Zheng +3Cycle-Consistent LearningImage Generation

    School of Cyber Science and Technology, University of Science and Technology of China · Anhui Province Key Laboratory of Digital Security · Tongji University +3

  20. A Proposal-Free Query-Guided Network for Grounded Multimodal Named Entity Recognition

    Mar 18, 2026Hongbing Li, Jiamin Liu, Shuo Zhang +1Named Entity RecognitionMultimodal Grounding

    School of Artificial Intelligence, Beijing University of Posts and Telecommunications, Beijing, China

  21. RealVLG-R1: A Large-Scale Real-World Visual-Language Grounding Benchmark for Robotic Perception and Manipulation

    Mar 16, 2026Linfei Li, Lin Zhang, Ying ShenVision-Language GroundingLanguage-Conditioned Robot Manipulation

    School of Computer Science and Technology, Tongji University, China

  22. GroundSet: A Cadastral-Grounded Dataset for Spatial Understanding with Vector Data

    Mar 15, 2026Roger Ferrod, Maël Lecene, Krishna Sapkota +4VLM EvaluationRemote Sensing Image Understanding

    Université Paris Cité, LIPADE, F-75006 Paris, France · Google Research

  23. Script-to-Slide Grounding: Grounding Script Sentences to Slide Objects for Automatic Instructional Video Generation

    Mar 14, 2026Rena Suzuki, Masato Kikuchi, Tadachika OzonoVideo GenerationVision-Language Grounding

    Nagoya Institute of Technology, Nagoya, Aichi, Japan

  24. Mastering Negation: Boosting Grounding Models via Grouped Opposition-Based Learning

    Mar 13, 2026Zesheng Yang, Xi Jiang, Bingzhang Hu +4Vision-Language Grounding

    Department of Computer Science, Southern University of Science and Technology, Shenzhen, China · Vision Intelligence Center, Hefei CAS Dihuge Automation Co., LTD, Hefei, China · School of Information Science and Technology, Harbin Institute of Technology (Shenzhen), China +2

  25. Spatial Reasoning is Not a Free Lunch: A Controlled Study on LLaVA

    Mar 13, 2026Nahid Alam, Leema Krishna Murali, Siddhant Bharadwaj +7Spatial Reasoning in Language ModelsVisual Spatial Reasoning

    Cohere Labs Community · Eisai Inc. · Indian Institute of Science, Bangalore +4

  26. SPARROW: Learning Spatial Precision and Temporal Referential Consistency in Pixel-Grounded Video MLLMs

    Mar 12, 2026Mohamad Alansari, Naufal Suryanto, Divya Velayudhan +3Multimodal Large Language ModelsVision-Language Grounding

    Khalifa University, Abu Dhabi, UAE

  27. See, Symbolize, Act: Grounding VLMs with Spatial Representations for Better Gameplay

    Mar 12, 2026Ashish Baghel, Paras ChopraVision-Language AgentsVision-Language Grounding

    Lossfunk

  28. GroundCount: Grounding Vision-Language Models with Object Detection for Mitigating Counting Hallucinations

    Mar 11, 2026Boyuan Chen, Minghao Shao, Siddharth Garg +2Vision-Language ModelsVLM Hallucination Mitigation

    Tandon School of Engineering, New York University, NY, USA · eBRAIN Lab, Division of Engineering, New York University Abu Dhabi, UAE

  29. WalkGPT: Grounded Vision-Language Conversation with Depth-Aware Segmentation for Pedestrian Navigation

    Mar 11, 2026Rafi Ibn Sultan, Hui Zhu, Xiangyu Zhou +4Vision-Language GroundingLarge Vision-Language Models

    Department of Computer Science, Wayne State University · Department of Radiation Oncology, Henry Ford Health · Department of Electrical and Computer Engineering, The Ohio State University +1

  30. AR2-4FV: Anchored Referring and Re-identification for Long-Term Grounding in Fixed-View Videos

    Mar 8, 2026Teng Yan, Yihan Liu, Jiongxu Chen +3Query Re-IdentificationVision-Language Grounding

    The Hong Kong University of Science and Technology (Guangzhou)