Vision-Language Grounding

Ask about this topic

Overview

Vision-language grounding studies how models connect language to specific visual evidence, enabling them to locate a described object, segment a region, track an entity through video, or act on an instruction. Common approaches align text with image regions using contrastive learning, detection, segmentation, and point or box prediction; newer multimodal models also use attention, structured scene representations, visual search, and step-by-step evidence grounding. Evaluation spans referring-expression benchmarks, high-resolution and video localization, document understanding, and embodied tasks such as navigation and robotic manipulation. A central challenge is that correct answers do not prove models used the image: language priors, ambiguity, small or changing targets, and long-horizon context can all undermine faithful and robust grounding.

Latest papers

CardsList

420 papers49 papers in the last four weeks, down 6% on the four weeks before. 0.3% of all new papers.

  1. Step-Level Visual Grounding Faithfulness Predicts Out-of-Distribution Generalization in Long-Horizon Vision-Language Models

    Mar 6, 2026Md Ashikur Rahman, Md Arifur Rahman, Niamul Hassan Samin +2Vision-Language GroundingVLM Robustness

    The KOW Company · University of Dhaka · American International University - Bangladesh (AIUB)

  2. Keeping the Evidence Chain: Semantic Evidence Allocation for Training-Free Token Pruning in Video Temporal Grounding

    Mar 5, 2026Jiaqi Li, Shuntian Zheng, Yixian Shen +4Temporal Video GroundingVision-Language Grounding

    University of Warwick, Coventry CV4 7AL, United Kingdom · University of Amsterdam, Amsterdam 1012 WX, Netherlands · Amazon AGI, The United States of America

  3. Seeing Clearly without Training: Mitigating Hallucinations in Multimodal LLMs for Remote Sensing

    Mar 3, 2026Yi Liu, Jing Zhang, Di Wang +3Remote Sensing VQAVLM Hallucination Mitigation

    School of Computer Science, Wuhan University, China · Zhongguancun Academy, China · School of Computer Science, Chongqing University, China

  4. Better Eyes, Better Thoughts: Why Vision Chain-of-Thought Fails in Medicine

    Mar 2, 2026Yuan Wu, Zongxian Yang, Jiayu Qian +5Medical VLMsVLM Reasoning

    Department of Computer Science, City University of Hong Kong (Dongguan) · College of Computing and Data Science (CCDS), Nanyang Technological University · School of Computer Science, Northwestern Polytechnical University

  5. Robust Skills, Brittle Grounding: Diagnosing Restricted Generalization in Vision-Language Action Policies via Multi-Object Picking

    Feb 27, 2026David Emukpere, Romain Deffayet, Jean-Michel RendersInstruction Generalization in VLA ModelsGeneralization in Robotic Manipulation

    Naver Labs Europe, 6 Chem. de Maupertuis, Meylan, France.

  6. Ref-Adv: Exploring MLLM Visual Reasoning in Referring Expression Tasks

    Feb 27, 2026Qihua Dong, Kuo Yang, Lin Ju +6Referring Expression ComprehensionVLM Evaluation

    Northeastern University

  7. Vision-Language Semantic Grounding for Multi-Domain Crop-Weed Segmentation

    Feb 27, 2026Nazia Hossain, Xintong Jiang, Yu Tian +3Domain Generalization for Semantic SegmentationSemantic Segmentation

    Department of Bioresource Engineering, McGill University, Sainte-Anne-de-Bellevue, QC, Canada · Department of Plant Science, McGill University, Sainte-Anne-de-Bellevue, QC, Canada

  8. Towards Long-Form Spatio-Temporal Video Grounding

    Feb 26, 2026Xin Gu, Bing Fan, Jiali Yao +5Temporal Video GroundingLong-Context Modeling

    University of Chinese Academy of Sciences · 2UNT · Institute of Software Chinese Academy of Sciences +2

  9. StoryMovie: A Dataset for Semantic Alignment of Visual Stories with Movie Scripts and Subtitles

    Feb 25, 2026Daniel Oliveira, David Martins de MatosVisual StorytellingVision-Language Alignment

    INESC-ID Lisboa, R. Alves Redol 9, 1000-029 Lisboa, Portugal · Instituto Superior Técnico, Universidade de Lisboa, Av. Rovisco Pais, 1049-001 Lisboa, Portugal

  10. A Multimodal Framework for Aligning Human Linguistic Descriptions with Visual Perceptual Data

    Feb 23, 2026Joseph BinghamReferring Expression ComprehensionMultimodal Grounding

    Faculty of Biology, Technion University, Haifa, Israel.

  11. Focusing Where Vision Matters: Selective Training for Large Vision Language Models via Visual Information Gain

    Feb 19, 2026Seulbi Lee, Sangheum HwangVision-Language ModelsLarge Vision-Language Models

    Department of Data Science, Seoul National University of Science and Technology, Seoul, Republic of Korea. · Currently with Tomocube, Daejeon, Republic of Korea.

  12. Saliency-Aware Multi-Route Thinking: Revisiting Vision-Language Reasoning

    Feb 18, 2026Mingjia Shi, Yinhan He, Yaochen Zhu +1VLM ReasoningReasoning for VLMs

    University of Virginia.

  13. Revealing and Enhancing Core Visual Regions: Harnessing Internal Attention Dynamics for Hallucination Mitigation in LVLMs

    Feb 17, 2026Guangtao Lyu, Qi Liu, Chenghao Xu +5Attention Intervention in VLMsVLM Hallucination Mitigation

    School of Electronic Engineering, Xidian University, Xi’an, China · College of Computer and Information, Hohai University, Nanjing, China · School of Computer Science and Technology, Xidian University, Xi’an, China +1

  14. Do MLLMs Really See It: Reinforcing Visual Attention in Multimodal LLMs

    Feb 9, 2026Siqu Ou, Tianrui Wan, Zhiyuan Zhao +2Reasoning for VLMsAttention Intervention in VLMs

    TeleAI · Shanghai Jiao Tong University · Northwestern Polytechnical University

  15. PixSearch: Region-Grounded Retrieval for Knowledgeable Large Multimodal Models

    Jan 27, 2026Jeonghwan Kim, Renjie Tao, Sanat Sharma +8Multimodal RAGVisual Question Answering

    Meta Reality Labs · University of Illinois Urbana-Champaign

  16. Visual-Language-Guided Task Planning for Horticultural Robots

    Jan 17, 2026Jose Cuaran, Kendall Koe, Aditya Potnis +2Large Language Model-Based Robot PlanningVision-Language Grounding

    The Siebel School of Computing and Data Science, University of Illinois, Urbana-Champaign · National Center for Supercomputing Applications at University of Illinois, Urbana-Champaign · The Department of Agricultural and Biological Engineering, University of Illinois, Urbana-Champaign

  17. PathFLIP: Fine-grained Language-Image Pretraining for Versatile Computational Pathology

    Dec 19, 2025Fengchun Liu, Songhan Jiang, Linghan Cai +2Computational PathologyVision-Language Grounding

    Harbin Institute of Technology, Shenzhen, School of Computer Science and Technology · National University of Singapore, Department of Electronic and Computer Engineering

  18. VLM2GeoVec: Toward Universal Multimodal Embeddings for Remote Sensing

    Dec 12, 2025Emanuel Sánchez Aimar, Gulnaz Zhambulova, Fahad Shahbaz Khan +2Multimodal UnderstandingCross-Modal Learning

    Linköping University, Sweden

  19. MMLongCite: A Benchmark for Evaluating Faithfulness of Long-Context Vision-Language Models

    Oct 15, 2025Keyan Zhou, Zecheng Tang, Lingfeng Ming +9VLM EvaluationVision-Language Grounding

    Soochow University · ByteDance · Harbin Institute of Technology +1

  20. Constructive Distortion: Improving MLLMs with Attention-Guided Image Warping

    Oct 10, 2025Dwip Dalal, Gautam Vashishtha, Utkarsh Mishra +6Object Hallucination in VLMsMultimodal Large Language Models

    University of Illinois Urbana–Champaign · Skan AI · Texas A&M University +1

  21. Talk in Pieces, See in Whole: Disentangled and Hierarchical Representation Learning in Language-based Object Detection

    Sep 29, 2025Sojung An, Kwanyong Park, Yong Jae Lee +1Open-Vocabulary Object DetectionDisentangled Representation Learning

    Korea University · University of Seoul · University of Wisconsin-Madison

  22. RefAM: Attention Magnets for Zero-Shot Referral Segmentation

    Sep 26, 2025Anna Kukleva, Enis Simsar, Alessio Tonioni +4Attention MechanismsImage Segmentation

    Max Planck Institute for Informatics, SIC · ETH Zürich · Google +1

  23. PixFoundation 2.0: Do Video Multi-Modal LLMs Use Motion in Visual Grounding?

    Sep 2, 2025Mennatullah SiamVision-Language GroundingVideo-Language Model Evaluation

  24. VMMU: A Vietnamese Multitask Multimodal Understanding and Reasoning Benchmark

    Aug 19, 2025Vy Tuong Dang, An Vo, Emilio Villa-Cueva +4VLM EvaluationMultimodal Reasoning

    KAIST · MBZUAI

  25. TAR: Temporal Anchor-Constrained Reasoning for Video Temporal Grounding

    Aug 11, 2025Chaohong Guo, Xun Mo, Yongwei Nie +3Temporal Video GroundingVision-Language Models

    South China University of Technology · Guangdong Laboratory of Artificial Intelligence and Digital Economy (SZ) · Bytedance Inc.

  26. MLLM-Guided Autonomous Object Detection for Open-World Understanding

    Jul 14, 2025Furkan Mumcu, Michael J. Jones, Anoop Cherian +1Open-Vocabulary Object DetectionVision-Language Grounding

    University of South Florida · Mitsubishi Electric Research Laboratories (MERL)

  27. FORGE: Forensic Reasoning with Grounded Evidence

    Mar 20, 2025Rohit Kundu, Shan Jia, Vishal Mohanty +2VLM AdaptationExplainable Deepfake Detection

    University of California, Riverside · Google LLC

  28. Fine-Grained Instruction-Guided Graph Reasoning for Vision-and-Language Navigation

    Mar 14, 2025Yaohua Liu, Binkai Ou, Rong Fu +2Vision-Language NavigationVision-Language Grounding

    Faculty of Science and Technology, University of Macau, Macau 999078, China

  29. PixFoundation: Are We Heading in the Right Direction with Pixel-level Vision Foundation Models?

    Feb 6, 2025Mennatullah SiamLarge Vision-Language ModelsVision-Language Grounding

  30. Paint Outside the Box: Synthesizing and Selecting Training Data for Visual Grounding

    Dec 1, 2024Zilin Du, Haoxin Li, Jianfei Yu +1Vision-Language GroundingTraining Data Selection

    Nanyang Technological University Singapore · Nanjing University of Science and Technology China