Vision-Language Grounding

Follow this topic Ask about this topic

Vision-language grounding studies how models connect language to specific visual evidence, enabling them to locate a described object, segment a region, track an entity through video, or act on an instruction. Common approaches align text with image regions using contrastive learning, detection, segmentation, and point or box prediction; newer multimodal models also use attention, structured scene representations, visual search, and step-by-step evidence grounding. Evaluation spans referring-expression benchmarks, high-resolution and video localization, document understanding, and embodied tasks such as navigation and robotic manipulation. A central challenge is that correct answers do not prove models used the image: language priors, ambiguity, small or changing targets, and long-horizon context can all undermine faithful and robust grounding.

Latest papers

420 papers49 papers in the last four weeks, down 6% on the four weeks before. 0.3% of all new papers.

Grounding Multimodal Reasoning with Evidence-Ablated Negatives
Oct 29, 2026Chenxi Liu, Tianyi Xiong, Yanshuo Chen +1Multimodal Reasoning RL for VLMs
Department of Computer Science, University of Maryland, College Park, College Park, MD 20742
inkGround: Benchmarking Fine-grained Visual Grounding Capabilities of Vision-Language Models for Cursive Handwriting in the Wild
Oct 28, 2026Sidra HanifVLM Evaluation Vision-Language Grounding
San Diego State University
LOCUS: Locate Once, Check Until Settled. Draft-and-Verify Visual Grounding with a Box Head and Coordinate-Token Verification
Oct 26, 2026Muhammad TalhaVision-Language Grounding Visual Grounding
Independent Researcher
Seeing Before Naming: Context-Grounded Learning of Novel Visual Concepts
Oct 26, 2026Gabriel Adineera, Jinoh KimVision-Language Grounding Few-Shot Learning
Texas A&M University-Commerce USA
What Sketches Tell Us about LVLMs: Conventions, Grounding, and Localisation
Oct 25, 2026Chaitat Utintu, Ahmed Bourouis, Arkaprabha Basu +1Vision-Language Models VLM Interpretability
SketchX, Surrey Institute for People-Centred Artificial Intelligence (PAI), University of Surrey · UKRI AI: CDT in AI for Digital Media Inclusion
Grounding on Request: Learning Modality Control in Vision-Language Models
Oct 24, 2026Maxime Dassen, Alexander Martin, Andrew Yates +2Vision-Language Grounding Visual Evidence Sensitivity in VLMs
University of Amsterdam · Johns Hopkins University · Human Language Technology Center of Excellence, Johns Hopkins University
PreView: Enhancing GUI Grounding with Preprocessed Views and Cues
Oct 24, 2026Wenjun Hou, Kaishuai XuGUI Grounding Vision-Language Grounding
Independent Researcher
MiD-VLM: Detecting and Grounding Global Semantic Misinformation in Image-Caption Pairs
Oct 23, 2026Tarunya Prasad, Bharatesh ChakravarthiFake News Detection Vision-Language Grounding
Arizona State University
PROVE: Probabilistic Visual Grounding via Patch-Level Evidence for Vision-Language Models
Oct 23, 2026Mayank Nautiyal, Tianru Zhang, Li Ju +2Vision-Language Grounding Confidence Estimation for VLMs
Department of Information Technology, Uppsala University, Sweden · Science for Life Laboratory, Uppsala University, Sweden
Angular Supervision for Visual Object Grounding in Autoregressive Vision-Language Models
Oct 23, 2026Parsa Esmaeilkhani, Longin Jan LateckiVision-Language Grounding VLM Interpretability
Temple University, Philadelphia, USA
SAGE: Sense-Aligned Multimodal Grounding Evaluation with Gloss-Augmented Reasoning
Oct 22, 2026Sunjae Kwon, Suhaas Kiran DG, Zonghai Yao +3VLM Evaluation Vision-Language Grounding
UMass Amherst · UMass Lowell · UMass Chan Medical School +1
Visual Grounding First, Multimodal In-context Learning Follows
Oct 22, 2026Minhyuk Seo, Minjae Lee, Chaeeun Lee +4Multimodal ICL Vision-Language Grounding
UC Berkeley · Seoul National University · Xero +2
Unlocking Spatial Grounding in Flow-Matching Models via SNR Tuning
Oct 21, 2026Martin Thißen, Elke HergenrötherFlow-Based Generative Modeling Vision-Language Grounding
Department of Computer Science, Darmstadt University of Applied Sciences, Darmstadt, Germany
Evaluating Visual Language Models on Visual Grounding for Visually Impaired People
Oct 21, 2026Fairooz Azim, Lydia Achour, Claudia Borg +1Vision-Language Grounding VLM Evaluation
Orange Research Lannion, France · University of Malta, Msida, Malta
LaMER: Language-Conditioned Multi-Expert Routing for Weakly Supervised Referring Expression Comprehension
Oct 19, 2026Yu Yu Chang, Tzu-Hsuan Weng, Youssef Mohamed Amin +4Vision-Language Grounding Mixture of Experts
National Central University, Taoyuan, Taiwan · Pharos University in Alexandria, Alexandria, Egypt
SL-Ground: Sentence-Level Grounding for Fine-Grained Open-Vocabulary Object Detection
Oct 17, 2026Yang-Syu Li, Cheng-Huan Yang, Chia-Yung Jui +1Open-Vocabulary Object Detection Vision-Language Grounding
Department of Computer Science and Information Engineering, National Chung Cheng University, Chiayi, Taiwan. · National Center for High-Performance Computing, Taichung, Taiwan.
Grounded Human-Attributed Description and Activity Recognition in Videos
Oct 15, 2026Jay N. Paranjape, Jurijs Nazarovs, Rohan Sarkar +5Human Activity Recognition Vision-Language Grounding
Johns Hopkins University, USA · Amazon
ReferFlow: Beyond Final-Target Localization for Complex Referring Expressions
Oct 14, 2026Tianyi Gao, Han Fang, Xin Wei +11Referring Expression Comprehension Vision-Language Grounding
State Key Laboratory of Human-Machine Hybrid Augmented Intelligence, Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University · Xingchen AGI Lab, China Telecom Artificial Intelligence Technology (Beijing) Co., Ltd. · Beijing Normal University +4
Recognizing Dialogue Ambiguity via Candidate Set Visual Grounding
Oct 14, 2026Shun Inadumi, Lee Sangmyeong, Koichiro YoshinoVisual Grounding Vision-Language Grounding
The University of Osaka · Guardian Robot Project, RIKEN · Nara Institute of Science and Technology
WinARDet: Compressed Window Auto-Regression for Long-Horizon Vision-Language Grounding
Oct 14, 2026Chenxu Wang, Hangyi Kuang, Bo-Wen Yin +6Vision-Language Grounding Autoregressive Decoding
Nankai University · Beijing Normal University
MHVF: Training-Free GUI Grounding via Verification-Guided Multi-Hypothesis Search
Oct 13, 2026Minghua Chen, Xingming Liao, Weiqiang Zhong +1GUI Grounding Vision-Language Grounding
School of Computer Science and Technology, Guangdong University of Technology, Guangzhou 510090, China
Geospatial Vector Geometry as a Native Spatial Modality for Vision-Language Models
Oct 12, 2026Huang Fei, Boyi Chen, Hao Xue +3Vision-Language Models Geospatial Reasoning
The Hong Kong University of Science and Technology (Guangzhou), China · The Hong Kong University of Science and Technology, Hong Kong SAR, China
A Persian Vision-Language-Action Dataset and Benchmark for Ambiguity- and Infeasibility-Aware Instruction-to-Plan Grounding
Oct 11, 2026Abdollah Pourasghari, Saeedeh Sadat Sadidpour, Mohammad RoustaeiVision-Language Grounding Vision-Language-Action Models
Malek Ashtar University of Technology, Iran · Isfahan University of Technology, Isfahan, Iran
GG-Nav: Goal-oriented Grounding Chain-of-Thought Elicits Reasoning in Navigation Foundation Models
Oct 10, 2026Shaohao Zhu, Wensi Huang, Siqi Zhang +6Vision-Language Navigation Robot Navigation
Zhejiang University · Shanghai AI Laboratory · University of Science and Technology of China +2
Agreement versus Softmax under Spatial Disambiguation
Oct 9, 2026Michał Zakrzewski, Mariia Shpir, Taras Rumezhak +1Vision-Language Grounding 3D Visual Grounding
AGH University of Krakow, Krakow, Poland · SoftServe · ETH Zurich
Jag: Direct Box Prediction for Efficient Visual Grounding
Oct 9, 2026Xiuyuan Zhu, Ke Lu, Hao Wu +3Visual Grounding Vision-Language Grounding
University of Chinese Academy of Sciences, Beijing, China · State Key Laboratory of Communication Content Cognition, Beijing, China · Peng Cheng Laboratory, Shenzhen, Guangdong, China
In-Context Learning Can Help Vision-Language Models Overcome Training Priors
Oct 9, 2026Kun Wang, Xindi Wu, Sanghyuk Chun +2Visual Evidence Sensitivity in VLMs In-Context Learning
Princeton University
Spa-Bench: A Real-Robot Benchmark for Spatial Grounding in Vision-Language-Action Models
Oct 8, 2026Justin Tien-Smith, Matthew Pidden, Rolandos Alexandros PotamiasSpatial Grounding for VLA Models Visual Grounding for Robotics
Imperial College London
From What to Which: Decoding Modifier Grounding in Frozen MLLMs
Oct 8, 2026Barbara Toniella Corradini, Caterina Gallegati, Ludovica Genovese +1Multimodal Large Language Models Vision-Language Grounding
AI for Good · University of Siena, Italy
SAGE: Sink-Aware Guided Emphasis for Visual Grounding in Vision-Language Decoders
Oct 8, 2026Jeonghyo Song, YoungJoon YooAttention Intervention in VLMs Vision-Language Grounding
Department of Artificial Intelligence, Chung-Ang University

Oct 29, 2026J/K move · Enter open · S save

Grounding Multimodal Reasoning with Evidence-Ablated Negatives

Chenxi Liu, Tianyi Xiong, Yanshuo Chen, Heng Huang

Department of Computer Science, University of Maryland, College Park, College Park, MD 20742

Multimodal Reasoning RL for VLMs Vision-Language Grounding Visual Evidence Sensitivity in VLMs Reinforcement Learning with Verifiable Rewards

Open paper AskPDF arXiv

Reinforcement learning with verifiable rewards (RLVR) has become a powerful tool for improving multimodal reasoning. However, standard outcome supervision suffers from credit assignment ambiguity, as a model may receive positive rewards by exploiting language priors or dataset shortcuts rather than grounding its reasoning in the requisite visual evidence. This makes high-quality negative responses critical, since effective negatives should remain plausible under the current policy while exposing the shortcut that should be suppressed. In this paper, we propose EVAN, a multimodal reinforcement learning framework utilizing EVidence-Ablated Negatives to train LMMs against their own language-prior failures. EVAN first identifies highly visually dependent examples using a scaled image information gain metric and preserves data diversity with a mixed-ranking strategy. It then constructs evidence-ablated images by masking the most discriminative image patches, eliciting plausible but visually ungrounded responses from the current policy. During policy optimization, these negative responses are incorporated as bounded auxiliary signals under the clean image, penalizing shortcut learning while maintaining RL stability. Experiments across challenging multimodal reasoning benchmarks demonstrate that EVAN mitigates modality imbalance and improves over strong RLVR baselines. Our code is available at https://github.com/DawnLIU35/EVAN.git.