Visual Grounding

Follow this topic Ask about this topic

Visual grounding studies how systems identify the image regions or objects referred to by language, often producing boxes, points, or masks. Common approaches include aligning text and visual features, selecting among detected objects, using attention or high-resolution crops, and training with localization annotations or reinforcement-learning rewards. The field spans referring-expression benchmarks such as RefCOCO, fine-grained spatial reasoning, medical and remote-sensing images, video, and embodied tasks such as navigation and manipulation. A central challenge is that correct answers do not prove a model used visual evidence: systems still struggle with precise localization, changing scenes, and reliable visual dependence while balancing accuracy, efficiency, and robustness.

Latest papers

155 papers23 papers in the last four weeks, up 28% on the four weeks before. 0.1% of all new papers.

See First, Answer Later: Visual Evidence Pre-Alignment via Sufficiency-Driven RL
Jun 16, 2026Yilian Liu, Sicong Leng, Guoshun Nan +7Multimodal Pretraining Visually Grounded Reasoning
Beijing University of Posts and Telecommunications, China · Nanyang Technological University, Singapore · China Telecom, China
DifferAD-R1: A Difference-Guided IndustrialAnomaly Localization with Multimodal LargeLanguage Models
Jun 15, 2026Dingrong Wang, Xian Tao, Zhen Qu +5Reinforcement Learning Industrial Anomaly Detection
Institute of Automation, Chinese Academy of Sciences (CAS), Beijing 100190, China · School of Artificial Intelligence, University of Chinese Sciences, Beijing 100049, China · Institute of Automation, CAS, Beijing 100190, China +4
Training-Free Open-Vocabulary Visual Grounding for Remote Sensing Images and Videos
Jun 15, 2026Ke Li, Di Wang, Yongshan Zhu +5Remote Sensing Image Understanding Temporal Video Grounding
School of Computer Science and Technology, Xidian University, Xi’an 710071, China · Interdisciplinary Institute of Artificial Intelligence, Xidian University, Xi’an, Shaanxi 710126, China · School of Artificial Intelligence, Xidian University, Xi’an 710071, China +2
Iterative Visual Thinking and the Self-Correction Mirage in VLM Grounding
Jun 11, 2026Animesh Tripathy, Aswanth KrishnanVLM Evaluation Vision-Language Grounding
QpiAI India Pvt. Ltd.
SalArt-VQA: Diagnosing Whether VLMs Understand Salient Artifacts in Generated Images
Jun 10, 2026Ruotian Zhang, Xiaoxiao Sun, Junzhe Huang +2VLM Evaluation VLM Hallucination
Zhejiang University · The University of Queensland
Helping Figures Tell their Story! Paper-Grounded Video Generation Explaining Complex Scientific Figures
Jun 10, 2026Ishani Mondal, Javad Baghirov, Jordan Boyd-GraberImage-to-Video Generation Video Generation
University of Maryland, College Park
PRPO: Perception-Reinforced Policy Optimization via Token-Level Dynamic Advantage Reshaping
Jun 7, 2026Qiming Li, Tianlun Li, Xiaolong Cheng +5Token-Level Credit Assignment VLM Reasoning
Amap CV Lab, Alibaba Group · Peking University
Seeing is Believing: Aligning Prompt Rewriting with Visual Anchors for Text-to-Image Generation
Jun 7, 2026Xuanyi Liu, Deyi Ji, Junyu Lu +6T2I Generation Prompt Optimization for T2I Generation
Peking University · Tencent · Dalian University of Technology +4
MedSIGHT: Towards Grounded Visual Comprehension in Medical Large Vision-Language Models
Jun 4, 2026Aofei Chang, Le Huang, Alex James Boyd +4Medical VLMs Medical Image Segmentation
GE HealthCare · College of Information Sciences and Technology, The Pennsylvania State University, State College, PA, USA · GE Healthcare, Bellevue, WA, USA.
Self-Improving Small Object Grounding in LVLMs
Jun 1, 2026Tianze Yang, Yucheng Shi, Ruitong Sun +2Large Vision-Language Models VLM Interpretability
University of Georgia
MM-Snowball: Evaluating and Mitigating Hallucination Snowballing in Multimodal Multi-Turn Dialogue
May 30, 2026Yue Jiang, Xue Jiang, Lihua Zhang +6Multimodal Large Language Models Multimodal Hallucination
College of Intelligent Robotics and Advanced Manufacturing, Fudan University · Southern University of Science and Technology · TMLR Group, Hong Kong Baptist University +3
Improving Visual Grounding in Remote Sensing via Cluster-Guided Refinement and Model Ensemble Voting
May 30, 2026Panav Shah, Geet Sethi, Ashutosh GandheRemote Sensing Vision-Language Grounding
Indian Institute of Technology Bombay Mumbai, India · Indian Institute of Technology Bombay2026 Mumbai, India
Attend to Evidence: Evidence-Anchored Spatial Attention Supervision for Multimodal RLVR
May 29, 2026Ruina Hu, Chen Wang, Lai Wei +5Reinforcement Learning with Verifiable Rewards Visually Grounded Reasoning
1Harbin Institute of Technology · 2Zhongguancun Academy · 4Nankai University +4
Mags-RL: Wearing Multimodal LLMs a Magnifying Glass via Agentic Reinforcement Learning For Complex Scene Reasoning
May 27, 2026Xuanzhao Dong, Wenhui Zhu, Peijie Qiu +11Visual Question Answering Tool-Using Vision-Language Agents
Arizona State University, Tempe, AZ, USA · Washington University in St. Louis, St. Louis, MO, USA · Clemson University, Clemson, SC, USA +3
LocateAnything: Fast and High-Quality Vision-Language Grounding with Parallel Box Decoding
May 26, 2026Shihao Wang, Shilong Liu, Yuanguo Kuang +10Efficient VLM Inference Vision-Language Grounding
The Hong Kong Polytechnic University · NVIDIA · Princeton University +2
Correcting Visual Blur Induced by Attention Distraction to Reduce Hallucinations: Algorithm and Theory
May 23, 2026Quanjiang Li, Zhiming Liu, Wei Luo +2LLM Hallucination Mitigation Hallucination in Language Models
National University of Defense Technology, China · Harbin Institute of Technology (Shenzhen), China · Xi’an Jiaotong University, China.
Faithful-MR1: Faithful Multimodal Reasoning via Anchoring and Reinforcing Visual Attention
May 21, 2026Changyuan Tian, Zhicong Lu, Huaxing Liu +7Multimodal Large Language Models Multimodal Reasoning
AMAP, Alibaba Group · University of Chinese Academy of Sciences · Tsinghua University +1
AgroVG: A Large-Scale Multi-Source Benchmark for Agricultural Visual Grounding
May 21, 2026Haocheng Li, Juepeng Zheng, Zenghao Yang +5Agricultural Image Analysis Set Prediction
China Agricultural University · Sun Yat-sen University · National Supercomputing Center in Shenzhen +3
VISTAQA: Benchmarking Joint Visual Question Answering and Pixel-Level Evidence
May 20, 2026Mozhgan Nasr Azadani, Yimu Wang, Yongpeng Zhu +5VLM Evaluation Visual Question Answering
University of Waterloo · Stanford University · NVIDIA
Leveraging Latent Visual Reasoning in Silence
May 18, 2026Dongyao Zhu, Zhen Wang, Xi Xiao +7Visual Spatial Reasoning Latent Visual Reasoning
North Carolina State University · UC, San Diego · University of Alabama at Birmingham +4
EPIC-Bench: A Perception-Centric Benchmark for Fine-Grained Embodied Visual Grounding in Vision-Language Models
May 16, 2026Haozhe Shan, Xiancong Ren, Han Dong +10VLM Evaluation Vision-Language Grounding
X-Humanoid · Fudan University · University of New South Wales +4
From Failure to Feedback: Group Revision Unlocks Hard Cases in Object-Level Grounding
May 15, 2026Yuyuan Liu, Yiping Ji, Anjie Le +6Vision-Language Grounding Visual Grounding
Department of Engineering Science, University of Oxford · Australian Institute for Machine Learning, Adelaide University · Technical University of Munich +2
EARL: Towards a Unified Analysis-Guided Reinforcement Learning Framework for Egocentric Interaction Reasoning and Pixel Grounding
May 14, 2026Yuejiao Su, Xinshen Zhang, Zhen Ye +3Egocentric Video Understanding Vision-Language Grounding
Department of Electrical and Electronic Engineering, The Hong Kong Polytechnic University, Hong Kong SAR. · Division of Emerging Interdisciplinary Areas (EMIA), The Hong Kong University of Science and Technology, Hong Kong SAR.
Mitigating Mask Prior Drift and Positional Attention Collapse in Large Diffusion Vision-Language Models
May 14, 2026Sujung Hong, Chanyong Yoon, Seong Jae HwangVision-Language Models Diffusion VLMs
Department of Artificial Intelligence, Yonsei University, Seoul, Republic of Korea.
CLIP Tricks You: Training-free Token Pruning for Efficient Pixel Grounding in Large VIsion-Language Models
May 13, 2026Sangin Lee, Yukyung ChoiEfficient VLM Inference Vision-Language Grounding
Dept. of Artificial Intelligence and Robotics, Sejong University, Seoul, Republic of Korea · Artificial Intelligence and Robotics Institute (AIRI), Sejong University, Seoul, Republic of Korea
From Web to Pixels: Bringing Agentic Search into Visual Perception
May 12, 2026Bokang Yang, Xinyi Sun, Kaituo Feng +3Visual Question Answering Vision-Language Grounding
1Shenzhen Loop Area Institute · 2Wuhan University · 3CUHK MMLab
Do Vision-Language-Models show human-like logical problem-solving capability in point and click puzzle games?
May 11, 2026Maximilian Triebel, Marco Menner, Dominik HelfensteinVLM Evaluation Physical Reasoning
Institute of Artificial Intelligence, University of Stuttgart, Stuttgart, Germany.
SoccerLens: Grounded Soccer Video Understanding Beyond Accuracy
May 10, 2026Ismael Elsharkawi, Ahmed Sait, Silvio Giancola +3VLM Evaluation Sports Video Analysis
Department of Computer Science and Engineering, The American University in Cairo · Image And Visual Understanding Lab (IVUL), KAUST
Investigating Anisotropy in Visual Grounding under Controlled Counterfactual Perturbations
May 9, 2026Gabriele Lombardo, Luigi Maiorana, Liliana Lo Presti +1Neural Representation Geometry VLM Interpretability
University of Palermo Palermo, Italy
ChartREG++: Towards Benchmarking and Improving Chart Referring Expression Grounding under Diverse referring clues and Multi-Target Referring
May 8, 2026Tianhao Niu, Ziyu Han, Xuan Dong +2Referring Image Segmentation Visual Grounding
Research Center for Social Computing and Interactive Robotics Harbin Institute of Technology, China

Jun 16, 2026J/K move · Enter open · S save

See First, Answer Later: Visual Evidence Pre-Alignment via Sufficiency-Driven RL

Yilian Liu, Sicong Leng, Guoshun Nan, Junyi Zhu+6

Beijing University of Posts and Telecommunications, China · Nanyang Technological University, Singapore · China Telecom, China

Multimodal Pretraining Visually Grounded Reasoning Visual Grounding Vision-Language Alignment RL for VLMs

Open paper AskPDF arXiv

Multimodal large language models (MLLMs) integrate strong text reasoning with visual inputs, yet their responses can be inconsistent with the underlying images, indicating ineffective utilization of visual evidence during inference. The prevailing training paradigm relies on large-scale caption-based pretraining for general alignment, followed by supervised fine-tuning and reinforcement learning to enable instruction following and complex reasoning. However, such pretraining provides only weak visual grounding: short, coarse captions bias models toward salient objects while neglecting fine-grained visual evidence. In this paper, we introduce Visual Evidence Pre-Alignment (VEPA), an intermediate stage between pretraining and post-training that explores a novel sufficiency-driven objective with Group Relative Policy Optimization (GRPO) to optimize question-conditioned visual evidence descriptions. Extensive experiments across diverse benchmarks show that our VEPA consistently enhances performance on visually demanding evaluations and complements standard supervised post-training. Further analyses show that the income stems from strengthened, transferable visual grounding, rather than from additional task-specific training.