Visual Grounding

Ask about this topic

Overview

Visual grounding studies how systems identify the image regions or objects referred to by language, often producing boxes, points, or masks. Common approaches include aligning text and visual features, selecting among detected objects, using attention or high-resolution crops, and training with localization annotations or reinforcement-learning rewards. The field spans referring-expression benchmarks such as RefCOCO, fine-grained spatial reasoning, medical and remote-sensing images, video, and embodied tasks such as navigation and manipulation. A central challenge is that correct answers do not prove a model used visual evidence: systems still struggle with precise localization, changing scenes, and reliable visual dependence while balancing accuracy, efficiency, and robustness.

Latest papers

CardsList

153 papers23 papers in the last four weeks, up 28% on the four weeks before. 0.1% of all new papers.

  1. OBB-PBD: Extending Parallel Box Decoding to Oriented Bounding Boxes for Referring Expression Grounding in Aerial Imagery

    Oct 26, 2026Chang Woo Song, Yeong Jun KohVisual GroundingRemote Sensing Visual Grounding

    Department of Computer Science & Engineering, Chungnam National University, Daejeon 34134, Republic of Korea

  2. LOCUS: Locate Once, Check Until Settled. Draft-and-Verify Visual Grounding with a Box Head and Coordinate-Token Verification

    Oct 26, 2026Muhammad TalhaVision-Language GroundingVisual Grounding

    Independent Researcher

  3. When Pointing Becomes Profiling: Social Bias and Refusal in Text-Steerable Visual Encoders

    Oct 26, 2026Seyedarmin Azizi, Parsa Razmara, Sajjad Ghiasvand +2VLM SafetyVisual Grounding

    University of Southern California, USA · UC Santa Barbara, USA

  4. What Sketches Tell Us about LVLMs: Conventions, Grounding, and Localisation

    Oct 25, 2026Chaitat Utintu, Ahmed Bourouis, Arkaprabha Basu +1Vision-Language ModelsVLM Interpretability

    SketchX, Surrey Institute for People-Centred Artificial Intelligence (PAI), University of Surrey · UKRI AI: CDT in AI for Digital Media Inclusion

  5. AgentVQA: A Unified Benchmark for Agentic Visual Understanding

    Oct 22, 2026Advait Gupta, Sachin Konan, Tom Yin +2VLM EvaluationAgent Benchmarks

    University of Maryland, College Park · Princeton University

  6. WeVisOne-VL: Unified Visual Perception from Objects to Pixels

    Oct 22, 2026Tianming Liang, Junyi Pan, Yuxi Li +5Image MattingFine-Grained Visual Perception

    WeChat Vision, Tencent Inc.

  7. A Visual Grounding Benchmark for Retinal OCT with Multimodal Large Language Models

    Oct 22, 2026Ronald Fecso, José Morano, Botond Fazekas +6Medical VLMsMedical Image Grounding

    Institute of Artificial Intelligence, Center for Medical Data Science, Medical University of Vienna, Vienna, Austria · Department of Ophthalmology and Optometry, Medical University of Vienna, Vienna, Austria

  8. Auditing Where Concept Bottleneck Accuracy Comes From on CUB

    Oct 20, 2026Sascha Saralajew, Eleonora Poeta, Eliana Pastor +1Concept Bottleneck ModelsVisual Grounding

    Politecnico di Torino Turin, Italy

  9. From Seeing to Reporting: A Theory of Reliable Visual Set Generation in Vision-Language Models

    Oct 18, 2026Junzhe Shi, Yuan Gan, Shida JiangVision-Language ModelsVisual Grounding

    Systems Engineering, University of California, Berkeley, Berkeley, CA, USA · Department of AI, Quotr AI, Emeryville, CA, USA

  10. Recognizing Dialogue Ambiguity via Candidate Set Visual Grounding

    Oct 14, 2026Shun Inadumi, Lee Sangmyeong, Koichiro YoshinoVisual GroundingVision-Language Grounding

    The University of Osaka · Guardian Robot Project, RIKEN · Nara Institute of Science and Technology

  11. What Breaks in Multi-Image Schema Application? Factorial Oracle Interventions Reveal an Instance-Information Bottleneck

    Oct 14, 2026Yiliu Xu, Xinjin Li, Yu MaMulti-Image ReasoningVisual Reasoning

    Columbia University New York, United States

  12. Thinking with Images without Coordinates

    Oct 9, 2026Meng Cao, Haoze Zhao, Yi Wang +2VLM ReasoningVisually Grounded Reasoning

    Mohamed bin Zayed University of Artificial Intelligence · Shanghai AI Lab · Sun Yat-sen University

  13. Knowing Whether, Not Just Where: Anchor-Probing Distillation for Open-Set Remote Sensing Visual Grounding

    Oct 9, 2026Zhihao Chang, Lei Zhai, Shuyuan Yang +1Visual GroundingRemote Sensing Visual Grounding

  14. Jag: Direct Box Prediction for Efficient Visual Grounding

    Oct 9, 2026Xiuyuan Zhu, Ke Lu, Hao Wu +3Visual GroundingVision-Language Grounding

    University of Chinese Academy of Sciences, Beijing, China · State Key Laboratory of Communication Content Cognition, Beijing, China · Peng Cheng Laboratory, Shenzhen, Guangdong, China

  15. From What to Which: Decoding Modifier Grounding in Frozen MLLMs

    Oct 8, 2026Barbara Toniella Corradini, Caterina Gallegati, Ludovica Genovese +1Multimodal Large Language ModelsVision-Language Grounding

    AI for Good · University of Siena, Italy

  16. Improving Image-Based Nutrition Estimation Through Multimodal Food-Item Verification and Recovery

    Oct 8, 2026Jingbo Yue, Bruce Coburn, Jinge Ma +2Visual Grounding

    Elmore Family School of Electrical and Computer Engineering, Purdue University West Lafayette, IN, USA

  17. Mixture of Layers: Dynamic Layer Routing for Visual Reasoning

    Oct 7, 2026Jeonghwan Kim, Sofia Stoica, Jiwan Chung +5VLM ReasoningLarge Vision-Language Models

    University of Illinois Urbana-Champaign · Yonsei University · Meta Reality Labs

  18. Adversarial Images Hijack Web Agents from Visual Grounding to Browser Execution

    Oct 7, 2026Wanjing Han, Levi Taiji Li, Mu Zhang +2AI Agent SecurityTool-Using Vision-Language Agents

    University of Utah

  19. Visual Grounding Safety in Vision-Language Models

    Oct 5, 2026Erfan Shayegani, Kundan Krishna, Yue Dong +3VLM EvaluationVLM Safety Alignment

    University of California, Riverside · Work done during internship at Apple. · Apple

  20. AgroGround: Multi-Granularity Grounded Recognition in Agriculture

    Oct 3, 2026Abdulla Alshehhi, Zongyan Han, Rao Muhammad AnwerAgricultural Image AnalysisVision-Language Grounding

    Mohamed bin Zayed University of Artificial Intelligence Abu Dhabi, United Arab Emirates

  21. Same Reward, Different Skills: When Multimodal RL Learns to Look

    Oct 1, 2026Haocun Ye, Xinlong Jiang, Qile Chen +6Reinforcement Learning with Verifiable RewardsVisual Grounding

    University of the Chinese Academy of Sciences · Institute of Computing Technology, Chinese Academy of Sciences · Independent Researcher

  22. CoEvolve: Construct-to-Edit Visual Grounding with Bidirectional State Refinement

    Oct 1, 2026Dongwei Sun, Yujie Zhang, Bowen Yao +3Object DetectionVisual Grounding

    School of Computer Science and Technology and the Ministry of Education Key Lab for Intelligent Networks and Network Security, Xi’an Jiaotong University, Xi’an 710049, China · School of Computer Science and Technology, Faculty of Electronic and Information Engineering, Xi’an Jiaotong University, Xi’an 710049, China · State Key Laboratory of Remote Sensing and Digital Earth, Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100094, China

  23. LEGO-OPD: Factorized Teacher Composition for Multimodal On-Policy Distillation

    Sep 29, 2026Jaeyun Shin, Hangeol Chang, Jong Chul YeVLM DistillationMulti-Teacher Knowledge Distillation

    Korea Advanced Institute of Science and Technology (KAIST)

  24. DSPO: Diversity-aware Subjective Policy Optimization for Robust Emotional Reasoning

    Sep 29, 2026Cheng Ye, Weidong Chen, Bingyan Xu +1VLM RobustnessMultimodal Reasoning

    University of Science and Technology of China, Hefei

  25. FigAct: Turning Scientific Figures into Active Canvases for Explanation

    Sep 28, 2026Shishi Xiao, Zichao Wang, Alexa Siu +2Scientific VisualizationVisually Grounded Reasoning

    Brown University · Adobe Research

  26. Look Before You Judge: Training-Free Region Mining for Grounded and Explainable Deepfake Detection

    Sep 28, 2026Chia-Ling Chen, Yu-Ting Ta, Jian-Yu Jiang-Lin +8Explainable Deepfake DetectionDeepfake Detection

    National Taiwan University · National Tsing Hua University · National Yang Ming Chiao Tung University

  27. Probe to Act: Elevating Browser-Use Agent via Active Visual Probing

    Sep 27, 2026Keliang Li, Heng Wang, Chen Hu +3Active PerceptionVisual Grounding

    Institute of Computing Technology, Chinese Academy of Sciences · University of Chinese Academy of Sciences · StepFun

  28. FORUM: Frozen Outputs Reconciled Using Model Agreement for Visual Grounding

    Sep 27, 2026Taiyo Sato, Takamasa Sanda, Keisuke Maeda +3Referring Expression ComprehensionVisual Grounding

    Hokkaido University, Sapporo, Japan · Neurogica Inc., Tokyo, Japan

  29. FocusDrive: Reasoning with Visual Focus for Autonomous Driving

    Sep 27, 2026Zhiyuan Liu, Zehong Ke, Yuanxin Tian +7Autonomous Driving PlanningVisually Grounded Reasoning

    Tsinghua University

  30. InterTab: Interleaved Visual-Structure Alignment for Multi-Modal Table Reasoning

    Sep 26, 2026Hanqian Li, Sirui Huang, Chen Ling +12Table QAMultimodal CoT Reasoning

    The Hong Kong University of Science and Technology (Guangzhou) · Ant Group · Hong Kong Polytechnic University +4