Vision-Language Grounding

Follow this topic Ask about this topic

Momentum

44 papers in the last four weeks, up 175% on the four weeks before. 0.4% of all new papers.

Jul 13Week of Sep 28

Latest papers 288

All topics

RefCaptioner: Multi-Reference Image-Grounded Video Captioning
Jul 30, 2026Tengfei Liu, Yang Shi, Yuran Wang +16Video Captioning Vision-Language Grounding
PKU · Kling Team · NUS +7
One Patch Is Enough: Reinforcement-Optimized Visual Token Grounding for MLLM-Based Scene Text Spotting
Jul 30, 2026Rui Tang, Wentao Yang, Peirong Zhang +4Visual Tokenization Scene Text Recognition
South China University of Technology · Guangzhou, China · HiThink Research +1
Hallucinations Leave a Grounding Signature:Verifier-Guided Decoding for Selective Object Correction
Jul 30, 2026Lei Yang, Xinze Liu, Dayan Wu +7Hallucination Detection in VLMs LLM Hallucination Mitigation
Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China · School of Cyber Security, University of Chinese Academy of Sciences, Beijing, China · Department of Applied Mathematics and Statistics, Johns Hopkins University, Baltimore, MD, USA
Prior Directions: Why GUI Grounding Gets Locked in the Past
Jul 29, 2026Weile Gong, Zijian Lu, Mingcai Chen +3Vision-Language Models GUI Grounding
Nanjing University of Posts and Telecommunications, Nanjing, China
Towards Faithful Sentimental Image Captioning via Evidence-Aware Multi-Agent Reasoning
Jul 28, 2026Tiecheng Cai, Zexian Yang, Chao Chen +2Image Captioning Vision-Language Grounding
College of Computer and Data Science, Fuzhou University Fuzhou, China · School of Computer Science and Technology, Harbin Institute of Technology Shenzhen, China
IMPRINT: Image-Conditioned Query Enrichment for Long-Tail Object Goal Navigation
Jul 27, 2026Jelin Raphael Akkara, Filippo Ziliotto, Luciano Serafini +2Vision-Language Grounding Object-Goal Navigation
University of Padova, Italy. · Fondazione Bruno Kessler (FBK), Trento, Italy.
Evidence Attribution in Visual Document Understanding without Coordinates or Region Labels
Jul 27, 2026Zhuchenyang Liu, Yao Zhang, Yu XiaoVLM Hallucination VLM Interpretability
Aalto University · Espoo, Finland
The Visual Bottleneck: Sparse-Frame Adaptation of MLLMs for Joint Spatial-Temporal Video Grounding
Jul 27, 2026Jiameng Zhang, Srikanth MadikeriTemporal Video Grounding VLM Adaptation
University of Zurich
DDVT: Dynamic Dual-level Vision Transformer Fusion Network for Answer Grounding in Visual Question Answering
Jul 27, 2026Yue Zhang, Xiangyu Li, Wanshu Fan +2Visual Question Answering Vision Transformer
National and Local Joint Engineering Laboratory of Computer Aided Design, School of Software Engineering, Dalian University, Dalian, China · School of Computer Science and Technology, Dalian University of Technology, Dalian, China
CRAG-MM-Diagnostics: Enabling Stage-Wise Analysis of Knowledge-Intensive VQA
Jul 23, 2026Hanseok Oh, Parishad BehnamGhader, Benno Krojer +4VLM Evaluation Multimodal RAG
New York University, New York NY, USA · Mila - Quebec AI Institute, Montreal, Canada · McGill University, Montreal, Canada +3
Do Pathology Vision-Language Models Truly See Pathology?
Jul 23, 2026Chengyang Zhang, Wenchuan Zhang, Bo Li +10VLM Evaluation Computational Pathology
College of Computer Science, Sichuan University · Department of Pathology and Institute of Clinical Pathology, West China Hospital, Sichuan University · Department of Computer Science, School of Computing, National University of Singapore +2
ProCap: Prominence-guided Object Rectification for Faithful and Comprehensive Video Captioning
Jul 23, 2026Debjyoti Das Adhikary, Aritra Hazra, Partha Pratim ChakrabartiVideo Captioning Vision-Language Grounding
Department of Computer Science and Engineering, Indian Institute of Technology Kharagpur, Kharagpur 721302, India
WeedExpert-R1: Incentivizing Botanical Reasoning in MLLMs with Reinforcement Learning for Precision Weed Grounding
Jul 17, 2026Zonglin Yang, Wei-Zhen Liang, Nevin Lawrence +4Open-Vocabulary Object Detection Weed Detection
aDepartment of Biological Systems Engineering, University of Nebraska-Lincoln, Lincoln, NE, USA · bPanhandle Research and Extension Center, University of Nebraska-Lincoln, Scottsbluff, NE, USA · cDepartment of Agronomy and Horticulture, University of Nebraska-Lincoln, Lincoln, NE, USA +1
How Do VLMs Fail? Vision-Operation Misalignment in Compositional VQA
Jul 17, 2026Navya Gupta, Bingjie Xu, Avinash Anand +2Visual Question Answering VLM Interpretability
Singapore Institute of Technology Singapore · NVIDIA Singapore
IoU-PD: IoU-Aware Privileged Distillation for Visual Grounding with Multimodal Large Language Models
Jul 17, 2026Xiuyuan Zhu, Ke Lu, Hao Wu +4Cross-Modal Knowledge Distillation Multimodal Large Language Models
University of Chinese Academy of Sciences, Beijing, China · State Key Laboratory of Communication Content Cognition, Beijing, China · Peng Cheng Laboratory, Shenzhen, Guangdong, China +1
Fine-grained CLIP fine-tuning with self-annotated region alignment
Jul 15, 2026Chenyang Zhao, Wei Lin, Antoni B. Chan +1VLM Adaptation Vision-Language Grounding
Department of Computer Science, City University of Hong Kong · Division of Social Science, Hong Kong University of Science & Technology
GHR-VLM: Making Zero-Shot Transit Video Analytics Realizable with Grounded Hybrid Reasoning
Jul 15, 2026Kaicong Huang, Weiheng Oh, Ruimin KeZero-Shot Learning Vision-Language Grounding
Rensselaer Polytechnic Institute Troy, NY, USA
ScanFocus: A Coarse-to-Fine Framework for Spatio-Temporal Video Grounding
Jul 15, 2026Kai Chen, Ming Dai, Wenxuan Cheng +1Temporal Video Grounding Vision-Language Grounding
Southeast University
CoRe: A Comprehensive Framework for Cross-Image Comparative Reasoning in Vision-Language Models
Jul 14, 2026Lin Peng, Cong Wan, Zeyu Guo +2VLM Evaluation Visual Reasoning
Xi’an Jiaotong University Xi’an, China
Instance-Enriched Semantic Maps for Visual Language Navigation
Jul 14, 2026Jiho Hong, Eunae Kang, Sanghyun Kim +1Embodied Navigation Vision-Language Navigation
Department of Mechanical Engineering, Kyung Hee University, Yongin-si, 17104, Republic of Korea · Advanced Institutes of Convergence Technology (AICT), Suwon, 16229, Republic of Korea · School of Mechanical Engineering, Kyungpook National University, Daegu, 41566, Republic of Korea
Actor as Its Own Critic: Unifying Region Understanding and Localization via CycleGRPO
Jul 13, 2026Xin Zhang, Haochen Wang, Yikang Zhou +3Group Relative Policy Optimization Multimodal Large Language Models
National University of Singapore · ByteDance
SynCLIP: Synonym-Coherent Language-Image Pretraining for Robust Open-Vocabulary Dense Perception
Jul 13, 2026Mingjie Xie, Guangjun He, Dongli Xu +5VLM Robustness Vision-Language Grounding
1Beihang University · 2State Key Laboratory of Space Information System and Integrated Application · 3Independent Researcher +3
Evaluation of Multilingual Ability to Use Spatial Deictic Expressions in Vision-Language Models
Jul 8, 2026Kaito Watanabe, Taisei Yamamoto, Tomoki Doi +1Visual Spatial Reasoning Multilingual VLM Evaluation
The University of Tokyo · Riken · Tohoku University
Analysis-by-Proxy: Localization Signals in VLMs Operating as Condition Encoders
Jul 7, 2026Yoav Baron, Sara Dorfman, Roni Paiss +2Visual Place Recognition VLM Interpretability
Tel Aviv University, Tel Aviv, Israel · Google DeepMind.
Segmentation before Answering: Pixel Grounding for MLLM Visual Reasoning
Jul 7, 2026Yake Wei, Yuan Wang, Fengyun Rao +2Vision-Language Grounding VLM Reasoning
Gaoling School of Artificial Intelligence, Renmin University of China · Laboratory of Research on Large Models and Intelligent Governance · Tsinghua University +1
Green for Go, Red for No: Visual Grounding via Semantic Segmentation for VLA Navigation Policies
Jul 6, 2026Adrian Szvoren, Dimitrios Kanoulas, Nilufer TuptukVisual Navigation Vision-Language Navigation
Department of Computer Science University College London · Department of Security and Crime Science University College London
Learning Structured Visual Compositional Representations for Weakly Supervised Referring Expression Comprehension
Jul 6, 2026Lian Xu, Mohammed Bennamoun, Farid Boussaid +3Visual Representation Learning Vision-Language Grounding
The University of Western Australia · Murdoch University · Sun Yat-sen University +1
Do Medical Vision Language Models Actually See? A Counterfactual Grounding Framework and Hard-Negative Contrastive Training for Visually-Reliant Medical VLMs
Jul 4, 2026Anas Zafar, Leema Krishna Murali, Siddhant Bharadwaj +2VLM Evaluation Counterfactual Evaluation
The University of Texas MD Anderson Cancer Center · Eisai Inc. · Cohere Labs Community +1
Token-Based Affordance Grounding with Large Vision-Language Models
Jul 3, 2026Seung Il Lee, Qinqian Lei, Daguang Xu +4Zero-Shot Learning Vision-Language Grounding
University of Mississippi · National University of Singapore · NVIDIA
GeoSearcher: Anchor-Guided Progressive Reasoning for Remote Sensing Visual Grounding with Process Supervision
Jul 1, 2026Dianyu Wang, Peirong Zhang, Xuyang Li +2Vision-Language Grounding VLM Reasoning
Aerospace Information Research Institute, the Key Laboratory of Network Information System Technology (NIST), and the Key Laboratory of Target Cognition and Application Technology (TCAT), Chinese Academy of Sciences, Beijing 100190, China · School of Electronic, Electrical and Communication Engineering, University of Chinese Academy of Sciences, Beijing 100190, China

Jul 30, 2026, cs.CVJ/K move · Enter open · S save

RefCaptioner: Multi-Reference Image-Grounded Video Captioning

Tengfei Liu, Yang Shi, Yuran Wang, Xiaohan Zhang+15

PKU · Kling Team · NUS+7

Video Captioning Vision-Language Grounding

Open paper AskPDF arXiv

Existing video captioning models generate natural descriptions of video content but cannot explicitly ground local visual elements to multiple reference images. We introduce multi-reference image-grounded video captioning, a new task requiring factual video descriptions with phrase-level reference grounding, and propose RefCaptioner, a two-stage post-training framework for this task. RefCaptioner combines mixed-data SFT with Hierarchical Coverage-Discounted GRPO to jointly improve reference selection, phrase-level binding, distractor rejection, and cross-reference consistency while preserving general video-captioning ability. To support training, we construct a corpus containing

20,000

videos and 171,354 reference images. We further introduce MRVBench, a benchmark for evaluating caption factuality and multi-reference grounding on both real-world and AI-generated videos. Experiments show that RefCaptioner achieves the best overall performance among the open-source models while remaining competitive on standard video captioning benchmarks. Human evaluation further confirms that its captions are preferred by annotators and enable more source-faithful video reconstruction with both open-source and proprietary video generators.

Momentum

Related topics

Latest papers 288