Large Vision-Language Models

Also known as LVLM

Follow this topic Ask about this topic

Momentum

30 papers in the last four weeks, up 36% on the four weeks before. 0.3% of all new papers.

Jul 13Week of Sep 28

Latest papers 253

All topics

Mitigating Action-Relation Hallucinations in LVLMs via Relation-aware Visual Enhancement
May 12, 2026Zhenxin Qin, Qiang Li, Qingzhuo Wang +3VLM Robustness VLM Hallucination
Tongji University, Shanghai, China
Allegory of the Cave: Measurement-Grounded Vision-Language Learning
May 12, 2026Kepeng Xu, Li Xu, Gang He +1Large Vision-Language Models Vision-Language Grounding
Xidian University · Southwest University of Science and Technology
20/20 Vision Language Models: A Prescription for Better VLMs through Data Curation Alone
May 12, 2026DatologyAI, :, Siddharth Joshi +32Training Data Curation Large Vision-Language Models
DatologyAI Team1
Towards a Large Language-Vision Question Answering Model for MSTAR Automatic Target Recognition
May 11, 2026David F. Ramirez, Tim L. Overman, Kristen Jaskie +2Remote Sensing Image Understanding Remote Sensing VQA
SenSIP Center, School of ECEE, Arizona State University, Tempe, Arizona · Prime Solutions Group, Goodyear, Arizona
Vocabulary Hijacking in LVLMs: Unveiling Critical Attention Heads by Excluding Inert Tokens to Mitigate Hallucination
May 11, 2026Yangneng Chen, Junlin Li, Weijun Yao +4Hallucination in Language Models VLM Hallucination
1Harbin Institute of Technology (Shenzhen), China · 2Huawei Technologies Co., Ltd. · 3The Hong Kong Polytechnic University +1
V-ABS: Action-Observer Driven Beam Search for Dynamic Visual Reasoning
May 11, 2026Zhiwei Ning, Xuanang Gao, Jiaxi Cao +6Large Vision-Language Models Multimodal Reasoning
School of Automation and Intelligent Sensing, Shanghai Jiao Tong University · Institute of Image Processing and Pattern Recognition, Shanghai Jiao Tong University · SenseTime Research +1
ERASE: Eliminating Redundant Visual Tokens via Adaptive Two-Stage Token Pruning
May 11, 2026Yuna Lee, Kyoungho Min, Yulhwa KimEfficient VLM Inference Large Vision-Language Models
Department of Electrical and Computer Engineering, Sungkyunkwan University, Republic of Korea · Department of Semiconductor Systems Engineering, Sungkyunkwan University, Republic of Korea
PACE: Policy-Native Adaptive Decision Timing for Long-Horizon Reasoning
May 11, 2026Chen Li, Zhantao Yang, Fangyi Chen +3Large Vision-Language Models Long-Horizon Planning
Carnegie Mellon University
How Many Visual Tokens Do Multimodal Language Models Need? Scaling Visual Token Pruning with F^3A
May 9, 2026YiJie Huang, Yiqun Zhang, Zhuoyue Jia +7Efficient VLM Inference Large Vision-Language Models
School of Computer Science and Engineering, Northeastern University, Shenyang 110819, China · Shanghai Artificial Intelligence Laboratory · School of Computer and Communication Engineering, Northeastern University, Qinhuangdao 066004, China
Built Environment Reasoning from Remote Sensing Imagery Using Large Vision--Language Models
May 8, 2026Dongdong Wang, Deepak Balakrishnan, Ravi Srinivasan +1Remote Sensing Image Understanding Large Vision-Language Models
Department of Urban and Regional Planning, University of Florida · M.E. Rinker, Sr. School of Construction Management, University of Florida
Operating Within the Operational Design Domain: Zero-Shot Perception with Vision-Language Models
May 8, 2026Berkehan Ünal, Hauke Dierend, Dren Fazlija +1Zero-Shot Learning Large Vision-Language Models
Volkswagen Aktiengesellschaft, Wolfsburg, Germany · L3S Research Center, Leibniz University Hannover, Hanover, Germany. · Faculty of Information Technology, University of Jyväskylä, Jyväskylä, Finland. +2
LithoBench: Benchmarking Large Multimodal Models for Remote-Sensing Lithology Interpretation
May 8, 2026Jun Wang, Fengpeng Li, Hang Dong +2Remote Sensing Image Understanding Large Vision-Language Models
School of Computer Science, China University of Geosciences · PRADA Lab, King Abdullah University of Science and Technology · Department of Computer Science, University of Exeter
From Pixels to Prompts: Vision-Language Models
May 8, 2026Khang Hoang Nhat VoVision-Language Models Large Vision-Language Models
Towards Annotation-Free Validation of MLLMs: A Vision-Language Logical Consistency Metric
May 7, 2026Ying Gu, Mei Chee Leong, Hui Li Tan +3VLM Evaluation Large Vision-Language Models
Institute for Infocomm Research (I2R), Agency for Science, Technology and Research (A*STAR), Singapore
Large Vision-Language Models Get Lost in Attention
May 7, 2026Gongli Xi, Ye Tian, Mengyu Yang +5Transformer FFNs Vision-Language Models
School of Cyberspace Security, Beijing University of Posts and Telecommunications, Beijing, China · State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications, Beijing, China · School of Computer Science (National Pilot Software Engineering School), Beijing University of Posts and Telecommunications, Beijing, China +3
DART: A Vision-Language Foundation Model for Comprehensive Rope Condition Monitoring
May 6, 2026Anju Rani, Daniel Ortiz-Arroyo, Petar DurdevicJoint-Embedding Predictive Architecture Large Vision-Language Models
Department of Energy Aalborg University Esbjerg, Denmark 6700
CAST: Mitigating Object Hallucination in Large Vision-Language Models via Caption-Guided Visual Attention Steering
May 6, 2026Qiming Li, Zekai Ye, Xiaocheng Feng +9Hallucination in Language Models Large Vision-Language Models
1Harbin Institute of Technology · 2Peng Cheng Laboratory · 3Harbin Institute of Technology, Shenzhen
Before Forgetting, Learn to Remember: Revisiting Foundational Learning Failures in LVLM Unlearning Benchmarks
May 5, 2026JuneHyoung Kwon, MiHyeon Kim, Eunju Lee +3VLM Evaluation VLM Unlearning
Department of Artificial Intelligence, Chung-Ang University · KT Corporation · Graduate School of Advanced Imaging Sciences, Multimedia and Film, Chung-Ang University
VIDA: A Dataset for Visually Dependent Ambiguity in Multimodal Machine Translation
May 3, 2026Jingheng Pan, Xintong Wang, Longyue Wang +3Large Vision-Language Models Multimodal Model Evaluation
Department of Informatics, Universität Hamburg · Alibaba Group
Two-Pass Zero-Shot Temporal-Spatial Grounding of Rare Traffic Events in Surveillance Video
May 2, 2026Jiantang HuangTemporal Video Grounding Large Vision-Language Models
Persistent Visual Memory: Sustaining Perception for Deep Generation in LVLMs
May 1, 2026Siyuan Huang, Xiaoye Qu, Yafu Li +6Memory-Augmented VLMs VLM Robustness
Shanghai AI Laboratory · Shanghai Jiao Tong University · The Chinese University of Hong Kong +3
Make Your LVLM KV Cache More Lightweight
May 1, 2026Xihao Chen, Yangyang Guo, Roger ZimmermannKV Caching Efficient VLM Inference
Integrative Sciences and Engineering Programme, National University of Singapore · School of Computing, National University of Singapore
Online Self-Calibration Against Hallucination in Vision-Language Models
May 1, 2026Minghui Chen, Chenxu Yang, Hengjie Zhu +3Vision-Language Models Hallucination in Language Models
Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China · School of Cyber Security, University of Chinese Academy of Sciences, Beijing, China
Revealing the Impact of Visual Text Style on Attribute-based Descriptions Produced by Large Visual Language Models
Apr 30, 2026Xiaomeng Wang, Martha Larson, Zhengyu ZhaoVLM Evaluation Vision-Language Models
Radboud University · Nijmegen, Netherlands · Xi’an Jiaotong University +1
Prefill-Time Intervention for Mitigating Hallucination in Large Vision-Language Models
Apr 28, 2026Chengsheng Zhang, Chenghao Sun, Xinyan Jiang +2Vision-Language Models Hallucination in Language Models
University of Science and Technology of China · Shanghai Advanced Research Institute, Chinese Academy of Sciences, Shanghai, China · University of Chinese Academy of Sciences, Beijing, China
Dynamic Decision Learning: Test-Time Evolution for Abnormality Grounding in Rare Diseases
Apr 27, 2026Jun Li, Mingxuan Liu, Jiazhen Pan +4Medical Imaging Test-Time Adaptation
Technical University of Munich · Munich Center for Machine Learning · University of Trento +3
Structural Pruning of Large Vision Language Models: A Comprehensive Study on Pruning Dynamics, Recovery, and Data Efficiency
Apr 27, 2026Yiran Huang, Lukas Thede, Massimiliano Mancini +2Large Vision-Language Models Structured Pruning
Technical University of Munich, Germany · Munich Center for Machine Learning, Germany. · Helmholtz Munich, Germany. +3
PushupBench: Your VLM is not good at counting pushups
Apr 25, 2026Shengzhi Li, Jiarun Chen, Karun Sharma +2VLM Evaluation Video-Language Model Evaluation
Evian: Towards Explainable Visual Instruction-tuning Data Auditing
Apr 22, 2026Zimu Jia, Mingjie Xu, Andrew Estornell +1Large Vision-Language Models Multimodal Instruction Tuning
1The Hong Kong University of Science and Technology (Guangzhou) · 2ByteDance Seed
Evaluating Remote Sensing Image Captions Beyond Metric Biases
Apr 22, 2026Ziyun Chen, Fan Liu, Liang Yao +3VLM Evaluation Remote Sensing Image Understanding
Hohai University Nanjing, China · Cardiff University Cardiff, UK
Mitigating Hallucinations in Large Vision-Language Models without Performance Degradation
Apr 22, 2026Xingyu Zhu, Junfeng Fang, Shuo Wang +4Vision-Language Models VLM Adaptation
MoE Key Lab of BIPC, University of Science and Technology of China · National University of Singapore
Multi-modal Reasoning with LLMs for Visual Semantic Arithmetic
Apr 21, 2026Chuou Xu, Liya Ji, Qifeng ChenLarge Vision-Language Models VLM Reasoning
The Hong Kong University of Science and Technology
VCE: A zero-cost hallucination mitigation method of LVLMs via visual contrastive editing
Apr 21, 2026Yanbin Huang, Yisen Li, Guiyao Tie +6Large Vision-Language Models Object Hallucination in VLMs
1Huazhong University of Science and Technology · Institute of Artificial Intelligence (TeleAI), China Telecom
ST-Prune: Training-Free Spatio-Temporal Token Pruning for Vision-Language Models in Autonomous Driving
Apr 21, 2026Lin Sha, Haiyun Guo, Tao Wang +4VLMs for Autonomous Driving Efficient VLM Inference
School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing 100049, China · Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China · Carizon, Beijing 100094, China
DUALVISION: RGB-Infrared Multimodal Large Language Models for Robust Visual Reasoning
Apr 20, 2026Abrar Majeedi, Zhiyuan Ruan, Ziyi Zhao +3Cross-Modal Learning Visual Question Answering
University of Wisconsin-Madison · Amazon · Northeastern University
SIF: Semantically In-Distribution Fingerprints for Large Vision-Language Models
Apr 18, 2026Yifei Zhao, Qian Lou, Mengxin ZhengVLM Robustness Language Model Fingerprinting
University of Central Florida
Where Do Vision-Language Models Fail? World Scale Analysis for Image Geolocalization
Apr 17, 2026Siddhant Bharadwaj, Ashish Vashist, Fahimul Aleem +1VLM Evaluation Large Vision-Language Models
University of Central Florida
HyperGVL: Benchmarking and Improving Large Vision-Language Models in Hypergraph Understanding and Reasoning
Apr 17, 2026Yanbin Wei, Chun Kang, Siwei Li +11VLM Evaluation Large Vision-Language Models
1Southern University of Science and Technology · 2Hong Kong University of Science and Technology · 4Beihang University +1
How Do LLMs and VLMs Understand Viewpoint Rotation Without Vision? An Interpretability Study
Apr 16, 2026Zhen Yang, Ping Jian, Zhongbin Guo +5Visual Spatial Reasoning Large Vision-Language Models
School of Computer Science and Technology, Beijing Institute of Technology, Beijing, China · Key Laboratory of Computing Power Network and Information Security, Ministry of Education, Shandong Computer Science Center (National Supercomputer Center in Jinan), Qilu University of Technology (Shandong Academy of Sciences), Jinan, China
IWP: Token Pruning as Implicit Weight Pruning in Large Vision Language Models
Apr 1, 2026Dong-Jae Lee, Sunghyun Baek, Junmo KimEfficient VLM Inference Large Vision-Language Models
KAIST, South Korea
When Sinks Help or Hurt: Unified Framework for Attention Sink in Large Vision-Language Models
Apr 1, 2026Jiho Choi, Jaemin Kim, Sanghwan Kim +2Large Vision-Language Models VLM Interpretability
KAIST, South Korea · Chung-Ang University, South Korea · Technical University of Munich, Germany
GeoHeight-Bench: Towards Height-Aware Multimodal Reasoning in Remote Sensing
Mar 26, 2026Xuran Hu, Zhitong Xiong, Zhongcheng Hong +3Remote Sensing Image Understanding Large Vision-Language Models
KTH Royal Institute of Technology · Technical University of Munich · The Hong Kong University of Science and Technology (Guangzhou)
Revealing Multi-View Hallucination in Large Vision-Language Models
Mar 25, 2026Wooje Park, Insu Lee, Soohyun Kim +4Visual Question Answering Hallucination in Language Models
Seoul National University · Sungkyunkwan University
DetPO: In-Context Learning with Multi-Modal LLMs for Few-Shot Object Detection
Mar 24, 2026Gautam Rajendrakumar Gare, Neehar Peri, Matvei Popov +3Large Vision-Language Models Few-Shot Object Detection
Carnegie Mellon University · Roboflow
Neural Gate: Mitigating Privacy Risks in LVLMs via Neuron-Level Gradient Gating
Mar 13, 2026Xiangkui Cao, Jie Zhang, Meina Kan +2Large Vision-Language Models Privacy Protection in VLMs
State Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences University of Chinese Academy of Sciences · University of Chinese Academy of Sciences
Cultural Counterfactuals: Evaluating Cultural Biases in Large Vision-Language Models with Counterfactual Examples
Mar 2, 2026Phillip Howard, Xin Su, Kathleen C. FraserVLM Evaluation Counterfactual Evaluation of VLMs
Thoughtworks · University of Ottawa
AGMark: Attention-Guided Dynamic Watermarking for Large Vision-Language Models
Feb 10, 2026Yue Li, Xin Yi, Dongsheng Shi +3Large Vision-Language Models Robust Watermarking
East China Normal University Shanghai, China · Hasso Plattner Institute Potsdam, Germany · University of Potsdam Potsdam, Germany +1
SPARC: Separating Perception And Reasoning Circuits for Test-time Scaling of VLMs
Feb 6, 2026Niccolo Avogaro, Nayanika Debnath, Li Mi +6Visual Question Answering Efficient VLM Inference
ETH Zürich · IBM Research · MIT-IBM Watson AI Lab.
Stage-wise Attention-Guided Region Sequencing for Adversarial Attacks on Large Vision-Language Models
Feb 4, 2026Jaehyun Kwak, Nam Cao, Boryeong Cho +3Adversarial Attacks on VLMs Adversarial Attacks
KAIST · KENTECH
A Comparative Evaluation of Large Vision-Language Models for 2D Object Detection under SOTIF Conditions
Jan 30, 2026Ji Zhou, Yilin Ding, Yongqi Zhao +4VLM Evaluation Large Vision-Language Models
Institute of Automotive Engineering, Graz University of Technology, 8010 Graz, Austria · Independent Researcher, 200000 Shanghai, China · School of Intelligent Connected Vehicle, Hubei University of Automotive Technology, 442002 Shiyan, China +1
R3G: A Reasoning-Retrieval-Reranking Framework for Vision-Centric Answer Generation
Jan 25, 2026Zhuohong Chen, Zhengxian Wu, Zirui Liao +6Multimodal RAG Visual Question Answering
1The Shenzhen International Graduate School, Tsinghua University, China · School of Computer Science and Information Engineering, Hefei University of Technology, China · 2State Key Laboratory of Nuclear Power Safety Technology and Equipment, China
From One-to-One to Many-to-Many: Dynamic Cross-Layer Injection for Deep Vision-Language Fusion
Jan 15, 2026Cheng Chen, Yuyu Guo, Pengpeng Zeng +4Vision-Language Models VLM Adaptation
Tongji University, China · Ant Group, China · Shanghai Innovation Institute, China +1
From Words to Wavelengths: VLMs for Few-Shot Multispectral Object Detection
Dec 17, 2025Manuel Nkegoum, Minh-Tan Pham, Élisa Fromont +2VLM Adaptation Multispectral Object Detection
Univ Bretagne Sud, IRISA, UMR 6074, Vannes, France · ATERMES, Montigny-le-Bretonneux, France · Univ Rennes, IRISA, UMR 6074, Rennes, France +1
Improving Semantic Uncertainty Quantification in LVLMs with Semantic Gaussian Processes
Dec 16, 2025Joseph Hoche, Andrei Bursuc, David Brellmann +4Vision-Language Models Uncertainty Quantification
AMIAD, Pôle Recherche, Palaiseau · Safran Tech · University of Liège +3
Efficient-VLN: A Simple yet Strong Baseline for Efficient Vision-Language Navigation
Dec 11, 2025Duo Zheng, Shijia Huang, Yanyang Li +1KV Caching Efficient VLM Inference
The Chinese University of Hong Kong · Weitu AI
Visual Species Recognition with Large Multimodal Models as Post-Hoc Correctors
Dec 10, 2025Tian Liu, Anwesha Basu, James Caverlee +1Large Vision-Language Models Multimodal Large Language Models
Department of Computer Science and Engineering, Texas A&M University · Department of AI, University of Macau
From Hindsight to Foresight: Self-Encouraged Hindsight Distillation for Knowledge-based Visual Question Answering
Nov 14, 2025Yu Zhao, Ying Zhang, Xuhui Sui +4Visual Question Answering CoT Distillation
College of Computer Science, VCIP, DISSec, Nankai University, Tianjin, China · School of Software, Tiangong University, Tianjin, China · Shenzhen Campus of Sun Yat-sen University, Shenzhen, China +1
GUI-AIMA: Aligning Intrinsic Multimodal Attention with a Context Anchor for GUI Grounding
Nov 2, 2025Shijie Zhou, Viet Dac Lai, Hao Tan +4Multimodal Grounding GUI Grounding
University at Buffalo, USA · Adobe Research, USA
Activation-Informed Pareto-Guided Low-Rank Compression for Efficient LLM/VLM
Oct 7, 2025Ryan Solgi, Parsa Madinei, Jiayi Tian +4Model Compression LLM Compression
University of California-Santa Barbara, USA · Amazon, USA
VISOR++: Universal Visual Inputs based Steering for Large Vision Language Models
Sep 29, 2025Ravikumar Balakrishnan, Mansi PhuteLanguage Model Steering Large Vision-Language Models
HiddenLayer, USA · Georgia Institute of Technology, USA

May 12, 2026, cs.CVJ/K move · Enter open · S save

Mitigating Action-Relation Hallucinations in LVLMs via Relation-aware Visual Enhancement

Zhenxin Qin, Qiang Li, Qingzhuo Wang, Ruiyang Qin+2

Tongji University, Shanghai, China

VLM Robustness VLM Hallucination Large Vision-Language Models Object Hallucination in VLMs

Open paper AskPDF arXiv

Large Vision-Language Models (LVLMs) have achieved remarkable performance on diverse vision-language tasks. However, LVLMs still suffer from hallucinations, generating text that contradicts the visual input. Existing research has primarily focused on mitigating object hallucinations, but often overlooks more complex relation hallucinations, particularly action relations involving interactions between objects. In this study, we empirically observe that the primary cause of action-relation hallucinations in LVLMs is the insufficient attention allocated to visual information. Thus, we propose a framework to locate action-relevant image regions and enhance the LVLM's attention to those regions. Specifically, we define the Action-Relation Sensitivity (ARS) score to identify attention heads that are most sensitive to action-relation changes, thereby localizing action-relevant image regions that contain key visual cues. Then, we propose the Relation-aware Visual Enhancement (RVE) method to enhance the LVLM's attention to these action-relevant image regions. Extensive experiments demonstrate that, compared to existing baselines, our method achieves superior performance in mitigating action-relation hallucinations with negligible additional inference cost. Furthermore, it effectively generalizes to spatial-relation hallucinations and object hallucinations.

Momentum

Related topics

Latest papers 253