Large Vision-Language Models

Also known as LVLM

Large vision-language models combine a visual encoder with a language model, aligning them through feature projection, multimodal pretraining, and instruction tuning. Researchers improve their abilities with reinforcement learning, latent visual reasoning, and tools for grounding answers in image regions, especially in visual question answering, OCR, charts, and long-video understanding. To cut inference costs, methods prune visual tokens, select frames, or inspect high-resolution images selectively; other work targets hallucinations, bias, privacy, and adversarial vulnerabilities through data curation, attention interventions, and model editing. Performance is improving across many benchmarks, but reliable fine-grained spatial and temporal reasoning remains difficult, and fluent answers do not always reflect correct use of visual evidence.

Latest papers

404 papers44 papers in the last four weeks, up 63% on the four weeks before. 0.3% of all new papers.

Mitigating Entangled Steering in Large Vision-Language Models for Hallucination Reduction
Apr 9, 2026Yuanhong Zhang, Zhaoyang Wang, Xin Zhang +2VLM Hallucination VLM Hallucination Mitigation
Ministry of Education Key Laboratory of Intelligent Networks and Network Security, Xi‘an Jiaotong University, Xi‘an, China · Centre for Frontier AI Research, Institute of High Performance Computing, Agency for Science, Technology and Research, Singapore
HAWK: Head Importance-Aware Visual Token Pruning in Multimodal Models
Apr 9, 2026Qihui Zhu, Tao Zhang, Yuchen Wang +9Visual Token Pruning Efficient VLM Inference
University of Science and Technology of China · ChangXin Memory Technologies, Inc · Huawei Noah’s Ark Lab
Q-Zoom: Query-Aware Adaptive Perception for Efficient Multimodal Large Language Models
Apr 8, 2026Yuheng Shi, Xiaohuan Pei, Linfeng Wen +2Efficient VLM Inference Large Vision-Language Models
University of Sydney, Australia · Sun Yat-sen University, China · City University of Hong Kong, China
From Inheritance to Saturation: Disentangling the Evolution of Visual Redundancy for Architecture-Aware MLLM Inference Acceleration
Apr 8, 2026Jiaqi Shi, Yuechan Li, Xulong Zhang +2Efficient VLM Inference Large Vision-Language Models
University of Science and Technology of China · Wuhan University · Ping An Technology (Shenzhen) Co., Ltd., Shenzhen, China
ID-Selection: Importance-Diversity Based Visual Token Selection for Efficient LVLM Inference
Apr 7, 2026Zhaohong Huang, Wenjing Liu, Yuxin Zhang +2Efficient VLM Inference Visual Token Pruning
Xiamen University, 361005, P.R. China.
Efficient Inference for Large Vision-Language Models: Bottlenecks, Techniques, and Prospects
Apr 7, 2026Jun Zhang, Yicheng Ji, Feiyang Ren +7Efficient VLM Inference Large Vision-Language Models
The State Key Laboratory of Blockchain and Data Security, Zhejiang University · Hangzhou High-Tech Zone (Binjiang) Institute of Blockchain and Data Security
Vero: An Open RL Recipe for General Visual Reasoning
Apr 6, 2026Gabriel Sarch, Linrong Cai, Qunzhong Wang +3Reasoning for VLMs Visual Reasoning
Princeton University
Responses Fall Short of Understanding: Revealing the Gap between Internal Representations and Responses in Visual Document Understanding
Apr 6, 2026Haruka Kawasaki, Ryota Tanaka, Kyosuke NishidaLarge Vision-Language Models Document Image Understanding
Human Informatics Labs., NTT, Inc.
RCP: Representation Consistency Pruner for Mitigating Distribution Shift in Large Vision-Language Models
Apr 4, 2026Jianwei Zhang, Chaoning Zhang, Sihan Cao +7Efficient VLM Inference Large Vision-Language Models
School of Computer Science and Engineering, University of Electronic Science and Technology of China, Qingshuihe Campus, No. 2006 Xiyuan Avenue, High-Tech Zone (West), Chengdu 611731, China · Machine Learning Department, Mohamed bin Zayed University of Artificial Intelligence, Building 1B, Masdar City, Abu Dhabi, United Arab Emirates · School of Robotics and Advanced Manufacture, Harbin Institute of Technology, Shenzhen, University Town of Shenzhen, Nanshan District, Shenzhen 518055, China +1
Leveraging Gaze and Set-of-Mark in VLLMs for Human-Object Interaction Anticipation from Egocentric Videos
Apr 4, 2026Daniele Materia, Francesco Ragusa, Giovanni Maria FarinellaHuman-Object Interaction Egocentric Video Understanding
Department of Mathematics and Computer Science – University of Catania, Italy · Next Vision s.r.l. – Spinoff of the University of Catania, Italy
Revealing Physical-World Semantic Vulnerabilities: Universal Adversarial Patch for Infrared Vision-Language Models
Apr 3, 2026Chengyin Hu, Yuxian Dong, Yikun Guo +8Adversarial Attacks on VLMs VLM Robustness
China University of Petroleum-Beijing at Karamay, No. 355 Anding Road, Karamay 834000, Xinjiang, China. · the Defense Innovation Institute, Chinese Academy of Military Science, Beijing, China. · Shenzhen University, Shenzhen, China. +1
WSVD: Weighted Low-Rank Approximation for Fast and Efficient Execution of Low-Precision Vision-Language Models
Apr 2, 2026Haiyu Wang, Yutong Wang, Jack Jiang +1Efficient VLM Inference Low-Rank Compression
Tandon School of Engineering, New York University · Courant Institute of Mathematical Sciences, New York University
Not All Tokens See Equally: Perception-Grounded Policy Optimization for Large Vision-Language Models
Apr 2, 2026Zekai Ye, Qiming Li, Xiaocheng Feng +6RL for VLMs Large Vision-Language Models
Harbin Institute of Technology · Peng Cheng Laboratory · Huawei Technologies Co., Ltd
ACT Now: Preempting LVLM Hallucinations via Adaptive Context Integration
Apr 1, 2026Bei Yan, Yuecong Min, Jie Zhang +2Attention Intervention in VLMs VLM Hallucination Mitigation
State Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China · University of Chinese Academy of Sciences, Beijing, China
IWP: Token Pruning as Implicit Weight Pruning in Large Vision Language Models
Apr 1, 2026Dong-Jae Lee, Sunghyun Baek, Junmo KimEfficient VLM Inference Large Vision-Language Models
KAIST, South Korea
When Sinks Help or Hurt: Unified Framework for Attention Sink in Large Vision-Language Models
Apr 1, 2026Jiho Choi, Jaemin Kim, Sanghwan Kim +2Large Vision-Language Models VLM Interpretability
KAIST, South Korea · Chung-Ang University, South Korea · Technical University of Munich, Germany
First Logit Boosting: Visual Grounding Method to Mitigate Object Hallucination in Large Vision-Language Models
Apr 1, 2026Jiwoo Ha, Jongwoo Baek, Jinhyun SoVLM Hallucination Mitigation Object Hallucination in VLMs
DGIST EECS
In Search of the Ingredients of Open-Endedness: Replicating Picbreeder with Large Vision-Language Models
Apr 1, 2026Sam Earle, Kai Arulkumaran, Andrew Dai +3Open-Ended Generation Large Vision-Language Models
New York University, Brooklyn, New York, USA · Work partially completed during an internship at Sakana AI. · Sakana AI, Tokyo, Japan +3
Hierarchical Pre-Training of Vision Encoders with Large Language Model
Mar 31, 2026Eugene Lee, Ting-Yu Chang, Jui-Huang Tsai +2Vision-Language Pretraining Large Vision-Language Models
University of Cincinnati · National Yang Ming Chiao Tung University
A Comprehensive Information-Decomposition Analysis of Large Vision-Language Models
Mar 31, 2026Lixin Xiu, Xufang Luo, Hideki NakayamaVisual Instruction Tuning Large Vision-Language Models
The University of Tokyo · Microsoft Research
Hallucination-aware intermediate representation edit in large vision-language models
Mar 31, 2026Wei Suo, Hanzu Zhang, Lijun Zhang +3VLM Hallucination Mitigation Vision-Language Models
School of Computer Science, Northwestern Polytechnical University, China.
MolmoPoint: Better Pointing for VLMs with Grounding Tokens
Mar 30, 2026Christopher Clark, Yue Yang, Jae Sung Park +8Vision-Language Grounding Large Vision-Language Models
Allen Institute for AI · University of Washington
Bridging Visual Representation and Reinforcement Learning from Verifiable Rewards in Large Vision-Language Models
Mar 28, 2026Yuhang Han, Yuyang Wu, Zhengbo Jiao +6VLM Reasoning RL for VLMs
EPIC Lab, SJTU · HKUST (GZ) · Huawei
Falcon Perception
Mar 28, 2026Aviraj Bevli, Sofian Chaybouti, Yasser Dahou +6Large Vision-Language Models Long-Context VLMs
Falcon Vision Team, TII
Diagnosing and Repairing Unsafe Channels in Vision-Language Models via Causal Discovery and Dual-Modal Safety Subspace Projection
Mar 28, 2026Jinhu Fu, Yihang Lou, Qingyi Si +3VLM Safety Vision-Language Models
Beijing University of Posts and Telecommunications · Huawei Technologies Ltd. · Peking University +1
The Limits of Learning from Pictures and Text: Vision-Language Models and Embodied Scene Understanding
Mar 27, 2026Gillian Rosenberg, Skylar Stadhard, Bruce C. Hansen +1Large Vision-Language Models VLM Evaluation
Barnard College, Columbia University · Colgate University
Mitigating the Reasoning Tax in Vision-Language Fine-Tuning with Input-Adaptive Depth Aggregation
Mar 27, 2026Yiming Ren, Yujiu Yang, Junjie WangVLM Adaptation VLM Reasoning
Tsinghua University
From Pixels to BFS: High Maze Accuracy Does Not Imply Visual Planning
Mar 27, 2026Alberto G. Rodriguez SalgadoVLM Reasoning Visual Spatial Reasoning
Independent Researcher
VLAgeBench: Benchmarking Large Vision-Language Models for Zero-Shot Human Age Estimation
Mar 27, 2026Rakib Hossain Sajib, Md Kishor Morol, Rajan Das Gupta +2Large Vision-Language Models VLM Evaluation
Department of Computer Science and Engineering, Begum Rokeya University, Rangpur, Rangpur, Bangladesh · EliteLab.AI, Queens, New York, United States · Department of Computer Science, American International University - Bangladesh, Dhaka, Bangladesh
LanteRn: Latent Visual Structured Reasoning
Mar 26, 2026André G. Viveiros, Nuno Gonçalves, Matthias Lindemann +1Latent Visual Reasoning VLM Reasoning
Instituto de Telecomunicações · Instituto Superior Técnico, Universidade de Lisboa · Carnegie Mellon University

Apr 9, 2026J/K move · Enter open · S save

Mitigating Entangled Steering in Large Vision-Language Models for Hallucination Reduction

Yuanhong Zhang, Zhaoyang Wang, Xin Zhang, Weizhan Zhang+1

Ministry of Education Key Laboratory of Intelligent Networks and Network Security, Xi‘an Jiaotong University, Xi‘an, China · Centre for Frontier AI Research, Institute of High Performance Computing, Agency for Science, Technology and Research, Singapore

VLM Hallucination VLM Hallucination Mitigation Large Vision-Language Models

Open paper AskPDF arXiv

Large Vision-Language Models (LVLMs) have achieved remarkable success across cross-modal tasks but remain hindered by hallucinations, producing textual outputs inconsistent with visual content. Existing methods mitigate hallucinations but often alter generation behavior, resulting in shorter outputs and shifted token distributions, especially in latent space steering approaches. We identify that this issue stems from entangled steering signals, where suppressing hallucinations inadvertently disrupts the model's intrinsic generation behavior. To address this, we propose MESA, an effective plug-and-play framework that performs controlled and selective latent intervention for hallucination mitigation. Specifically, MESA targets hallucination-relevant responses while preserving the model's original token distribution, enabling effective hallucination reduction without compromising generation behavior. Extensive experiments across diverse generative and discriminative benchmarks demonstrate that MESA consistently reduces hallucinations while better preserving generation behavior, outperforming prior methods across multiple LVLM families.