Large Vision-Language Models

Also known as LVLM

Ask about this topic

Overview

Large vision-language models combine a visual encoder with a language model, aligning them through feature projection, multimodal pretraining, and instruction tuning. Researchers improve their abilities with reinforcement learning, latent visual reasoning, and tools for grounding answers in image regions, especially in visual question answering, OCR, charts, and long-video understanding. To cut inference costs, methods prune visual tokens, select frames, or inspect high-resolution images selectively; other work targets hallucinations, bias, privacy, and adversarial vulnerabilities through data curation, attention interventions, and model editing. Performance is improving across many benchmarks, but reliable fine-grained spatial and temporal reasoning remains difficult, and fluent answers do not always reflect correct use of visual evidence.

Latest papers

CardsList

404 papers44 papers in the last four weeks, up 63% on the four weeks before. 0.3% of all new papers.

  1. GeoHeight-Bench: Towards Height-Aware Multimodal Reasoning in Remote Sensing

    Mar 26, 2026Xuran Hu, Zhitong Xiong, Zhongcheng Hong +3Remote Sensing Image UnderstandingLarge Vision-Language Models

    KTH Royal Institute of Technology · Technical University of Munich · The Hong Kong University of Science and Technology (Guangzhou)

  2. LLaVA-LE: Large Language-and-Vision Assistant for Lunar Exploration

    Mar 25, 2026Gokce Inal, Pouyan Navard, Alper YilmazVision-Language ModelsLarge Vision-Language Models

    The Ohio State University Columbus, OH, USA

  3. Unlocking Few-Shot Capabilities in LVLMs via Prompt Conditioning and Head Selection

    Mar 25, 2026Adhemar de Senneville, Xavier Bou, Jérémy Anger +2Large Vision-Language ModelsZero-Shot Image Classification

    Université Paris-Saclay, CNRS, ENS Paris-Saclay, Centre Borelli, France · École Polytechnique, AMIAD, France · Institut Universitaire de France

  4. Mitigating Object Hallucinations in LVLMs via Attention Imbalance Rectification

    Mar 25, 2026Han Sun, Qin Li, Peixin Wang +1Object Hallucination in VLMsVLM Hallucination Mitigation

    Shanghai Key Laboratory of Trustworthy Computing, East China Normal University

  5. Revealing Multi-View Hallucination in Large Vision-Language Models

    Mar 25, 2026Wooje Park, Insu Lee, Soohyun Kim +4Visual Question AnsweringHallucination in Language Models

    Seoul National University · Sungkyunkwan University

  6. Attention-aware Inference Optimizations for Large Vision-Language Models with Memory-efficient Decoding

    Mar 25, 2026Fatih Ilhan, Gaowen Liu, Ramana Rao Kompella +5Efficient VLM InferenceLarge Vision-Language Models

    Georgia Institute of Technology · Cisco Research

  7. VISion On Request: Enhanced VLLM efficiency with sparse, dynamically selected, vision-language interactions

    Mar 24, 2026Adrian Bulat, Alberto Baldrati, Ioannis Maniadis Metaxas +2Efficient VLM InferenceVision-Language Models

    Samsung AI Cambridge · Technical University of Iasi · Queen Mary University of London

  8. DetPO: In-Context Learning with Multi-Modal LLMs for Few-Shot Object Detection

    Mar 24, 2026Gautam Rajendrakumar Gare, Neehar Peri, Matvei Popov +3Large Vision-Language ModelsFew-Shot Object Detection

    Carnegie Mellon University · Roboflow

  9. Focus, Don't Prune: Identifying Instruction-Relevant Regions for Information-Rich Image Understanding

    Mar 24, 2026Mincheol Kwon, Minseung Lee, Seonga Choi +7Efficient VLM InferenceLarge Vision-Language Models

    Korea University · KT Corporation · Soongsil University +1

  10. Unified Spatiotemporal Token Compression for Video-LLMs at Ultra-Low Retention

    Mar 23, 2026Junhao Du, Jialong Xue, Anqi Li +2Video Token CompressionLarge Vision-Language Models

    Shanghai Jiao Tong University · Beijing University of Posts and Telecommunications

  11. Getting to the Point: Pointing Improves LVLMs at Counting

    Mar 23, 2026Simone Alghisi, Massimo Rizzoli, Seyed Mahed Mousavi +1VLM ReasoningLarge Vision-Language Models

    Signals and Interactive Systems Lab, University of Trento, Italy

  12. Rethinking Token Reduction for Large Vision-Language Models

    Mar 23, 2026Yi Wang, Haofei Zhang, Qihan Huang +7Efficient VLM InferenceVisual Token Compression

    College of Computer Science and Technology, Zhejiang University · State Key Laboratory of Blockchain and Data Security, Zhejiang University · Hangzhou High-Tech Zone (Binjiang) Institute of Blockchain and Data Security +3

  13. Which Concepts to Forget and How to Refuse? Decomposing Concepts for Continual Unlearning in Large Vision-Language Models

    Mar 23, 2026Hyundong Jin, Dongyoon Han, Eunwoo KimVLM UnlearningMachine Unlearning

    Chung-Ang University · NAVER AI Lab

  14. ResPrune: Text-Conditioned Subspace Reconstruction for Visual Token Pruning in Large Vision-Language Models

    Mar 22, 2026Xu Li, Yi Zheng, Yuxuan Liang +5Visual Token PruningEfficient VLM Inference

    College of Computer Science and Artificial Intelligence, Fudan University, Shanghai 200433, China

  15. Adaptive Greedy Frame Selection for Long Video Understanding

    Mar 20, 2026Yuning Huang, Xiaoyu Ji, Joseph Huang +2Query-Aware Video Frame SelectionAdaptive Video Frame Sampling

    Purdue University, West Lafayette, IN 47906

  16. CurveStream: Boosting Streaming Video Understanding in MLLMs via Curvature-Aware Hierarchical Visual Memory Management

    Mar 20, 2026Chao Wang, Xudong Tan, Jianjian Cao +2Streaming Video UnderstandingMemory-Augmented Video Understanding

    College of Future Information Technology, Fudan University, Shanghai, China · Shanghai Innovation Institute, Shanghai, China

  17. Instruction-Free Tuning of Large Vision Language Models for Medical Instruction Following

    Mar 19, 2026Myeongkyun Kang, Soopil Kim, Xiaoxiao Li +1Medical VLMsLarge Vision-Language Models

    Department of Electrical and Computer Engineering, The University of British Columbia, Vancouver, BC V6T 1Z4, Canada · Department of Robotics and Mechatronics Engineering, Daegu Gyeongbuk Institute of Science and Technology (DGIST), Daegu 42988, Republic of Korea · Division of Intelligent Robot, Daegu Gyeongbuk Institute of Science and Technology (DGIST), Daegu 42988, Republic of Korea +2

  18. Perceptio: Perception Enhanced Vision Language Models via Spatial Token Generation

    Mar 19, 2026Yuchen Li, Amanmeet Garg, Shalini Chaudhuri +2Large Vision-Language ModelsVisual Spatial Reasoning

    Amazon

  19. Counting Circuits: Mechanistic Interpretability of Visual Reasoning in Large Vision-Language Models

    Mar 19, 2026Liwei Che, Zhiyu Xue, Yihao Quan +7VLM ReasoningLarge Vision-Language Models

    Rutgers University · UC Santa Barbara · University of Washington +1

  20. Kestrel: Grounding Self-Refinement for LVLM Hallucination Mitigation

    Mar 17, 2026Jiawei Mao, Hardy Chen, Haoqin Tu +7VLM Hallucination MitigationLarge Vision-Language Models

    UC Santa Cruz · UC Berkeley · UNC-Chapel Hill +1

  21. Understanding Pruning Regimes in Vision-Language Models Through Domain-Aware Layer Selection

    Mar 17, 2026Saeed Khaki, Nima Safaei, Kamal GinotraLVLM PruningNeural Network Pruning

    Ohio State University

  22. Segmentation-Based Attention Entropy: Detecting and Mitigating Object Hallucinations in Large Vision-Language Models

    Mar 17, 2026Jiale Song, Jiaxin Luo, Xue-song Tang +2Object Hallucination in VLMsVLM Hallucination Mitigation

    School of Information and Intelligent Science, Donghua University, Shanghai, China

  23. Parallel In-context Learning for Large Vision Language Models

    Mar 17, 2026Shin'ya Yamaguchi, Daiki Chijiwa, Tamao Sakao +1Product of ExpertsIn-Context Learning

    NTT

  24. Mostly Text, Smart Visuals: Asymmetric Text-Visual Pruning for Large Vision-Language Models

    Mar 16, 2026Sijie Li, Biao Qian, Jungong HanLVLM PruningNeural Network Pruning

    University of Sheffield, UK · Tsinghua University, China

  25. Multi-turn Physics-informed Vision-language Model for Physics-grounded Anomaly Detection

    Mar 16, 2026Yao Gu, Xiaohao Xu, Yingna WuReasoning for VLMsVideo Anomaly Detection

    Shanghaitech University · University of Michigan, Ann Arbor

  26. MMSpec: Benchmarking Speculative Decoding for Vision-Language Models

    Mar 16, 2026Hui Shen, Xin Wang, Ping Zhang +11Speculative DecodingEfficient VLM Inference

    University of Michigan · The Ohio State University · Independent +5

  27. UAVBench and UAVIT-1M: Benchmarking and Enhancing MLLMs for Low-Altitude UAV Vision-Language Understanding

    Mar 15, 2026Yang Zhan, Yuan YuanVision-Language ModelsVLM Evaluation

    School of Artificial Intelligence, Optics, and Electronics (iOPEN), Northwestern Polytechnical University, Xi’an 710072, China.

  28. UVLM: A Universal Vision-Language Model Loader for Reproducible Multimodal Benchmarking

    Mar 14, 2026Joan Perez, Giovanni FuscoVLM EvaluationVLM Inference

    Urban Geo Analytics, France · Université Côte d’Azur–CNRS–AMU–Avignon Université, ESPACE, France

  29. Visual-ERM: Reward Modeling for Visual Equivalence

    Mar 13, 2026Ziyu Liu, Shengyuan Ding, Xinyu Fang +7Reward ModelingRL for VLMs

    Shanghai Jiao Tong University · Shanghai AI Laboratory · Fudan University +1

  30. Neural Gate: Mitigating Privacy Risks in LVLMs via Neuron-Level Gradient Gating

    Mar 13, 2026Xiangkui Cao, Jie Zhang, Meina Kan +2Large Vision-Language ModelsPrivacy Protection in VLMs

    State Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences University of Chinese Academy of Sciences · University of Chinese Academy of Sciences