Large Vision-Language Models

Also known as LVLM

Ask about this topic

Overview

Large vision-language models combine a visual encoder with a language model, aligning them through feature projection, multimodal pretraining, and instruction tuning. Researchers improve their abilities with reinforcement learning, latent visual reasoning, and tools for grounding answers in image regions, especially in visual question answering, OCR, charts, and long-video understanding. To cut inference costs, methods prune visual tokens, select frames, or inspect high-resolution images selectively; other work targets hallucinations, bias, privacy, and adversarial vulnerabilities through data curation, attention interventions, and model editing. Performance is improving across many benchmarks, but reliable fine-grained spatial and temporal reasoning remains difficult, and fluent answers do not always reflect correct use of visual evidence.

Latest papers

CardsList

404 papers44 papers in the last four weeks, up 63% on the four weeks before. 0.3% of all new papers.

  1. WalkGPT: Grounded Vision-Language Conversation with Depth-Aware Segmentation for Pedestrian Navigation

    Mar 11, 2026Rafi Ibn Sultan, Hui Zhu, Xiangyu Zhou +4Vision-Language GroundingLarge Vision-Language Models

    Department of Computer Science, Wayne State University · Department of Radiation Oncology, Henry Ford Health · Department of Electrical and Computer Engineering, The Ohio State University +1

  2. Fighting Hallucinations with Counterfactuals: Diffusion-Guided Perturbations for LVLM Hallucination Suppression

    Mar 11, 2026Hamidreza Dastmalchi, Aijun An, Ali Cheraghian +1VLM HallucinationVLM Hallucination Mitigation

    York University Toronto, Canada · Macquarie University Sydney, Australia · Northern Illinois University Illinois, USA

  3. Does the Question Really Matter? Training-Free Data Selection for Vision-Language SFT

    Mar 10, 2026Peng Sun, Yi Yang, Huawen Shen +4Multimodal Instruction TuningLarge Vision-Language Models

    Nanjing University · Institute of Information Engineering · North University of China +1

  4. Granulon: Awakening Pixel-Level Visual Encoders with Adaptive Multi-Granularity Semantics for MLLM

    Mar 9, 2026Junyuan Mao, Qiankun Li, Linghao Meng +5Large Vision-Language ModelsFine-Grained Visual Perception

    National University of Singapore · Nanyang Technological University

  5. Energy-Driven Adaptive Visual Token Pruning for Efficient Vision-Language Models

    Mar 6, 2026Jialuo He, Huangxun ChenEfficient VLM InferenceVisual Token Pruning

    The Hong Kong University of Science and Technology (Guangzhou)

  6. AdaIAT: Adaptively Increasing Attention to Generated Text to Alleviate Hallucinations in LVLM

    Mar 5, 2026Li'an Zhong, Ziqiang He, Jibin Zheng +3Attention Intervention in VLMsVLM Hallucination Mitigation

    Sun Yat-Sen University · Foshan University · University of British Columbia

  7. DeepScan: A Training-Free Framework for Visually Grounded Reasoning in Large Vision-Language Models

    Mar 4, 2026Yangfu Li, Hongjian Zhan, Jiawei Chen +3VLM ReasoningVisually Grounded Reasoning

    East China Normal University · Sichuan University · Shanghai AI Laboratory

  8. EvoPrune: Early-Stage Visual Token Pruning for Efficient MLLMs

    Mar 4, 2026Yuhao Chen, Bin Shan, Xin Ye +1Efficient VLM InferenceVisual Token Pruning

    ByteDance, Beijing, China

  9. Specificity-aware reinforcement learning for fine-grained open-world classification

    Mar 3, 2026Samuele Angheben, Davide Berasi, Alessandro Conti +2Image ClassificationFine-Grained Image Classification

    University of Trento · Fondazione Bruno Kessler

  10. Chart Deep Research in LVLMs via Parallel Relative Policy Optimization

    Mar 3, 2026Jiajin Tang, Gaoyang, Wenjie Wang +2Chart UnderstandingLarge Vision-Language Models

    ByteDance · ShanghaiTech University · School of Computer Science and Engineering, Sun Yat-sen University

  11. Think-as-You-See: Streaming Chain-of-Thought Reasoning for Large Vision-Language Models

    Mar 3, 2026Jialiang Zhang, Junlong Tong, Junyan Lin +4VLM ReasoningStreaming Inference

    Institute of Digital Twin, Eastern Institute of Technology, Ningbo · Ocean University of China · Shanghai Jiao Tong University +3

  12. Nodes Are Early, Edges Are Late: Probing Diagram Representations in Large Vision-Language Models

    Mar 3, 2026Haruto Yoshida, Keito Kudo, Yoichi Aoki +4Vision-Language ModelsLarge Vision-Language Models

    Tohoku University · Human Informatics Labs., NTT, Inc.

  13. iGVLM: Dynamic Instruction-Guided Vision Encoding for Question-Aware Multimodal Understanding

    Mar 3, 2026Hanpeng Liu, Yaqian Li, Zidan Wang +5Large Vision-Language ModelsVLM Reasoning

    School of Computer Science and Technology, Huazhong University of Science and Technology, Wuhan, China · Li Auto Inc. · Institute of AI for Industries, Chinese Academy of Sciences.

  14. OmniFashion: Towards Generalist Fashion Intelligence via Multi-Task Vision-Language Learning

    Mar 3, 2026Zhengwei Yang, Andi Long, Hao Li +3Vision-Language ModelsLarge Vision-Language Models

    National Engineering Research Center for Multimedia Software, Institute of Artificial Intelligence, School of Computer Science, Wuhan University · Harbin Institute of Technology

  15. Cultural Counterfactuals: Evaluating Cultural Biases in Large Vision-Language Models with Counterfactual Examples

    Mar 2, 2026Phillip Howard, Xin Su, Kathleen C. FraserVLM EvaluationCounterfactual Evaluation of VLMs

    Thoughtworks · University of Ottawa

  16. Securing the Floor and Raising the Ceiling: A Merging-based Paradigm for Multi-modal Search Agents

    Mar 2, 2026Zhixiang Wang, Jingxuan Xu, Dajun Chen +3Multimodal Search AgentsModel Merging

    Ant Group, Hangzhou, China · Peking University, Beijing, China

  17. AG-VAS: Anchor-Guided Zero-Shot Visual Anomaly Segmentation with Large Multimodal Models

    Mar 1, 2026Zhen Qu, Xian Tao, Xiaoyi Bao +4Zero-Shot Anomaly DetectionMultimodal Anomaly Detection

    Institute of Automation, Chinese Academy of Sciences · School of Artificial Intelligence, University of Chinese Academy of Sciences · Casivision +1

  18. AgilePruner: An Empirical Study of Attention and Diversity for Adaptive Visual Token Pruning in Large Vision-Language Models

    Mar 1, 2026Changwoo Baek, Jouwon Song, Sohyeon Kim +1Visual Token PruningLarge Vision-Language Models

  19. Wavelet-based Frame Selection by Detecting Semantic Boundary for Long Video Understanding

    Feb 28, 2026Wang Chen, Yuhui Zeng, Yongdong Luo +5Long-Video UnderstandingQuery-Aware Video Frame Selection

    Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University · College of Computer and Data Science, Fuzhou University

  20. Self-Correction Inside the Model: Leveraging Layer Attention to Mitigate Hallucinations in Large Vision Language Models

    Feb 28, 2026April FuCross-Layer AttentionSelf-Correction in VLMs

    Independent Researcher

  21. Quant Experts: Token-aware Adaptive Error Reconstruction with Mixture of Experts for Large Vision-Language Models Quantization

    Feb 27, 2026Chenwei Jia, Baoting Li, Xuchong Zhang +3VLM QuantizationNeural Network Quantization

    State Key Laboratory of Human-Machine Hybrid Augmented Intelligence Institute of Artificial Intelligence and Robotics Xi’an Jiaotong University

  22. Venus: Benchmarking and Empowering Multimodal Large Language Models for Aesthetic Guidance and Cropping

    Feb 27, 2026Tianxiang Du, Hulingxiao He, Yuxin PengLarge Vision-Language ModelsImage Editing

    Wangxuan Institute of Computer Technology, Peking University

  23. HiDrop: Hierarchical Vision Token Reduction in MLLMs via Late Injection, Concave Pyramid Pruning, and Early Exit

    Feb 27, 2026Hao Wu, Yingqi Fan, Jinyang Dai +3Efficient VLM InferenceLarge Vision-Language Models

    Institute of Digital Twin, Eastern Institute of Technology, Ningbo · Ningbo Key Laboratory of Spatial Intelligence and Digital Derivative · University of Science and Technology of China +2

  24. LookWise: Knowing When and Where to Look for Fine-Grained Visual Reasoning in Multimodal Large Language Models

    Feb 26, 2026Yuxiang Shen, Hailong Huang, Zhenkun Gao +6VLM ReasoningLarge Vision-Language Models

    the Institute of Intelligent Machines, Hefei Institutes of Physical Science, Chinese Academy of Sciences, Hefei 230031, China, and also with the University of Science and Technology of China, Hefei 230026, China · Zhejiang University, Hangzhou 310027, China · East China Normal University, Shanghai 200241, China +3

  25. HulluEdit: Single-Pass Evidence-Consistent Subspace Editing for Mitigating Hallucinations in Large Vision-Language Models

    Feb 26, 2026Yangguang Lin, Quan Fang, Yufei Li +3VLM Hallucination MitigationObject Hallucination in VLMs

    Beijing University of Posts and Telecommunications, Beijing, China · Institute of Automation, Chinese Academy of Sciences, Beijing, China · Beijing Jiaotong University, Beijing, China

  26. Spectrally Distilled Representations Aligned with Instruction-Augmented LLMs for Satellite Imagery

    Feb 26, 2026Minh Kha Do, Wei Xiang, Kang Han +5Remote Sensing VLMsRemote Sensing Image Understanding

    La Trobe University, Melbourne, VIC 3086, Australia · Cisco Research, San Jose, CA, USA

  27. NoLan: Mitigating Object Hallucinations in Large Vision-Language Models via Dynamic Suppression of Language Priors

    Feb 25, 2026Lingfeng Ren, Weihao Yu, Runpeng Yu +1Object Hallucination in VLMsVLM Hallucination Mitigation

    National University of Singapore, Singapore · Peking University Shenzhen Graduate School, China

  28. TranX-Adapter: Bridging Artifacts and Semantics within MLLMs for Robust AI-generated Image Detection

    Feb 25, 2026Wenbin Wang, Yuge Huang, Jianqing Xu +5AI-Generated Image DetectionMultimodal Fusion

    Wuhan University · Tencent YouTu Lab · Singapore Management University

  29. Dynamic Multimodal Activation Steering for Hallucination Mitigation in Large Vision-Language Models

    Feb 25, 2026Jianghao Yin, Qin Chen, Kedi Chen +3VLM Hallucination MitigationLarge Vision-Language Models

    East China Normal University

  30. Dr. Seg: Revisiting GRPO Training for Visual Large Language Models through Perception-Oriented Design

    Feb 25, 2026Haoxiang Sun, Tao Wang, Chenwei Tang +2Large Vision-Language ModelsRL for VLMs

    College of Computer Science, Sichuan University · School of Electronic and Computer Engineering, Peking University