Vision-Language Models

Also known as VLM

Ask about this topic

Overview

Vision-language models study how systems can connect images or videos with language to recognize, describe, retrieve, and reason about visual content. Common approaches include contrastive image-text encoders such as CLIP, generative models that connect vision encoders to language models, and instruction tuning, sometimes combined with retrieval or tools. Researchers evaluate them on image-text retrieval, visual question answering, OCR and document understanding, visual grounding, and video benchmarks. A central challenge is that fluent answers do not guarantee visual grounding: hallucinations, reliance on textual shortcuts, weak compositional and temporal reasoning, and the costs of efficient deployment remain important concerns.

Latest papers

CardsList

738 papers77 papers in the last four weeks, up 20% on the four weeks before. 0.4% of all new papers.

  1. Look Before Acting: Enhancing Vision Foundation Representations for Vision-Language-Action Models

    Mar 16, 2026Yulin Luo, Hao Chen, Zhuangzhe Wu +10Spatial Grounding for VLA ModelsVision-Language-Action Models

    State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University · The Chinese University of Hong Kong · Simplexity Robotics

  2. LightZeroNav: Zero-Shot Vision Language Navigation in Continuous Environments Based on Lightweight VLMs

    Mar 16, 2026Kun Luo, Xiangyu Dong, Xiaoguang Ma +2Vision-Language NavigationVision-Language Models

    Foshan Graduate School of Innovation, Northeastern University, Foshan 528311, China · Faculty of Robot Science and Engineering, Northeastern University, Shenyang 110819, China · QingniaoAI, China +1

  3. Efficient Document Parsing via Parallel Token Prediction

    Mar 16, 2026Lei Li, Ze Zhao, Meng Li +6Efficient VLM InferenceDocument Parsing

    Platform and Content Group, Tencent · Renmin University of China

  4. Confusion-Aware In-Context-Learning for Vision-Language Models in Robotic Manipulation

    Mar 16, 2026Yayun He, Zuheng Kang, Botao Zhao +3VLM RobustnessVision-Language Models

    Ping An Technology (Shenzhen) Co., Ltd. · Shenzhen Bao’an Middle School

  5. UAVBench and UAVIT-1M: Benchmarking and Enhancing MLLMs for Low-Altitude UAV Vision-Language Understanding

    Mar 15, 2026Yang Zhan, Yuan YuanVision-Language ModelsVLM Evaluation

    School of Artificial Intelligence, Optics, and Electronics (iOPEN), Northwestern Polytechnical University, Xi’an 710072, China.

  6. AgriChat: A Multimodal Large Language Model for Agriculture Image Understanding

    Mar 14, 2026Abderrahmene Boudiaf, Irfan Hussain, Sajid JavedAgricultural Image AnalysisVision-Language Models

    Department of Computer Science, Khalifa University of Science and Technology, Abu Dhabi, United Arab Emirates

  7. QTrack: Query-Driven Reasoning for Multi-modal MOT

    Mar 14, 2026Tajamul Ashraf, Tavaheed Tariq, Sonia Yadav +4Referring MOTMultimodal Visual Object Tracking

    King Abdullah University of Science and Technology (KAUST), Saudi Arabia · Gaash Research Lab, National Institute of Technology Srinagar, India · The University of Queensland, Australia

  8. Vision Verification Enhanced Fusion of VLMs for Efficient Visual Reasoning

    Mar 13, 2026Selim Furkan Tekin, Yichang Xu, Gaowen Liu +3VLM Hallucination MitigationVLM Reasoning

    Georgia Institute of Technology · Cisco Systems

  9. OSMDA: OpenStreetMap-based Domain Adaptation for Remote Sensing VLMs

    Mar 12, 2026Stefan Maria Ailuro, Mario Markov, Mohammad Mahdi +3Remote Sensing VLMsVLM Adaptation

    INSAIT, Sofia University "St. Kliment Ohridski" · INSAIT, Sofia University “St. Kliment Ohridski”

  10. Explicit Logic Channel for Validation and Enhancement of MLLMs on Zero-Shot Tasks

    Mar 12, 2026Mei Chee Leong, Ying Gu, Hui Li Tan +2VLM EvaluationVision-Language Models

    Institute for Infocomm Research (I2R) · Present address: Institute of Advanced Intelligence and Computing (IAIC), Agency for Science, Technology and Research (A*STAR), Singapore

  11. DOne: Decoupling Structure and Rendering for High-Fidelity Design-to-Code Generation

    Mar 12, 2026Xinhao Huang, Jinke Yu, Wenhao Xu +5Code GenerationVision-Language Models

    HKUST (Guangzhou), Guangzhou, China · Alibaba Group, Hangzhou, China · HKUST, Hong Kong, China +2

  12. Grammar of the Wave: Towards Explainable Multivariate Time Series Event Detection via Neuro-Symbolic VLM Agents

    Mar 12, 2026Sky Chenwei Wan, Yifei Y. Wang, Tianjun Hou +2Neuro-Symbolic AIVision-Language Models

    AI Lab, SLB, France · LTCI, Télécom Paris, Institut Polytechnique de Paris, France

  13. Enhancing Lightweight Vision Language Models through Group Competitive Learning for Socially Compliant Navigation

    Mar 12, 2026Xinyu Zhang, Atsushi Konno, Toshihiko Yamasaki +1VLMs for RoboticsVision-Language Models

    Hokkaido University, Sapporo, Japan · The University of Tokyo, Tokyo, Japan

  14. GroundCount: Grounding Vision-Language Models with Object Detection for Mitigating Counting Hallucinations

    Mar 11, 2026Boyuan Chen, Minghao Shao, Siddharth Garg +2Vision-Language ModelsVLM Hallucination Mitigation

    Tandon School of Engineering, New York University, NY, USA · eBRAIN Lab, Division of Engineering, New York University Abu Dhabi, UAE

  15. Qianfan-OCR: A Unified End-to-End Model for Document Intelligence

    Mar 11, 2026Daxiang Dong, Mingming Zheng, Dong Xu +17Document Image UnderstandingOptical Character Recognition

  16. R4-CGQA: Retrieval-based Vision Language Models for Computer Graphics Image Quality Assessment

    Mar 11, 2026Zhuangzi Li, Jian Jin, Shilv Cai +1Image Quality AssessmentVision-Language Models

    College of Computing and Data Science, Nanyang Technological University, Singapore

  17. High-Fidelity Text-to-Image Generation from Pre-Trained Vision-Language Models via Distribution-Conditioned Diffusion Decoding

    Mar 11, 2026Ji Woo Hong, Hee Suk Yoon, Gwanhyeong Koo +5T2I GenerationDiffusion-Based Image Generation

    Korea Advanced Institute of Science and Technology (KAIST), Daejeon, Republic of Korea · Microsoft Research Asia (MSRA), Beijing, China

  18. More than the Sum: Panorama-Language Models for Adverse Omni-Scenes

    Mar 10, 2026Weijia Fan, Ruiping Liu, Jiale Wei +7Vision-Language ModelsVisual Question Answering

    Karlsruhe Institute of Technology · Shenzhen University · UCL +1

  19. IntroSVG: Learning from Rendering Feedback for Text-to-SVG Generation via an Introspective Generator-Critic Framework

    Mar 10, 2026Feiyu Wang, Jiayuan Yang, Zhiyuan Zhao +4Text-to-SVG GenerationVision-Language Models

    Fudan University · TeleAI · Northwestern Polytechnical University +1

  20. MM-Zero: Self-Evolving Multi-Model Vision Language Models From Zero Data

    Mar 10, 2026Zongxia Li, Hongyang Du, Chengsong Huang +8Vision-Language ModelsVLM Reasoning

    University of Maryland · Brown University · Washington University in St. Louis +4

  21. Automating Crash Diagram Generation Using Vision-Language Models: A Case Study on Multi-Lane Roundabouts

    Mar 9, 2026Xiao Lu, Hao Zhen, Jidong J. YangVision-Language ModelsIntelligent Transportation Systems

    Smart Mobility and Infrastructure Lab, College of Engineering University of Georgia Athens

  22. Reading ≠\neq Seeing: Diagnosing and Closing the Typography Gap in Vision-Language Models

    Mar 9, 2026Heng Zhou, Ao Yu, Li Kang +5Vision-Language ModelsVLM Evaluation

  23. Visual Self-Fulfilling Alignment: Shaping Safety-Oriented Personas via Threat-Related Images

    Mar 9, 2026Qishun Yang, Shu Yang, Lijie Hu +1VLM Safety AlignmentVLM Safety

    King Abdullah University of Science and Technology · Provable Responsible AI and Data Analytics Lab · China University of Petroleum-Beijing at Karamay +1

  24. Novel Semantic Prompting for Zero-Shot Action Recognition

    Mar 9, 2026Salman Iqbal, Waheed RehmanZero-Shot Action RecognitionVideo Action Recognition

  25. MJ1: Multimodal Judgment via Grounded Verification

    Mar 9, 2026Bhavesh Kumar, Dylan Feng, Leonard TangPosition Bias in VLMsMLLM Evaluation

    Haize Labs

  26. AgriPath: A Systematic Exploration of Architectural Trade-offs for Crop Disease Classification

    Mar 8, 2026Hamza Mooraj, George Pantazopoulos, Alessandro SugliaContrastive Learning for VLMsPlant Disease Classification

    School of Mathematical and Computer Sciences Heriot-Watt University · Institute for Language, Cognition and Computation, School of Informatics University of Edinburgh

  27. Cross-Modal Taxonomic Generalization in (Vision-) Language Models

    Mar 8, 2026Tianyang Xu, Marcelo Sandoval-Castaneda, Karen Livescu +2Vision-Language Models

    Toyota Technological Institute at Chicago · The University of Texas at Austin

  28. The Model Knows Which Tokens Matter: Automatic Token Selection via Noise Gating

    Mar 7, 2026Landi He, Xiaoyu Yang, Lijian XuEfficient VLM InferenceVisual Token Pruning

    Shenzhen University of Advanced Technology

  29. Deep Expert Injection for Anchoring Retinal VLMs with Domain-Specific Knowledge

    Mar 7, 2026Shuai Lu, Meng Wang, Jia Guo +6Vision-Language ModelsMedical VLMs

    Beijing Institute of Technology, Beijing, China · Institute of High Performance Computing, Agency for Science, Technology and Research, Singapore · National University of Singapore, Singapore +2

  30. Vision Language Models Cannot Reason About Physical Transformation

    Mar 7, 2026Dezhi Luo, Yijiang Li, Maijunxian Wang +7Physical ReasoningVision-Language Models

    University of Michigan · University of California San Diego · University of California Berkeley +6