Vision-Language Models

Also known as VLM

Overview

Vision-language models study how systems can connect images or videos with language to recognize, describe, retrieve, and reason about visual content. Common approaches include contrastive image-text encoders such as CLIP, generative models that connect vision encoders to language models, and instruction tuning, sometimes combined with retrieval or tools. Researchers evaluate them on image-text retrieval, visual question answering, OCR and document understanding, visual grounding, and video benchmarks. A central challenge is that fluent answers do not guarantee visual grounding: hallucinations, reliance on textual shortcuts, weak compositional and temporal reasoning, and the costs of efficient deployment remain important concerns.

Latest papers

CardsList

774 papers37 papers in the last four weeks, down 57% on the four weeks before. 0.3% of all new papers.

  1. Probing the "Conscious" Mind: Selective Access and Global Workspace in Vision-Language Models

    Nov 19, 2026Jiahao Huang, Jun WangVision-Language ModelsVLM Interpretability

  2. Learning to Think with Images Faithfully via Adaptive Data Selection

    Nov 19, 2026Yu Zhao, Jiaxin Zhang, Rohit Saxena +3Faithfulness in VLMsTool Use in VLMs

  3. RegionAgent: Two-Stage Preference Optimization for Regional Image Restoration

    Nov 19, 2026Weiyu Zhou, Jiacong Tang, Tao Hu +5Image RestorationAgent-Based Image Restoration

  4. What Does Anti-Hallucination Training Buy? Saying Less, Cheap Pruning, and the Rest

    Nov 19, 2026Lingkai Bu, Qian GaoObject Hallucination in VLMsVLM Hallucination Mitigation

  5. Recurrent Vision Language Models Solve the Shell Game

    Nov 18, 2026Dingqiang Ye, Jiacong Xu, Wen Zhang +2Vision-Language ModelsTemporal Reasoning in VLMs

  6. Teaching VLMs What to Say, Not How to Reason: Rethinking Counterfactual Reasoning in Autonomous Driving

    Nov 18, 2026Hayeon Oh, Donghwan LeeVLMs for Autonomous DrivingVLM Reasoning

  7. ViLFlow: Parallel Vision-Language Generation with Continuous Latent Flows

    Nov 18, 2026Kevin Zhai, Bian Sun, Zhenyi Wang +1Vision-Language ModelsImage Captioning

  8. PReMind: When Personal Memory Meets Visual Hazards

    Nov 18, 2026Yeongtak Oh, Donghyeok Lee, Joonhee Oh +2Memory-Augmented VLMsVision-Language Models

  9. MoMoVL: Modality-Aware Adaptive Compression for On-Device Sparsely-Activated Vision-Language Models

    Nov 18, 2026Dongkeun Lee, Wonjun LeeEfficient VLM InferenceVLM Compression

  10. Know When to Zoom: How Small Is Too Small for Vision-Language Models?

    Nov 17, 2026Siddharth Vohra, Min XuVision-Language ModelsVisual Evidence Sensitivity in VLMs

  11. UNITE: Consolidating Embodied VLM Specialists via Shared-Anchor One-Bit Factorization

    Nov 16, 2026Tao Tao, Anke Tang, Enneng Yang +3VLM MergingVision-Language Models

  12. GEM-MM: Reducing VLM Hallucination with Entropy-Guided Preference Alignment

    Nov 15, 2026Yuanshan Chua, Xuejiao ZhaoVision-Language ModelsVLM Hallucination Mitigation

  13. LogT: Logically Think with Images for Visual Search

    Nov 14, 2026Yanjun Fu, Quanzeng You, Jiadong Guo +6Tool Use in VLMsReasoning for VLMs

  14. VEPO: Mitigating Multilingual Hallucinations in LVLMs via Visual Evidence-guided Preference Optimization

    Nov 13, 2026Chengpeng Fu, Xiaocheng Feng, Wenshuai Huo +4VLM Hallucination MitigationVision-Language Models

  15. Provable Cross-Modal Transfer of Hierarchical Concept Structure in CLIP Embeddings

    Nov 13, 2026Nghia Nguyen, Tianjiao Ding, Lachlan Ewen MacDonald +1Vision-Language ModelsMultimodal Embedding

  16. A General Concept-based Decomposition for Vision-Language Embeddings

    Nov 11, 2026Simone Alberto Peirone, Ortal Senouf, Francesca Pistilli +2Vision-Language ModelsVLM Interpretability

  17. How Hyperbolic Are Hyperbolic Vision-Language Models?

    Nov 10, 2026Freek Byrman, Partha Das, Stijn Verdenius +1Vision-Language ModelsRepresentation Geometry in VLMs

  18. GHiML: VLM-Guided Hierarchical Multi-Label Learning with Applications to Benthic Imagery

    Nov 9, 2026Isaac Xu, Devi Ayyagari, Martin Gillis +3Hierarchical Multi-Label ClassificationImage Classification

  19. When to Trust Adaptation: Contextual Fusion for Transferable Vision-Language Prompting

    Nov 9, 2026Zhenyuan Chen, Wenshuo Tao, Zhuo Huang +3VLM EnsemblingVLM Adaptation

  20. TIB: Sample-wise Tempered Information Bottleneck for Multimodal Attribution beyond Alignment Assumption

    Nov 8, 2026Qiming Huang, Peixi Liu, Jianbo JiaoMultimodal Model InterpretabilityInformation Bottleneck

  21. Vision Language Models for Primary Structure Threat Identification in Wildland Urban Interface Fires

    Nov 7, 2026Zhangding Liu, Neda Mohammadi, John E. TaylorVision-Language ModelsVLM Evaluation

  22. Decoupled Mode Connectivity for Base-to-Novel Generalization in Vision-Language Models

    Nov 6, 2026Imad Eddine Marouf, Khalid Oublal, Enzo Tartaglione +1VLM AdaptationPrompt Learning for VLMs

  23. Describe Your Facial Expressions by Linking Image Encoders and Large Language Models

    Nov 5, 2026Yujian Yuan, Jiabei Zeng, Shiguang ShanImage CaptioningVision-Language Models

  24. Concertina: Embodying Manipulation and Cognition in One Diffusion Vision-Language Model

    Nov 5, 2026Chengcheng Wang, Wei He, Wenhao Li +4Vision-Language-Action ModelsVision-Language Models

  25. UniCompress: From Multi-LoRA Experts to One Compact Model for Edge Intelligence

    Nov 4, 2026Mingkang Liu, Peipei Jiao, Weiming Meng +3VLM CompressionVLM Quantization

  26. RI-Prune: Relation and Information Aware Visual Token Pruning for Multi-Image Vision-Language Models

    Nov 4, 2026Dongjin Lee, Eunju Lee, Jinhee Jang +2Visual Token PruningLVLM Pruning

  27. OCR in the Foundation Model Era: From Specialist Models to Unified Models

    Nov 3, 2026Zhiheng Li, Zongyang Ma, Jiaxian Chen +7Optical Character RecognitionUnified Multimodal Models