Vision-Language Models

Also known as VLM

Overview

Vision-language models study how systems can connect images or videos with language to recognize, describe, retrieve, and reason about visual content. Common approaches include contrastive image-text encoders such as CLIP, generative models that connect vision encoders to language models, and instruction tuning, sometimes combined with retrieval or tools. Researchers evaluate them on image-text retrieval, visual question answering, OCR and document understanding, visual grounding, and video benchmarks. A central challenge is that fluent answers do not guarantee visual grounding: hallucinations, reliance on textual shortcuts, weak compositional and temporal reasoning, and the costs of efficient deployment remain important concerns.

Latest papers

CardsList

774 papers37 papers in the last four weeks, down 57% on the four weeks before. 0.3% of all new papers.

  1. UniDrive-WM: Unified Understanding, Planning and Generation World Model for Autonomous Driving

    Jan 7, 2026Zhexiao Xiong, Xin Ye, Burhan Yaman +5Vision-Language ModelsWorld Model Learning

  2. Disentangling Perception and Reasoning in Multimodal LLMs via Reward Design

    Jan 1, 2026Omar Sharif, Eftekhar Hossain, Nikhil Singh +1Reasoning for VLMsVision-Language Models

  3. Parameter Efficient Multimodal Instruction Tuning for Romanian Vision Language Models

    Dec 16, 2025George-Andrei Dima, Răzvan-Alexandru Smădu, Dumitru-Clementin CercelVLM AdaptationImage Captioning

  4. Improving Semantic Uncertainty Quantification in LVLMs with Semantic Gaussian Processes

    Dec 16, 2025Joseph Hoche, Andrei Bursuc, David Brellmann +4Vision-Language ModelsUncertainty Quantification

  5. SpaceDrive: Infusing Spatial Awareness into VLM-based Autonomous Driving

    Dec 11, 2025Peizheng Li, Zhenghao Zhang, David Holtz +6Vision-Language ModelsVLMs for Autonomous Driving

  6. How (Mis)calibrated is your Federated CLIP and what to do about it?

    Dec 3, 2025Mainak Singha, Masih Aminbeidokhti, Paolo Casari +3Vision-Language ModelsVLM Adaptation

  7. Vision-Language Memory for Spatial Reasoning

    Nov 25, 2025Zuntao Liu, Yi Du, Taimeng Fu +3Memory-Augmented VLMsVision-Language Models

  8. T2T-VICL: Cross-Task Visual In-Context Learning via Implicit Text-Driven VLMs

    Nov 20, 2025Shao-Jun Xia, Huixin Zhang, Zhengzhong TuVision-Language ModelsIn-Context Learning

  9. GraphPilot: Grounded Scene Graph Conditioning for Language-Based Autonomous Driving

    Nov 14, 2025Fabian Schmidt, Markus Enzweiler, Abhinav ValadaVision-Language ModelsVLMs for Autonomous Driving

  10. TokenSwap: Backdoor Attack on the Compositional Understanding of Large Vision-Language Models

    Sep 29, 2025Zhifang Zhang, Qiqi Tao, Jiaqi Lv +3VLM RobustnessAdversarial Attacks on VLMs

  11. HERMAN: Hierarchical Representation Matching for CLIP-based Class-Incremental Learning

    Sep 26, 2025Zhen-Hao Xie, Yan Wang, Lan Li +3Class-Incremental LearningVision-Language Models

  12. Image Recognition with Vision and Language Embeddings of VLMs

    Sep 11, 2025Illia Volkov, Nikita Kisel, Klara Janouskova +1Image ClassificationVLM Evaluation

  13. Prompting with Sign Parameters for Low-resource Sign Language Instruction Generation

    Aug 22, 2025Md Tariquzzaman, Md Farhan Ishmam, Saiyma Sittul Muna +2VLM EvaluationVision-Language Models

  14. TAR: Temporal Anchor-Constrained Reasoning for Video Temporal Grounding

    Aug 11, 2025Chaohong Guo, Xun Mo, Yongwei Nie +3Temporal Video GroundingVision-Language Models

  15. SalQ-VLM: Fine-Grained Saliency-Guided Quantization for Vision-Language Models

    Aug 5, 2025Yufei Xue, Yushi Huang, Lunjie Zhu +2Vision-Language ModelsVLM Quantization

  16. LaViDa: A Large Diffusion Language Model for Multimodal Understanding

    May 22, 2025Shufan Li, Konstantinos Kallidromitis, Hritik Bansal +7Multimodal UnderstandingVision-Language Models

  17. Mitigating Hallucinations via Inter-Layer Consistency Aggregation in Large Vision-Language Models

    May 18, 2025Kai Tang, Jinhao You, Yichen Guo +8Vision-Language ModelsHallucination in Language Models

  18. Concept-as-Tree: A Controllable Synthetic Data Framework Makes Stronger Personalized VLMs

    Mar 17, 2025Ruichuan An, Kai Zeng, Ming Lu +5Synthetic Data AugmentationVision-Language Models

  19. Human Cognitive Benchmarks Reveal Foundational Visual Gaps in MLLMs

    Feb 23, 2025Jen-Tse Huang, Dasen Dai, Jen-Yuan Huang +7VLM EvaluationVision-Language Models

  20. REVEAL: Robust Evolution of Vision-Language Models for Explainable AI-Video Detection

    Feb 20, 2025Yun-Yun Tsai, Qingyuan Liu, Ruijian Zha +4Tool Use in VLMsVision-Language Models

  21. Enhanced Vision-Language Models for Diverse Sensor Understanding: Cost-Efficient Optimization and Benchmarking

    Dec 30, 2024Sangyun Chung, Youngjoon Yu, Se Yeon Kim +2VLM EvaluationVision-Language Models

  22. VLMGuard: Bootstrapping Malicious Prompt Detectors from Unlabeled Vision-Language Prompts in the Wild

    Oct 1, 2024Junlin Fang, Wenyu Chen, Reshmi Ghosh +7VLM RobustnessVision-Language Models