Vision-Language Models

Also known as VLM

Vision-language models study how systems can connect images or videos with language to recognize, describe, retrieve, and reason about visual content. Common approaches include contrastive image-text encoders such as CLIP, generative models that connect vision encoders to language models, and instruction tuning, sometimes combined with retrieval or tools. Researchers evaluate them on image-text retrieval, visual question answering, OCR and document understanding, visual grounding, and video benchmarks. A central challenge is that fluent answers do not guarantee visual grounding: hallucinations, reliance on textual shortcuts, weak compositional and temporal reasoning, and the costs of efficient deployment remain important concerns.

Latest papers

738 papers77 papers in the last four weeks, up 20% on the four weeks before. 0.4% of all new papers.

Integration of Object Detection and Small VLMs for Construction Safety Hazard Identification
Apr 6, 2026Muhammad Adil, Mehmood Ahmed, Muhammad Aqib +3Construction Safety Vision-Language Models
Infrastructure Human Tech (IHT) Lab, Department of Civil and Environmental Engineering, University of Alberta, Edmonton, Alberta, Canada
Assessing Privacy Preservation and Utility in Online Vision-Language Models
Apr 6, 2026Karmesh Siddharam Chaudhari, Youxiang Zhu, Amy Feng +2Privacy Protection in VLMs Vision-Language Models
Department of Computer Science, University of Massachusetts Boston, MA, USA
Watch Before You Answer: Learning from Visually Grounded Post-Training
Apr 6, 2026Yuxuan Zhang, EunJeong Hwang, Huaisong Zhang +8VLM Post-Training Vision-Language Models
University of British Columbia · Vector Institute · Etude AI +4
Is a Picture Worth a Thousand Words? Adaptive Multimodal Fact-Checking with Visual Evidence Necessity
Apr 6, 2026Jaeyoon Jung, Yejun Yoon, Kunwoo ParkAutomated Fact-Checking Vision-Language Models
School of AI Convergence, Soongsil University · MAUM AI Inc. · Department of Intelligent Semiconductors, Soongsil University
Beyond Semantics: Uncovering the Physics of Fakes via Universal Physical Descriptors for Cross-Modal Synthetic Detection
Apr 6, 2026Mei Qiu, Jianqiang Zhao, Yanyun QuAI-Generated Image Detection Vision-Language Models
SDIC Intelligence Information Technology Co., Ltd. · Xia’men University, China
Don't Blink: Evidence Collapse during Multimodal Reasoning
Apr 5, 2026Suresh Raghu, Satwik PandeySelective Prediction for VLMs Vision-Language Models
Independent Researcher, New Delhi, India
Single-agent vs. Multi-agents for Automated Video Analysis of On-Screen Collaborative Learning Behaviors
Apr 4, 2026Likai Peng, Shihui FengVision-Language Models Multi-Agent Collaboration
Faculty of Education, The University of Hong Kong, Pokfulam, Hong Kong · Institute of Data Science, The University of Hong Kong, Pokfulam, Hong Kong
CoME-VL: Scaling Complementary Multi-Encoder Vision-Language Learning
Apr 3, 2026Ankan Deria, Komal Kumar, Xilin He +4Vision-Language Models Multimodal Fusion
Mohamed bin Zayed University of Artificial Intelligence Abu Dhabi, UAE
EffiMiniVLM: A Compact Dual-Encoder Regression Framework
Apr 3, 2026Yin-Loon Khor, Yi-Jie Wong, Yan Chai HumMultimodal Regression Efficient VLM Training
Universiti Malaya, Kuala Lumpur 50603, Malaysia · Universiti Tunku Abdul Rahman, Selangor 43000, Malaysia
Collaborative Multi-Mode Pruning for Vision-Language Models
Apr 3, 2026Zimeng Wu, Yunhong Wang, Donghao Wang +1LVLM Pruning Efficient VLM Inference
State Key Laboratory of Virtual Reality Technology and Systems, Beihang University, Beijing, China · School of Computer Science and Engineering, Beihang University, Beijing, China
Overconfidence and Calibration in Medical VQA: Empirical Findings and Hallucination-Aware Mitigation
Apr 2, 2026Ji Young Byun, Young-Jin Park, Jean-Philippe Corbeil +1Vision-Language Models VLM Hallucination
Johns Hopkins University, School of Medicine · Massachusetts Institute of Technology · Microsoft Healthcare & Life Sciences
Jagle: Building a Large-Scale Japanese Multimodal Post-Training Dataset for Vision-Language Models
Apr 2, 2026Issa Sugiura, Keito Sasagawa, Keisuke Nakao +8Multimodal Dataset Vision-Language Models
Institute of Science Tokyo · NII LLMC · Waseda University +4
Sparse Spectral LoRA: Routed Experts for Medical VLMs
Apr 1, 2026Omid Nejati Manzari, Hojat Asgariandehkordi, Taha Koleilat +2Medical VLMs Vision-Language Models
Concordia University, Montreal, Canada
PDA: Text-Augmented Defense Framework for Robust Vision-Language Models against Adversarial Image Attacks
Apr 1, 2026Jingning Xu, Haochen Luo, Chen LiuVLM Robustness Vision-Language Models
Department of Computer Science City University of Hong Kong
Prompt-Guided Prefiltering for VLM Image Compression
Mar 31, 2026Bardia Azizian, Ivan V. BajicVision-Language Models Learned Image Compression
School of Engineering Science, Simon Fraser University, Burnaby, BC, Canada
Q-Mask: Query-driven Causal Masks for Text Anchoring in OCR-Oriented Vision-Language Models
Mar 31, 2026Longwei Xu, Feng Feng, Shaojie Zhang +7Optical Character Recognition Vision-Language Models
MiLM Plus, Xiaomi Inc
From Skeletons to Semantics: Design and Deployment of a Hybrid Edge-Based Action Detection System for Public Safety
Mar 31, 2026Ganen Sethupathy, Lalit Dumka, Jan SchagenVideo Surveillance Temporal Action Detection
Sopra Steria Germany, Hamburg, Germany · Sopra Steria India, Noida, India
M-MiniGPT4: Multilingual VLLM Alignment via Translated Data
Mar 31, 2026Seung Hun Han, Youssef Mohamed, Mohamed ElhoseinyVision-Language Models Vision-Language Alignment
MBZUAI · KAUST
Hallucination-aware intermediate representation edit in large vision-language models
Mar 31, 2026Wei Suo, Hanzu Zhang, Lijun Zhang +3VLM Hallucination Mitigation Vision-Language Models
School of Computer Science, Northwestern Polytechnical University, China.
ConInfer: Context-Aware Inference for Training-Free Open-Vocabulary Remote Sensing Segmentation
Mar 31, 2026Wenyang Chen, Zhanxuan Hu, Yaping Zhang +2Image Segmentation Remote Sensing Image Segmentation
Yunnan Normal University · Xi’an University of Posts & Telecommunications
Hydra: Unifying Document Retrieval and Generation in a Single Vision-Language Model
Mar 30, 2026Athos GeorgiouVision-Language Models Visual Document Retrieval
Independent Researcher
BUZZY: Contrastive Scoring to Mitigate Text-Induced Bias in Multimodal Multiple-Choice QA
Mar 30, 2026Taeyun Roh, Suhyeong Park, Dongyoung Lee +4Vision-Language Models Multimodal QA
Korea University · AIGEN Sciences · Myongji University +1
Learning to See through Illumination Extremes with Event Streaming in Multimodal Large Language Models
Mar 29, 2026Baoheng Zhang, Jiahui Liu, Gui Zhao +7Vision-Language Models Multimodal Large Language Models
The University of Hong Kong
Seeing the Scene Matters: Revealing Forgetting in Video Understanding Models with a Scene-Aware Long-Video Benchmark
Mar 28, 2026Seng Nam Chen, Hao Chen, Chenglam Ho +4Long-Video Understanding Vision-Language Models
CUHK (SZ) · University of Cambridge · UESTC +3
Diagnosing and Repairing Unsafe Channels in Vision-Language Models via Causal Discovery and Dual-Modal Safety Subspace Projection
Mar 28, 2026Jinhu Fu, Yihang Lou, Qingyi Si +3VLM Safety Vision-Language Models
Beijing University of Posts and Telecommunications · Huawei Technologies Ltd. · Peking University +1
Make Geometry Matter for Spatial Reasoning
Mar 27, 2026Shihua Zhang, Qiuhong Shen, Shizun Wang +2Visual Spatial Reasoning 3D Spatial Reasoning
National University of Singapore, Singapore
Towards GUI Agents: Vision-Language Diffusion Models for GUI Grounding
Mar 27, 2026Shrinidhi Kumbhar, Haofu Liao, Srikar Appalaraju +1Diffusion VLMs GUI Grounding
Arizona State University · AWS Agentic AI
No Hard Negatives Required: Concept Centric Learning Leads to Compositionality without Degrading Zero-shot Capabilities of Contrastive Models
Mar 26, 2026Hai X. Pham, David T. Hoffmann, Ricardo Guerrero +1Vision-Language Models Compositional Reasoning in VLMs
Samsung AI Center Cambridge, UK
CLIP-RD: Relational Distillation for Efficient CLIP Knowledge Distillation
Mar 26, 2026Jeannie Chung, Hanna Jang, Ingyeong Yang +2Cross-Modal Learning Cross-Modal Knowledge Distillation
Ewha Womans University, Seoul 03760, South Korea
LLaVA-LE: Large Language-and-Vision Assistant for Lunar Exploration
Mar 25, 2026Gokce Inal, Pouyan Navard, Alper YilmazVision-Language Models Large Vision-Language Models
The Ohio State University Columbus, OH, USA

Apr 6, 2026J/K move · Enter open · S save

Integration of Object Detection and Small VLMs for Construction Safety Hazard Identification

Muhammad Adil, Mehmood Ahmed, Muhammad Aqib, Vicente A. Gonzalez+2

Infrastructure Human Tech (IHT) Lab, Department of Civil and Environmental Engineering, University of Alberta, Edmonton, Alberta, Canada

Construction Safety Vision-Language Models VLM Reasoning Efficient VLM Inference

Open paper AskPDF arXiv

Accurate and timely identification of construction hazards around workers is essential for preventing workplace accidents. While large vision-language models (VLMs) demonstrate strong contextual reasoning capabilities, their high computational requirements limit their applicability in near real-time construction hazard detection. In contrast, small vision-language models (sVLMs) with fewer than 4 billion parameters offer improved efficiency but often suffer from reduced accuracy and hallucination when analyzing complex construction scenes. To address this trade-off, this study proposes a detection-guided sVLM framework that integrates object detection with multimodal reasoning for contextual hazard identification. The framework first employs a YOLOv11n detector to localize workers and construction machinery within the scene. The detected entities are then embedded into structured prompts to guide the reasoning process of sVLMs, enabling spatially grounded hazard assessment. Within this framework, six sVLMs (Gemma-3 4B, Qwen-3-VL 2B/4B, InternVL-3 1B/2B, and SmolVLM-2B) were evaluated in zero-shot settings on a curated dataset of construction site images with hazard annotations and explanatory rationales. The proposed approach consistently improved hazard detection performance across all models. The best-performing model, Gemma-3 4B, achieved an F1-score of 50.6%, compared to 34.5% in the baseline configuration. Explanation quality also improved significantly, with BERTScore F1 increasing from 0.61 to 0.82. Despite incorporating object detection, the framework introduces minimal overhead, adding only 2.5 ms per image during inference. These results demonstrate that integrating lightweight object detection with small VLM reasoning provides an effective and efficient solution for context-aware construction safety hazard detection.