Vision-Language Models

Also known as VLM

Follow this topic Ask about this topic

Momentum

55 papers in the last four weeks, up 120% on the four weeks before. 0.5% of all new papers.

Jul 13Week of Sep 28

Latest papers 480

All topics

Enhancing Continual Learning of Vision-Language Models via Dynamic Prefix Weighting
Apr 20, 2026Hyeonseo Jang, Hyuk Kwon, Kibok LeeClass-Incremental Learning Vision-Language Models
Yonsei University
Mitigating Multimodal Hallucination via Phase-wise Self-reward
Apr 20, 2026Yu Zhang, Chuyang Sun, Kehai Chen +3Vision-Language Models Hallucination in Language Models
Harbin Institute of Technology · Peng Cheng Laboratory · Shenzhen, China
RS-HyRe-R1: A Hybrid Reward Mechanism to Overcome Perceptual Inertia for Remote Sensing Images Understanding
Apr 19, 2026Gaozhi Zhou, Hu He, Peng Shen +8Remote Sensing Image Understanding Visual Question Answering
School of Mechanical and Electrical Engineering, Central South University, Changsha 410083, China · School of Geosciences and Info-Physics, Central South University, Changsha 410083, China
More Than Meets the Eye: Measuring the Semiotic Gap in Vision-Language Models via Semantic Anchorage
Apr 19, 2026Wei HeVLM Evaluation Figurative Language Understanding
IDSAI, University of Exeter Exeter, UK
Towards Joint Quantization and Token Pruning of Vision-Language Models
Apr 19, 2026Xinqing Li, Xin He, Xindong Zhang +3Vision-Language Models VLM Quantization
VCIP, College of Computer Science, Nankai University · School of Computer Science and Engineering, Tianjin University of Technology · OPPO Research Institute +3
LLaTiSA: Towards Difficulty-Stratified Time Series Reasoning from Visual Perception to Semantics
Apr 19, 2026Yueyang Ding, HaoPeng Zhang, Rui Dai +4Vision-Language Models Time Series Reasoning
Amap, Alibaba Group
PestVL-Net: Enabling Multimodal Pest Learning via Fine-grained Vision-Language Interaction
Apr 19, 2026Xueheng Li, Tao Hu, Ke Cao +5Agricultural Image Analysis Vision-Language Models
Institute of Intelligent Machines, Hefei Institutes of Physical Science, Chinese Academy of Sciences · University of Science and Technology of China · 3Zhongke Hefei Institute of Technology Innovation Engineering
Cross-Modal Attention Analysis and Optimization in Vision-Language Models: A Study on Visual Reliability
Apr 19, 2026Lijie ZhouCross-Modal Learning VLM Robustness
School of Computer Science University of Nottingham Ningbo China Ningbo, China
DO-Bench: An Attributable Benchmark for Diagnosing Object Hallucination in Vision-Language Models
Apr 18, 2026JiYang Wang, Jiawei Chen, Mengqi Xiao +3VLM Evaluation Hallucination Detection in VLMs
Shanghai Key Laboratory of Multidimensional Information Processing, East China Normal University · Zhongguancun Academy
Channel Attention-Guided Cross-Modal Knowledge Distillation for Referring Image Segmentation
Apr 18, 2026Chen YangCross-Modal Learning Cross-Modal Knowledge Distillation
School of Information Science and Technology, TaiShan University, Taian, China
Information Router for Mitigating Modality Dominance in Vision-Language Models
Apr 17, 2026Seulgi Kim, Mohit Prabhushankar, Ghassan AlRegibVLM Robustness Vision-Language Models
OLIVES at the Center for Signal and Information Processing CSIP, School of Electrical and Computer Engineering, Georgia Institute of Technology, Atlanta, GA, USA
Do Vision-Language Models Truly Perform Vision Reasoning? A Rigorous Study of the Modality Gap
Apr 17, 2026Yige Xu, Yongjie Wang, Zizhuo Wu +3Vision-Language Models VLM Adaptation
College of Computing and Data Science, Nanyang Technological University, Singapore · Alibaba-NTU Global e-Sustainability CorpLab (ANGEL) · Tongyi Lab, Alibaba Group, China
AstroVLM: Expert Multi-agent Collaborative Reasoning for Astronomical Imaging Quality Diagnosis
Apr 17, 2026Yaohui Han, Tianshuo Wang, Zixi Zhao +6Vision-Language Models VLM Reasoning
The Chinese University of Hong Kong · Central South University · Huawei Technologies Co., Ltd +1
VisPCO: Visual Token Pruning Configuration Optimization via Budget-Aware Pareto-Frontier Learning for Vision-Language Models
Apr 16, 2026Huawei Ji, Yuanhao Sun, Yuan Jin +4Vision-Language Models Efficient VLM Inference
Shanghai Jiao Tong University, Shanghai, China · University of Edinburgh, Edinburgh, UK
Reasoning Dynamics and the Limits of Monitoring Modality Reliance in Vision-Language Models
Apr 16, 2026Danae Sánchez Villegas, Samuel Lewis-Lim, Nikolaos Aletras +1CoT Faithfulness Vision-Language Models
University of Copenhagen, Department of Computer Science · University of Sheffield, School of Computer Science
AIM: Asymmetric Information Masking for Visual Question Answering Continual Learning
Apr 16, 2026Peifeng Zhang, Zice Qiu, Donghua Yu +4Visual Question Answering Vision-Language Models
Sun Yat-Sen University · National Supercomputing Center in Shenzhen · Tsinghua University
Switch-KD: Visual-Switch Knowledge Distillation for Vision-Language Models
Apr 16, 2026Haoyi Sun, Xiaoxiao Wang, Ning Mao +5Cross-Modal Knowledge Distillation Vision-Language Models
Li Auto Inc.
BARD: Bridging AutoRegressive and Diffusion Vision-Language Models Via Highly Efficient Progressive Block Merging and Stage-Wise Distillation
Apr 15, 2026Baoyou Chen, Hanchen Xia, Peng Tu +5Vision-Language Models VLM Adaptation
Shanghai Academy of AI for Science · Fudan University · Shanghai Innovation Institute +1
Beyond Attention Scores: SVD-Based Vision Token Pruning for Efficient Vision-Language Models
Apr 13, 2026Yvon Apedo, Martyna Poreba, Michal Szczepanski +1Vision-Language Models Efficient VLM Inference
Universit´e Paris-Saclay, Univ Evry, IBISC, 91020, Evry-Courcouronnes, France · Universit´e Paris-Saclay, CEA, List, F-91120, Palaiseau, France
VisionFoundry: Teaching VLMs Visual Perception with Synthetic Images
Apr 10, 2026Guanyu Zhou, Yida Yin, Wenhao Chai +3Visual Question Answering Synthetic Data Augmentation
Princeton University · New York University
Overconfidence and Calibration in Medical VQA: Empirical Findings and Hallucination-Aware Mitigation
Apr 2, 2026Ji Young Byun, Young-Jin Park, Jean-Philippe Corbeil +1Vision-Language Models VLM Hallucination
Johns Hopkins University, School of Medicine · Massachusetts Institute of Technology · Microsoft Healthcare & Life Sciences
BUZZY: Contrastive Scoring to Mitigate Text-Induced Bias in Multimodal Multiple-Choice QA
Mar 30, 2026Taeyun Roh, Suhyeong Park, Dongyoung Lee +4Vision-Language Models Multimodal QA
Korea University · AIGEN Sciences · KAIST +1
Uncertainty-guided Compositional Alignment with Part-to-Whole Semantic Representativeness in Hyperbolic Vision-Language Models
Mar 23, 2026Hayeon Kim, Ji Ha Jang, Junghun James Kim +1Vision-Language Models Hierarchical Representation Learning
Dept. of Electrical and Computer Engineering, Seoul National University, Republic of Korea · INMC & IPAI, Seoul National University, Republic of Korea
ME-IQA: Memory-Enhanced Image Quality Assessment via Re-Ranking
Mar 21, 2026Kanglong Fan, Tianhe Wu, Wen Wen +6Memory-Augmented VLMs Vision-Language Models
City University of Hong Kong, Hong Kong SAR, China · ByteDance Inc., China
FMMC: Harnessing the Power of Foundation Models for Accurate Material Classification
Mar 18, 2026Qingran Lin, Fengwei Yang, Chaolun ZhuVision-Language Models Synthetic Data Generation
Georgia Institute of Technology · Duke University · Waseda University
Explicit Logic Channel for Validation and Enhancement of MLLMs on Zero-Shot Tasks
Mar 12, 2026Mei Chee Leong, Ying Gu, Hui Li Tan +2VLM Evaluation Vision-Language Models
Institute for Infocomm Research (I2R) · Present address: Institute of Advanced Intelligence and Computing (IAIC), Agency for Science, Technology and Research (A*STAR), Singapore
GreenRFM: Learning a resource-efficient radiology vision-language foundation model via supervision-centric pre-training
Mar 6, 2026Yingtai Li, Shuai Ming, Qiuli Wang +13Medical Imaging Foundation Models Radiology VLMs
School of Biomedical Engineering, Division of Life Sciences and Medicine, University of Science and Technology of China (USTC), Hefei, Anhui, 230026, China · Center for Medical Imaging, Robotics, Analytic Computing & Learning (MIRACLE), Suzhou Institute for Advanced Research, USTC, Suzhou, Jiangsu, 215123, China · Department of Radiology, The First Affiliated Hospital of USTC, Division of Life Sciences and Medicine, USTC, Hefei, Anhui, 230001, China +7
VisNec: Measuring and Leveraging Visual Necessity for Multimodal Instruction Tuning
Mar 1, 2026Mingkang Dong, Hongyi Cai, Jie Li +4Vision-Language Models Data Selection
SJTU · Universiti Malaya · CMU +3
How to Train Your Long-Context Visual Document Model
Feb 16, 2026Austin VeselkaVisual Question Answering Vision-Language Models
LightOn
Robustness of Vision Language Models Against Split-Image Harmful Input Attacks
Feb 8, 2026Md Rafi Ur Rashid, MD Sadik Hossain Shanto, Vishnu Asutosh Dasu +1VLM Robustness Adversarial Attacks on VLMs
Pennsylvania State University · Bangladesh University of Engineering and Technology

Apr 20, 2026, cs.CVJ/K move · Enter open · S save

Enhancing Continual Learning of Vision-Language Models via Dynamic Prefix Weighting

Hyeonseo Jang, Hyuk Kwon, Kibok Lee

Yonsei University

Class-Incremental Learning Vision-Language Models VLM Adaptation Domain-Incremental Learning Prompt Tuning Incremental Learning+2

Open paper AskPDF arXiv

We investigate recently introduced domain-class incremental learning scenarios for vision-language models (VLMs). Recent works address this challenge using parameter-efficient methods, such as prefix-tuning or adapters, which facilitate model adaptation to downstream tasks by incorporating task-specific information into input tokens through additive vectors. However, previous approaches often normalize the weights of these vectors, disregarding the fact that different input tokens require different degrees of adjustment. To overcome this issue, we propose Dynamic Prefix Weighting (DPW), a framework that dynamically assigns weights to prefixes, complemented by adapters. DPW consists of 1) a gating module that adjusts the weights of each prefix based on the importance of the corresponding input token, and 2) a weighting mechanism that derives adapter output weights as a residual of prefix-tuning weights, ensuring that adapters are utilized only when necessary. Experimental results demonstrate that our method achieves state-of-the-art performance in domain-class incremental learning scenarios for VLMs. The code is available at: https://github.com/YonseiML/dpw.

Momentum

Related topics

Latest papers 480