Vision-Language Models

Also known as VLM

Follow this topic Ask about this topic

Momentum

55 papers in the last four weeks, up 120% on the four weeks before. 0.5% of all new papers.

Jul 13Week of Sep 28

Latest papers 480

All topics

Low-Rank Prompt Learning for Vision-Language Models with Fixed-Token Bases
Sep 8, 2026Tanvir Muntakim Tonoy, Sajjad Ghiasvand, Mahnoosh Alizadeh +1Vision-Language Models Prompt Learning
Department of Electrical and Computer Engineering UC Santa Barbara Santa Barbara, CA 93106, USA
Layer Selection in VLMs for Zero-Shot OOD Detection via Multi-Resolution Entropy Estimation
Sep 8, 2026Shyam Nandan Rai, Francesco Di Salvo, Sebastian Doerrich +1Vision-Language Models Medical VLMs
xAILab Bamberg, University of Bamberg, Bamberg, Germany
SafeRI: Recognition and Intervention for Token-Level Safety Intervention in Large Vision Language Models
Sep 3, 2026Caoyuan Ma, Tian Gu, Wenpu Liu +11Vision-Language Models Large Vision-Language Models
The University of Tokyo · Wuhan University · Shanghai AI Laboratory +4
Deeply Interleaved Text-Image Contexts for Multimodal LLMs Assessment
Sep 2, 2026Zihao Wang, Xi Xiang, Yuwen Sun +5Vision-Language Models Multimodal Model Evaluation
1Harbin Institute of Technology · 2Huawei Noah’s Ark Lab
TrajMind: Chaining Role-Specialized LoRAs for Fast-and-Slow Collective Trajectory Anomaly Diagnosis
Sep 2, 2026Jiahao Wu, Zhenqun Yang, Chen Jason Zhang +1Anomaly Detection Agent Trajectory Analysis
The Hong Kong Polytechnic University · Hong Kong, China
EdiTikZ: Scientific Figure Editing from Revision Trajectories
Sep 1, 2026Christian Greisinger, Zhixue Zhao, Steffen EgerVision-Language Models
University of Technology Nuremberg · University of Sheffield
Reliability Challenges in Diffusion Vision-Language Models
Sep 1, 2026Md. Atabuzzaman, Chris ThomasVLM Evaluation VLM Robustness
Department of Computer Science Virginia Tech
Text Capability Loss in Vision-Language Adaptation: An Attention-Sink Diagnosis
Sep 1, 2026Minsik Choi, Geewook Kim, Young Geun KimVision-Language Models VLM Adaptation
Korea University · NAVER Cloud AI
You Cannot Photograph the Same Street Twice: Reliability Limits in Vision-Language Measurement of Urban Change
Sep 1, 2026Kaizhen TanVLM Evaluation Vision-Language Models
New York University
(V)LMs generalize beyond surface co-occurrence: Evidence from cross-modal number agreement
Aug 31, 2026Zach Studdiford, Kanishka MisraVision-Language Models
Department of Psychology · University of Wisconsin-Madison · Department of Linguistics +1
Slow to See, Slow to Suppress: Understanding the Effects of Modality in Context-Memory Conflicts
Aug 31, 2026Athulith Paraselli, Etha Tianze Hua, Ellie PavlickVision-Language Models Knowledge Conflicts in Language Models
Brown University
Frontier vision-language models have overtaken young adults at detecting AI-generated portraits -- but not their calibration
Aug 31, 2026Sunwhi Kim, Sunyul Kim, Meounggun Jo +1VLM Evaluation Vision-Language Models
Hwasung Medi-Science University, Dept. of Bio-Healthcare · Hwasung Medi-Science University, Dept. of Bio-Healthcare, Republic of Korea · Yonsei University, Graduate School of Engineering, Dept. of Artificial Intelligence +5
Hallucination Mitigation for Large Vision-Language Models via Implicit Feature Stabilization
Aug 30, 2026Aditi Sarker, Rafi Ibn Sultan, Hui Zhu +2VLM Robustness Vision-Language Models
Department of Computer Science, Wayne State University, MI, USA · Institute for AI and Data Science (AIDaS), Wayne State University, MI, USA
Conditional Visual Evidence Utility: State-Dependent Rank Reversals in Frozen Vision-Language Encoders
Aug 28, 2026Yunxuan Fang, Xinhe WangVision-Language Models Visual Search
Beihang University
Do Vision-Language Models Agree on the Affective Qualities of Shape? A Cross-Model Audit for Generative Design Interfaces
Aug 26, 2026Luca Bux, Thiago Rios, Ingo Scholtes +2VLM Evaluation Vision-Language Models
Honda Research Institute Europe GmbH Offenbach am Main, Germany · Julius-Maximilians-Universität Würzburg Würzburg, Germany
CL4D: Contrastive Language-4D Pretraining for Vision-Language Reasoning in Dynamic Scenes
Aug 19, 2026Kumal Hewagamage, Isuranga Senavirathne, Sasika Amarasinghe +4Vision-Language Models Vision-Language Alignment
University of Moratuwa, Sri Lanka · Singapore-MIT Alliance for Research & Technology (SMART) Centre, Singapore · Agency for Science, Technology and Research (A*STAR), Singapore
TRAPSBench: Vision-Language Models Encode but Fail to Express Epistemic Restraint
Aug 13, 2026Fnu Pramono, John Cai, Sourabh KulkarniVLM Evaluation Vision-Language Models
Meta Superintelligence Labs · Reflection AI
Heterogeneous Vision-Language Ensemble with Disagreement-Aware Reranking for Text-Based Person Anomaly Retrieval
Aug 13, 2026Huu-An Vu, Cam Tu Tran Thi, Thanh Toan Le Ngo +5Vision-Language Models Multimodal Reranking
Hanoi University of Science and Technology, Hanoi, Vietnam · GenAI4E Lab · University of Information Technology, VNU-HCM, Ho Chi Minh City, Vietnam +4
Scaling Representation Diversity: Modulated Attention and Reconstructive Regularization for Visual Grounding
Aug 13, 2026Junyi Hu, Tian Bai, Fengyi Wu +7Vision-Language Models Vision-Language Grounding
Department of Automation, Tsinghua University · School of Information and Communication Engineering, University of Electronic Science and Technology of China · Chinalco Digital Intelligence (Chengdu) Technology Co., Ltd. +2
Click2Poly: A VLM for vector mapping buildings and walls
Aug 11, 2026Nicolas Girard, Jawher Ben Abdallah, Arno Gobbin +2Vision-Language Models Building Footprint Extraction
LuxCarta
Once Poisoned, Arbitrarily Controlled: A Programmable Backdoor in VLMs
Aug 11, 2026Tao Lin, Gaojie Jin, Zongxin Liu +2Adversarial Attacks on VLMs Vision-Language Models
Key Laboratory of System Software (Chinese Academy of Sciences), Beijing, China · Institute of Software, Chinese Academy of Sciences, Beijing, China · University of Chinese Academy of Sciences, Beijing, China +2
Order Matters: LVLMs as Judges for Temporal Reasoning in Image Sequences
Aug 11, 2026Martina Ianaro, Guilherme Fernandes, Maurizio Gabbrielli +1VLM Evaluation Vision-Language Models
University of Bologna Bologna, Italy · NOVA School of Science and Technology NOVA Laboratory for Computer Science and Informatics Lisbon, Portugal
MMArt: A Multi-Perspective Multimodal Dataset for Visual Art Understanding
Aug 11, 2026Shuai Wang, Wangyuan Ding, Yixian Shen +5Image Captioning Vision-Language Models
University of Amsterdam, Amsterdam, The Netherlands · Amazon AGI, United States of America · College of Business and Economics, University of Johannesburg, South Africa
Dynamic Context Adapters: Efficiently Infusing History into Vision-and-Language Models
Aug 11, 2026Yuhang Song, Bor-Jiun Lin, Jiaxu Liu +3Memory-Augmented VLMs Vision-Language Models
Department of Computer Science, University of Liverpool · Department of Computer Science, National Tsinghua University · Imperial College London +1
GRASP: Granularity-Aware Region Alignment and Semantic Prototype Learning for Fine-Grained Cross-Modal Understanding in Drone Views
Aug 10, 2026Jiahui Cui, Yan Zhao, Kan Wei +4Vision-Language Models Fine-Grained Image Retrieval
Aerospace Information Research Institute, Chinese Academy of Sciences · Key Laboratory of Target Cognition and Application Technology (TCAT), Beijing, China · University of Chinese Academy of Sciences, Beijing, China +1
Right Answer, Wrong Heat: Explanation-Aware Evaluation and Thermal-Grounded Feedback for MLLMs on Infrared Images
Aug 10, 2026Yongsong Huang, Xiaofeng Liu, Tomo Miyazaki +2Vision-Language Models Vision-Language Grounding
Tohoku University · Yale University
Real Data Closes Synthetic-to-Real Gap in Optical Chemical Structure Recognition
Aug 10, 2026Yani Guan, Dengpan Dong, Zi Wei +4Synthetic-to-Real Domain Adaptation Vision-Language Models
SES AI Corporation
TomaMMU: A Comprehensive Multimodal Understanding Benchmark for Tomato Leaf Diseases
Aug 9, 2026Gia-Han Truong, Khang Nguyen Quoc, Luyl-Da QuachVision-Language Models Plant Disease Classification
Department of Information Technology, FPT University, Can Tho, Vietnam · School of Electrical Engineering, Korea University, Seoul, South Korea
VLZip: Unified Visual and Textual Compression for Interleaved Long-Context Modeling
Aug 9, 2026Yuqi Zhang, Cheng Chen, Yuyu Guo +6Vision-Language Models Efficient VLM Inference
Shanghai Key Lab of Intell. Info. Processing, School of CS, Fudan University, China · Shanghai Innovation Institute, China · Ant Group, China
VectraYX-Vision-1B: A Sub-2B Spanish/LATAM Cybersecurity Vision-Language Model with Structured Visual Reasoning and Native Tool Use
Aug 9, 2026Juan S. SantillanaVLM Evaluation Tool Use in VLMs
Globant

Sep 8, 2026, cs.CVJ/K move · Enter open · S save

Low-Rank Prompt Learning for Vision-Language Models with Fixed-Token Bases

Tanvir Muntakim Tonoy, Sajjad Ghiasvand, Mahnoosh Alizadeh, Ramtin Pedarsani

Department of Electrical and Computer Engineering UC Santa Barbara Santa Barbara, CA 93106, USA

Vision-Language Models Prompt Learning VLM Adaptation Few-Shot Learning Visual Prompt Learning

Open paper AskPDF arXiv

Prompt learning adapts CLIP to downstream recognition by replacing hand-written templates with learned continuous context vectors, which in Context Optimization (CoOp) form a dense prompt matrix

\mathbf{P}\in\mathbb{R}^{m\times d}

trained from only a few examples per class. We study whether this matrix is over-parameterized by factorizing it as

\mathbf{P}=\mathbf{B}\mathbf{A}

, which cuts the trainable prompt parameters from

md

r(m+d)

, and to

rd

once the token-side factor

\mathbf{B}

is fixed. Across seven few-shot benchmarks and two CLIP backbones, low-rank prompts match or improve dense CoOp at far fewer parameters, with the clearest gains on low-shot base-to-new generalization. We then find that the token-side factor need not be learned at all: fixing

\mathbf{B}

to a Gaussian, orthogonal, SVD-derived, or even random basis and training only the embedding-side factor

\mathbf{A}

stays on par with the fully trainable factorization, and a source-trained

\mathbf{B}

offers no advantage over a random one. A prompt-factor asymmetry and a local update-space dimension gap show why fixing

\mathbf{B}

is far less restrictive than fixing

\mathbf{A}

, and a smoothness-only guarantee certifies that optimizing

\mathbf{A}

over a fixed

\mathbf{B}

converges. In the CLIP prompt setting, the embedding-side coefficients carry the adaptation while the token basis can simply be fixed.

Momentum

Related topics

Latest papers 480