Vision-Language Models

Also known as VLM

Follow this topic Ask about this topic

Momentum

55 papers in the last four weeks, up 120% on the four weeks before. 0.5% of all new papers.

Jul 13Week of Sep 28

Latest papers 480

All topics

Same Answer, Different Representations: Hidden instability in VLMs
Feb 6, 2026Farooq Ahmad Wani, Alessandro Suglia, Rohit Saxena +6VLM Evaluation VLM Robustness
Sapienza University of Rome · University of Edinburgh · University of Oxford +5
Model Specific Task Similarity for Vision Language Model Selection via Layer Conductance
Feb 1, 2026Wei Yang, Hong Xie, Tao Tan +3VLM Evaluation Vision-Language Models
School of AI and Data Science USTC
Gated Relational Alignment via Confidence-based Distillation for Efficient VLMs
Jan 30, 2026Yanlong Chen, Amirhossein Habibian, Luca Benini +1Cross-Modal Knowledge Distillation Vision-Language Models
Department of Information Technology and Electrical Engineering, ETH Zurich, Zurich, Switzerland · School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore · Qualcomm AI Research, Amsterdam, the Netherlands +1
LightOnOCR: A 1B End-to-End Multilingual Vision-Language Model for State-of-the-Art OCR
Jan 20, 2026Said Taghadouini, Adrien Cavaillès, Baptiste AubertinVision-Language Models Optical Character Recognition
LightOn
From One-to-One to Many-to-Many: Dynamic Cross-Layer Injection for Deep Vision-Language Fusion
Jan 15, 2026Cheng Chen, Yuyu Guo, Pengpeng Zeng +4Vision-Language Models VLM Adaptation
Tongji University, China · Ant Group, China · Shanghai Innovation Institute, China +1
Zero-Shot Distracted Driver Detection via Vision Language Models with Double Decoupling
Jan 13, 2026Takamichi Miyata, Sumiko Miyata, Andrew MorrisVision-Language Models Autonomous Driving Perception
Chiba Institute of Technology · Chiba, Japan · Institute of Science Tokyo +3
CoMa: Contextual Massing Generation with Vision-Language Models
Jan 13, 2026Evgenii Maslov, Alexandra Vabnits, Vladimir Vorona +6Vision-Language Models Multimodal Generation
FusionBrain Lab, Artificial Intelligence Research Institute, 123112 Moscow, Russia · Research Center of the Artificial Intelligence Institute, Innopolis University, 420500 Innopolis, Russia · ITMO University, Institute of Design and Urban Studies, 197101 Saint Petersburg, Russia +2
UniDrive-WM: Unified Understanding, Planning and Generation World Model for Autonomous Driving
Jan 7, 2026Zhexiao Xiong, Xin Ye, Burhan Yaman +5Vision-Language Models World Model Learning
Bosch Research North America · Bosch Center for Artificial Intelligence (BCAI) · Arizona State University +2
Parameter Efficient Multimodal Instruction Tuning for Romanian Vision Language Models
Dec 16, 2025George-Andrei Dima, Răzvan-Alexandru Smădu, Dumitru-Clementin CercelVisual Question Answering Image Captioning
National University of Science and Technology POLITEHNICA Bucharest, Bucharest, Romania
Improving Semantic Uncertainty Quantification in LVLMs with Semantic Gaussian Processes
Dec 16, 2025Joseph Hoche, Andrei Bursuc, David Brellmann +4Vision-Language Models Uncertainty Quantification
AMIAD, Pôle Recherche, Palaiseau · Safran Tech · University of Liège +3
SpaceDrive: Infusing Spatial Awareness into VLM-based Autonomous Driving
Dec 11, 2025Peizheng Li, Zhenghao Zhang, David Holtz +6Vision-Language Models VLMs for Autonomous Driving
Mercedes-Benz AG · TU Munich · Karlsruhe Institute of Technology +4
How (Mis)calibrated is your Federated CLIP and what to do about it?
Dec 3, 2025Mainak Singha, Masih Aminbeidokhti, Paolo Casari +3Vision-Language Models VLM Adaptation
University of Trento, Italy · ´Ecole de technologie sup´erieure, QC, Canada · AMIAD, Pˆole Recherche, Palaiseau, France +2
Vision-Language Memory for Spatial Reasoning
Nov 25, 2025Zuntao Liu, Yi Du, Taimeng Fu +3Memory-Augmented VLMs Vision-Language Models
Spatial AI & Robotics (SAIR) Lab, University at Buffalo, USA · Stanford University, USA
T2T-VICL: Cross-Task Visual In-Context Learning via Implicit Text-Driven VLMs
Nov 20, 2025Shao-Jun Xia, Huixin Zhang, Zhengzhong TuVision-Language Models In-Context Learning
Duke University · Texas A&M University
GraphPilot: Grounded Scene Graph Conditioning for Language-Based Autonomous Driving
Nov 14, 2025Fabian Schmidt, Markus Enzweiler, Abhinav ValadaVision-Language Models VLMs for Autonomous Driving
Esslingen University of Applied Sciences · University of Freiburg
TokenSwap: Backdoor Attack on the Compositional Understanding of Large Vision-Language Models
Sep 29, 2025Zhifang Zhang, Qiqi Tao, Jiaqi Lv +3VLM Robustness Adversarial Attacks on VLMs
School of Computer Science and Engineering, Southeast University, China · School of Electrical Engineering and Computer Science, University of Queensland, Australia · Information Systems Technology and Design Pillar, Singapore University of Technology and Design, Singapore +2
HERMAN: Hierarchical Representation Matching for CLIP-based Class-Incremental Learning
Sep 26, 2025Zhen-Hao Xie, Yan Wang, Lan Li +3Class-Incremental Learning Vision-Language Models
School of Artificial Intelligence and the State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, China
TAR: Temporal Anchor-Constrained Reasoning for Video Temporal Grounding
Aug 11, 2025Chaohong Guo, Xun Mo, Yongwei Nie +3Temporal Video Grounding Vision-Language Models
South China University of Technology · Guangdong Laboratory of Artificial Intelligence and Digital Economy (SZ) · Bytedance Inc.
SalQ-VLM: Fine-Grained Saliency-Guided Quantization for Vision-Language Models
Aug 5, 2025Yufei Xue, Yushi Huang, Lunjie Zhu +2Vision-Language Models VLM Quantization
Institute of Artificial Intelligence (TeleAI), China Telecom · The Hong Kong University of Science and Technology
ID-Align: RoPE-Conscious Position Remapping for Dynamic High-Resolution Adaptation in Vision-Language Models
May 27, 2025Bozhou Li, Wentao ZhangRotary Positional Embeddings Vision-Language Models
Peking University Beijing, China
LaViDa: A Large Diffusion Language Model for Multimodal Understanding
May 22, 2025Shufan Li, Konstantinos Kallidromitis, Hritik Bansal +7Vision-Language Models Efficient VLM Inference
1UCLA · 2Panasonic AI Research · 4Salesforce Research +1
Mitigating Hallucinations via Inter-Layer Consistency Aggregation in Large Vision-Language Models
May 18, 2025Kai Tang, Jinhao You, Yichen Guo +8Vision-Language Models Hallucination in Language Models
Peking University · University of Pennsylvania · Nanyang Technological University +4
Concept-as-Tree: A Controllable Synthetic Data Framework Makes Stronger Personalized VLMs
Mar 17, 2025Ruichuan An, Kai Zeng, Ming Lu +5Synthetic Data Augmentation Vision-Language Models
1Peking University · 2CUHK MMLab · 3Zhongguancun Academy +1
REVEAL: Robust Evolution of Vision-Language Models for Explainable AI-Video Detection
Feb 20, 2025Yun-Yun Tsai, Qingyuan Liu, Ruijian Zha +4Tool Use in VLMs Vision-Language Models
Columbia University · Rutgers University
Enhanced Vision-Language Models for Diverse Sensor Understanding: Cost-Efficient Optimization and Benchmarking
Dec 30, 2024Sangyun Chung, Youngjoon Yu, Se Yeon Kim +2VLM Evaluation Vision-Language Models
Integrated Vision Language Lab., School of Electrical Engineering, Korea Advanced Institute of Science and Technology (KAIST), 291 Daehak-ro, Yuseong-gu, Daejeon, 34141, Republic of Korea
VLMGuard: Bootstrapping Malicious Prompt Detectors from Unlabeled Vision-Language Prompts in the Wild
Oct 1, 2024Junlin Fang, Wenyu Chen, Reshmi Ghosh +7VLM Robustness Vision-Language Models
College of Computing and Data Science Nanyang Technological University · School of Physical and Mathematical Sciences Nanyang Technological University · Microsoft Corp. +1
Prompting with Sign Parameters for Low-resource Sign Language Instruction Generation
Date pendingMd Tariquzzaman, Md Farhan Ishmam, Saiyma Sittul Muna +2VLM Evaluation Vision-Language Models
Department of Computer Science and Engineering Islamic University of Technology, Bangladesh
CLIP-RD: Relational Distillation for Efficient CLIP Knowledge Distillation
Date pendingJeannie Chung, Hanna Jang, Ingyeong Yang +2Cross-Modal Learning Cross-Modal Knowledge Distillation
Ewha Womans University, Seoul 03760, South Korea
TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
Date pendingPeng Cai, Zhaofan Zou, Shifa Liu +7Vision-Language Models Document Parsing
China Telecom Artificial Intelligence Technology (Beijing) Co., Ltd.
StreamTTT: Reconciling Real-Time Perception and Long-Term Memory in Streaming VLMs
Date pendingJoya Chen, Zeyun Zhong, Mike Zheng ShouMemory-Augmented VLMs Streaming Video Understanding
National University of Singapore · Karlsruhe Institute of Technology

Feb 6, 2026, cs.AIJ/K move · Enter open · S save

Same Answer, Different Representations: Hidden instability in VLMs

Farooq Ahmad Wani, Alessandro Suglia, Rohit Saxena, Aryo Pradipta Gema+5

Sapienza University of Rome · University of Edinburgh · University of Oxford+5

VLM Evaluation VLM Robustness Vision-Language Models Neural Network Robustness Adversarial Robustness

Open paper AskPDF arXiv

The robustness of Vision Language Models (VLMs) is commonly assessed through output-level invariance, implicitly assuming that stable predictions reflect stable multimodal processing. In this work, we argue that this assumption is insufficient. We introduce a representation-aware and frequency-aware evaluation framework that measures internal embedding drift, spectral sensitivity, and structural smoothness (spatial consistency of vision tokens), alongside standard label-based metrics. Applying this framework to modern VLMs across the SEEDBench, MMMU, and POPE datasets reveals three distinct failure modes. First, models frequently preserve predicted answers while undergoing substantial internal representation drift; for perturbations such as text overlays, this drift approaches the magnitude of inter-image variability, indicating that representations move to regions typically occupied by unrelated inputs despite unchanged outputs. Second, robustness does not improve with scale; larger models achieve higher accuracy but exhibit equal or greater sensitivity, consistent with sharper yet more fragile decision boundaries. Third, we find that perturbations affect tasks differently: they harm reasoning when they disrupt how models combine coarse and fine visual cues, but on the hallucination benchmarks, they can reduce false positives by making models generate more conservative answers.

Momentum

Related topics

Latest papers 480