VLM Robustness

VLM: Vision-Language Model

Follow this topic Ask about this topic

Momentum

26 papers in the last four weeks, up 117% on the four weeks before. 0.3% of all new papers.

Jul 13Week of Sep 28

Latest papers 295

All topics

Mechanistic Diagnostics of Spatial Lexical Bias in Multimodal Large Language Model Spatial Reasoning
Jun 1, 2026Chuang Ma, Qianying Liu, Tomoyuki Obuchi +6VLM Robustness Visual Spatial Reasoning
Kyoto University · NII LLMC · RIKEN AIP +4
Density-Aware Translation of Spurious Correlations in Zero-Shot VLMs
Jun 1, 2026Afsaneh Hasanebrahimi, Hanxun Huang, Christopher Leckie +1VLM Robustness Vision-Language Models
School of Computing and Information Systems, The University of Melbourne, Victoria, Australia.
Make Your VLA More Robust Without More Data By Interleaving Motion Planning
May 31, 2026Dan BW Choe, Sundhar Vinodh Sangeetha, Samuel Coogan +1VLM Robustness Long-Horizon Robotic Manipulation
Georgia Institute of Technology, Atlanta, GA, USA
RoboStressBench: Benchmarking VLM Robustness to Physical Visual Stress in Embodied Scenes
May 30, 2026Leyi Wu, Yifan Zhao, Jinjie Zhang +11VLM Evaluation VLM Robustness
HKUST(GZ) · Knowin · HKUST
The Regularizing Power of Language-Training Deepfake Detectors
May 29, 2026Benedikt Hopf, Zongwei Wu, Radu TimofteVLM Robustness Explainable Deepfake Detection
Computer Vision Lab, CAIDAS, University of Würzburg, Germany
What Makes LVLMs Hallucinate Less? Unveiling the Architectural Factors Behind Hallucination Robustness
May 29, 2026Yusheng He, Jizhe Zhou, Xia Du +3VLM Evaluation VLM Robustness
School of Computer Science, Engineering Research Center of Machine Learning and Industry Intelligence, Sichuan University, Chengdu, 610020, China · School of Computer and Information Engineering, Xiamen University of Technology, Xiamen, 361000, China · Department of Electrical and Computer Engineering, University of Hong Kong, Pok Fu Lam, Hong Kong, China +1
Immuno-VLM: Immunizing Large Vision-Language Models via Generative Semantic Antibodies for Open-World Trustworthiness
May 29, 2026Xiang Fang, Wanlong Fang, Wei JiVLM Robustness Large Vision-Language Models
School of Software Engineering, Huazhong University of Science and Technology · Nanyang Technological University, Singapore · Nanjing University
LoMo: Local Modality Substitution for Deeper Vision-Language Fusion
May 28, 2026Feng Han, Zhixiong Zhang, Zheming Liang +2Cross-Modal Learning VLM Robustness
Fudan University · Shanghai Innovation Institute · Shanghai Jiao Tong University +1
MuPHI: Learning Implicit Multimodal Harm Reasoning via Semantically Grounded Reward Optimization
May 28, 2026Anisha Saha, Varsha Suresh, Teodora Kamova +3VLM Robustness Multimodal Reasoning
Max Planck Institute for Informatics, Saarland Informatics Campus · Saarland University · The University of Edinburgh +1
ReactBench: A Cause-Driven Benchmark for Multimodal Hallucination via Systematic Evaluation
May 28, 2026Shizhe Zhou, Bohan Jia, Kai Wu +4VLM Robustness Hallucination Detection
East China Normal University
TRACER: Persistent Regularization for Robust Multimodal Finetuning
May 28, 2026Hesam Asadollahzadeh, Feng Liu, Christopher Leckie +1VLM Robustness Neural Network Robustness
School of Computing and Information Systems (CIS), Faculty of Engineering and IT (FEIT), University of Melbourne, Australia.
JECA^2: Judgment-Explanation Consistent Adversarial Attack against Forensic Vision-Language Models
May 27, 2026Jiachen QianVLM Robustness Adversarial Attacks on VLMs
City University of Hong Kong
When Think-with-Image Meets Safety: What Determines Multimodal Jailbreak Robustness?
May 27, 2026Yuan Tian, Bing Hu, Fang Wu +3VLM Robustness Tool Use in VLMs
Independent Researcher · Stanford University · Harvard University +2
Structure-Guided Visual Perturbation Neutralization for LVLMs
May 27, 2026Yuanhe Zhang, Xueting Wang, YanBin Ren +6VLM Robustness Adversarial Attacks on VLMs
1Beijing University of Posts and Telecommunications · University of Science and Technology of China · 3JIUTIAN Research +2
Towards Unified Vision-Language Models with Incomplete Multi-Modal Inputs
May 27, 2026Xiang Fang, Wanlong Fang, Changshuo Wang +4Missing-Modality Learning VLM Robustness
School of Software Engineering, Huazhong University of Science and Technology · Nanyang Technological University, Singapore · University College London +3
On the Robustness of Machine Unlearning for Vision-Language Models
May 26, 2026Yujie Lin, Kaidi Jia, Jiayao Ma +2VLM Evaluation VLM Unlearning
1Xiamen University
Unveiling the Fragility of Vision-Language Models: Multi-Modal Adversarial Synergy via Texture-Constrained Perturbations and Cross-Modal Optimization
May 26, 2026Xiang Fang, Wanlong Fang, Changshuo WangVLM Robustness Adversarial Attacks on VLMs
School of Software Engineering, Huazhong University of Science and Technology · Nanyang Technological University, Singapore · University College London
EVIDENT: Routing MLLM Adaptation through Entity-Grounded Visual Evidence for Cross-Domain Video Temporal Grounding
May 25, 2026Geo Ahn, Jiwook Han, Youngrae Kim +2Temporal Video Grounding VLM Robustness
Seoul National University
Closed-Loop Bidirectional Prompting for Adversarial Robustness of Vision Language Models
May 25, 2026Xiao Liu, Jiaxiang Liu, Boci Peng +6VLM Robustness Prompt Learning
University of Macau · Guangdong Institute of Intelligence Science and Technology · Peking University +4
Localization then Neutralization: Gradient-guided Token Suppression against Visual Prompt Injection Attack
May 24, 2026Dongpeng Zhang, Ke Ma, Yangbangyan Jiang +4Gradient-Based Attribution VLM Robustness
School of Advanced Interdisciplinary Sciences, UCAS · School of Electronic, Electrical and Communication Engineering, UCAS · School of Computer Science and Technology, UCAS +4
Mitigating Object Hallucinations in Vision-Language Models through Region-Aware Attention Recalibration
May 24, 2026Yuanzhi Xu, Qian Gao, Jun Fan +4VLM Robustness Vision-Language Models
Qilu University of Technology (Shandong Academy of Sciences) Jinan, China · China Telecom Digital Intelligence Technology Co, Ltd Jinan, China · Shenyang Aerospace University Shenyang, China +1
DUEL: Adversarial Self-Play for Multimodal Reasoning
May 24, 2026Lin Qiu, Hanqing Zeng, Yao Liu +3VLM Robustness VLM Reasoning
Meta AI
SpaceDG: Benchmarking Spatial Intelligence under Visual Degradation
May 21, 2026Xiaolong Zhou, Yifei Liu, Ziyang Gong +8VLM Robustness Spatial Reasoning Benchmarks
2Shanghai Artificial Intelligence Laboratory · University of Electronic Science and Technology of China · 6Beihang University +4
Exposing Vulnerabilities in Visible-Infrared VLMs: A Unified Geometric Adversarial Framework with Cross-Task Transferability
May 21, 2026Xiang Chen, Yuxian Dong, Chao Li +6VLM Robustness Adversarial Patch Attacks
Lost in Fog: Sensor Perturbations Expose Reasoning Fragility in Driving VLAs
May 20, 2026Abhinaw Priyadershi, Jelena FrtunikjVLM Robustness VLMs for Autonomous Driving
NVIDIA Corporation, USA · NVIDIA GmbH, Germany
Mitigating Hallucinations in Large Vision-Language Models via Causal Route Gating
May 20, 2026Zhe Cheng, Wenyu Chen, Fode Zhang +1VLM Robustness Hallucination in Language Models
Center of Statistical Research, School of Statistics and Data Science, Southwestern University of Finance and Economics, Chengdu, China. · Department of Biomedical Engineering, College of Design and Engineering, National University of Singapore, Singapore.
The Yes-Man Syndrome: Benchmarking Abstention in Embodied Robotic Agents
May 19, 2026Doguhan Yeke, Elif Su Temirel, Ananth Shreekumar +3VLM Robustness Large Language Model-Based Robot Planning
Purdue University · Bilkent University · Work performed while an intern at Purdue University.
Towards Fine-Grained Robustness: Attention-Guided Test-Time Prompt Tuning for Vision-Language Models
May 19, 2026Jia-Wei Hai, Yijun Wang, Xiu-Shen WeiVLM Robustness Adversarial Attacks on VLMs
School of Computer Science and Engineering, and Key Laboratory of New Generation Artificial Intelligence Technology and Its Interdisciplinary Applications, Southeast University, China. · School of Intelligence Science and Engineering, Southeast University, China.
Structured Layout Priors for Robust Out-of-Distribution Visual Document Understanding
May 19, 2026Peter El Hachem, Ahmed Nassar, A. Said Gurbuz +2VLM Robustness LLM Prompting
ETH Zurich Rämistrasse, Switzerland · IBM Research Rüschlikon, Switzerland
RoVLA: Multi-Consistency Constraints for Robust Vision-Language-Action Models
May 19, 2026Jingzhou Luo, Yifan Wen, Yongjie Bai +3VLM Robustness Language-Conditioned Robot Manipulation
Sun Yat-sen University, China · Peng Cheng Laboratory · Guangdong Key Laboratory of Big Data Analysis and Processing +1

Jun 1, 2026, cs.CLJ/K move · Enter open · S save

Mechanistic Diagnostics of Spatial Lexical Bias in Multimodal Large Language Model Spatial Reasoning

Chuang Ma, Qianying Liu, Tomoyuki Obuchi, Fei Cheng+5

Kyoto University · NII LLMC · RIKEN AIP+4

VLM Robustness Visual Spatial Reasoning Multimodal Large Language Models Linguistic Bias in Language Models Causal Interventions in Language Models

Open paper AskPDF arXiv

Multimodal large language models (MLLMs) remain unreliable on spatial multiple-choice questions, and their failures are often attributed to poorly attended visual information. We identify a complementary failure mode, spatial lexical bias: a spatial relation word added to the answer options can act as a lexical-semantic distractor that draws the model's decision toward that option. Using nine open-weight MLLMs, we show that this phenomenon is widespread. We then isolate diagnostic cases in which a model answers a binary spatial question correctly yet consistently chooses a newly added third spatial option, which we call binary-stable but ternary-fragile cases. Leveraging mechanistic interpretability tools on these cases, we find that the failure arises on the language side rather than the visual side: visual attention analyses and residual-stream probes show the correct spatial relation remains internally available, while irrelevant-option controls, activation patching, and sparse component interventions trace the bias to specific LLM-side channels and neurons. Accordingly, we show that a lightweight LLM-only DPO update on tiny single-object-pair synthetic data mitigates the bias, lifting four-way robust accuracy by up to 100 points on synthetic data, and by 68.0, 32.6, and 20.1 points on broader evaluation datasets WhatsUp, SpatialMQA-Direct, and VSR.

Momentum

Related topics

Latest papers 295