VLM Robustness

VLM: Vision-Language Model

Follow this topic Ask about this topic

Momentum

26 papers in the last four weeks, up 117% on the four weeks before. 0.3% of all new papers.

Jul 13Week of Sep 28

Latest papers 295

All topics

One Anchor for All: Unified Multilingual and Multimodal Safety Alignment for LVLMs
Jul 30, 2026Enyi Shi, Fei Shen, Chuancheng Shi +4VLM Robustness Multimodal Large Language Models
1Nanjing University of Science and Technology · 2National University of Singapore · 3The University of Sydney +2
A Cross-Architecture Audit of Direction-Based Inference-Time Defences in Vision-Language Models
Jul 30, 2026Xiangyu Yin, Tora Bodin, Rohan Menon +1VLM Robustness Adversarial Attacks on VLMs
Chalmers University of Technology, Gothenburg, Sweden · Carl von Ossietzky University of Oldenburg, Oldenburg, Germany
Unifying Adversarially Robust Model Experts in Vision-Language Models
Jul 30, 2026Nguyen Duc Thai, Junhao Dong, Sua Qi Rong +2Adversarial Training VLM Robustness
College of Computing and Data Science, Nanyang Technological University, Singapore
Calibrate Before Reason: Robust Visual Token Reduction against Semantic Drift in VLMs
Jul 30, 2026Jiasheng Li, Zhong Ji, Yan Zhang +1VLM Robustness Efficient VLM Inference
Tianjin University, Tianjin, China · Tsinghua University, Beijing, China
MMOOC: A Comprehensive Benchmark for Out-of-Context Evaluation in Multimodal Large Language Models
Jul 30, 2026Wenjie Zhu, Yabin Zhang, Wenjun Zeng +1VLM Robustness Multimodal Large Language Models
The Hong Kong Polytechnic University · Harbin Institute of Technology (Shenzhen) · Eastern Institute of Technology, Ningbo
Objective-Aligned Direct Answer SFT for Robust Multi-Frame Medical VQA
Jul 30, 2026Site Li, Jianyi Hao, Xiaofeng LiuSupervised Fine-Tuning VLM Robustness
Yale University
Attack Ensembles Expose a Safety-Utility Trade-off in Black-Box Guard Defenses Against Encoded VLM Jailbreaks
Jul 29, 2026Haoyu Zhang, Zhuoxi Wang, Shibo Zheng +7VLM Robustness Adversarial Attacks on VLMs
Northeastern University
TPCD: Tone-Pressure Contrastive Decoding and the Label-Free Gating Bottleneck in Vision-Language Models
Jul 29, 2026Jinkun Zhao, Kui Zhang, Wenjun WuVLM Robustness Adversarial Attacks on VLMs
Beihang University
Evaluating VLMs for Autonomous Agent-Driven Geometry Clipping Detection in Video Game QA
Jul 28, 2026Carlos Celemin, Benedict Wilkins, Adrián Barahona-Ríos +2VLM Evaluation VLM Robustness
Sony Interactive Entertainment, London, United Kingdom
Bigger or Cheaper? Scale and Quantization Effects on Uncertainty Signals in Vision-Language Models Under Image Degradation
Jul 27, 2026M M Asif FerdousVLM Robustness Selective Prediction
Independent Researcher
Face Age Verification Vulnerabilities Under Simple Appearance Manipulations
Jul 27, 2026Ioannis Sarridis, Ioannis Kompatsiaris, Symeon PapadopoulosVLM Robustness Algorithmic Bias
Information Technologies Institute, Centre for Research and Technology Hellas, 6th km Charilaou-Thermi Rd, Thessaloniki, 57001, Greece
ObsDriveBench: Benchmarking Multimodal Understanding under Adverse Weather with Observability Awareness
Jul 26, 2026Qiao Yan, Yihan Wang, Zhenghao Xing +2Autonomous Driving Benchmarks VLM Robustness
Department of Computer Science and Engineering, The Chinese University of Hong Kong · Institute of Medical Intelligence and XR, The Chinese University of Hong Kong
Small Vision-Language Models Know When They Are Wrong But Cannot Say So: A Two-Model Study of Stated versus Internal Confidence Under Realistic Image Degradation
Jul 24, 2026M M Asif FerdousVLM Robustness Vision-Language Models
Independent Researcher
V-DEAL: Diagnosing Video Safety De-Calibration as an Understanding-Refusal Coupling Failure
Jul 23, 2026Zhetong Zhang, Honghao Fu, Miao Xu +2VLM Evaluation VLM Robustness
University of Queensland · University of California, Merced
Test-Time Training for Modality Order Consistency in Vision-Language Models
Jul 22, 2026Aditi Gupta, Yossi GandelsmanVLM Robustness VLM Adaptation
University of Chicago · Reve
Visual Token Compression Enhances Robustness of MLLMs
Jul 21, 2026Shishen Gu, Jiequan Cui, Wenbo Hu +3VLM Robustness Adversarial Attacks on VLMs
Hefei University of Technology · Hefei, China
Dual Adversarial Fine-tuning for Enhancing Robustness of Large Vision Language Model
Jul 21, 2026Sibo Wang, Jie Zhang, Shiguang Shan +2VLM Robustness Fine-Tuning
Can Experts Adapt Without Training? On Test-Time Modality Generalization in MVLMs
Jul 18, 2026Raza Imam, Darakshan Rashid, Yutong Xie +3VLM Robustness VLM Adaptation
Mohamed bin Zayed University of Artificial Intelligence, Abu Dhabi, UAE · Indian Institute of Technology Delhi, India · Khalifa University, Abu Dhabi, UAE
One Modality to Forget Them All: Enhancing Cross-Modal Unlearning in Vision-Language Models
Jul 17, 2026Sudharshan Balaji, Yili Ren, Guangjing Wang +2VLM Unlearning VLM Robustness
University of South Florida · USA · University of Massachusetts Lowell
On Success and Simplicity: A Second Look at Transferable Vision-Language Attack Pipeline
Jul 16, 2026Yuchen Ren, Zhengyu Zhao, Chenhao Lin +2VLM Robustness Adversarial Attacks on VLMs
School of Cyber Science and Engineering, Xi’an Jiaotong University, Xi’an, China · State Key Laboratory of Mathematical Engineering and Advanced Computing, Information Engineering University, Zhengzhou, China
GeoDetect: Geometric Adversarial Detection for VLPs
Jul 16, 2026Afsaneh Hasanebrahimi, Hanxun Huang, Christopher Leckie +2VLM Robustness Adversarial Attacks on VLMs
The University of Melbourne, Melbourne, Australia · Monash University, Melbourne, Australia
SynCLIP: Synonym-Coherent Language-Image Pretraining for Robust Open-Vocabulary Dense Perception
Jul 13, 2026Mingjie Xie, Guangjun He, Dongli Xu +5VLM Robustness Vision-Language Grounding
1Beihang University · 2State Key Laboratory of Space Information System and Integrated Application · 3Independent Researcher +3
MED-DSLC: Multi-Expert-Domain Classification via Domain Supervision and Logit Calibration
Jul 13, 2026Zheng Zeng, Deepak Sridhar, Nuno VasconcelosVLM Robustness VLM Adaptation
University of California, San Diego, USA
Imperceptible and Reversible Adversarial Examples against Vision-Language Models for Privacy Protection
Jul 11, 2026Qi Lu, Ziqi Zhou, Yufei Song +5VLM Robustness Adversarial Attacks on VLMs
School of Cyber Science and Engineering, Huazhong University of Science and Technology · College of Computer Science, Chongqing University · School of Software and engineering, Huazhong University of Science and Technology +2
Robustifying Vision-Language Models via Test-Time Prompt Adaptation
Jul 10, 2026Xingyu Zhu, Huanshen Wu, Shuo Wang +4VLM Robustness VLM Adaptation
National University of Singapore · University of Science and Technology of China · The Hong Kong University of Science and Technology
On Adversarial Vulnerability of Vision-Language Models through the Lens of Intermediate Spectral Subspaces
Jul 8, 2026Chethan Krishnamurthy Ramanaik, Tobias Callies, Michael Hecht +1VLM Robustness Adversarial Attacks on VLMs
University of the Bundeswehr Munich, Germany · University of Wrocław, Poland
InfraQR: Edge-Placed QR-Inspired Structured Patch Attacks on Infrared Vision-Language Models
Jul 8, 2026Xin Li, Jiaju Han, Ma Yaqi +5VLM Robustness Adversarial Patch Attacks
China University of Petroleum-Beijing at Karamay, Karamay, Xinjiang, China · Shenzhen Research Institute of Big Data, Shenzhen, China · Guizhou University, Guiyang, China +1
PolicyShiftGuard: Benchmarking and Improving Policy-Adaptive Image Guardrails
Jul 7, 2026Mingyang Song, Luxin Xu, Haoyu Sun +3VLM Robustness VLM Adaptation
1Fudan University · 2Tongji University · 3Virtue AI +3
AVA-VLM: Adaptive Visual Attention-Vision Language Model for In-the-Wild Construction Site Monitoring
Jul 7, 2026Younggun Kim, Taeheon Kim, Youngseo Kim +1VLM Robustness Efficient VLM Inference
Department of Civil and Environmental Engineering, University of California, Los Angeles, 405 Hilgard Avenue, Los Angeles, 90095, CA, USA · AI+KCIR Global Resilience Research Center, Suwon, 16419, Gyeonggi-do, South Korea · SmartInside AI Co., Ltd., Suwon, 16419, Gyeonggi-do, South Korea +1
Benchmarking the Robustness of Autonomous Driving to Environmental Illusions: A Lane Perception Perspective
Jul 7, 2026Tianyuan Zhang, Xianglong Liu, Aishan Liu +6Autonomous Driving Benchmarks VLM Robustness
SKLCCSE, the School of Computer Science and Engineering, Beihang University, Beijing 100191, China · School of Cyber Science and Technology, Sun Yat-sen University, Shenzhen 518107, China · Henan University of Science and Technology, Luoyang 471023, China +2

Jul 30, 2026, cs.AIJ/K move · Enter open · S save

One Anchor for All: Unified Multilingual and Multimodal Safety Alignment for LVLMs

Enyi Shi, Fei Shen, Chuancheng Shi, Linxia Zhu+3

1Nanjing University of Science and Technology · 2National University of Singapore · 3The University of Sydney+2

VLM Robustness Multimodal Large Language Models LLM Safety Alignment

Open paper AskPDF arXiv

As large vision-language models (LVLMs) are deployed globally, the combination of multilingual instructions and visual information makes malicious attacks more covert and sophisticated than ever before. However, existing methods isolate language and modality defenses, which, coupled with the scarcity of safety data and high fine-tuning costs, makes it difficult for models to defend against compound attacks. To address this severe challenge, we propose a neuron-level cross-dimensional safety alignment framework driven by modality- and language-shared safety neurons (MLS-Neurons). First, we identify monolingual and unimodal safety neurons by comparing responses to harmful and benign samples, quantifying functional saliency through activation strength and downstream impact. Then, by intersecting these unimodal neurons within each language, we extract modality-shared safety neurons (MS-Neurons) responsive to both visual and textual risks, bridging the safety representation gap between modalities. Furthermore, using English as a semantic anchor, we intersect MS-Neurons across languages to identify modality- and language-shared safety neurons (MLS-Neurons), serving as key defenses against compound attacks. Finally, we update only this minimal subset of shared neurons (~0.03% of parameters), transferring English-only safety supervision to multilingual and multimodal scenarios. Extensive experiments show that our method significantly outperforms state-of-the-art approaches across diverse multilingual and multimodal safety benchmarks while preserving general utility.

Momentum

Related topics

Latest papers 295