VLM Robustness

VLM: Vision-Language Model

Follow this topic Ask about this topic

Momentum

26 papers in the last four weeks, up 117% on the four weeks before. 0.3% of all new papers.

Jul 13Week of Sep 28

Latest papers 295

All topics

Pseudocode-Guided Structured Reasoning for Automating Reliable Inference in Vision-Language Models
May 19, 2026Weicong Ni, Tianbao Jiang, Linlin WangVLM Robustness Object Hallucination in VLMs
All authors are with East China Normal University, Shanghai, China.
RE-VLM: Event-Augmented Vision-Language Model for Scene Understanding
May 19, 2026Hanqing Liu, Mingjie Liu, Luoping Cui +3Visual Question Answering Event-Based Vision
School of Artificial Intelligence, Beijing University of Posts and Telecommunications, Beijing, China · School of Artificial Intelligence, Beijing University of Posts and Telecommunications, Beijing, China 2State Key Laboratory of General Artificial Intelligence, BIGAI, Beijing, China
Geometry-Aware Uncertainty Coresets for Robust Visual In-Context Learning in Histopathology
May 18, 2026Franciskus Xaverius Erick, Johanna Paula Müller, Bernhard KainzVLM Robustness In-Context Learning
FAU Erlangen-Nürnberg, Erlangen, DE · Department of Computing, Imperial College London, London, UK
Self-Evolving Spatial Reasoning in Vision Language Models via Geometric Logic Consistency
May 18, 2026Junming Liu, Yuqi Li, Yifei Sun +6VLM Robustness Vision-Language Models
Shanghai Artificial Intelligence Laboratory · The City University of New York · The Chinese University of Hong Kong +3
SkyNative: A Native Multimodal Framework for Remote Sensing Visual Evidence Reasoning
May 18, 2026Xiao Yang, Ronghao Fu, Zhiwen Lin +10VLM Evaluation VLM Robustness
College of Computer Science and Technology, Jilin University, China · Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education
When a Zero-Shooter Cheats: Improving Age Estimation via Activation Steering
May 17, 2026Erik Imgrund, Pia Hanfeld, Klim Kireev +1VLM Robustness Large Vision-Language Models
BIFOLD & TU Berlin
Employing Vision-Language Models for Face Image Quality Assessment
May 17, 2026Erdi Sarıtaş, Eren Onaran, Vitomir Štruc +1VLM Evaluation VLM Robustness
Department of Computer Engineering, Istanbul Technical University, Istanbul, Türkiye · Faculty of Electrical Engineering, University of Ljubljana, Ljubljana, Slovenia · Division of Engineering, NYU Abu Dhabi, Abu Dhabi, UAE
Medical Context Distorts Decisions in Clinical Vision Language Models
May 17, 2026David Restrepo, Ira Ktena, Maria Vakalopoulou +2VLM Evaluation VLM Robustness
1MICS, CentraleSup´elec - Universit´e Paris-Saclay, France · 2Cancer Data Science Unit, IHU PRISM, National Institute in Precision Oncology, University Paris-Saclay, CentraleSupelec, Gustave Roussy, INSERM, Gif-sur-Yvette · 3Ellison Institute of Technology (EIT), Oxford, UK +1
SHED: Style-Homogenized Embedding Alignment for Domain Generalization
May 16, 2026Kai Gan, Tong WeiVLM Robustness VLM Adaptation
School of Computer Science and Engineering, Southeast University, Nanjing 210096, China · Key Laboratory of Computer Network and Information Integration (Southeast University), Ministry of Education, China
Right Predictions, Misleading Explanations: On the Vulnerability of Vision-Language Model Explanations
May 15, 2026Narges Babadi, Hadis KarimipourVLM Robustness Adversarial Attacks on VLMs
Department of Electrical and Software Engineering, University of Calgary, Calgary, Canada.
A Cross-Modal Prompt Injection Attack against Large Vision-Language Models with Image-Only Perturbation
May 15, 2026Hao Yang, Zhuo Ma, Yang Liu +3VLM Robustness Adversarial Attacks on VLMs
Xidian University Xi’an, Shaanxi, China
Sparse Autoencoders enable Robust and Interpretable Fine-tuning of CLIP models
May 15, 2026Fabian Morelli, Arnas Uselis, Ankit Sonthalia +1Distribution Shift Robustness VLM Robustness
University of Tübingen ∗ · University of Tübingen
Are VLMs Seeing or Just Saying? Uncovering the Illusion of Visual Re-examination
May 15, 2026Chufan Shi, Cheng Yang, Yaokang Wu +4VLM Evaluation VLM Robustness
University of Southern California · University of California San Diego · Carnegie Mellon Univer +1
BiomedAP: A Vision-Informed Dual-Anchor Framework with Gated Cross-Modal Fusion for Robust Medical Vision-Language Adaptation
May 15, 2026Huanyang Tong, Kai Liu, Fangjun Kuang +1VLM Robustness Prompt Learning
Wenzhou University · Wenzhou Business College
VLMs Trace Without Tracking: Diagnosing Failures in Visual Path Following
May 15, 2026Hyesoo Hong, Minsoo Kim, Wonje Jeung +3VLM Evaluation Visual Navigation
Yonsei University
AGC: Adaptive Geodesic Correction for Adversarial Robustness on Vision-Language Models
May 15, 2026Zhiwei Li, Jiacheng Xue, Weining Wang +4VLM Robustness Adversarial Robustness
1NLPR & MAIS, Institute of Automation, Chinese Academy of Sciences · School of Computer Science and Engineering, Central South University · University of Chinese Academy of Sciences
EVA: Editing for Versatile Alignment against Jailbreaks
May 14, 2026Yi Wang, Hongye Qiu, Yue Xu +4VLM Robustness LLM Safety Alignment
To See is Not to Learn: Protecting Multimodal Data from Unauthorized Fine-Tuning of Large Vision-Language Model
May 14, 2026Chengshuai Zhao, Zhen Tan, Dawei Li +2VLM Robustness Multimodal Robustness
School of Computing and Augmented Intelligence, Arizona State University, Tempe, AZ, USA · Department of Computer Science and Engineering, Texas A&M University, College Station, TX, USA.
Reducing Hallucination in Vision-Language Models via Stage-wise Preference Optimization under Distribution Shift
May 13, 2026Qinwu XuVLM Robustness Hallucination in Language Models
Meta AI
CA-GCL: Cross-Anatomy Global-Local Contrastive Learning for Robust 3D Medical Image Understanding
May 13, 2026Hanwen Zhang, Yao Liu, Die Dai +4VLM Robustness Medical Image Anomaly Detection
University of Electronic Science and Technology of China, Chengdu, China · Mohamed bin Zayed University of Artificial Intelligence, Abu Dhabi, UAE
What to Ignore, What to React: Visually Robust RL Fine-Tuning of VLA Models
May 13, 2026Yuanfang Peng, Jingjing Fu, Chuheng Zhang +6VLM Robustness Domain Generalization
1Hong Kong University of Science and Technology · 2Microsoft Research Asia · 3Zhejiang University
Revealing Interpretable Failure Modes of VLMs
May 12, 2026Isha Chaudhary, Vedaant V Jain, Kavya Sachdeva +2VLM Robustness Vision-Language Models
UIUC · Kumo AI · IIT Delhi
GuidedVLA: Specifying Task-Relevant Factors via Plug-and-Play Action Attention Specialization
May 12, 2026Xiaosong Jia, Bowen Yang, Zuhao Ge +17VLM Robustness Robotic Manipulation
Institute of Trustworthy Embodied AI (TEAI), Fudan University · 2Shanghai Key Laboratory of Multimodal Embodied AI · Shanghai Jiao Tong University +1
Chronicles-OCR: A Cross-Temporal Perception Benchmark for the Evolutionary Trajectory of Chinese Characters
May 12, 2026Gengluo Li, Shangpin Peng, Xingyu Wan +16VLM Evaluation VLM Robustness
Institute of Information Engineering, Chinese Academy of Sciences · Tencent · Anyang Normal University +2
Mitigating Action-Relation Hallucinations in LVLMs via Relation-aware Visual Enhancement
May 12, 2026Zhenxin Qin, Qiang Li, Qingzhuo Wang +3VLM Robustness VLM Hallucination
Tongji University, Shanghai, China
Dynamic Execution Commitment of Vision-Language-Action Models
May 12, 2026Feng Chen, Xianghui Wang, Yuxuan Chen +4VLM Robustness Efficient VLM Inference
University of Adelaide · Sichuan University · Shanghai Jiao Tong University +3
CheXTemporal: A Dataset for Temporally-Grounded Reasoning in Chest Radiography
May 11, 2026Eva Prakash, Yunhe Gao, Chong Wang +10VLM Robustness Chest X-Ray Classification
Stanford University · University of Oxford · University of California, Berkeley +2
Birds of a Feather Flock Together: Background-Invariant Representations via Linear Structure in VLMs
May 11, 2026Youssef Zaazou, Mark ThomasVLM Robustness Vision-Language Models
Independent Researcher
C-CoT: Counterfactual Chain-of-Thought with Vision-Language Models for Safe Autonomous Driving
May 11, 2026Kefei Tian, Yuansheng Lian, Kai Yang +2VLM Robustness Counterfactual Evaluation of VLMs
College of Transportation, Tongji University, Shanghai, China · Department of Civil Engineering, Tsinghua University, Beijing 100084, China · School of Vehicle and Mobility, Tsinghua University, Beijing, China +1
Not Blind but Silenced: Rebalancing Vision and Language via Adversarial Counter-Commonsense Equilibrium
May 11, 2026Qingxin Xiao, Peilin Zhao, Yangyang Zhao +2VLM Robustness Hallucination in Language Models
South China University of Technology · Institute for Super Robotics (Huangpu) · Shanghai Jiao Tong University +1

May 19, 2026, cs.AIJ/K move · Enter open · S save

Pseudocode-Guided Structured Reasoning for Automating Reliable Inference in Vision-Language Models

Weicong Ni, Tianbao Jiang, Linlin Wang

All authors are with East China Normal University, Shanghai, China.

VLM Robustness Object Hallucination in VLMs VLM Reasoning LLM Reasoning

Open paper AskPDF arXiv

Vision-Language Models (VLMs) are becoming the cornerstone of high-level reasoning for robotic automation, enabling robots to parse natural language commands and perceive their environments. However, their susceptibility to hallucinations introduces critical failures in decision-making, posing significant safety and reliability risks in physical deployments. This challenge is exacerbated by the open-ended nature of real-world tasks, where questions vary vastly in difficulty and modality, demanding robust and adaptable reasoning strategies. To tackle this, we propose the Pseudocode-guided Structured Reasoning framework (PStar), which adaptively selects structured pseudocode reasoning paths to help VLMs perform flexible and step-by-step reasoning. We first design a set of abstract reasoning functions and formulate a structured pseudocode library to represent modular reasoning strategies. Crucially, we design a Difficulty Feature Vector (DFV) that allows the model to assess question complexity and adaptively choose appropriate reasoning strategies-enhancing robustness and interpretability. Extensive experiments demonstrate that PStar significantly reduces hallucination rates, achieving state-of-the-art scores of 87.1% on POPE and 68.0% on MMStar, outperforming even GPT-4V. By providing a validated mechanism to reduce visual-language errors, PStar offers a critical step toward deploying more trustworthy and deterministic VLMs for real-world automated systems, where such errors can lead to catastrophic outcomes.

Momentum

Related topics

Latest papers 295