VLM Evaluation

VLM: Vision-Language Model

Follow this topic Ask about this topic

Momentum

71 papers in the last four weeks, up 69% on the four weeks before. 0.7% of all new papers.

Jul 13Week of Sep 28

Latest papers 649

All topics

SenseBench: A Benchmark for Remote Sensing Low-Level Visual Perception and Description in Large Vision-Language Models
May 11, 2026Chen Zhong, Xiao An, Jiaxing Sun +3VLM Evaluation Remote Sensing
Wuhan University · Shanghai Artificial Intelligent Laboratory
Sens-VisualNews: A Benchmark Dataset for Sensational Image Detection
May 11, 2026Andreas Goulas, Damianos Galanopoulos, Evlampios Apostolidis +1VLM Evaluation VLM Robustness
Information Technologies Institute (ITI), CERTH, Thessaloniki, Greece · Queen Mary University of London, London, UK
SleepWalk: A Three-Tier Benchmark for Stress-Testing Instruction-Guided Vision-Language Navigation
May 11, 2026Niyati Rawal, Sushant Ravva, Shah Alam Abir +5VLM Evaluation Spatial Reasoning Benchmarks
† Indian AI Research Organization (IAIRO), India · ⋆ĸragya Lab, BITS Pilani Goa, India · ¶ University of Dhaka, Bangladesh +3
RW-Post: Auditable Evidence-Grounded Multimodal Fact-Checking in the Wild
May 11, 2026Danni Xu, Shaojing Fan, Harry Cheng +1VLM Evaluation Multimodal Grounding
School of Computing (SoC), National University of Singapore (NUS), Singapore · Department of Electrical and Computer Engineering (ECE), National University of Singapore (NUS), Singapore
The Gordian Knot for VLMs: Diagrammatic Knot Reasoning as a Hard Benchmark
May 11, 2026Hao Liu, Jicheng LiuVLM Evaluation Vision-Language Grounding
Department of Psychology New York University · Department of Computer Science University of Southern California
The Cartesian Shortcut: Re-evaluate Vision Reasoning in Polar Coordinate Space
May 11, 2026Xia Hu, Zhenrui Yue, Brian Potetz +4VLM Evaluation Spatial Reasoning Benchmarks
Google DeepMind · Stanford University · Google Research
SAGE: Scalable Agentic Grounded Evaluation for Crop Disease Diagnosis
May 10, 2026Muhammad Arbab Arshad, Tirtho Roy, Yanben Shen +7VLM Evaluation Agricultural Image Analysis
Iowa State University, USA · New York University, USA
DeepTumorVQA: A Hierarchical 3D CT Benchmark for Stage-Wise Evaluation of Medical VLMs and Tool-Augmented Agents
May 10, 2026Yixiong Chen, Wenjie Xiao, Pedro R. A. S. Bassi +7VLM Evaluation Tool Use in VLMs
Johns Hopkins University · University of Bologna · Center for Biomolecular Nanotechnologies, Istituto Italiano di Tecnologia +3
SoccerLens: Grounded Soccer Video Understanding Beyond Accuracy
May 10, 2026Ismael Elsharkawi, Ahmed Sait, Silvio Giancola +3VLM Evaluation Sports Video Analysis
Department of Computer Science and Engineering, The American University in Cairo · Image And Visual Understanding Lab (IVUL), KAUST
From Pixels to Concepts: Do Segmentation Models Understand What They Segment?
May 10, 2026Shuang Liang, Zeqing Wang, Yuxian Li +2VLM Evaluation Image Segmentation
Department of Electrical and Computer Engineering, The University of Hong Kong · CASIC · School of Computer Science and Engineering, Sun Yat-sen University +1
Language-Conditioned Visual Grounding with CLIP Multilingual
May 9, 2026J. de Curtò, Mauro Liz, I. de ZarzàVLM Evaluation Multilingual VLM Evaluation
Department of Computer Applications in Science & Engineering, BARCELONA Supercomputing Center, Barcelona, Spain · Estudis d’Informàtica, Multimèdia i Telecomunicació, Universitat Oberta de Catalunya, Barcelona, Spain · Escuela T´ecnica Superior de Ingenier´ıa (ICAI), Universidad Pontificia Comillas, Madrid, Spain +2
Tracking the Truth: Object-Centric Spatio-Temporal Monitoring for Video Large Language Models
May 9, 2026Tri Cao, Khoi Le, Thong Nguyen +7VLM Evaluation Object Hallucination in VLMs
National University of Singapore · VinUniversity · Nanyang Technological University
Illusion-Aware Visual Preprocessing and Anti-Illusion Prompting for Classic Illusion Understanding in Vision-Language Models
May 9, 2026Junli Zha, Jiahui Wang, Xinkai Lu +1VLM Evaluation VLM Robustness
1SF Technology Co., Ltd.
PPU-Bench:Real World Benchmark for Personalized Partial Unlearning in Vision Language Models
May 9, 2026Jiahui Guang, Zexun Zhan, Zhenlin Xu +5VLM Evaluation VLM Unlearning
Harbin Institute of Technology, Shenzhen, China · Sichuan University, Chengdu, China · Harbin Institute of Technology, Weihai, China +3
Lost in Volume: The CT-SpatialVQA Benchmark for Evaluating Semantic-Spatial Understanding of 3D Medical Vision-Language Models
May 9, 2026Mashrafi Monon, Umaima Rahman, Asif Hanif +2VLM Evaluation 3D Spatial Reasoning
Mohamed Bin Zayed University of Artificial Intelligence, Abu Dhabi, UAE · New York University Abu Dhabi, Abu Dhabi, UAE
NICE FACT: Diagnosing and Calibrating VLMs in Quantitative Reasoning for Kinematic Physics
May 8, 2026Jian Lan, Zhicheng Liu, Xinpeng Wang +5VLM Evaluation Physical Reasoning
University of Munich (LMU), Germany 2Munich Center of Machine Learning3 Sichuan University
MedVIGIL: Evaluating Trustworthy Medical VLMs Under Broken Visual Evidence
May 8, 2026Hanqi Jiang, Junhao Chen, Mingyu Kang +12VLM Evaluation VLM Robustness
University of Georgia · Chungbuk National University · Chungnam National University Hospital +5
RuleSafe-VL: Evaluating Rule-Conditioned Decision Reasoning in Vision-Language Content Moderation
May 8, 2026Zhifeng Lu, Dianyuan Wang, Yuhu Shang +1VLM Evaluation Logical Reasoning
Beijing University of Posts and Telecommunications, Beijing, China
Towards Annotation-Free Validation of MLLMs: A Vision-Language Logical Consistency Metric
May 7, 2026Ying Gu, Mei Chee Leong, Hui Li Tan +3VLM Evaluation Large Vision-Language Models
Institute for Infocomm Research (I2R), Agency for Science, Technology and Research (A*STAR), Singapore
CXR-ContraBench: Benchmarking Negated-Option Attraction in Medical VLMs
May 7, 2026Zhengru Fang, Yanan Ma, Yu Guo +5VLM Evaluation Chest X-Ray Classification
City University of Hong Kong · Tsinghua Shenzhen International Graduate School
iTRIALSPACE: Programmable Virtual Lesion Trials for Controlled Evaluation of Lung CT Models
May 7, 2026Fakrul Islam Tushar, Umme Hafsa Momy, Joseph Y. Lo +1VLM Evaluation Radiology VLMs
Department of Radiology and Imaging Sciences, University of Arizona · Department of Biomedical Engineering, Florida International University · Center for Virtual Imaging Trials, Department of Radiology, Duke University Medical Center
How Far Are VLMs from Privacy Awareness in the Physical World? An Empirical Study
May 6, 2026Junran Wang, Xinjie Shen, Zehao Jin +1VLM Evaluation Vision-Language Models
Georgia Institute of Technology
StableI2I: Spotting Unintended Changes in Image-to-Image Transition
May 6, 2026Jiayang Li, Shuo Cao, Xiaohui Li +6VLM Evaluation Image Editing Evaluation
Peking University · Shanghai Artificial Intelligence Laboratory · University of Science and Technology of China +1
Where Reliability Lives in Vision-Language Models: A Mechanistic Study of Attention, Hidden States, and Causal Circuits
May 5, 2026Logan Mann, Ajit Saravanan, Ishan Dave +4VLM Evaluation VLM Interpretability
UC Santa Barbara · UC Berkeley · NVIDIA +2
Before Forgetting, Learn to Remember: Revisiting Foundational Learning Failures in LVLM Unlearning Benchmarks
May 5, 2026JuneHyoung Kwon, MiHyeon Kim, Eunju Lee +3VLM Evaluation VLM Unlearning
Department of Artificial Intelligence, Chung-Ang University · KT Corporation · Graduate School of Advanced Imaging Sciences, Multimedia and Film, Chung-Ang University
Erase Persona, Forget Lore: Benchmarking Multimodal Copyright Unlearning in Large Vision Language Models
May 5, 2026JuneHyoung Kwon, JungMin Yun, YoungBin KimVLM Evaluation VLM Unlearning
Department of Artificial Intelligence, Chung-Ang University · Graduate School of Advanced Imaging Sciences, Multimedia and Film, Chung-Ang University
MHPR: Multidimensional Human Perception and Reasoning Benchmark for Large Vision-Languate Models
May 5, 2026Kangkang Wang, Qinting Jiang, Wanping Zhang +2VLM Evaluation Vision-Language Models
AI Compute Group, Baidu · Tsinghua University
VideoNet: A Large-Scale Dataset for Domain-Specific Action Recognition
May 4, 2026Tanush Yadav, Mohammadreza Salehi, Jae Sung Park +6VLM Evaluation VLM Adaptation
University of Washington · Stanford University · Allen Institute for AI
Does it Really Count? Assessing Semantic Grounding in Text-Guided Class-Agnostic Counting
May 4, 2026Giacomo Pacini, Luca Ciampi, Nicola Messina +3VLM Evaluation VLM Robustness
Institute of Information Science and Technologies of the National Research Council (ISTI-CNR), Via G. Moruzzi 1, 56124, Pisa, Italy · University of Pisa - Department of Information Engineering, Via G. Caruso 16, 56122 Pisa, Italy
Metric Unreliability in Multimodal Machine Unlearning: A Systematic Analysis and Principled Unified Score
May 4, 2026Abdullah Ahmad Khan, Hamid Laga, Ferdous SohelVLM Evaluation VLM Unlearning
Murdoch University

May 11, 2026, cs.CVJ/K move · Enter open · S save

SenseBench: A Benchmark for Remote Sensing Low-Level Visual Perception and Description in Large Vision-Language Models

Chen Zhong, Xiao An, Jiaxing Sun, Zihan Gui+2

Wuhan University · Shanghai Artificial Intelligent Laboratory

VLM Evaluation Remote Sensing Image Quality Assessment Remote Sensing Benchmarks

Open paper AskPDF arXiv

Low-level visual perception underpins reliable remote sensing (RS) image analysis, yet current image quality assessment (IQA) methods output uninterpretable scalar scores rather than characterizing physics-driven RS degradations, deviating markedly from the diagnostic needs of RS experts. While Vision-Language Models (VLMs) present a compelling alternative by delivering language-grounded IQA, their visual priors are heavily biased toward ground-level natural images. Consequently, whether VLMs can overcome this domain gap to perceive and articulate RS artifacts remains insufficiently studied. To bridge this gap, we propose \textbf{SenseBench}, the first dedicated diagnostic benchmark for RS low-level visual perception and description. Driven by a physics-based hierarchical taxonomy that unifies both non-reference and reference-based paradigms, SenseBench features over 10K meticulously curated instances across 6 major and 22 fine-grained RS degradation categories. Specifically, two complementary protocols are designed for evaluation: objective low-level visual \textit{perception} and subjective diagnostic \textit{description}. Comprehensive evaluation of 29 state-of-the-art VLMs reveals not only skewed domain priors and multi-distortion collapse, but also \textit{fluency illusion} and a \textit{perception-description inversion} effect. We hope SenseBench provides a robust evaluation testbed and high-quality diagnostic data to advance the development of VLMs in RS low-level perception. Code and datasets are available \href{https://github.com/Zhong-Chenchen/SenseBench}{\textcolor{blue}{here}}.

Momentum

Related topics

Latest papers 649