VLM Evaluation

VLM: Vision-Language Model

Follow this topic Ask about this topic

Momentum

71 papers in the last four weeks, up 69% on the four weeks before. 0.7% of all new papers.

Jul 13Week of Sep 28

Latest papers 649

All topics

What Matters for Grocery Product Retrieval with Open Source Vision Language Models
May 18, 2026Emmanuel G. Maminta, Rowel O. AtienzaVLM Evaluation Fine-Grained Image Retrieval
AI Graduate Program, University of the Philippines, Diliman, Quezon City · EEEI, University of the Philippines, Diliman, Quezon City
SkyNative: A Native Multimodal Framework for Remote Sensing Visual Evidence Reasoning
May 18, 2026Xiao Yang, Ronghao Fu, Zhiwen Lin +10VLM Evaluation VLM Robustness
College of Computer Science and Technology, Jilin University, China · Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education
CounterCount: A Diagnostic Framework for Counting Bias in Vision Language Models
May 18, 2026Reem Alzahrani, Hassan Alshanqiti, Bushra Bin Hemid +3VLM Evaluation Vision-Language Models
King Abdullah University of Science and Technology, Thuwal, Saudi Arabia. · University of Edinburgh · KAUST
AutoRubric-T2I: Robust Rule-Based Reward Model for Text-to-Image Alignment
May 17, 2026Kuei-Chun Kao, Daixuan Huo, Yuanhao Ban +1VLM Evaluation Rubric-Based Evaluation
University of California, Los Angeles · Arena
Employing Vision-Language Models for Face Image Quality Assessment
May 17, 2026Erdi Sarıtaş, Eren Onaran, Vitomir Štruc +1VLM Evaluation VLM Robustness
Department of Computer Engineering, Istanbul Technical University, Istanbul, Türkiye · Faculty of Electrical Engineering, University of Ljubljana, Ljubljana, Slovenia · Division of Engineering, NYU Abu Dhabi, Abu Dhabi, UAE
Medical Context Distorts Decisions in Clinical Vision Language Models
May 17, 2026David Restrepo, Ira Ktena, Maria Vakalopoulou +2VLM Evaluation VLM Robustness
1MICS, CentraleSup´elec - Universit´e Paris-Saclay, France · 2Cancer Data Science Unit, IHU PRISM, National Institute in Precision Oncology, University Paris-Saclay, CentraleSupelec, Gustave Roussy, INSERM, Gif-sur-Yvette · 3Ellison Institute of Technology (EIT), Oxford, UK +1
UCSF-PDGM-VQA: Visual Question Answering dataset for brain tumor MRI interpretation
May 16, 2026Shiv Ghosh, Junayd Lateef, Chih-Hua Liu +3VLM Evaluation Medical VQA
Fung Institute for Engineering Leadership University of California, Berkeley Berkeley, CA 94709 · Department of Radiology University of California, San Francisco San Francisco, CA 94158 · Department of Radiology University of California, San Francisco San Francisco, CA 94143 +1
EPIC-Bench: A Perception-Centric Benchmark for Fine-Grained Embodied Visual Grounding in Vision-Language Models
May 16, 2026Haozhe Shan, Xiancong Ren, Han Dong +9VLM Evaluation Vision-Language Grounding
X-Humanoid · University of New South Wales · University of Science and Technology of China +4
Can Vision Language Models Be Adaptive in Mathematics Education? A Learner Model-based Rubric Study
May 15, 2026Jie Gao, Yongan Yu, Junzhu Su +3VLM Evaluation Intelligent Tutoring Systems
1McGill University · 2Mila – Quebec AI Institute · 1McGill +1
Are VLMs Seeing or Just Saying? Uncovering the Illusion of Visual Re-examination
May 15, 2026Chufan Shi, Cheng Yang, Yaokang Wu +4VLM Evaluation VLM Robustness
University of Southern California · University of California San Diego · Carnegie Mellon Univer +1
VCG-Bench: Towards A Unified Visual-Centric Benchmark for Structured Generation and Editing
May 15, 2026Xiaoyan Su, Peijie Dong, Zhenheng Tang +8VLM Evaluation Code Generation
The Hong Kong University of Science and Technology (GuangZhou) · The Hong Kong University of Science and Technology · Huawei Technologies Co., Ltd +2
VLMs Trace Without Tracking: Diagnosing Failures in Visual Path Following
May 15, 2026Hyesoo Hong, Minsoo Kim, Wonje Jeung +3VLM Evaluation Visual Navigation
Yonsei University
MI-CXR: A Benchmark for Longitudinal Reasoning over Multi-Interval Chest X-rays
May 15, 2026Sunghwan Steve Cho, Yunseok Han, Jaeyoung DoVLM Evaluation Healthcare
AIDAS Laboratory, 1ECE & 2IPAI, Seoul National University
On the Cultural Anachronism and Temporal Reasoning in Vision Language Models
May 14, 2026Mukul Ranjan, Prince Jha, Khushboo Kumari +1VLM Evaluation VLM Reasoning
MBZUAI, UAE · Inception, UAE
MemLens: Benchmarking Multimodal Long-Term Memory in Large Vision-Language Models
May 14, 2026Xiyu Ren, Zhaowei Wang, Yiming Du +11VLM Evaluation Memory-Augmented VLMs
CSE Deparment, HKUST · CUHK · OmniMemory (Shenzhen) Intelligent Technology Co., Ltd. +1
Exploring Vision-Language Models for Online Signature Verification: A Zero-Shot Capability Study
May 14, 2026Marta Robledo-Moreno, Ruben Vera-Rodriguez, Ruben Tolosana +1VLM Evaluation Zero-Shot Learning
BiometricsAI, School of Engineering, Universidad Autonoma de Madrid, Spain
SceneFunRI: Reasoning the Invisible for Task-Driven Functional Object Localization
May 14, 2026Posheng Chen, Powen Cheng, Gueter Josmy Faure +2VLM Evaluation Visual Spatial Reasoning
National Taiwan University · Delta Robotics Innovation Center
CurveBench: A Benchmark for Exact Topological Reasoning over Nested Jordan Curves
May 13, 2026Amirreza Mohseni, Mona Mohammadi, Morteza Saghafian +1VLM Evaluation Spatial Reasoning Benchmarks
Maastricht University · Cornell University · TU Wien +1
KamonBench: A Grammar-Based Dataset for Evaluating Compositional Factor Recovery in Vision-Language Models
May 13, 2026Richard Sproat, Stefano PeluchettiVLM Evaluation Compositional Generalization
FIKA-Bench: From Fine-grained Recognition to Fine-Grained Knowledge Acquisition
May 13, 2026Geng Li, Yuxin PengVLM Evaluation Fine-Grained Visual Perception
1Wangxuan Institute of Computer Technology, Peking University, China
Revealing the Gap in Human and VLM Scene Perception through Counterfactual Semantic Saliency
May 13, 2026Ziqi Wen, Parsa Madinei, Miguel P. EcksteinVLM Evaluation Vision-Language Models
Department of Computer Science, University of California, Santa Barbara · Department of Psychological and Brain Sciences, University of California, Santa Barbara
CiteVQA: Benchmarking Evidence Attribution for Trustworthy Document Intelligence
May 13, 2026Dongsheng Ma, Jiayu Li, Zhengren Wang +9VLM Evaluation Citation Verification
Peking University · Shanghai Artificial Intelligence Laboratory
UHR-Micro: Diagnosing and Mitigating the Resolution Illusion in Earth Observation VLMs
May 12, 2026Shuo Ni, Tong Wang, Jing Zhang +5VLM Evaluation Remote Sensing Image Understanding
National Key Laboratory of Science and Technology on Space-Born Intelligent Information Processing, Beijing Institute of Technology, Beijing, China · Zhongguancun Academy, Beijing, China · School of Computer Science, Wuhan University, Wuhan, China +2
Chronicles-OCR: A Cross-Temporal Perception Benchmark for the Evolutionary Trajectory of Chinese Characters
May 12, 2026Gengluo Li, Shangpin Peng, Xingyu Wan +16VLM Evaluation VLM Robustness
Institute of Information Engineering, Chinese Academy of Sciences · Tencent · Anyang Normal University +2
Unlocking UML Class Diagram Understanding in Vision Language Models
May 12, 2026Artem Naboichenko, René PeinlVLM Evaluation Visual Question Answering
Hof University of Applied Sciences, Hof, Germany
Vision2Code: A Multi-Domain Benchmark for Evaluating Image-to-Code Generation
May 11, 2026Ajay Vikram Periasami, Junlin Wang, Bhuwan DhingraVLM Evaluation Code Generation
Duke University
Do Vision-Language-Models show human-like logical problem-solving capability in point and click puzzle games?
May 11, 2026Maximilian Triebel, Marco Menner, Dominik HelfensteinVLM Evaluation Physical Reasoning
Institute of Artificial Intelligence, University of Stuttgart, Stuttgart, Germany.
Don't Look at the Numbers: Visual Anchoring Bias and Layer-wise Representation in VLMs
May 11, 2026M. ShalankinVLM Evaluation Vision-Language Models
Grounded or Guessing? LVLM Confidence Estimation via Blind-Image Contrastive Ranking
May 11, 2026Reza Khanmohammadi, Erfan Miahi, Simerjot Kaur +4VLM Evaluation Vision-Language Models
Michigan State University · Independent AI Researcher · JPMorgan AI Research +1
Verification Mirage: Mapping the Reliability Boundary of Self-Verification in Medical VQA
May 11, 2026Ruinan Jin, Beidi Zhao, Myeongkyun Kang +2VLM Evaluation Vision-Language Models
1The University of British Columbia · 2Vector Institute · 3Redmin University of China

May 18, 2026, cs.CVJ/K move · Enter open · S save

What Matters for Grocery Product Retrieval with Open Source Vision Language Models

Emmanuel G. Maminta, Rowel O. Atienza

AI Graduate Program, University of the Philippines, Diliman, Quezon City · EEEI, University of the Philippines, Diliman, Quezon City

VLM Evaluation Fine-Grained Image Retrieval Information Retrieval Vision-Language Retrieval Image Retrieval Multimodal Understanding

Open paper AskPDF arXiv

Multimodal product retrieval (MPR) underpins checkout-free retail and automated inventory systems, yet it demands fine-grained SKU discrimination that standard vision-language benchmarks fail to capture. We present the first systematic zero-shot evaluation of 190 open-source VLMs on the MPR task of the GroceryVision Challenge, isolating pre-training data, architecture, and input resolution. Our analysis yields three actionable findings. \textbf{(1) Data quality trumps scale.} Switching from raw web-scrapes to filtered datasets delivers up to 16.6% accuracy gains, exceeding the benefit of doubling model parameters. \textbf{(2) Efficient models can win.} MobileCLIP-B (150M parameters) outperforms 351M counterparts trained on noisy data. We introduce \textit{semantic power density} (

φ

), an efficiency metric that penalizes sub-threshold accuracy. \textbf{(3) A precision gap persists.} State-of-the-art models achieve 94.5% Recall@5 but suffer a 17.5% drop at Recall@1, revealing that contrastive embeddings cluster categories effectively but fail to rank visually similar SKUs. Code and evaluation scripts are available at https://github.com/upeee/openmpr.

Momentum

Related topics

Latest papers 649