VLM Evaluation

VLM: Vision-Language Model

Follow this topic Ask about this topic

Momentum

71 papers in the last four weeks, up 69% on the four weeks before. 0.7% of all new papers.

Jul 13Week of Sep 28

Latest papers 658

All topics

Animation2Code: Evaluating Temporal Visual Reasoning in Video-to-Code Generation
Jun 26, 2026Anya Ji, Abhijith Varma Mudunuri, David M. Chan +1VLM Evaluation
University of California, Berkeley
DataComp-VLM: Improved Open Datasets for Vision-Language Models
Jun 26, 2026Matteo Farina, Vishaal Udandarao, Thao Nguyen +33VLM Evaluation Training Data Curation
University of Trento · 2Tübingen AI Center, University of Tübingen · University of Cambridge +12
HumanMoveVQA: Can Video MLLMs reason about human movement in videos?
Jun 26, 2026Pulkit Gera, Faegheh Sardari, Asmar Nadeem +4VLM Evaluation Spatial Reasoning Benchmarks
CVSSP, University of Surrey, Guildford, UK · Tesco, UK
Video-MME-Logical: A Controlled Diagnostic Benchmark for Video Temporal-Logical Reasoning
Jun 26, 2026Hohin Kwan, Hongyu Li, Ray Zhang +5VLM Evaluation Multimodal Large Language Models
1HKUST · 2Colab, Beihang University · 3CUHK
Aloe-Vision: Robust Vision-Language Models for Healthcare
Jun 25, 2026Jaume Guasch-Martí, Enrique Lopez-Cuena, Martín Suárez-Fernández +3VLM Evaluation VLM Robustness
Barcelona Supercomputing Center (BSC)
FlameVQA: A Physically-Grounded UAV Wildfire VQA Benchmark with Radiometric Thermal Supervision
Jun 25, 2026Mobin Habibpour, John Spodnik, Niloufar Alipour Talemi +1VLM Evaluation Visual Question Answering
Clemson University · Clemson, SC, USA
GAVEL: Grounded Caption Error Verification and Localization
Jun 25, 2026Zixian Gao, Atsushi Hashimoto, Kuniaki SaitoVLM Evaluation Text-Image Alignment
OMRON SINIC X Corporation
From Hallucination to Grounding: Diagnosing Visual Spatial Intelligence via CRISP
Jun 25, 2026Zhixing Li, Yinan YuVLM Evaluation Spatial Reasoning Benchmarks
Chalmers University of Technology, Gothenburg, Sweden
How Robust is OCR-Reasoning? Evaluating OCR-Reasoning Robustness of Vision-Language Models under Visual Perturbations
Jun 24, 2026Yuxing Cheng, Yuan Wu, Yi ChangVLM Evaluation VLM Robustness
School of Artificial Intelligence, Jilin University · Engineering Research Center of Knowledge-Driven Human-Machine Intelligence, MOE, China · International Center of Future Science, Jilin University
What Transfers from Text to Vision? Capability Scaling Laws and Transfer Dynamics for VLMs
Jun 24, 2026Ziran Li, Qiang Wang, Zhengyu Chen +4Vision-Language Models VLM Evaluation
Meituan · Tsinghua University
ShutterMuse: Capture-Time Photography Guidance with MLLMs
Jun 24, 2026Jiayu Li, Yixiao Fang, Tianyu Hu +5VLM Evaluation Multimodal Large Language Models
1Fudan University · 2StepFun
HG-Bench: A Benchmark for Multi-Page Handwritten Answer-Region Grounding in Automated Homework Assessment
Jun 24, 2026Chuangxin Zhao, Boyan Shi, Yanling Wang +7VLM Evaluation Multimodal Grounding
C3-Bench: A Context-Aware Change Captioning Benchmark
Jun 24, 2026Jae-Woo Kim, Hyeongbeom Kim, Ue-Hwan KimVLM Evaluation
Gwangju Institute of Science and Technology, Gwangju, Republic of Korea · GIST InnoCORE AI-Nano Convergence Institute for Early Detection of Neurodegenerative Diseases, Gwangju Institute of Science and Technology, 61005 Gwangju, Republic of Korea
Physics Question Scene Graph: Fine-grained Evaluation of Physical Plausibility in Text-to-Video Generation
Jun 24, 2026Atin Pothiraj, Jaemin Cho, Yue Zhang +2VLM Evaluation Physical Consistency in Video Generation
University of North Carolina at Chapel Hill, USA · AI2, USA · Johns Hopkins University, USA +1
Are We There Yet? Exploring the Capabilities of MLLMs in Assistive AI Applications
Jun 23, 2026Shayon Dasgupta, Avijit Dasgupta, C. V. JawaharVLM Evaluation Scene Text Recognition
IIT BHU · India · CVIT, IIIT Hyderabad
Do vision-language models search like humans? Reasoning tokens as a reaction-time analog in classic visual-search paradigms
Jun 23, 2026Farahnaz WickVLM Evaluation Vision-Language Models
1Independent Researcher
EG-VQA: Benchmarking Verifiable Video Question Answering with Grounded Temporal Evidence
Jun 23, 2026Linpeng Huang, Weixing Chen, Zexin Chen +2VLM Evaluation Temporal Video Grounding
Sun Yat-sen University · Shenzhen University · Peng Cheng Laboratory
EgoSAT: A Comprehensive Benchmark of Egocentric Streaming Interaction Understanding
Jun 23, 2026Yijia Lei, Jinzhao Li, Yichi Zhang +3VLM Evaluation Streaming Video Understanding
College of AI, Tsinghua University · University of Wisconsin–Madison
PHANTOM: A Large-Scale Dataset of Multimodal Adversarial Attacks for Vision-Language Models
Jun 23, 2026Simone Gallivanone, Hossein Khodadadi, Mauro Dore +2VLM Evaluation VLM Robustness
The Italian Institute of Artificial Intelligence (AI4I), Turin, Italy · HikmaAI S.r.l., Pula, Italy
Ill-Posed by Design: Probing Evidence Use in VLMs
Jun 23, 2026Boaz Meivar, Shaked Perek, Shani Shvartzman +2VLM Evaluation Counterfactual Evaluation of VLMs
1Tel Aviv University · 2IBM Research
A Benchmark for Hallucination Detection in VLMs for Gastrointestinal Endoscopy
Jun 23, 2026Aminu Lawal, Niyoj Oli, Sachin Acharya +3VLM Evaluation Hallucination Detection in VLMs
University of Aberdeen, Aberdeen, United Kingdom · Nepal Applied Mathematics and Informatics Institute for Research, Lalitpur, Nepal · West Virginia University, Morgantown, WV, United States
REALM: A Unified Red-Teaming Benchmark for Physical-World VLMs
Jun 22, 2026Yifei Zhao, Qian Lou, Mengxin ZhengVLM Evaluation VLM Robustness
University of Central Florida
Listening makes Vision Clear for VLMs
Jun 22, 2026Yiyang Chen, Yixin Tan, Binrui ShenVLM Evaluation Visual Attention
Beijing Normal University
MotionHalluc: Diagnosing Kinematic Hallucinations in Fine-Grained Motion Reasoning
Jun 22, 2026Weile Guo, Shenghong He, Danying Mo +3VLM Evaluation Spatiotemporal Reasoning
1Sun Yat-Sen University · 2Shenzhen University
Each Judge Its Own Yardstick: Discovering Per-VLM Taxonomies for Physical Video Evaluation
Jun 22, 2026Yu Cao, Ziquan Liu, Zhensong Zhang +3VLM Evaluation Physical Consistency in Video Generation
Queen Mary University of London · Huawei Darwin Research Center · Imperial College London
RoboGaze: Evaluating Robot World Models via Structured Vision-Language Analysis
Jun 22, 2026Minh-Loi Nguyen, Nghiem Tuong Diep, Hung Khang Nguyen +10VLM Evaluation Robot Manipulation Benchmarks
Center for AI Research, VinUniversity · 2VinRobotics · 3DFKI +3
Chains That See, Answers That Don't: A Multi-Aspect Evaluation Recipe for Forced Chain-of-Thought on Video-MME
Jun 22, 2026Zhichao Fan, Yanhang Li, Zexin ZhuangVLM Evaluation Video QA
University of Illinois Urbana-Champaign Urbana, Illinois, USA · Northeastern University Boston, Massachusetts, USA · Southern Methodist University Dallas, Texas, USA
MapReason-OSM: Can Vision-Language Models Make Graph-Verifiable Mobility Decisions from Street Maps ?
Jun 21, 2026Srinivas Venkatanarayanan, Clement Pakkam IsaacVLM Evaluation Spatial Reasoning Benchmarks
NVIDIA · University of Central Florida · University of South Florida
Benchmarking Vision-Language Models for Microscopic Plant Image Understanding
Jun 21, 2026Tianqi Wei, Xin Yu, Zhi Chen +2VLM Evaluation Microscopy Image Analysis
The University of Queensland · Adelaide University · University of Southern Queensland
CVSBench: A Comprehensive Benchmark for Cross-view Spatial Reasoning and Dreaming
Jun 21, 2026Ruixun Liu, Lingyu Zhang, Lanxuan Xue +3VLM Evaluation Spatial Reasoning Benchmarks
Faculty of Electronic and Information Engineering, Xi’an Jiaotong University · Ministry of Education Key Laboratory of Intelligent Networks and Network Security

Jun 26, 2026, cs.CVJ/K move · Enter open · S save

Animation2Code: Evaluating Temporal Visual Reasoning in Video-to-Code Generation

Anya Ji, Abhijith Varma Mudunuri, David M. Chan, Alane Suhr

University of California, Berkeley

VLM Evaluation

Open paper AskPDF arXiv

While recent vision-language models (VLMs) have achieved significant improvements on static visual-to-code tasks such as generating code for webpages, charts, or SVGs, it remains unclear whether they can recover temporal dynamics when motion is present. To this end, we introduce Animation2Code, a benchmark for evaluating temporal visual reasoning via reconstructing executable web animation code from videos. Animation2Code consists of 1,069 web animation videos with diverse visual appearances and motion patterns, paired with corresponding HTML/CSS/JavaScript implementations. We propose two human-aligned metrics, appearance similarity and temporal similarity, which allow us to disentangle visual fidelity from temporal alignment when comparing rendered animations against ground-truth samples. Benchmarking state-of-the-art VLMs on this dataset shows that current VLMs struggle to maintain temporal consistency in reconstruction, even when achieving high appearance similarity, including under finetuning and iterative refinement settings. Code and data are available at https://anya-ji.github.io/animation2code-website .

Momentum

Related topics

Latest papers 658