Multimodal Benchmarks

Follow this topic Ask about this topic

Recent momentum

-42%

11 papers in the last 28 days · 0.2% of indexed attention

Twelve weeks of publication activity for this topic as it is defined today.

Weekly history

Recent digests

What was published in this topic, kept on the site without email delivery.

Period ending 2026-09-21

3 new papers

A weekly snapshot of new work published in Multimodal Benchmarks.

Period ending 2026-09-14

2 new papers

A weekly snapshot of new work published in Multimodal Benchmarks.

Period ending 2026-09-07

5 new papers

A weekly snapshot of new work published in Multimodal Benchmarks.

171 papers

Latest in Multimodal Benchmarks

Browse all topics →

Chronicles-OCR: A Cross-Temporal Perception Benchmark for the Evolutionary Trajectory of Chinese Characters
May 12, 2026Gengluo Li, Shangpin Peng, Xingyu Wan +16Optical Character Recognition Deepseek-Ocr
Human-Grounded Multimodal Benchmark with 900K-Scale Aggregated Student Response Distributions from Japan's National Assessment of Academic Ability
May 12, 2026Kyosuke Takami, Yuka Tateisi, Satoshi Sekine +1Multimodal Benchmarks Computerized Adaptive Testing
When Looking Is Not Enough: Visual Attention Structure Reveals Hallucination in MLLMs
May 12, 2026Fanpu Cao, Xin Zou, Xuming Hu +1Large Language Model Hallucination Multi-Modal Large Language Models
Vision2Code: A Multi-Domain Benchmark for Evaluating Image-to-Code Generation
May 11, 2026Ajay Vikram Periasami, Junlin Wang, Bhuwan DhingraCode Generation Multimodal Benchmarks
CADBench: A Multimodal Benchmark for AI-Assisted CAD Program Generation
May 11, 2026Anna C. Doris, Jacob Thomas Sony, Ghadi Nehme +3Text-To-Cad Computer-Aided Design
MulTaBench: Benchmarking Multimodal Tabular Learning with Text and Image
May 11, 2026Alan Arazi, Eilam Shapira, Shoham Grunblat +8Tabular Learning Tabular Foundation Models
SenseBench: A Benchmark for Remote Sensing Low-Level Visual Perception and Description in Large Vision-Language Models
May 11, 2026Chen Zhong, Xiao An, Jiaxing Sun +3Remote Sensing Visual Perception
Multi-domain Multi-modal Document Classification Benchmark with a Multi-level Taxonomy
May 11, 2026Denghao Ma, Qing Liu, Zulong Chen +5Multimodal Documents Multimodal Benchmarks
SciVQR: A Multidisciplinary Multimodal Benchmark for Advanced Scientific Reasoning Evaluation
May 11, 2026Longteng Guo, Xuanxu Lin, Dongze Hao +5Multimodal Reasoning Benchmarks Multimodal Reasoning
Omni-Persona: Systematic Benchmarking and Improving Omnimodal Personalization
May 11, 2026Yeongtak Oh, Dongwook Lee, Sangkwon Park +2Multimodal Benchmarks Omni
SKG-VLA: Scene Knowledge Graph Priors for Structured Scene Semantics and Multimodal Reasoning for Decision Making
May 10, 2026Zeyu Li, Lei LiMultimodal Reasoning Intermediate Scene Representations
EmoS: A High-Fidelity Multimodal Benchmark for Fine-grained Streaming Emotional Understanding
May 9, 2026Pengze Guo, Jingxi Liang, Zhiwen Xie +2Emotion Recognition Empathy
FraudBench: A Multimodal Benchmark for Detecting AI-Generated Fraudulent Refund Evidence
May 9, 2026Xinyu Yan, Boyang Chen, Jiaming Zhang +12Ai-Generated Image Detection Authenticity
McNdroid: A Longitudinal Multimodal Benchmark for Robust Drift Detection in Android Malware
May 7, 2026Md Mahmuduzzaman Kamol, Jesus Lopez, Saeefa Rubaiyet Nowmi +5Malware Families Huawei
Are We Making Progress in Multimodal Domain Generalization? A Comprehensive Benchmark Study
May 7, 2026Hao Dong, Hongzhao Li, Shupan Li +3Multimodal Domain Generalization Multimodal Benchmarks
AstroAlertBench: Evaluating the Accuracy, Reasoning, and Honesty of Multimodal LLMs in Astronomical Classification
May 7, 2026Claire Chen, Jiabao Sean Xiao, Shuze Daniel Liu +7Multimodal Benchmarks Multimodal Classification
HNC: Leveraging Hard Negative Captions towards Models with Fine-Grained Visual-Linguistic Comprehension Capabilities
May 6, 2026Esra Dönmez, Pascal Tilli, Hsiu-Yu Yang +2Image-Text Pairs Fine-Grained Perception
DiffCap-Bench: A Comprehensive, Challenging, Robust Benchmark for Image Difference Captioning
May 6, 2026Yuancheng Wei, Haojie Zhang, Linli Yao +7Remote Sensing Image Change Captioning Multimodal Benchmarks
CC-OCR V2: Benchmarking Large Multimodal Models for Literacy in Real-world Document Processing
May 5, 2026Zhipeng Xu, Junhao Ji, Zulong Chen +10Optical Character Recognition Deepseek-Ocr
MILE: Mixture of Incremental LoRA Experts for Continual Semantic Segmentation across Domains and Modalities
May 5, 2026Shishir Muralidhara, Didier Stricker, René SchusterLow-Rank Adaptation Multimodal Benchmarks
MULTITEXTEDIT: Benchmarking Cross-Lingual Degradation in Text-in-Image Editing
May 4, 2026Liwei Cheng, Shibo Feng, Lunjie Zhou +2Multilingual Benchmark Cross-Lingual Consistency
VIDA: A dataset for Visually Dependent Ambiguity in Multimodal Machine Translation
May 3, 2026Jingheng Pan, Xintong Wang, Longyue Wang +3Multimodal Benchmarks Out-Of-Distribution Detection
VT-Bench: A Unified Benchmark for Visual-Tabular Multi-Modal Learning
May 3, 2026Zi-Yi Jia, Zi-Jian Cheng, Xin-Yue Zhang +4Multimodal Benchmarks Tabular Learning
Mitigating Multimodal LLMs Hallucinations via Relevance Propagation at Inference Time
May 3, 2026Itai Allouche, Joseph KeshetLarge Language Model Hallucination Multi-Modal Large Language Models
Omni-Fake: Benchmarking Unified Multimodal Social Media Deepfake Detection
May 2, 2026Tianxiao Li, Zhenglin Huang, Haiquan Wen +10Explainable Deepfake Detection Multimodal Benchmarks
WildTableBench: Benchmarking Multimodal Foundation Models on Table Understanding In the Wild
May 1, 2026Junzhe Huang, Xiaoxiao Sun, Yan Yang +6Tabular Data Multimodal Benchmarks
Let ViT Speak: Generative Language-Image Pre-training
May 1, 2026Yan Fang, Mengcheng Lan, Zilong Huang +7Large Vision Language Models Vision Encoders
InteractWeb-Bench: Can Multimodal Agent Escape Blind Execution in Interactive Website Generation?
Apr 30, 2026Qiyao Wang, Haoran Hu, Longze Chen +4Web Mock-Interface Synthesis
TRIP-Evaluate: An Open Multimodal Benchmark for Evaluating Large Models in Transportation
Apr 29, 2026Han Gong, Zhen Zhou, Yunyang Shi +4Multimodal Benchmarks Tour
Evian: Towards Explainable Visual Instruction-tuning Data Auditing
Apr 22, 2026Zimu Jia, Mingjie Xu, Andrew Estornell +1Large Vision Language Models Multimodal Benchmarks
DistortBench: Benchmarking Vision Language Models on Image Distortion Identification
Apr 21, 2026Divyanshu Goyal, Akhil Eppa, Vanya Bannihatti KumarDistortion Multimodal Benchmarks
Lost in Translation: Do LVLM Judges Generalize Across Languages?
Apr 21, 2026Md Tahmid Rahman Laskar, Mohammed Saidul Islam, Mir Tafseer Nayeem +5Multilingual Benchmark Large Language Model Judges
MathNet: a Global Multimodal Benchmark for Mathematical Reasoning and Retrieval
Apr 20, 2026Shaden Alshammari, Kevin Wen, Abrar Zainal +5Olympiad-Level Problem Math Problems
WebCompass: Towards Multimodal Web Coding Evaluation for Code Language Models
Apr 20, 2026Xinping Lei, Xinyu Che, Junqi Xiong +16Web Evaluation Benchmarks
MACS: Modality-Aware Capacity Scaling for Efficient Multimodal MoE Inference
Apr 19, 2026Bo Li, Chuan Wu, Shaolin ZhuMixture-Of-Expert Inference Multimodal Benchmarks
Beyond Text-Dominance: Understanding Modality Preference of Omni-modal Large Language Models
Apr 18, 2026Xinru Yan, Boxi Cao, Yaojie Lu +4Omni-Modal Multi-Modal Large Language Models
MetaDent: Labeling Clinical Images for Vision-Language Models in Dentistry
Apr 16, 2026Meng-Xun Li, Wen-Hui Deng, Zhi-Xing Wu +6Medical Vision-Language Models Multimodal Benchmarks
MedLayBench-V: A Large-Scale Benchmark for Expert-Lay Semantic Alignment in Medical Vision Language Models
Apr 7, 2026Han Jang, Junhyeok Lee, Heeseong Eum +1Medical Vision-Language Models Multimodal Benchmarks
TSHA: A Benchmark for Visual Language Models in Trustworthy Safety Hazard Assessment Scenarios
Mar 31, 2026Qiucheng Yu, Ruijie Xu, Mingang Chen +2Hazard Large Language Model Safety
OmniACBench: A Benchmark for Evaluating Context-Grounded Acoustic Control in Omni-Modal Models
Mar 25, 2026Seunghee Kim, Bumkyu Park, Kyudan Jung +5Omni-Modal Multimodal Benchmarks
Designing UNICORN: a Unified Benchmark for Imaging in Computational Pathology, Radiology, and Natural Language
Mar 3, 2026Michelle Stegeman, Lena Philipp, Fennie van der Graaf +20Pathology Foundation Models Multimodal Clinical Data
VectorGym: A Multi-Task Benchmark for SVG Code Generation, Sketching and Editing
Feb 22, 2026Joan Rodriguez, Haotian Zhang, Abhay Puri +14Scalable Vector Graphic Sketches
Multimodal Benchmark for Safety Assessment in Industrial Inspection Scenarios
Jan 29, 2026Zeyi Liu, Shuang Liu, Jihai Min +7Inspection Multi-Modal Sensing
6 Fingers, 1 Kidney: Natural Adversarial Medical Images Reveal Critical Weaknesses of Vision-Language Models
Dec 3, 2025Leon Mayer, Piotr Kalinowski, Caroline Ebersbach +6Medical Vision-Language Models Multimodal Benchmarks
Johnson-Lindenstrauss Lemma Guided Network for Efficient 3D Medical Segmentation
Sep 26, 2025Jinpeng Lu, Linghan Cai, Yinda Chen +43D Semantic Segmentation Medical Image Segmentation
The P $^3$ Dataset: Pixels, Points and Polygons for Multimodal Building Vectorization
May 21, 2025Raphael Sulzer, Liuyun Duan, Nicolas Girard +1Light Detection And Ranging Point Clouds Real3D-Ad Datasets
HumaniBench: A Human-Centric Framework for Large Multimodal Models Evaluation
May 16, 2025Shaina Raza, Aravind Narayanan, Vahid Reza Khazaie +6Multimodal Benchmarks Human-Annotated Benchmark
Deepfake Media Generation and Detection in the Generative AI Era: A Survey and Outlook
Nov 29, 2024Florinel-Alin Croitoru, Andrei-Iulian Hiji, Vlad Hondru +7Deepfake Detection Multimodal Benchmarks
VLMEvalKit: An Open-Source Toolkit for Evaluating Large Multi-Modality Models
Jul 16, 2024Haodong Duan, Xinyu Fang, Junming Yang +41Multimodal Benchmarks Evaluation Benchmarks
Solve the Missing First Step: Can VLMs Standardize Raw Heterogeneous Medical Data?
Date pendingXin Chen, Dongliang Xu, Cunhao Zhu +5Diagnostic Benchmark Multimodal Clinical Data
MedGEN-Bench: A Contextually Entangled Benchmark for Open-ended Multimodal Medical Generation
Date pendingJunjie Yang, Yuhao Yan, Gang Wu +12Medical Visual Question Answering Medical Vision-Language Models

May 12, 2026 · cs.CVJ/K move · Enter open · S save

Chronicles-OCR: A Cross-Temporal Perception Benchmark for the Evolutionary Trajectory of Chinese Characters

Gengluo Li, Shangpin Peng, Xingyu Wan, Chengquan Zhang+15

Institute of Information Engineering, Chinese Academy of Sciences · Tencent · Anyang Normal University · The Palace Museum · Nankai University

Vision Large Language Models (VLLMs) have achieved remarkable success in modern text-rich visual understanding. However, their perceptual robustness in the face of the continuous morphological evolution of historical writing systems remains largely unexplored. Existing ancient text datasets typically focus on isolated historical periods, failing to capture the systematic visual distribution shifts spanning thousands of years. To bridge this gap and empower Digital Humanities, we introduce Chronicles-OCR, the first comprehensive benchmark specifically designed to evaluate the cross-temporal visual perception capabilities of VLLMs across the complete evolutionary trajectory of Chinese characters, known as the Seven Chinese Scripts. Curated in collaboration with top-tier institutional domain experts, the dataset comprises 2,800 strictly balanced images encompassing highly diverse physical media, ranging from tortoise shells to paper-based calligraphy. To accommodate the drastic morphological and topological variations across different historical stages, we propose a novel Stage-Adaptive Annotation Paradigm. Based on this, Chronicles-OCR formulates four rigorous quantitative tasks: cross-period character spotting, fine-grained archaic character recognition via visual referring, ancient text parsing, and script classification. By isolating visual perception from semantic reasoning, Chronicles-OCR provides an authoritative platform to expose the limitations of current VLLMs, paving the way for robust, evolution-aware historical text perception. Chronicles-OCR is publicly available at https://github.com/VirtualLUOUCAS/Chronicles-OCR.

Optical Character Recognition Deepseek-Ocr Latin Multimodal Benchmarks Characterization Evolution

Open paper Ask about it PDF

Recent momentum

Neighbouring directions

Recent digests

3 new papers

2 new papers

5 new papers

Latest in Multimodal Benchmarks