Multimodal Benchmarks

Ask about this topic

Momentum

16 papers in the last four weeks, up 60% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 166

All topics
CardsList

  1. MMPostTrainBench: Benchmarking Autonomous Research for Multimodal Post-Training

    Oct 4, 2026Yuxin Liu, Yuxuan Wang, Zhenxin Lei +9Multimodal Benchmarks

    University of Science and Technology of China · Alibaba Token Hub, Alibaba Group · University of the Chinese Academy of Sciences +2

  2. Architectural Sampling: Test-Time Scaling via Computational Diversity in Frozen Vision-Language Models

    Oct 1, 2026Akshit Singh, Shyam Marjit, Wei Lin +2Test-Time ScalingFrozen Vision-Language Models

    Dartmouth College · IISc Bangalore · Xero

  3. EviRover: Reinforcing Agentic Perception Beyond a Glance

    Sep 30, 2026Kaixuan Fan, Kaituo Feng, Tianshuo Peng +4Visual PerceptionPerception

    MMLab, The Chinese University of Hong Kong · Fudan University

  4. MM-FinEval: A Multi-Task Multimodal Benchmark for Real-World Financial Forecasting

    Sep 29, 2026Dong Shu, Yanguang Liu, Huopu Zhang +4Multimodal Benchmarks

    Northwestern University · New Jersey Institute of Technology · Georgia Institute of Technology +2

  5. What Comes Next? Omni-StoryBench for Evaluating Story-Grounded Omnimodal Generation

    Sep 29, 2026Sieun Hyeon, Yejoon Lee, Mintaek Lim +3Omni-ModalMultimodal Generation

    AIDAS Laboratory, 1ECE & 2IPAI, Seoul National University

  6. VISTA-Bench: Benchmarking Multilingual Image Translation with Image-Specific Rubrics

    Sep 29, 2026Bo Lv, Mao Zheng, Zheng Li +3Multimodal BenchmarksUnpaired Image-To-Image Translation

    Foundation Model Department, Tencent

  7. Beyond Binary Preferences: Graded Preference Optimization for Limb-Motion Captioning

    Sep 29, 2026Yanan Wang, Tingsong Li, Kaixun Jiang +3Video CaptioningOpen-Vocabulary Action Recognition

    Zhejiang University · Alibaba Token Hub, Alibaba Group · University of Science and Technology of China +2

  8. ARCH-B: Architectural Representation, Comprehension and Hierarchy Benchmark

    Sep 28, 2026Kieran Sagar Parikh, Jose Luis Garcia del Castillo y LopezMultimodal BenchmarksBuilding Information Modeling

    Harvard University · Northeastern University

  9. EngIntervene: Benchmarking Multimodal Engineering State Understanding and Design Intervention Reasoning

    Sep 27, 2026Jinchang Zhang, Yingda Tao, Jiakai Lin +1EngineeringMultimodal Benchmarks

    Intelligent Vision and Sensing (IVS) Lab Indiana University Bloomington Bloomington, IN, USA

  10. PRISM-VLM: A Multi-Axis Discriminative Benchmark for Compact Vision-Language Models

    Sep 23, 2026Sanghee Park, Kee-Eung KimRecent Vision-Language ModelsMultimodal Benchmarks

    NAVER Cloud AI · KAIST AI

  11. BAS-OPD: Budget-Aware Selective On-Policy Self-Distillation for Fine-Grained Multimodal Perception

    Sep 22, 2026Zihan Chen, Hengguang Zhou, Yuan Kang +5Unsupervised On-Policy Self-DistillationRecent Vision-Language Models

    HFIPS, Chinese Academy of Sciences · University of Science and Technology of China · University of California, Los Angeles

  12. TAC-Time: Texts as Channels For Multimodal Time Series Forecasting

    Sep 21, 2026Jiayi Liang, Xiaotian Gu, Xinyu Xie +2Multivariate Time Series ForecastingTemporal Dynamics

    East China Normal University Shanghai, China

  13. MiX: Micro-Inverted-Scaling for End-to-End Low-Bit Vision-Language Model Acceleration

    Sep 17, 2026Yuan Liao, Jae-sun SeoGradient QuantizationMix

    Cornell Tech, Cornell University New York, NY, USA

  14. MUSE: Benchmarking Large Vision-Language Models on Multi-Modal Understanding in Situated Education

    Sep 16, 2026Luyao Zhu, Xun Wei Yee, Wei Li +2Multimodal BenchmarksMultimodal Understanding

    AI Singapore, National University of Singapore, Singapore · Institute of Advanced Intelligence and Computing, A*STAR · School of Computing, National University of Singapore, Singapore

  15. Beyond Numerical Time Series: A Unified Benchmark for Multimodal Forecasting with Heterogeneous Context

    Sep 14, 2026Peng Chen, Zhihao Zhuang, Hongzhou Chen +6Time Series Foundation ModelsMultimodal Benchmarks

    Ant International · Hangzhou, China

  16. Sci-MMR: Benchmarking Multi-Step Evidence-Grounded Scientific Reasoning in Multimodal Agents

    Sep 12, 2026Jiaqiang Li, Yajie Yang, Zhiheng Xi +15Scientific FigureMultimodal Benchmarks

    Fudan NLP Lab · Fudan NLP Group

  17. Urban Boundaries, Social Barriers: A Benchmark and Vision-Centric Framework for Mapping Gated Communities and Equity Implications

    Sep 3, 2026Minwei Zhao, Weiming Zhang, Jiawang Du +4Urban EnvironmentsDelineation

    The Hong Kong University of Science and Technology (Guangzhou) · School of Public Administration and Policy, Renmin University of China · Sony AI +1

  18. The Visual Insensitivity Gap: Diagnosing When Vision-Language Models Fail to Use Visual Evidence

    Sep 1, 2026Genpei ZhangRecent Vision-Language ModelsVisual Perception

    University of Wisconsin–Madison

  19. Do VLMs Share Safety Neurons Across Modalities?

    Aug 31, 2026Jiaxuan Li, Jiahao Zhang, Duc Minh Vo +3Multimodal BenchmarksNeurons

    SB Intuitions Corp., Tokyo, Japan

  20. Read the Room, Read the Image: Understanding Indirect Speech Acts in Multimodal Visual Contexts

    Aug 31, 2026Jaehee Kim, Ji Hoon Chung, Seoyoon Park +5Multimodal BenchmarksDialogue Benchmarks

    Department of Korean Language and Literature, Yonsei University · Interdisciplinary Graduate Program of Linguistics and Informatics, Yonsei University · LG AI Research +1

  21. When History Is Multimodal: Rethinking Context Management for Long-Horizon Agents

    Aug 30, 2026Jiaqi Su, Cong Pang, Jiawei Hong +4Multimodal MemoryLong-Horizon Agents

    National University of Singapore · SenseTime Research · ShanghaiTech University

  22. MVC-Bench: Benchmarking Calibration of Medical Vision-Language Models

    Aug 27, 2026Ashshak Sharifdeen, Shihab Aaqil Ahamed, Ufaq Khan +6Medical Vision-Language ModelsMultimodal Benchmarks

    1Mohamed bin Zayed University of AI · 2Sabaragamuwa University of Sri Lanka · 3Digital Platform Development, SLT PLC

  23. Diagram-MMU: A Multi-Modal Benchmark for Scientific Diagrams

    Aug 12, 2026Weihao Bo, Shan Zhang, Yanpeng Sun +7DiagramsDiagnostic Benchmark

    Nanjing University of Science and Technology · Baidu Inc · AIML, Adelaide University +4

  24. MBA: Multimodal Benchmark and Agents for Real-World Business Ideation

    Aug 12, 2026Hojun Choi, Jaeyo Shin, Suin Lee +1Multimodal AgentsMultimodal Benchmarks

    KAIST AI, Republic of Korea

  25. Rethinking Text-Based Image Retrieval in Specific Domain

    Aug 11, 2026Jingyang Tan, Sheng Yang, Yuanpeng Chen +4Multimodal RetrievalFine-Grained Perception

    Department of Electronic and Communication Engineering, Harbin Institute of Technology · School of Data Science, Fudan University · Independent Researcher +1

  26. Hyperbolic Multimodal Continual Learning

    Aug 10, 2026Jiahong Liu, Ming Shen, Xiaohao Liu +4Hyperbolic LearningReplay-Based Continual Learning

    Department of Computer Science and Engineering, The Chinese University of Hong Kong, Hong Kong, China · School of Computing, National University of Singapore, Singapore · Department of Computer Science, Yale University, New Haven, CT, United States +1

  27. Jako Tako or Fluent? Presenting PoVisLE: A Polish Vision-Language Evaluation

    Aug 7, 2026Anna Kołos, Grzegorz Statkiewicz, Karolina Seweryn +3Multilingual BenchmarkMultimodal Benchmarks

    NASK National Research Institute, Warsaw, Poland

  28. An AI4AI Framework for Visual Token Pruning

    Aug 7, 2026Zhen Liu, Wenli Huang, Wei Song +3Visual Token PruningMultimodal Benchmarks

    Xi’an Jiaotong University · Ningbo University of Technology · 3North China University of Technology +3

  29. OmniMech: All-in-one Multimodal Mechanical Benchmark for 3D Reconstruction

    Aug 6, 2026Taiting Lu, Runze Liu, Ziwei Dong +18Large-Scale Benchmark3D Reconstruction

    Pennsylvania State University · Independent Researcher · Shanghai Jiao Tong University +2

  30. OmniRouting: A Semantic-Coupled Multimodal Benchmark for Constraint-Aware Spatial Reasoning in PCB Routing

    Aug 5, 2026Taiting Lu, Kaiyuan Lin, Ziwei Dong +18Electronic Design AutomationLarge Language Model Routing

    1Pennsylvania State University · 2Independent Researcher · 3Shanghai Jiao Tong University +3

  31. BanglaWild: An In-the-Wild Bengali Scene Text Recognition Benchmark for OCR and Vision-Language Models

    Aug 4, 2026Sadab Shiper, Tawsif Tashwar Dipto, Mir Md Inzamam +1Optical Character RecognitionScene Text Recognition

    Islamic University of Technology, Bangladesh · BRAC University, Bangladesh

  32. LDU-Bench: Multimodal LLM Evaluation for Lithography Defect Understanding under Layout-Varying Circuit Backgrounds

    Aug 4, 2026Huanglong Ji, Botong Zhao, Shujing Lv +1LithographyDiagnostic Benchmark

  33. Remember-R1: Mitigating Long-Context Visual Forgetting through Reinforcement Learning

    Aug 2, 2026Jianmin Chen, Jiaqi Tang, Wei Wei +9Latent Visual ReasoningVisual Memory

    Hong Kong University of Science and Technology Hong Kong, China · Northwestern Polytechnical University Xi’an, China · Zhejiang University Hangzhou, China

  34. Distill What the Student Can See: Fisher-Projected On-Policy Distillation for Vision-Language Models

    Aug 2, 2026Leyan Xue, Feng Xiong, Mingjun Ma +1Token-Level UncertaintyMultimodal Benchmarks

    College of Intelligence and Computing Tianjin University Tianjin, China

  35. PathView-Bench: Can Multimodal Large Language Models Achieve Fine-grained Multiscale Understanding of Pathology Images?

    Jul 30, 2026Zongyi Chen, Yu Liang, Jie Lin +1Multimodal BenchmarksMultimodal Clinical Data

    National Institute for Data Science in Health and Medicine, Xiamen University · Institute of Artificial Intelligence Xiamen University, Xiamen, China · Department of Computer Science School of Informatics Xiamen University, Xiamen, China

  36. MMHBench: A Multi-Perspective Benchmark for Mental Health Understanding in Long-Form Videos

    Jul 30, 2026Jinpeng Hu, Erqiang Wang, Shan Wang +4Video Multimodal Large Language ModelsMental Health

    Hefei University of Technology · Faculty of Psychology, Beijing Normal University · The Chinese University of Hong Kong, Shenzhen +1

  37. AHA-Memes: A Fine-Grained Multimodal Benchmark for Understanding Hate in Arabic Memes

    Jul 29, 2026Mohamed Bayan Kmainasi, Ali Ezzat Shahroor, Abul Hasnat +3MemesHate Speech

    Qatar Computing Research Institute, Qatar · 2APAVI.AI, France · 3Hamad Bin Khalifa University, Qatar +1

  38. Understanding Knowledge Transfer Mechanism in Heterogeneous MLLM Fusion: A Simple Linear Approach

    Jul 29, 2026Yinghao Hou, Jiahe Fan, Yuanhao Pu +2Cross-Task Knowledge TransferMultimodal Benchmarks

    University of Science and Technology of China

  39. Co-Learning for Missing Arbitrary Modalities in Multi-modal Classification

    Jul 27, 2026Francisco Mena, Dino Ienco, Roberto Interdonato +2Missing ModalitiesMultimodal Classification

    GFZ Helmholtz Center for Geosciences, Potsdam, Germany · INRAE, UMR TETIS, University of Montpellier, Montpellier, France · INRIA, EVERGREEN, University of Montpellier, Montpellier, France +1

  40. MulRobBench: A Decision-Level Benchmark for Safe and Security-Policy-Compliant Multimodal UAV Agents

    Jul 26, 2026Belal S. Alsinglawi, Weizheng Wang, Junyi Wu +3Multi-UavUnmanned Aerial Vehicles

    College of Technological Innovation, Zayed University, Abu Dhabi, United Arab Emirates

  41. ObsDriveBench: Benchmarking Multimodal Understanding under Adverse Weather with Observability Awareness

    Jul 26, 2026Qiao Yan, Yihan Wang, Zhenghao Xing +2Autonomous DrivingMultimodal Benchmarks

    Department of Computer Science and Engineering, The Chinese University of Hong Kong · Institute of Medical Intelligence and XR, The Chinese University of Hong Kong

  42. Twins: Learn to Predict Unified Representations with Focal Loss

    Jul 24, 2026Kaixiong Gong, Xin Cai, Bin Lin +9Conditional Variational AutoencoderUnified Representation

    The Chinese University of Hong Kong · Tencent, Hunyuan · City University of Hong Kong +2

  43. Time-Reversed Imaging: A Multimodal Benchmark and Framework for Reconstructing Past Human-Environment Interactions

    Jul 24, 2026Jorge Bacca, Kebin Contreras, Luis Toscano-Palomino +1Scene ReconstructionComplementary Modalities

  44. RadSight: Towards Perceptually Reliable Multimodal Radiology Image Understanding

    Jul 24, 2026Jianqin Liu, Weiwei Cao, Wanxing Chang +7Medical ImagesMultimodal Clinical Data

    1DAMO Academy, Alibaba Group · University of Electronic Science and Technology of China

  45. Medical-Checklist: Assessing the Comprehension of Medical Images by Multimodal Models

    Jul 24, 2026Bannapol Limanond, Masanori Suganuma, Takayuki OkataniMultimodal Clinical DataMedical Vision-Language Models

  46. Khondo: A Multimodal Benchmark for Document Packet Splitting of Bangla Forms

    Jul 23, 2026Abu Tyeb Azad, Fahim Ahmed, Ishita Sur Apan +7DocumentBangla

    Wichita State University, USA · CCDS, Independent University, Bangladesh · Amazon GenAI, USA

  47. Can Multimodal Large Language Models Understand OCT?

    Jul 18, 2026Baochen Fu, Wenzhi Deng, Baihao Jin +5Optical Coherence TomographyMultimodal Benchmarks

    1Shandong University · 4Shandong Provincial Qianfoshan Hospital · University of Science and Technology of China +2

  48. MM-IssueLoc: A Controlled Benchmark for Evaluating Visual Evidence in Multimodal Repository-Level Issue Localization

    Jul 16, 2026Shaoxiong Zhan, Shi Hu, Boyu Feng +7Multimodal BenchmarksMultimodal Retrieval

    Tsinghua University

  49. VIABench: A Comprehensive Video Benchmark Collected from Blind Individuals for Visual Impairment Assistance

    Jul 16, 2026Yunfeng Liu, Yuandong Yang, Jiarui Han +5Assistive AgentsMultimodal Benchmarks

    1Nanjing University · 2Shanghai AI Laboratory

  50. CoRe: A Comprehensive Framework for Cross-Image Comparative Reasoning in Vision-Language Models

    Jul 14, 2026Lin Peng, Cong Wan, Zeyu Guo +2Fine-Grained PerceptionMultimodal Benchmarks

    Xi’an Jiaotong University Xi’an, China

  51. What Does Your Short-Answer VQA Score Actually Measure? Evaluator-Dependent Instability in Multimodal Short-Answer Benchmarks

    Jul 11, 2026Guanhua Ye, Niu Jingbin, Yan Li +4Multiple-Choice Visual Question AnsweringAnswer

    Beijing University of Posts and Telecommunications Beijing, China

  52. MedRealMM: A Real-World Multimodal Benchmark for Chinese Online Medical Consultation

    Jul 10, 2026Runhan Shi, Quan Zhou, Yuqian Xu +14Multimodal Clinical DataMultimodal Benchmarks

    JD Health International Inc. · Shanghai Jiao Tong University · National University of Singapore +3

  53. UniRef-UAV: A Multimodal Benchmark for Universal Referring in UAV Imagery

    Jul 9, 2026Haibin Tian, Huichao Xie, Xuelin Qian +3Aerial ImageryVisual Grounding Benchmarks

    School of Automation, Northwestern Polytechnical University, Xi’an 710072, China · College of Missile Engineering, Rocket Force University of Engineering, Xi’an 710038, China · School of Artificial Intelligence, Chongqing University of Posts and Telecommunications, Chongqing 400065, China

  54. Does AI Understand Imaging? A Systematic Benchmark of Agentic AI for Computational Imaging Tasks

    Jul 8, 2026Ethan Chung, Chuanjun Zheng, Jasper Tan +3Multi-Image Editing BenchmarksMultimodal Benchmarks

    University of Hawaii at Manoa · Glass Imaging

  55. Music I Care About: Automated Multimodal Benchmarking of LLM Music Perception Skills on (Almost) Any Music

    Jul 7, 2026Tomáš Sourada, Katia Vendrame, Jan HajičMultimodal Benchmarks

    Charles University · Brno University of Technology

  56. Evaluating and Understanding Model Editing for Medical Vision Language Models

    Jul 6, 2026Guli Zhu, Chenwei Wu, Liyue ShenMedical Vision-Language ModelsMultimodal Benchmarks

    EECS, University of Michigan, Ann Arbor, MI 48105, USA

  57. RADIO1D: Elastic Representations for Condensed Vision Modeling

    Jul 3, 2026Greg Heinrich, Mike Ranzinger, Collin McCarthy +6Vision Foundation ModelsMultimodal Benchmarks

    1NVIDIA

  58. OmniLayout: A Schematic-Coupled Multimodal Benchmark for Constraint-Aware Geometric Reasoning in PCB Layout

    Jul 3, 2026Taiting Lu, Kaiyuan Lin, Mingjia Wang +12Printed Circuit BoardLayouts

    Pennsylvania State University · Shanghai Jiao Tong University · Independent Researcher +2

  59. MMBench-Live: A Continuously Evolving Benchmark for Multimodal Models

    Jul 2, 2026Yuanzhi Liu, Shousheng Zhao, Bo Zhou +2Multimodal BenchmarksEvaluation Benchmarks

    1Beijing University of Posts and Telecommunications, Beijing, China

  60. MS-Resampler: Multi-Scope Visual Resampling for Efficient Multimodal LLMs

    Jun 30, 2026Zhongyang Li, Yaqian Li, Faming Fang +6Multimodal Large Language ModelsRecent Vision-Language Models

    Li Auto Inc. · East China Normal University.