Multimodal Model Evaluation

Ask about this topic

Momentum

49 papers in the last four weeks, up 48% on the four weeks before. 0.5% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 496

All topics
CardsList

  1. Two-Level Meta-Rubrics for Evaluating Open-Ended Generation: GAMUT, a Benchmark for Factual Completeness

    Jul 21, 2026Xilun Chen, Zhaleh Feizollahi, Ross Goodwin +5Open-Ended GenerationLLM-as-a-Judge

    Meta AI

  2. MeetingToM: Evaluating Multimodal LLMs on Theory-of-Mind Reasoning in Multi-Party Meetings

    Jul 21, 2026Ziyi Wang, Yuhang Wu, Dongxu Piao +3Theory of MindMultimodal Large Language Models

    College of AI, Tsinghua University · Department of Biostatistics and Bioinformatics, Duke University

  3. Computational Humor with Multimodal LLMs: Methods, Datasets, Evaluation, and Challenges

    Jul 21, 2026Tuo Liang, Zhe Hu, Disheng Liu +2Multimodal Large Language ModelsComputational Creativity

    Case Western Reserve University · The Hong Kong Polytechnic University

  4. Style over Substance: A Shortcut Audit of Emotion-Description Preference Evaluation

    Jul 20, 2026Jiabing Yang, Yixiang Chen, Yuan Xu +6Human Preference EvaluationPairwise Preference Evaluation

    University of Chinese Academy of Sciences Institute of Automation, Chinese Academy of Sciences Beijing, China

  5. Now We Know? A Systematic Comparison of TerraMind and THOR

    Jul 20, 2026Frederick Schindlegger, Kenzo Bounegta, Eva Gmelich Meijling +5Geospatial Foundation ModelsRemote Sensing

    University of Münster, Münster, Germany · 2ESA Φ-lab, ESRIN, Frascati, Italy · 3Rubicon, Paris, France +2

  6. How Reliable Are Multimodal Signals of Conversational State? Evidence from Remote Dyadic Collaborative Tasks

    Jul 20, 2026Tahiya ChowdhurySpoken Dialogue SystemsMultimodal Model Evaluation

    Colby College Waterville, Maine, United States

  7. Can Multimodal Large Language Models Understand OCT?

    Jul 18, 2026Baochen Fu, Wenzhi Deng, Baihao Jin +5Medical ImagingMedical Image Analysis

    1Shandong University · 4Shandong Provincial Qianfoshan Hospital · University of Science and Technology of China +2

  8. Knowing the Self, Understanding the World: A Dual-Cognition Benchmark for UAV Spatio-temporal Reasoning with MLLMs

    Jul 17, 2026Like Liu, Zhengzheng Xu, Haitao He +3Multimodal GroundingAerial Robotics

    Northwestern Polytechnical University Xi’an, Shaanxi, China

  9. SLAPBench: Benchmarking Multimodal Large Language Models for Four-Finger SLAP Fingerprint Verification

    Jul 17, 2026Bibesh Pyakurel, M. G. Sarwar MurshedBiometric IdentificationMultimodal Large Language Models

    Computer Science Department, University of Wisconsin–Green Bay, Green Bay, WI, USA

  10. MM-IssueLoc: A Controlled Benchmark for Evaluating Visual Evidence in Multimodal Repository-Level Issue Localization

    Jul 16, 2026Shaoxiong Zhan, Shi Hu, Boyu Feng +7Multimodal Model Evaluation

    Tsinghua University

  11. Benchmarking Multimodal Large Language Models for Scientific Visualization Literacy

    Jul 16, 2026Patrick Phuoc Do, Chau M. Ta, Chaoli WangScientific VisualizationData Visualization

    University of Notre Dame

  12. Contextualized Evaluation of Vision Language Models through Dynamic, Multi-turn Interactions

    Jul 16, 2026Yijiang Li, Huiqi Zou, Bingyang Wang +1VLM EvaluationVLM Hallucination

    UC San Diego · Northeastern University · Georgia Institute of Technology +1

  13. AVSCap: Orchestrating Audio-Visual Synergy for Omni-modal Video Captioning

    Jul 14, 2026Yanghai Wang, Jiahao Wang, Jiafu Tang +9Audio-Visual UnderstandingAudio-Visual Alignment

    NJU-LINK Team, Nanjing University · Kling Team, Kuaishou Technology · Institute of Automation, Chinese Academy of Sciences

  14. Beyond the Single Camera: Agentic Multi-View Reasoning in Sports Video Understanding

    Jul 13, 2026Kerui Chen, Jinglu Wang, Xiaoyi Zhang +1Active PerceptionSports Video Analysis

    Zhejiang University · Microsoft Research Asia

  15. GDP.pdf: Benchmarking Grounded Multimodal Reasoning over Professional PDF Documents

    Jul 13, 2026Suhaas Garre, Emily Ritchie, Sushant Mehta +1Multimodal GroundingMultimodal Reasoning

    Surge AI

  16. Do Video-LLMs Actually Watch? Diagnosing Character-Tracking Failures in Long-Form Video

    Jul 13, 2026Mohammad Al-Ratrout, Shayla Sharmin, Aditya Raikwar +1Visual Object TrackingVideo-Language Model Evaluation

  17. Capabilities of Claude Fable 5 on Biomedical Challenge Problems

    Jul 12, 2026Dominic Okonkwo, Magnus Hodgson, Temitope I. David +1Biomedical QALLM Refusal Behavior

    School of Computing, University of Georgia · Department of Chemistry, University of Illinois · Institute of Bioinformatics, University of Georgia

  18. Promptable Concept Segmentation from Above: Evaluating SAM 3's Zero-Shot and One-Shot Capabilities in Remote Sensing

    Jul 10, 2026Mohammad Dabaja, Turgay CelikRemote Sensing Image UnderstandingRemote Sensing Small Object Detection

    Department of ICT, University of Agder, Grimstad, Norway

  19. MedRealMM: A Real-World Multimodal Benchmark for Chinese Online Medical Consultation

    Jul 10, 2026Runhan Shi, Quan Zhou, Yuqian Xu +14HealthcareClinical Language Model Evaluation

    JD Health International Inc. · Shanghai Jiao Tong University · National University of Singapore +3

  20. Blind-Spots-Bench: Evaluating Blind Spots in Multimodal Models

    Jul 9, 2026Matteo Santelmo, Xiuying Wei, Israa Fakih +5VLM EvaluationLLM Evaluation

    Ecole Polytechnique Fédérale de Lausanne (EPFL), Switzerland

  21. UniRef-UAV: A Multimodal Benchmark for Universal Referring in UAV Imagery

    Jul 9, 2026Haibin Tian, Huichao Xie, Xuelin Qian +3Multimodal GroundingMultimodal Model Evaluation

    School of Automation, Northwestern Polytechnical University, Xi’an 710072, China · College of Missile Engineering, Rocket Force University of Engineering, Xi’an 710038, China · School of Artificial Intelligence, Chongqing University of Posts and Telecommunications, Chongqing 400065, China

  22. Rethinking Multimodal Time-Series Forecasting Evaluation

    Jul 8, 2026Haoxin Liu, Yichen Zhou, Rajat Sen +2Multivariate Time Series ForecastingZero-Shot Time Series Forecasting

    Georgia Institute of Technology · Google Research

  23. HoloCount: A Holistic Visual Counting Benchmark for MLLMs

    Jul 7, 2026Jinhong Deng, Limeng Qiao, Guanglu WanMultimodal RobustnessObject Counting

    Meituan

  24. Overview of the NLPCC 2026 Shared Task 1: Difficulty-Aware Multilingual and Multimodal Medical Instructional Video Understanding Evaluation

    Jul 7, 2026Shenxi Liu, Kan Li, Mingyang Zhao +2Multimodal IRTemporal Video Grounding

    School of Computer Science and Technology, Beijing Institute of Technology · Department of Computing, The Hong Kong Polytechnic University · Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences

  25. Music I Care About: Automated Multimodal Benchmarking of LLM Music Perception Skills on (Almost) Any Music

    Jul 7, 2026Tomáš Sourada, Katia Vendrame, Jan HajičAudio-Language Model EvaluationSynthetic Benchmark Generation

    Charles University · Brno University of Technology

  26. Propose and Attend: Training-free MLLM Grounding Confidence via Multi-Token Localized Attention

    Jul 7, 2026Daniel Shalam, Emanuel Ben Baruch, Avi Ben Cohen +1Multimodal GroundingLLM Uncertainty Estimation

    Amazon

  27. Evaluating and Understanding Model Editing for Medical Vision Language Models

    Jul 6, 2026Guli Zhu, Chenwei Wu, Liyue ShenKnowledge EditingMedical VLMs

    EECS, University of Michigan, Ann Arbor, MI 48105, USA

  28. Dashboard2Code: Evaluating Multimodal Models on Reconstructing Interactive Dashboards

    Jul 6, 2026Tianhao Niu, Ziyu Han, Qiguang Chen +5Data VisualizationCode Generation Evaluation

    Research Center for Social Computing and Interactive Robotics Harbin Institute of Technology, China

  29. PPE-Bench: A Benchmark for Evaluating MLLM Unlearning under Private-Public Entanglement

    Jul 3, 2026Xianren Zhang, Delvin Ce Zhang, Dongwon Lee +1Privacy Leakage in Language ModelsMLLM Unlearning

    The Pennsylvania State University · University of Sheffield

  30. K9-Bench: Evaluating Multimodal LLMs on Canine-Centric Videos

    Jul 2, 2026Khush Attarde, Yusuf Ali, Megha Thukral +3Video UnderstandingMultimodal Large Language Models

    Ogmen Robotics Inc., Massachusetts, USA · Georgia Institute of Technology, Atlanta, USA