Multimodal Model Evaluation

Follow this topic Ask about this topic

Momentum

49 papers in the last four weeks, up 44% on the four weeks before. 0.5% of all new papers.

Jul 13Week of Sep 28

Latest papers 496

All topics

Can MLLMs Reason About Visual Persuasion? Evaluating the Efficacy and Faithfulness of Reasoning
May 9, 2026Naeun Lee, Hyunjong Kim, Sunghwan Choi +2VLM Reasoning Multimodal Model Evaluation
Seoul National University
DocScope: Benchmarking Verifiable Reasoning for Trustworthy Long-Document Understanding
May 9, 2026Xiang Feng, Jiawei Zhou, Zhangfeng Huang +6LLM Grounding LLM Answer Verification
School of Computer Science, National Engineering Research Center for Multimedia Software and Hubei Key Laboratory of Multimedia and Network Communication Engineering, Wuhan University, China · Alibaba Group, Hangzhou, China · Independent Researcher +1
EmoS: A High-Fidelity Multimodal Benchmark for Fine-grained Streaming Emotional Understanding
May 9, 2026Pengze Guo, Jingxi Liang, Zhiwen Xie +2Multimodal Large Language Models Multimodal Emotion Recognition
NLP2CT Lab, Department of Computer and Information Science, University of Macau · School of Computer Science, Central China Normal University · University of Macau
Omni-DeepSearch: A Benchmark for Audio-Driven Omni-Modal Deep Search
May 9, 2026Tao Yu, yiming ding, Shenghua Chai +16Multi-Hop QA Multimodal IR
CASIA · UCAS · BAAI +2
Measuring What Matters: Benchmarking Generative, Multimodal, and Agentic AI in Healthcare
May 8, 2026Prasanna Desikan, Harshit Rajgarhia, Shivali Dalmia +1Healthcare Benchmark Design
Centific AI Research Redmond, WA, USA
SYNCR: A Cross-Video Reasoning Benchmark with Synthetic Grounding
May 8, 2026Sara Ghazanfari, Siddharth Garg, Prashanth Krishnamurthy +1Multimodal Grounding Video Reasoning
New York University
LithoBench: Benchmarking Large Multimodal Models for Remote-Sensing Lithology Interpretation
May 8, 2026Jun Wang, Fengpeng Li, Hang Dong +2Remote Sensing Image Understanding Large Vision-Language Models
School of Computer Science, China University of Geosciences · PRADA Lab, King Abdullah University of Science and Technology · Department of Computer Science, University of Exeter
TraceAV-Bench: Benchmarking Multi-Hop Trajectory Reasoning over Long Audio-Visual Videos
May 8, 2026Hengyi Feng, Hao Liang, Mingrui Chen +6Audio-Visual Understanding Multimodal Hallucination
University of Electronic Science and Technology of China · Peking University · Zhongguancun Academy +1
InterLV-Search: Benchmarking Interleaved Multimodal Agentic Search
May 8, 2026Bohan Hou, Jiuning Gu, Jiayan Guo +5Multimodal IR Multimodal Search Agents
Nanyang Technological University · Damo Academy, Alibaba Group · Shandong University +1
ChartREG++: Towards Benchmarking and Improving Chart Referring Expression Grounding under Diverse referring clues and Multi-Target Referring
May 8, 2026Tianhao Niu, Ziyu Han, Xuan Dong +2Referring Image Segmentation Visual Grounding
Research Center for Social Computing and Interactive Robotics Harbin Institute of Technology, China
EgoPro-Bench: Benchmarking Personalized Proactive Interaction in Egocentric Video Streams
May 8, 2026Dongchuan Ran, Linyu Ou, Xueheng Li +5Egocentric Video Understanding AI Agent Benchmarks
Sensetime Reasearch · Beijing Institute of Technology · University of Science and Technology of China
Uneven Evolution of Cognition Across Generations of Generative AI Models
May 7, 2026Isaac Galatzer-Levy, Daniel McDuff, Xin Liu +1Cognitive Modeling Multimodal Model Evaluation
1Google DeepMind · 2Google Research · University of Washington
Are We Making Progress in Multimodal Domain Generalization? A Comprehensive Benchmark Study
May 7, 2026Hao Dong, Hongzhao Li, Shupan Li +3Multimodal Robustness Domain Generalization
ETH Zürich · Zhengzhou University · MBZUAI +1
MedHorizon: Towards Long-context Medical Video Understanding in the Wild
May 7, 2026Bodong Du, Bowen Liu, Yang Yu +8Healthcare Medical VQA
The Hong Kong University of Science and Technology · Xidian University · Baidu Inc.
Uncovering Entity Identity Confusion in Multimodal Knowledge Editing
May 7, 2026Shu Wu, Xiaotian Ye, Xinyu Mou +3Multimodal Grounding Multimodal Large Language Models
New Laboratory of Pattern Recognition (NLPR) State Key Laboratory of Multimodal Artificial Intelligence Systems (MAIS) Institute of Automation, Chinese Academy of Sciences · Beijing University of Posts and Telecommunications · School of Artificial Intelligence, University of Chinese Academy of Sciences +3
TableVista: Benchmarking Multimodal Table Reasoning under Visual and Structural Complexity
May 7, 2026Zheyuan Yang, Liqiang Shang, Junjie Chen +6Multimodal Robustness Multimodal Reasoning
Tongji University · University of Bristol · Tianjin University +1
ICU-Bench:Benchmarking Continual Unlearning in Multimodal Large Language Models
May 7, 2026Yuhang Wang, Wenjie Mei, Junkai Zhang +3Privacy-Preserving Language Models Multimodal Large Language Models
School of Computer Science and Technology Xidian University
Unifying Scientific Communication: Fine-Grained Correspondence Across Scientific Media
May 7, 2026Megha Mariam K. M, Vineeth N. Balasubramanian, C. V. JawaharCross-Modal Alignment Scientific Communication
IIIT Hyderabad · Microsoft Research India & IIT Hyderabad
AstroAlertBench: Evaluating the Accuracy, Reasoning, and Honesty of Multimodal LLMs in Astronomical Classification
May 7, 2026Claire Chen, Jiabao Sean Xiao, Shuze Daniel Liu +7Multimodal Reasoning Multimodal Model Evaluation
1California Institute of Technology · 2Massachusetts Institute of Technology · 3Purdue University
DiffCap-Bench: A Comprehensive, Challenging, Robust Benchmark for Image Difference Captioning
May 6, 2026Yuancheng Wei, Haojie Zhang, Linli Yao +7Image Captioning Multimodal Model Evaluation
South China University of Technology · Peking University · The University of Hong Kong +2
CC-OCR V2: Benchmarking Large Multimodal Models for Literacy in Real-world Document Processing
May 5, 2026Zhipeng Xu, Junhao Ji, Zulong Chen +10Document Understanding Multimodal Large Language Models
Alibaba Group · Northeastern University
Can Multimodal Large Language Models Understand Pathologic Movements? A Pilot Study on Seizure Semiology
May 5, 2026Lina Zhang, Tonmoy Monsoor, Mehmet Efe Lorasdagi +8Video Understanding Multimodal Large Language Models
University of California, Los Angeles · Mercedes-Benz AG · Zhejiang University
VEBench:Benchmarking Large Multimodal Models for Real-World Video Editing
May 5, 2026Andong Deng, Dawei Du, Zhenfang Chen +7Video Editing Benchmarks Video Editing
ByteDance Intelligent Creation · ByteDance, San Jose, USA · CRCV, University of Central Florida
MULTITEXTEDIT: Benchmarking Cross-Lingual Degradation in Text-in-Image Editing
May 4, 2026Liwei Cheng, Shibo Feng, Lunjie Zhou +2Image Editing Evaluation Text-Guided Image Editing
1Harbin Institute of Technology
Metric Unreliability in Multimodal Machine Unlearning: A Systematic Analysis and Principled Unified Score
May 4, 2026Abdullah Ahmad Khan, Hamid Laga, Ferdous SohelVLM Evaluation VLM Unlearning
Murdoch University
From Where Things Are to What They Are For: Benchmarking Spatial-Functional Intelligence in Multimodal LLMs
May 4, 2026Le Zhang, Jihan Yang, Soundarya Krishnan +11Spatial Reasoning Benchmarks Visual Spatial Reasoning
1Mila - Qu´ebec AI Institute, UdeM · 2NYU · 3Apple +1
VIDA: A Dataset for Visually Dependent Ambiguity in Multimodal Machine Translation
May 3, 2026Jingheng Pan, Xintong Wang, Longyue Wang +3Large Vision-Language Models Multimodal Model Evaluation
Department of Informatics, Universität Hamburg · Alibaba Group
TMD-Bench: A Multi-Level Evaluation Paradigm for Music-Dance Co-Generation
May 3, 2026Xiaoda Yang, Majun Zhang, Changhao Pan +10Human Motion Generation Audio-Visual Synchronization
Department of Zhejiang, University of China · Tecent, China · School of National University of Singapore.
VT-Bench: A Unified Benchmark for Visual-Tabular Multi-Modal Learning
May 3, 2026Zi-Yi Jia, Zi-Jian Cheng, Xin-Yue Zhang +4Benchmark Design Multimodal Model Evaluation
National Key Laboratory for Novel Software Technology, Nanjing University, China
OralMLLM-Bench: Evaluating Cognitive Capabilities of Multimodal Large Language Models in Dental Practice
May 2, 2026Rongyang Wang, Shuang Zhou, Jiashuo Wang +2Medical Imaging Cognitive Modeling
Department of Orthodontics, Beijing Stomatological Hospital, School of Stomatology, Capital Medical University, Fanjiacun Road #9, Fengtai District, Beijing 100070, China · Division of Computational Health Sciences, Department of Surgery, University of Minnesota, Minneapolis, MN, USA · Department of Computing, The Hong Kong Polytechnic University, Hong Kong SAR, China +1

May 9, 2026, cs.CVJ/K move · Enter open · S save

Can MLLMs Reason About Visual Persuasion? Evaluating the Efficacy and Faithfulness of Reasoning

Naeun Lee, Hyunjong Kim, Sunghwan Choi, Injin Kong+1

Seoul National University

VLM Reasoning Multimodal Model Evaluation Multimodal Understanding

Open paper AskPDF arXiv

Persuasive visuals play a central role in advertising, public communication, and online media, making it increasingly important to understand whether an image is persuasive and why. However, current Multimodal Large Language Models (MLLMs) have limited ability to reason about visual persuasion. Our analysis reveals that models often rely on a shortcut---selectively citing easily recognizable visual elements and treating their mere presence as evidence of persuasiveness---rather than reasoning over the diverse visual cues in an image to determine how and why they contribute to persuasiveness. To address this limitation, we propose (1) a fine-tuning approach that trains models on rationales reflecting diverse perspectives on visual persuasiveness, and (2) an evaluation framework that measures the faithfulness of models' rationales through three complementary metrics. Fine-tuning on multi-perspective rationales improves persuasiveness prediction and reasoning effectiveness. However, our evaluation framework reveals a discrepancy between prediction performance and rationale faithfulness, showing that higher prediction performance does not necessarily correspond to more faithful reasoning. These findings highlight the need to evaluate effectiveness and faithfulness separately and inform future directions for improving both the training and evaluation of visual persuasion reasoning.

Momentum

Related topics

Latest papers 496