MLLM Evaluation

MLLM: Multimodal Large Language Model

Latest papers

CardsList

61 papers22 papers in the last four weeks, with none the four weeks before. 0.2% of all new papers.

  1. CaseScope: A Multi-Field Benchmark for Case-Level Pathology Diagnosis with Multimodal LLMs

    Nov 11, 2026Wenchuan Zhang, Chengyang Zhang, Fengling Li +9Computational PathologyMedical Image Benchmarks

  2. VIGOR: Benchmarking Visual Rationale Correctness in Multimodal Large Language Models

    Nov 9, 2026Yanlin Chen, Tang Li, Xi PengMLLM EvaluationVision-Language Grounding

  3. HumanSightBench: A Benchmark for Human Visual Perception in MLLMs

    Nov 9, 2026Zixin Su, Yangguang Lin, Zegang Peng +10VLM EvaluationMLLM Evaluation

  4. EndoSCOP-V: A Multi-Turn Video Understanding Evaluation Framework for Multimodal Models in Endoscopy Reporting and Clinical Reasoning

    Nov 7, 2026Abhishek Tyagi, Tanmay Jain, Vilas Biradar +15MLLM EvaluationEndoscopy

  5. MaRiO: Multi-agent Collaborative Reasoning via Shared Observations in MLLMs

    Nov 7, 2026Nathaniel Redmond, Fidel Omar Tito Cruz, Devansh Sharma +4Multi-Agent ReasoningMultimodal Large Language Models

  6. BEAKER: An Expert-Curated Benchmark for Embodied Brains in Self-Driving Chemical Laboratories

    Nov 4, 2026Fei Lin, Tengchao Zhang, Ziyang Gong +15Embodied Agent BenchmarksEmbodied Reasoning

  7. A Systematic Benchmark for Evaluating Multimodal Personalized Reasoning of Vision-Language Models

    Nov 4, 2026Dohyun Kim, Junyong Lhim, Kwanyong ParkMultimodal ReasoningPersonalized Language Models

  8. Overview of the NTCIR-19 SciClaimEval Task: Cross-modal Claim Verification in Scientific Papers

    Nov 4, 2026Xanh Ho, André Greiner-Petter, Florian Boudin +6Scientific Claim VerificationScientific Reasoning in Language Models

  9. What Does the Model See? Representation Validity in Multimodal LLM-Based Video Analysis

    Nov 3, 2026Christian Pipal, Eva-Maria Vogel, Morgan WackModality Reliance in Multimodal ModelsVideo-Language Model Evaluation

  10. NightIQA: A Comprehensive Dataset and Benchmark for MLLM-Based Nighttime Image Quality Assessment

    Nov 3, 2026Yiding Tian, Jiaojiao Yi, Jiayuan Lu +4Image Quality AssessmentMLLM Evaluation

  11. Same Outputs, Different Scores: Auditing the Unchecked Graders of Multimodal LLM Benchmarks

    Nov 2, 2026Ahmad Jarjis Hasan, Jonathan LwowskiMLLM EvaluationVLM Evaluation

  12. Open-Weight MLLMs Cannot Compare What They Cannot Name

    Oct 30, 2026Wenxin Su, Anna KreshukMLLM EvaluationVision-Language Grounding

  13. PhysCount: Do MLLMs Count Visual Appearances or Physical Entities?

    Oct 29, 2026Lidong Lu, Jiaxin Lv, Liangrui Dong +5Visual CountingObject Counting

  14. Benchmarking color judgment in multimodal LLMs with CIE-computable ground truth

    Oct 29, 2026Shuwei Yue, Kaida Xiao, Zhihua Wang +1MLLM EvaluationVLM Calibration

  15. VideoMemBench: How Well Do MLLMs Remember in Video Conversations?

    Oct 29, 2026Zhengdong Hu, Chao Wang, Kaituo Feng +4MLLM EvaluationVideo Question Answering

  16. Diagnosing the Reference Gap: What Visual Scaffolds Can and Cannot Fix in Multimodal Reasoning

    Oct 29, 2026Guizhen Chen, Weiwen Xu, Wotao Yin +2Multimodal ReasoningVisually Grounded Reasoning

  17. What do MLLMs know about 'Waldo'? Evaluating Visual Reasoning under Multi-Instance Multi-Object Challenges

    Oct 28, 2026Saumya Mundra, Ajoy Mondal, Vineeth N Balasubramanian +1Visual CountingMLLM Evaluation

  18. PRISM-BENCH: Diagnosing Grounded and Faithful Multimodal Reasoning Beyond Final-Answer Accuracy

    Oct 27, 2026Vansh Bhatia, Mehdi Rezagholizadeh, Parsa Ashrafi Fashi +3MLLM EvaluationMultimodal Model Evaluation

  19. SOMA: A Benchmark for Fine-Grained Egocentric Behavior and Body-Motion Understanding

    Oct 26, 2026Matea Tashkovska, Naël Ouerghemi, Haozhe Qi +1Human Motion UnderstandingEgocentric Video Understanding

  20. Omni-InteractScope: Benchmarking Proactive and Agentic Interaction in Continuous Omnimodal Streams

    Oct 26, 2026Yinsong Yan, Ling Wang, Yuxuan Wang +11Multimodal Agent EvaluationMLLM Evaluation

  21. AdShot: Benchmarking Multimodal Large Language Models for Video Advertisement Clipping

    Oct 24, 2026Wen Xie, Om Rastogi, Sai Siddhartha Vivek Dhir Rangoju +3MLLM Evaluation

  22. Melange: A Synthetic Visual Bias Benchmark for Multi-modal Large Language Models

    Oct 21, 2026Razieh Rezaei, Ehsaneddin Asgary, Masoud Jalili SabetMLLM EvaluationDemographic Bias in VLMs

  23. A Composable Evaluation System for Reproducible Omni-Modal Foundation Model Evaluation

    Sep 1, 2026Hodong Lee, Sanghee Park, Dohoon Ryu +4ML ReproducibilityMultimodal Foundation Models

  24. VERDICT: Training-Free Step-Wise Verification of Multimodal Reasoning via Disagreement-Aware Consensus

    Aug 11, 2026Rohit Sinha, Kunal Tilaganji, Tanuja Ganu +3Multimodal Large Language ModelsMultimodal CoT Reasoning

  25. SIGNPOST-Bench: Benchmarking Text-Vision Conflict Resolution in Multimodal Large Language Models

    Aug 4, 2026Sirun Li, Minghao Liu, Ling Dai +4VLM RobustnessMultimodal Large Language Models

  26. Contextualized Evaluation of Vision Language Models through Dynamic, Multi-turn Interactions

    Jul 16, 2026Yijiang Li, Huiqi Zou, Bingyang Wang +1VLM EvaluationVLM Hallucination