Multimodal Model Evaluation

Ask about this topic

Momentum

49 papers in the last four weeks, up 48% on the four weeks before. 0.5% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 496

All topics
CardsList

  1. AnesTRACE: Benchmarking Intraoperative Anesthesia from Multimodal Perception to Multi-step Decision-Making

    Sep 26, 2026Ziwei Huang, Qi Gao, Zhe Ji +5HealthcareClinical Decision Support

    School of Software Technology, Zhejiang University · Department of Anesthesiology, The Second Affiliated Hospital of Zhejiang University School of Medicine · School of Software, Central South University +1

  2. LAYERSCOPE: A Layerwise Characterization of Video and Multimodal Learned Representations

    Sep 23, 2026Sandra Arcos-Holzinger, Debashish Chakraborty, Rohita Mocharla +7Representation GeometryRepresentation Learning

    Johns Hopkins University · University of Melbourne · Human Language Technology Center of Excellence +2

  3. Benchmarking Neural Defend ARCAS 1B: A Foundational Multimodal Deepfake Detection Model

    Sep 21, 2026Sivashankar Selvarajan, Piyush Verma, Sumit Kumar +1Deepfake DetectionMultimodal Model Evaluation

    Neural Defend Inc.

  4. Benchmarking Off-the-Shelf Multimodal AI Models Against Dermatologists on Patient-Captured Skin Images

    Sep 21, 2026Rian Dolphin, Laura KnowlesSkin Lesion ClassificationMedical VLMs

    Independent Research, Dublin, Ireland · School of Medicine, University of Limerick, Ireland

  5. Omni2Web: Benchmarking Audiovisual Website Development

    Sep 20, 2026Minghao Han, Zhenghao Xing, Xize Cheng +7Multimodal GroundingAudio-Visual Understanding

    FDU · Alibaba Token Hub, Alibaba Group · CUHK +2

  6. AVTrace: Diagnosing Audio-Visual Temporal Reasoning in Omni Models

    Sep 17, 2026Longyin Zhang, Parth Sakhare Mahendra, Chengwei Wei +4Temporal Video GroundingAudio-Visual Understanding

    Institute of Advanced Intelligence and Computing (IAIC), A*STAR, Singapore · Centre for Frontier AI Research (CFAR), A*STAR, Singapore

  7. A Unified Evaluation Framework for Trustworthy Large Language Models, Agentic AI, and Multimodal Systems

    Sep 17, 2026Shaina Raza, Ahmed Y. Radwan, Imran Liaquat +1LLM EvaluationAI Agent Evaluation

    Vector Institute for Artificial Intelligence, Toronto, ON M5G 0C6, Canada

  8. MUSE: Benchmarking Large Vision-Language Models on Multi-Modal Understanding in Situated Education

    Sep 16, 2026Luyao Zhu, Xun Wei Yee, Wei Li +2VLM EvaluationLarge Vision-Language Models

    AI Singapore, National University of Singapore, Singapore · Institute of Advanced Intelligence and Computing, A*STAR · School of Computing, National University of Singapore, Singapore

  9. ReFigBench: Benchmarking Scientific Figure Reconstruction as Editable PowerPoint Artifacts

    Sep 16, 2026Liyang Fan, Chi Wei, Yitai Li +8Scientific VisualizationAI Agent Benchmarks

    SZU · SIAT, CAS · SUAT +2

  10. Rethinking Domain Specialization for Open-Ended Scientific Reasoning in Astronomy Language Models

    Sep 15, 2026Vanessa Lama, Sanjay Das, Emily Herron +5LLM EvaluationScientific QA

    Oak Ridge National Laboratory, Oak Ridge, Tennessee, USA · The Ohio State University, Columbus, Ohio, USA · Max-Planck-Institut für Astronomie, Königstuhl 17, D-69117 Heidelberg, Germany +1

  11. Beyond Numerical Time Series: A Unified Benchmark for Multimodal Forecasting with Heterogeneous Context

    Sep 14, 2026Peng Chen, Zhihao Zhuang, Hongzhou Chen +6Multivariate Time Series ForecastingTime Series Forecasting

    Ant International · Hangzhou, China

  12. TwinICL: Diagnosing Multimodal In-Context Learning through Paired Counterfactuals

    Sep 14, 2026Zihan Xue, Po-Yi Lu, Serhii Honcharenko +5Multimodal ICLModality Gap in VLMs

    University of California, Los Angeles · 2National Taiwan University · 3Texas A&M University +2

  13. On-Device Language Models for Privacy-Preserving Stress Prediction: A Multimodal Evaluation on Mobile Health

    Sep 14, 2026Ibukunoluwa Soyebo, Alyssa Donawa, Rodrigo Aguilar Barrios +2On-Device Language Model InferencePrivacy-Preserving Language Models

    Ming Hsieh Department of Electrical & Computer Engineering, University of Southern California · Thomas Lord Department of Computer Science, University of Southern California

  14. Physics as the label for measuring and correcting materials reasoning in multimodal models

    Sep 14, 2026Hasan Kurban, Rasul Khanbayov, Mustafa KurbanMultimodal ReasoningMultimodal Model Evaluation

    College of Science and Engineering, Hamad Bin Khalifa University, Doha, Qatar · Ankara University, Ankara, Turkey

  15. LatentVerse: A Framework for Understanding Shared and Modality-Specific Information in Multimodal Latent Representations

    Sep 14, 2026Majd Alafrange, Samuel Friedman, John Kitonyo +2Multimodal EmbeddingMultimodal Model Interpretability

    Machine Learning for Health (ML4H), Broad Institute of MIT and Harvard, Cambridge, Massachusetts, USA · Massachusetts Institute of Technology, Cambridge, Massachusetts, USA · The Schmidt Center, Broad Institute of MIT and Harvard, Cambridge, Massachusetts, USA

  16. Calibrated Ambiguity in Multimodal Language Models: Humans reach for cultural references, while models describe the picture

    Sep 14, 2026Cody Kommers, Mingrui Ye, Evelyn Gius +4Pragmatic Reasoning in Language ModelsMultimodal Large Language Models

    King’s College London London, United Kingdom · University of Edinburgh Edinburgh, United Kingdom

  17. Anchoring Clinical Events in Time: UID-Preserving Multimodal Reconstruction and Source-Grounded Adjudication

    Sep 14, 2026Sayantan Kumar, Nicolas Grimaldi, Jack Cummins +1Multimodal GroundingDocument Information Extraction

    National Library of Medicine, National Institutes of Health, USA · Princeton University, USA

  18. Sci-MMR: Benchmarking Multi-Step Evidence-Grounded Scientific Reasoning in Multimodal Agents

    Sep 12, 2026Jiaqiang Li, Yajie Yang, Zhiheng Xi +15Evidence-Grounded ReasoningScientific Reasoning

    Fudan NLP Lab · Fudan NLP Group

  19. When Does Text Inform? Benchmarking Information-Theoretic Metrics for Multimodal Time-Series Forecasting

    Sep 12, 2026Emma Andrews, Gianmarco MengaldoForecasting BenchmarksMutual Information Estimation

    National University of Singapore

  20. Beyond Generation and Accuracy: Diagnosing and Enhancing Visual Chain-of-Thought for Geometry Problem Solving

    Sep 11, 2026Zhitong Dong, Jicai Pan, Yingguo Gao +3Mathematical ReasoningMultimodal CoT Reasoning

    1Key Laboratory of New Generation Artificial Intelligence Technology, Southeast University Group · 2Ant

  21. OmniHallu: Unified Hallucination Detection for Cross-Modal Comprehension and Generation in Multimodal Large Language Models

    Sep 11, 2026Jianjiang Yang, Peihang Li, Shanqing Xu +3Hallucination Detection in VLMsHallucination Detection

    The University of Manchester · The University of Hong Kong · Huazhong University of Science and Technology +2

  22. The Illusion of Balanced Multimodal Sentiment Analysis: Beyond the Limits of Optimization-Based Methods

    Sep 11, 2026Ioanna Kaffeza, Efthymios Georgiou, Alexandros PotamianosModality ImbalanceMultimodal Optimization

    PERSEE Center, Mines Paris-PSL University, Sophia-Antipolis, France · National Technical University of Athens, Athens, Greece · University of Bern, Bern, Switzerland +2

  23. ReGround: Grounding Reviewer Comments in Multimodal Evidence

    Sep 11, 2026Serwar Basch, Lizhen Qu, Iryna GurevychMultimodal IRScholarly Peer Review

    Ubiquitous Knowledge Processing Lab (UKP Lab), Department of Computer Science and Hessian Center for AI (hessian.AI), TU Darmstadt · Department of Data Science & AI, Monash University, Australia

  24. CHIMERA Challenge Task 2 and 3: Response Subtypes Classification and Progression Survival Prediction in Bladder Cancer Patients using Multimodal Datasets

    Sep 8, 2026Catherine Chia, Tongjie Wang, Robert Spaans +12Multimodal RobustnessCancer Survival Prediction

  25. Studying Image Tokenizers as Visual Languages in Unified Multimodal Models

    Sep 8, 2026Siting Li, Zhengyang Wang, Simon Shaolei Du +2Image TokenizationUnified Multimodal Models

    University of Washington · Amazon FAR

  26. SAFIRE: Safety-Critical Benchmark for Fine-grained Fire and Smoke Understanding in Multimodal LLMs

    Sep 7, 2026Pengfei Li, Naufal Suryanto, Sicheng Zhang +2LLM Safety BenchmarksLLM Safety Evaluation

    Khalifa University · University of Western Australia

  27. Where Should Language Sit in a Multimodal Model? Lessons from What Language Does to Human Perception and Cognition

    Sep 7, 2026Peng Xie, Amr AlanwarUnified Multimodal ModelsMultimodal Fusion

    TUM School of Computation, Information and Technology, Department of Computer Engineering, Technical University of Munich

  28. Whose record is this? Diagnosing and authorizing record use in personalized multimodal models

    Sep 7, 2026Xinyu Mao, Junsi Li, Chenyang Liu +2Memory-Augmented VLMsMultimodal Model Evaluation

    University of Electronic Science and Technology of China

  29. MM-IFEval-Pro: A Multilingual and Attack-Resistant Benchmark for Instruction-Following in Vision-Language Models

    Sep 4, 2026Changming Xiao, Zhenliang Ni, Jinhui He +2VLM RobustnessAdversarial Attacks on VLMs

    Huawei Technologies Ltd. · Beijing, China · Shenzhen, China +1

  30. InSituMeasure: Probing Situated Measurement Grounding in Industrial Scenes with Multimodal Large Language Models

    Sep 3, 2026Chao Shen, Xinyuan Li, Yunfan Zhou +4Multimodal GroundingMultimodal Robustness

    Shanghai Key Laboratory of Scalable Computing and Systems · School of Computer Science, Shanghai Jiao Tong University · Qwen Team, Alibaba Group