Multimodal Evaluation

Ask about this topic

Momentum

11 papers in the last four weeks, against 2 the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 59

All topics
CardsList

  1. Multimodal Safety Evaluation Should Measure Controllability Beyond Classification

    Oct 5, 2026Junhyeong Park, Hanwool Lee, DongGeon Lee +4Safety BenchmarksMultimodal Evaluation

    KAIST · AIM Intelligence · Seoul National University +3

  2. PortraitAes: Intent-Conditioned Structured Portrait Aesthetics Assessment

    Oct 4, 2026Junzhou Xie, Haozhong Xiong, Xunyun Tian +8AestheticsMulti-Image Editing Benchmarks

    Southeast University · Qwen Business Unit of Alibaba

  3. Do MLLM Judges Judge the Edit? Auditing Bias in Image Editing Evaluation with Verified Quality Preservation

    Oct 1, 2026Yuan Huang, Zirui Song, Xiuying ChenLarge Language Model JudgesMulti-Image Editing Benchmarks

    Northeastern University · Mohamed bin Zayed University of Artificial Intelligence

  4. Multidimensional Observer Model and Perceptual Dimensions of Human Image Quality Assessment

    Sep 29, 2026Sheng Zhao, Weikai Lin, Yuhao ZhuImage Quality AssessmentMultimodal Evaluation

    Department of Computer Science University of Rochester

  5. Hierarchical Compression of Vision-Language Model Benchmarks

    Sep 29, 2026Hyunjong Ok, Seunggu Kang, Jaeho LeeVisual Token PruningMultimodal Evaluation

    Pohang University of Science and Technology · Upstage AI

  6. Feed the Panel Dimensions, Not Verdicts: Rubric-Decomposed Fusion of Vision-Language Aesthetic Judges

    Sep 22, 2026Amit Jadhav, Shaurya Beriwala, Beomjin KimAestheticsMultimodal Evaluation

    Purdue University Fort Wayne

  7. Benchmarking Off-the-Shelf Multimodal AI Models Against Dermatologists on Patient-Captured Skin Images

    Sep 21, 2026Rian Dolphin, Laura KnowlesSkin Lesion ClassificationMultimodal Clinical Data

    Independent Research, Dublin, Ireland · School of Medicine, University of Limerick, Ireland

  8. E-AVI: Evidence-Grounded Multimodal Assessment for Automated Video Interviews

    Sep 17, 2026Haoshen Wang, Dongbo Che, Zeyi Xie +3Multimodal EvaluationFine-Grained Video Understanding

    The Hong Kong Polytechnic University · Independent Researcher

  9. Scientific Image Quality Assessment via Multi-modal Retrieval-Augmented Generation

    Sep 17, 2026Yinuo Zhang, Bingshuo Liu, Zhiying Tu +6Image Quality AssessmentMultimodal Retrieval Augmented Generation

    Harbin Institute of Technology · Tencent · Nanyang Technological University

  10. Online Multimodal Workload Assessment in Contact-Rich Physical Human-Robot Interaction

    Sep 16, 2026Yanyi Chen, Fan Yang, Min DengHuman-Robot InteractionHaptic Feedback

    Department of Civil and Environmental Engineering, University of Tennessee, Knoxville, TN 37996, USA · College of Information and Communications, University of South Carolina, SC 29208, USA

  11. A multimodal large language model for evidence-based autism spectrum disorder screening

    Sep 15, 2026Jun Chen, Qi Zhao, Yunliang Jiang +8Autism Spectrum DisorderScreening

    National Special Education Resource Center for Children with Autism, Zhejiang Normal University, China · School of Computer Science and Technology, Zhejiang Normal University, China · School of Mathematics and Statistics, Xi’an Jiaotong University, China +5

  12. On-Device Language Models for Privacy-Preserving Stress Prediction: A Multimodal Evaluation on Mobile Health

    Sep 14, 2026Ibukunoluwa Soyebo, Alyssa Donawa, Rodrigo Aguilar Barrios +2Readmission PredictionStress

    Ming Hsieh Department of Electrical & Computer Engineering, University of Southern California · Thomas Lord Department of Computer Science, University of Southern California

  13. SurgSkill-Bench: A Benchmark for Multimodal Surgical Skill Assessment

    Aug 31, 2026Chaohui Dang, Zheheng Jiang, James Glasbey +3SurgeryMultimodal Evaluation

    Department of Electronic, Electrical and Systems Engineering, School of Engineering, University of Birmingham, Birmingham, UK · School of Computing and Mathematical Science, University of Leicester, Leicester, UK · Department of Applied Health Sciences, College of Medicine and Health, University of Birmingham, Birmingham, UK +1

  14. ImageEval 2026: Culturally Grounded Arabic Multimodal Evaluation

    Aug 31, 2026Samir Abdaljalil, Hunzalah Hassan Bhatti, Ahlam Bashiti +11Multimodal EvaluationCultural Awareness

    Texas A&M University · Qatar Computing Research Institute, Qatar · Birzeit University, Palestine +2

  15. FitAQA: A Benchmark of Fitness Action Quality Assessment for Multimodal Large Language Models

    Aug 9, 2026Kaili Zheng, Kaiwen Wang, Xun Zhu +3AthleticismPhysical Activities

    Department of Electronic Engineering, Tsinghua University · Xinjiang Region Sports Science Research Center · College of AI, Tsinghua University +1

  16. Walkable to Whom? Capturing Subjective Variability in Walkability Perception Using Multimodal Deep Learning

    Aug 7, 2026Moloud Damandeh, Meead SaberiMultimodal EvaluationUrban Environments

    School of Civil and Environmental Engineering, University of New South Wales (UNSW), Sydney, NSW, Australia, and the Research Centre for Integrated Transport Innovation (rCITI)

  17. SciQNet: Two-Stage Multimodal Adaptation for Scientific Image Quality Assessment

    Aug 6, 2026Yin-Loon Khor, Yi-Jie Wong, Jing Jie Tan +1Image Quality AssessmentMultimodal Evaluation

    Universiti Malaya · Universiti Tunku Abdul Rahman · National University of Singapore

  18. Dynamically Allocating Evaluation Effort for Model Ranking

    Aug 4, 2026Vilém Zouhar, Julia Kreutzer, Alon Lavie +4Model EvaluationEvaluation Benchmarks

    ETH Zurich · Cohere Labs · Carnegie Mellon University +3

  19. CALM-AH: An ABAW11-Calibrated Multimodal Ensemble with Reliability-Gated Multi-Expert Consensus for Video-Level Ambivalence and Hesitancy Recognition

    Jul 31, 2026Wenzhuo Sun, Mingjian Liang, Richard Attfield +3Affective ComputingMultimodal Evaluation

    Monash University, Melbourne, Australia · Monash Suzhou Research Institute, Suzhou, China

  20. SciFigAlign: Scoring Scientific Figures by Fine-tuned Alignment of Visuals with Manuscript Evidence

    Jul 29, 2026Chuanzhi Xu, Zihan Deng, Huiqi Liang +4Scientific FigurePeer Review

    The University of Sydney · The University of Hong Kong · Indiana University +2

  21. SpatialQ: Understanding 3D Gaussian Splatting Scene Quality via Visual-based MLLM

    Jul 29, 2026Jingxuan Su, Shenglin Wang, Tiesong Zhao +2Novel View SynthesisVideo Multimodal Large Language Models

    Guangdong Provincial Key Laboratory of Ultra High Definition Immersive Media Technology, School of Electronic and Computer Engineering, Peking University, Shenzhen 518055, China · Peng Cheng Laboratory, Shenzhen, 518055, China · College of Physics and Information Engineering, Fuzhou University, Fuzhou 350108, China

  22. PhaseAware: Interpretable Human-in-the-Loop Rehabilitation Scoring with Boundary Monitoring

    Jul 22, 2026Yankai Zheng, Yuhe Liu, Yuxin Ma +8RehabilitationMultimodal Evaluation

    Department of Medical Imaging, The First Affiliated Hospital of Xi'an Jiaotong University, Xi'an, 710061, China · Faculty of Electronic and Information Engineering, Xi'an Jiaotong University, Xi'an, 710049, China · School of Future Technology, Xi'an Jiaotong University, Xi'an, 710049, China +2

  23. Style over Substance: A Shortcut Audit of Emotion-Description Preference Evaluation

    Jul 20, 2026Jiabing Yang, Yixiang Chen, Yuan Xu +6Emotion RecognitionMultimodal Evaluation

    University of Chinese Academy of Sciences Institute of Automation, Chinese Academy of Sciences Beijing, China

  24. Debiasing Text-to-Image Evaluation via Implicit Cultural Alignment Reward Modeling

    Jul 17, 2026Bo-An Chang, Yu-Chih ChenModern Text-To-Image ModelsMultimodal Evaluation

    National Tsing Hua University, Hsinchu 300044, Taiwan · National Yang Ming Chiao Tung University, Hsinchu 300093, Taiwan

  25. Multimodal Assessment of Pancreatic Cancer Resectability Using Deep Learning

    Jul 15, 2026Vincent Ochs, Christoph Kuemmerli, Florentin Bieder +12Deep LearningMultimodal Evaluation

    University of Basel, Department of Biomedical Engineering, 4123 Allschwil, Switzerland · Clarunis, University Digestive Health Centre, Basel, Switzerland · Department of General Surgery, Kantonsspital Aarau, Tellstrasse, 5000, Aarau, Switzerland +3

  26. Accuracy and Normalized Accuracy under Length Bias: Analysis, Guidelines, and a Bayesian Alternative

    Jul 14, 2026Koen OostermeijerPosition BiasMultimodal Evaluation

    Aleph Alpha Research

  27. A Calibrated Multimodal Ensemble for Ambivalence/Hesitancy Recognition: System Description and Private-Test Submission Strategy

    Jul 13, 2026Josep Cabacas-Maso, Ismael Benito-Altamirano, Carles VenturaMultimodal EvaluationEnsemble

    eHealth Center, Faculty of Computer Science, Multimedia and Telecommunicactions, Universitat Oberta de Catalunya, 08016 Barcelona, Spain · MIND/IN2UB, Department of Electronic and Biomedical Engineeering, Universitat de Barcelona, 08028 Barcelona, Spain · Institute of Semiconductor Technology (IHT) & Laboratory for Emerging Nanometrology (LENA), Technische Universität Braunschweig, Braunschweig, Germany

  28. Pluralis v0.1: Towards a Multicultural, Multimodal, Multilingual Benchmark for AI Risk and Reliability

    Jul 7, 2026Alicia Parrish, Rajat Shinde, Sanket Badhe +57Multilingual SafetyMultilingual Benchmark

  29. Towards Developing a Multimodal Chat Assistant for University Stakeholders: RAG-based Approach

    Jul 1, 2026Md Abu Hanif Shaikh, Abdullah Al ShafiMultimodal Retrieval Augmented GenerationChatbots

    Institute of Information and Communication Technology, Khulna University of Engineering & Technology, Khulna, Bangladesh

  30. PerceptionRubrics: Calibrating Multimodal Evaluation to Human Perception

    Jun 26, 2026Yana Wei, Hongbo Peng, Yanlin Lai +14Multimodal EvaluationPerception

    Johns Hopkins University · StepFun · Tsinghua University +1

  31. Cross-view Multimodal Vision-Based Assessment Framework for Traditional Chinese Medicine Rehabilitation Training

    Jun 26, 2026Francis Xiatian Zhang, Hao Yao, Shengxuan Chen +4Multimodal EvaluationCross-View

    Department of Computer Science, Durham University, UK · Institute for Regeneration and Repair, The University of Edinburgh, UK · Ningbo Hospital of Traditional Chinese Medicine, Ningbo, China +2

  32. TraMP-LLaMA: Generative Interpretability with Decoupled Instruction Tuning for Facial Expression Quality Assessment

    Jun 25, 2026Shuchao Duan, Alan Whone, Hossein Rahmani +2Facial Expression RecognitionMultimodal Evaluation

    School of Computer Science University of Bristol Bristol, UK · Translational Health Sciences University of Bristol Bristol, UK · School of Computing and Communications Lancaster University Lancaster, UK

  33. Evaluation Pitfalls and Challenges in Multimedia Event Extraction

    Jun 25, 2026Philipp Seeberger, Steffen Freisinger, Tobias Bocklet +1Multimodal EvaluationChallenges

    Technische Hochschule Nürnberg Georg Simon Ohm

  34. When AUC 0.998 Is Not Enough: A Candidate Evaluation Protocol for Hidden-State Probes of Indirect Prompt Injection in Multimodal Computer-Use Agents

    Jun 22, 2026Yanhang Li, Zhichao Fan, Zexin ZhuangIndirect Prompt InjectionMultimodal Evaluation

    Northeastern University Boston, Massachusetts, USA · University of Illinois Urbana-Champaign Urbana, Illinois, USA · Southern Methodist University Dallas, Texas, USA

  35. Evaluation of Medical Vision Language Models HuluMed and MedGemma, and general purpose chatbots Gemma 3, ChatGPT Plus, and Claude Pro on real previously unseen wound images

    Jun 16, 2026Yunzhe Xue, Mohammed Saim Ahmed Quadri, Neal Panse +2Medical Vision-Language ModelsMultimodal Clinical Data

    Department of Data Science, New Jersey Institute of Technology, Newark, NJ, USA · Department of Computer Science, New Jersey Institute of Technology, Newark, NJ, USA · Vascular and Endovascular Surgery, Robert Wood Johnson Hospital, New Brunswick, NJ, USA

  36. Skill-Augmented AI Agents for Medical Research Analysis: An Exploratory Multi-Model Human Evaluation in an NSCLC Transcriptomic Biomarker Task

    Jun 10, 2026Qianyu Yao, Fei Sun, Bocheng Huang +10Multidisciplinary Tumor BoardsTranscriptomics

    AIPOCH PTE. LTD., Singapore

  37. Community-Aware Assessment of Social Textual Engagement and Resonance: A Human-Centric Perspective on User-Generated Content Evaluation

    Jun 1, 2026Tianjiao Li, Kai Zhao, Xiang Li +2Multimodal EvaluationVideo Quality

    Bilibili Index LLM Team

  38. Bounded-Compute Multimodal Regression for Product-Rating Prediction

    May 26, 2026William Leach, Ru He, Sizhuo Ma +4Multimodal EvaluationMultimodal Classification

    Snap Inc.

  39. Structured Visual Evidence Decomposition for Evidence-Grounded Multimodal Screening of Obstructive Sleep Apnea-Hypopnea Syndrome

    May 23, 2026Chen Zhan, Yingchen Wei, Xiaoyu Tan +2ScreeningMultimodal Clinical Data

    School of Electronic and Electrical Engineering, Shanghai University of Engineering Science, Shanghai 200438, China · Tencent Youtu Lab, Shanghai 200233, China · ENT Institute and Department of Otorhinolaryngology, Eye & ENT Hospital of Fudan University, Shanghai 200031, China +1

  40. QwenSafe: Multimodal Content Rating Description Identification via Preference-Aligned VLMs

    May 20, 2026Dishanika Denipitiyage, Aruna Seneviratne, Suranga SeneviratneMultimodal EvaluationContent Moderation

    University of Sydney, Sydney, Australia · University of New South Wales, Australia

  41. Physics-R1: An Audited Olympiad Corpus and Recipe for Visual Physics Reasoning

    May 13, 2026Shan YangOlympiad-Level ProblemMultimodal Evaluation

    Independent Researcher

  42. Robust Checkpoint Selection for Multimodal LLMs via Agentic Evaluation and Stability-Aware Ranking

    May 13, 2026Qinwu Xu, Zhuoheng Li, Jessie SalasMultimodal EvaluationLarge Language Model Evaluation

    Meta AI · Independent Researcher, previously Meta AI

  43. Measuring What Matters Beyond Text: Evaluating Multimodal Summaries by Quality, Alignment, and Diversity

    May 12, 2026Abid Ali, Diego Molla-Aliod, Usman NaseemMultimodal EvaluationSummarization

    School of Computing, Macquarie University, Sydney, Australia

  44. MLCR: Multi-Level Cue Refinement for Long-Term Multimodal Action Quality Assessment

    May 9, 2026Qiqi Li, Pengfei Wang, Hongyu Chen +1Multimodal EvaluationSpatial Encoding

    Qiushi Academy for Advanced Studies (QAAS), Zhejiang University, Hangzhou, Zhejiang, 310007, China · College of Computer Science and Technology, Zhejiang University, Hangzhou, Zhejiang, 310007, China · School of Software Technology, Zhejiang University, Hangzhou, Zhejiang, 310007, China +3

  45. Erase Persona, Forget Lore: Benchmarking Multimodal Copyright Unlearning in Large Vision Language Models

    May 5, 2026JuneHyoung Kwon, JungMin Yun, YoungBin KimUnlearning MethodMultimodal Evaluation

    Department of Artificial Intelligence, Chung-Ang University · Graduate School of Advanced Imaging Sciences, Multimedia and Film, Chung-Ang University

  46. MultiSense-Pneumo: A Multimodal Learning Framework for Pneumonia Screening in Resource-Constrained Settings

    May 4, 2026Dineth Jayakody, Pasindu Thenahandi, Chameli DommanigeMultimodal Clinical DataLung-Nodule Artificial Intelligence

    Department of Computer Science, Old Dominion University, VA, USA

  47. Metric Unreliability in Multimodal Machine Unlearning: A Systematic Analysis and Principled Unified Score

    May 4, 2026Abdullah Ahmad Khan, Hamid Laga, Ferdous SohelLarge Language Model UnlearningMultimodal Evaluation

    Murdoch University

  48. When Jokes Cross the Line: Analyzing Regular Humor and Dark Humor in YouTube Shorts

    Apr 30, 2026Sydney Johns, Sanjeev Parthasarathy, Shantnu Bhalla +1HumorShort Video Bursts

    Virginia Polytechnic Institute and State University, Alexandria, Virginia, USA

  49. HotComment: A Benchmark for Evaluating Popularity of Online Comments

    Apr 28, 2026Yafeng Wu, Yunyao Zhang, Liliang Ye +4Social Media Timeline DynamicsContent Moderation

    Huazhong University of Science and Technology Wuhan, China · Beijing Institute of Computer Technology and Applications Beijing, China

  50. VLM Judges Can Rank but Cannot Score: Task-Dependent Uncertainty in Multimodal Evaluation

    Apr 28, 2026Divake Kumar, Sina Tayebati, Devashri Naik +2Multimodal EvaluationLarge Language Model Judges

    University of Illinois at Chicago · 2AI Labs at Capital One

  51. QEVA: A Reference-Free Evaluation Metric for Narrative Video Summarization with Multimodal Question Answering

    Apr 27, 2026Woojun Jung, Junyeong KimSummarizationMultimodal Evaluation

    Department of Artificial Intelligence, Chung-Ang University

  52. FAIR_XAI: Improving Multimodal Foundation Model Fairness via Explainability for Wellbeing Assessment

    Apr 26, 2026Sophie Chiang, Tom Brennan, Fethiye Irmak Dogan +2Medical Vision-Language ModelsMultimodal Clinical Data

    Department of Computer Science & Technology, University of Cambridge. · Harvard University.

  53. WebCompass: Towards Multimodal Web Coding Evaluation for Code Language Models

    Apr 20, 2026Xinping Lei, Xinyu Che, Junqi Xiong +16Multimodal EvaluationEvaluation Benchmarks

    Nanjing University · Kuaishou Technology

  54. SciVisAgentBench: A Benchmark for Evaluating Scientific Data Analysis and Visualization Agents

    Mar 31, 2026Kuangshi Ai, Haichao Miao, Kaiyuan Tang +13MemoryagentbenchEvaluation Benchmarks

    University of Notre Dame · Lawrence Livermore National Laboratory · University of Utah +4

  55. Toward Scalable Audio Description Quality Control: A Workflow for Evaluating Human and VLM Raters

    Feb 1, 2026Lana Do, Gio Jung, Juvenal Francisco Barajas +5Video QualityMultimodal Evaluation

    Northeastern University · United States · San Francisco State University +2

  56. From Global to Granular: Revealing IQA Model Performance via Correlation Surface

    Jan 29, 2026Baoliang Chen, Danni Huang, Hanwei Zhu +5Image Quality AssessmentSpearman Correlation

    School of Computer Science, South China Normal University · College of Computing and Data Science, Nanyang Technological University · School of Computer Science, City University of Hong Kong +2

  57. Eval-Actions: Fine-Grained Execution Quality Evaluation for Robotic Manipulation

    Jan 26, 2026Mengyuan Liu, Juyi Sheng, Peiming Li +4Robotic ManipulationMultimodal Evaluation

  58. Redemption Score: A Multi-Modal Evaluation Framework for Image Captioning via Distributional, Perceptual, and Linguistic Signal Triangulation

    May 22, 2025Ashim Dahal, Ankit Ghimire, Saydul Akbar Murad +1Multimodal EvaluationImage Difference Captioning

    University of Southern Mississippi, Hattiesburg, MS, USA · School of Business & Technology, Emporia State University, Emporia, KS, USA

  59. CatSIM: A Categorical Image Similarity Metric

    Apr 20, 2020Geoffrey Z. Thompson, Ranjan MaitraImage Quality AssessmentSimilarity and Metrics

    Department of Statistics at Iowa State University, Ames, Iowa 50011, USA