cs.AINov 10, 2026

Beyond Proxy Metrics: MLLMs as Human Surrogates for Explainable AI Evaluation

Authors: Eoin M. Kenny, Salim I. Amoukou, Tom Bewley

Organizations: AI Research, JP Morgan, 25 Bank Street, London, UK · Mistral AI, London, UK

Abstract

Explainable AI (XAI) has expanded into a broad and impactful research area, but the fundamental question of how best to evaluate explanation techniques has proven frustratingly elusive. Methodologies typically revolve around either proxy metrics which tell us little about the utility of an explanation, or human evaluation which---while generally considered the gold standard---is expensive, difficult to scale, and challenging to reproduce. In this paper, we investigate if Multimodal Large Language Models (MLLMs) can act as surrogate human participants to automate the evaluation of XAI. To do this, we first propose a ``purpose-grounded'' evaluation framework with metrics applicable to both MLLMs and human users alike. Then, we offer some conceptual insights as to why and when we can trust these models to automate this process. Lastly, we conduct extensive empirical testing across five purpose-grounded areas such as model improvement and fourteen popular MLLMs. Results indicate that all models trend positively, but OpenAI and Qwen models generally show higher alignment compared to other variants, which would be the recommended choices in practice. We expect our insights not to replace user testing, but rather to help XAI researchers complement their current evaluation methodologies and move the field away from heavy reliance on proxy metrics for measuring explanation utility.

Figures & tables

Appendix figures & tables28 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. XAI-Arena: Can LLMs Assess the Quality of XAI Explanations?

    Sep 8, 2026Yanfei Hu Fleischhauer, Alona Zharova, Nadja Klein +1Explainable Artificial IntelligenceLLM-as-a-Judge

  2. Are explainable AI (XAI) evaluation strategies aligned? Comparing subjective, objective, and mathematical evaluation measures using saliency maps

    Apr 23, 2025Felix Kares, Timo Speith, Hanwei Zhang +1Saliency Map EvaluationExplainability Evaluation

  3. A Two-Stage LLM Framework for Accessible and Verified XAI Explanations

    Apr 14, 2026Georgios Mermigkis, Dimitris Metaxakis, Marios Tyrovolas +4Explainable Artificial IntelligenceFaithfulness of Language Model Explanations