cs.AIAug 2, 2024

A Comprehensive Review of Multimodal Large Language Models: Performance and Challenges Across Different Tasks

Authors: Jiaqi Wang, Hanqi Jiang, Yiheng Liu, Chong Ma, Xu Zhang, Yi Pan, Mengyuan Liu, Peiran Gu, 

Organizations: School of Computer Science, Northwestern Polytechnical University, Xi’an 710072, China · School of Computing, The University of Georgia, Athens 30602, USA · School of Automation, Northwestern Polytechnical University, Xi’an 710072, China · School of Physics and Information Technology, Shaanxi Normal University, Xi’an 710119 China · 

Abstract

In an era defined by the explosive growth of data and rapid technological advancements, Multimodal Large Language Models (MLLMs) stand at the forefront of artificial intelligence (AI) systems. Designed to seamlessly integrate diverse data types-including text, images, videos, audio, and physiological sequences-MLLMs address the complexities of real-world applications far beyond the capabilities of single-modality systems. In this paper, we systematically sort out the applications of MLLM in multimodal tasks such as natural language, vision, and audio. We also provide a comparative analysis of the focus of different MLLMs in the tasks, and provide insights into the shortcomings of current MLLMs, and suggest potential directions for future research. Through these discussions, this paper hopes to provide valuable insights for the further development and application of MLLM.

Figures & tables

Explore similar work

CardsList
  1. What We are Missing in Multimodal LLM Evaluation?

    Jun 24, 2026Po-han Li, Shenghui Chen, Sandeep Chinchali +1Spatiotemporal ReasoningMultimodal Large Language Models

  2. MIBench: Evaluating LMMs on Multimodal Interaction

    Mar 13, 2026Yu Miao, Zequn Yang, Yake Wei +5Multimodal Large Language ModelsMultimodal Model Evaluation