cs.CLAug 14, 2025

When Language Overrules: Revealing Text Dominance in Multimodal Large Language Models

Authors: Huyu Wu, Meng Tang, Xinhan Zheng, Haiyun Jiang

Organizations: Institute of Computing Technology, Chinese Academy of Sciences · Department of Computer Science, Aberystwyth University · Beijing University of Posts and Telecommunications · Shanghai Jiao Tong University

Abstract

Multimodal Large Language Models (MLLMs) have demonstrated remarkable capabilities across a diverse range of multimodal tasks. However, these models suffer from a core problem known as text dominance: they depend heavily on text for their inference, while underutilizing other modalities. While prior work has acknowledged this phenomenon in vision-language tasks, often attributing it to data biases or model architectures. In this paper, we conduct the first systematic investigation of text dominance across diverse data modalities, including images, videos, audio, time-series, and graphs. To measure this imbalance, we propose two evaluation metrics: the Modality Dominance Index (MDI) and the Attention Efficiency Index (AEI). Our comprehensive analysis reveals that text dominance is both significant and pervasive across all tested modalities. Our in-depth analysis identifies three underlying causes: attention dilution from severe token redundancy in non-textual modalities, the influence of fusion architecture design, and task formulations that implicitly favor textual inputs. Furthermore, we propose a simple token compression method that effectively rebalances model attention. Applying this method to LLaVA-7B, for instance, drastically reduces its MDI from 10.23 to a well-balanced value of 0.86. Our analysis and methodological framework offer a foundation for the development of more equitable and comprehensive multimodal language models.

Figures & tables

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Reading, Not Thinking: Understanding and Bridging the Modality Gap When Text Becomes Pixels in Multimodal LLMs

    Mar 10, 2026Kaiser Sun, Xiaochuang Yuan, Hongjun Liu +4Multimodal Large Language ModelsModality Gap in VLMs

  2. Beyond Text-Dominance: Understanding Modality Preference of Omni-modal Large Language Models

    Apr 18, 2026Xinru Yan, Boxi Cao, Yaojie Lu +4Unified Multimodal ModelsNeural Network Interpretability

  3. Information Router for Mitigating Modality Dominance in Vision-Language Models

    Apr 17, 2026Seulgi Kim, Mohit Prabhushankar, Ghassan AlRegibVLM RobustnessVision-Language Models