cs.LGDec 28, 2025

Fusion or Confusion? Multimodal Complexity Is Not All You Need

Authors: Tillmann Rheude, Roland Eils, Benjamin Wild

Organizations: Berlin Institute of Health, Charité - Universitätsmedizin Berlin · Department of Mathematics and Computer Science, Freie Universität Berlin · Intelligent Medicine Institute, Fudan University

Abstract

Multimodal learning has become a prominent research area, with the potential of substantial performance gains by combining information across modalities. At the same time, model development has trended toward increasingly complex deep learning architectures, motivated by the assumption that multimodal-specific methods improve performance. We challenge this assumption through a large-scale empirical study by reimplementing 19 high-impact multimodal methods across nine diverse datasets with up to 23 modalities. Under standardized experimental conditions, including hyperparameter tuning, weight initialization, cross-validation, and statistical testing, increased multimodal complexity often yields confusion rather than effective fusion of data modalities. Accordingly, complex multimodal architectures do not reliably outperform unimodal baselines and a Simple Baseline for Multimodal Learning (SimBaMM). Through a focused case study, we further demonstrate concrete methodological shortcomings even in top-tier multimodal learning publications, underscoring the need for standardized evaluation practices. In summary, we argue for a shift in focus for multimodal learning: away from the pursuit of architectural novelty and toward methodological rigor.

Figures & tables

Appendix figures & tables15 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Mitigating Strong-Modality Collapse in Multimodal Learning via Inverted Asymmetric Fusion

    Aug 27, 2026Mary Ogbuka Kenneth, Foaad Khosmood, Abbas EdalatMultimodal Fusion

  2. Confidence Falls Short: Asymmetric Certainty Gains from Optimization Hinder Multimodal Classification

    Sep 23, 2026Longfei Huang, Xiangyu Wu, Yang YangModality ImbalanceMultimodal Optimization