Overview

Multimodal classification studies how to combine different kinds of evidence—for example, medical images with clinical records, or meme images with text—to predict categories. Common approaches include feature or decision fusion, cross-attention and contrastive alignment, mixture-of-experts models, and large vision-language models; benchmarks range from medical diagnosis and sentiment analysis to document, time-series, and content moderation tasks. Current work also addresses missing or unreliable inputs through modality dropout, imputation, adaptive routing, and selective acquisition, while evaluating fairness, calibration, and interpretability. A central tension is that adding a modality can improve predictions when it contributes complementary information, but can also hurt when it is noisy, redundant, or unavailable at deployment.

Latest papers

CardsList

212 papers17 papers in the last four weeks, down 29% on the four weeks before. 0.1% of all new papers.

  1. Batch Augmentation with Unimodal Fine-tuning for Multimodal Fusion of Large Language Models

    May 10, 2025H M Dipu Kabir, Subrota Kumar Mondal, Mohammad Ali MoniMultimodal FusionMultimodal Classification

  2. GestaltMML: Enhancing Rare Genetic Disease Diagnosis through Multimodal Machine Learning Combining Facial Images and Clinical Text

    Dec 23, 2023Da Wu, Zhanliang Wang, Hongzhuo Chen +12Rare Disease DiagnosisMultimodal Classification