Overview

Multimodal fusion studies how to combine inputs such as images, text, audio, medical scans, sensor streams, and structured records so models can use their complementary information. Common approaches include early or late fusion, cross-attention and contrastive alignment, and adaptive gates or mixtures of experts that weight modalities by task, sample, or estimated reliability. Methods are tested across settings such as BraTS medical segmentation, CMU-MOSI and CMU-MOSEI sentiment analysis, RGB-thermal detection, and clinical prediction, with growing attention to missing, noisy, or misaligned inputs. A central open tension is that additional modalities can help when they add useful evidence, but can also introduce noise or let a dominant modality suppress weaker ones; robust gains and generalization beyond benchmark conditions remain difficult to establish.

Latest papers

CardsList

810 papers50 papers in the last four weeks, down 43% on the four weeks before. 0.4% of all new papers.

  1. Effective and Robust Multimodal Medical Image Analysis

    Feb 17, 2026Joy Dhar, Nayyar Zaidi, Maryam HaghighatMedical Image AnalysisMultimodal Fusion

  2. Feature Recalibration Based Olfactory-Visual Multimodal Model for Enhanced Rice Deterioration Detection

    Feb 16, 2026Rongqiang Zhao, Hengrui Hu, Yijing Wang +2Multimodal ClassificationMultimodal Fusion

  3. MambaVF: State Space Model for Efficient Video Fusion

    Feb 5, 2026Zixiang Zhao, Yukun Cui, Lilun Deng +4Multimodal Fusion

  4. Combining scEEG and PPG for reliable sleep staging

    Feb 4, 2026Jiawei Wang, Liang Xu, Shuntian Zheng +7Sleep Stage ClassificationWearable Health Monitoring

  5. Spectral Text Fusion: A Frequency-Aware Approach to Multimodal Time-Series Forecasting

    Feb 2, 2026Huu Hiep Nguyen, Minh Hoang Nguyen, Dung Nguyen +1Multivariate Time Series ForecastingFrequency-Domain Time Series Modeling

  6. Multi-Modal Time Series Prediction via Mixture of Modulated Experts

    Jan 29, 2026Lige Zhang, Ali Maatouk, Jialin Chen +3Multivariate Time Series ForecastingTime Series Forecasting

  7. Where is the multimodal goal post? On the Ability of Foundation Models to Recognize Highlights in Videos

    Jan 22, 2026Aditya K Surikuchi, Raquel Fernández, Sandro PezzelleMultimodal Foundation ModelsTemporal Video Understanding

  8. ThermoSplat: Cross-Modal 3D Gaussian Splatting with Feature Modulation and Geometry Decoupling

    Jan 22, 2026Zhaoqi Su, Shihai Chen, Xinyan Lin +3Thermal Imaging3D Gaussian Splatting

  9. Multimodal rumor detection enhanced by external evidence and forgery features

    Jan 21, 2026Han Li, Hua SunFake News DetectionCross-Modal Alignment

  10. From One-to-One to Many-to-Many: Dynamic Cross-Layer Injection for Deep Vision-Language Fusion

    Jan 15, 2026Cheng Chen, Yuyu Guo, Pengpeng Zeng +4Vision-Language ModelsVLM Adaptation

  11. Imaging-anchored multiomics in cardiovascular disease: integrating cardiac imaging, bulk, single-cell, and spatial transcriptomics

    Jan 10, 2026Minh H. N. Le, Thanh-Huy Nguyen, Tao Li +7Multimodal FusionSpatial Transcriptomics

  12. RGBT-GroundBench: Visual Grounding Beyond RGB in Complex Real-World Scenarios

    Dec 31, 2025Tianyi Zhao, Jiawen Xi, Linhui Xiao +4VLM RobustnessVisual Grounding

  13. Fusion or Confusion? Multimodal Complexity Is Not All You Need

    Dec 28, 2025Tillmann Rheude, Roland Eils, Benjamin WildMultimodal LearningMultimodal Fusion

  14. GPF-Net: Gated Progressive Fusion Learning for Polyp Re-Identification

    Dec 25, 2025Suncheng Xiang, Xiaoyang Wang, Junjie Jiang +2Multi-Scale Feature FusionMultimodal Fusion

  15. Audio-Visual Camera Pose Estimation with Passive Scene Sounds and In-the-Wild Video

    Dec 13, 2025Daniel Adebi, Sagnik Majumder, Kristen GraumanRelative Pose EstimationCamera Pose Estimation

  16. A Spatial-Spectral-Frequency Interactive Network for Multimodal Remote Sensing Classification

    Oct 6, 2025Hao Liu, Yunhao Gao, Wei Li +3Remote Sensing Image UnderstandingFrequency-Domain Feature Learning

  17. Frequency-Domain Decomposition and Recomposition for Robust Audio-Visual Segmentation

    Sep 23, 2025Kai Peng, Yunzhe Shen, Miao Zhang +6Audio-Visual UnderstandingMultimodal Semantic Segmentation

  18. Diff-Feat: Diffusion-Based Cross-Modal Feature Extraction for Multi-Label Classification

    Sep 19, 2025Tian Lan, Yiming Zheng, Jianxin YinMultimodal ClassificationMulti-Label Classification

  19. Invisible Yet Detected: PelFANet with Attention-Guided Anatomical Fusion for Pelvic Fracture Diagnosis

    Sep 17, 2025Siam Tahsin Bhuiyan, Rashedur Rahman, Sefatul Wasi +4Medical Image AnalysisMedical Image Classification

  20. SignBind-LLM: Multi-Stage Modality Fusion for Sign Language Translation

    Aug 20, 2025Marshall Thomas, Edward Fish, Richard BowdenFingerspelling RecognitionSign Language Translation

  21. GlaBoost: A Multimodal Structured Framework for Glaucoma Risk Stratification

    Aug 3, 2025Cheng Huang, Zeyu Han, Weizheng Xie +4Multimodal FusionClinical Risk Prediction

  22. The NTNU System at the S&I Challenge 2025 SLA Open Track

    Jun 5, 2025Hong-Yun Lin, Tien-Hong Lo, Yu-Hsuan Fang +4Audio-Language ModelsAutomated Evaluation

  23. The P3^3 Dataset: Pixels, Points and Polygons for Multimodal Building Vectorization

    May 21, 2025Raphael Sulzer, Liuyun Duan, Nicolas Girard +1Remote Sensing Image SegmentationBuilding Footprint Extraction

  24. Decoding the Multimodal Mind: Generalizable Brain-to-Text Translation via Multimodal Alignment and Adaptive Routing

    May 15, 2025Chunyu Ye, Yunhao Zhang, Jingyuan Sun +3Brain-Computer InterfacesNeural Decoding

  25. Batch Augmentation with Unimodal Fine-tuning for Multimodal Fusion of Large Language Models

    May 10, 2025H M Dipu Kabir, Subrota Kumar Mondal, Mohammad Ali MoniMultimodal FusionMultimodal Classification

  26. A Multimodal Feature Distillation with Mamba-Transformer Network for Brain Tumor Segmentation with Incomplete Modalities

    Apr 22, 2024Ming Kang, Fung Fung Ting, Shier Nee Saw +3Cross-Modal Knowledge DistillationCross-Modality Medical Image Segmentation

  27. GestaltMML: Enhancing Rare Genetic Disease Diagnosis through Multimodal Machine Learning Combining Facial Images and Clinical Text

    Dec 23, 2023Da Wu, Zhanliang Wang, Hongzhuo Chen +12Rare Disease DiagnosisMultimodal Classification