Uncertainty-Aware Multimodal Fusion

Ask about this topic

Overview

Uncertainty-aware multimodal fusion studies how to combine signals such as images, audio, text, and sensor measurements when their quality or relevance varies. Common approaches estimate uncertainty with Bayesian or evidential models, predict variance or confidence scores, and use them to gate features, route experts, or combine evidence with methods such as Dempster-Shafer theory. Researchers evaluate these methods on tasks including medical imaging, sentiment analysis, robotics, and autonomous sensing, often testing missing modalities, noise, corruption, and sensor mismatch. A key open challenge is ensuring that estimated reliability is calibrated and actually predicts which inputs are useful, especially beyond controlled benchmarks.

Latest papers

CardsList

82 papers18 papers in the last four weeks, up 64% on the four weeks before. 0.1% of all new papers.

  1. Asymmetric Cross-Modal Fine-Grained Visual Categorization: ACF-Net and the BirdPro Benchmark

    Aug 26, 2026Bohan Deng, Shuo Ye, Zitong YuAudio-Visual UnderstandingAudio-Visual Representation Learning

    Great Bay University

  2. Variance-Guided Spatial Attention Fusion for Robust End-to-End Driving under Asymmetric Sensor Degradation

    Aug 25, 2026Weizhi Tao, Zengwang Jin, Xiao Wang +1End-to-End Autonomous DrivingMultimodal Robustness

    Department of Aeronautical and Aviation Engineering, The Hong Kong Polytechnic University, 999077, Hong Kong, China · School of Robotics, Anhui University, Hefei 230039, China; with the Anhui Provincial Engineering Research Center for Unmanned System and Intelligent Technology, Anhui University, Hefei 230039, China; and also with the Shenzhen Research Institute of Northwestern Polytechnical University, Shenzhen 518057, China · School of Robotics and Institute of Embodied Intelligence, Anhui University, Hefei 230039, China

  3. Mind the Student: Behavioral and Contextual Cues for Automated Engagement Prediction in Online Learning

    Aug 25, 2026Alperen Kantarci, Visvanathan Ramesh, Gemma RoigOnline LearningEducational Data Mining

    Institute of Computer Science Goethe University Frankfurt Frankfurt am Main, Germany · Institute of Computer Science Goethe University Frankfurt The Hessian Center for Artificial Intelligence Frankfurt am Main, Germany

  4. OmicSync: Reliability-Aware Spatial Multi-Omics Clustering with Evidence-Constrained LLM Reasoning

    Aug 24, 2026Rabeya Tus Sadia, Qiang Ye, Qiang ChengClusteringMulti-Omics Data Integration

    Department of Computer Science, University of Kentucky · Department of Mathematics, University of Kentucky

  5. Fusing Perceptual Vision Experts with Multimodal LLMs for Explainable Plant Disease Diagnosis: From Benchmark Imagery to Real-World Robotic Field Validation

    Aug 23, 2026Ranjan Sapkota, Konstantinos I. Roumeliotis, Pengyao Xie +3Plant Disease ClassificationExplainable Image Classification

    Cornell University, Department of Biological and Environmental Engineering, Ithaca, 14850, NY, USA · University of the Peloponnese, Department of Informatics and Telecommunications, Tripoli, 22131, Greece · Agricultural University of Athens, Department of Agribusiness and Supply Chain Management, Athens, 11855, Greece

  6. MARCUS: Missing-Aware Region Representation with Contextual Urban Signals for Rent Prediction

    Aug 19, 2026Chenya Huang, Bin Liang, Zhidong Li +4Urban Representation LearningLearning with Missing Data

    Data Science Institute University of Technology Sydney Sydney, Australia · The Property Investors Alliance Sydney, Australia

  7. Rethinking Auxiliary Modalities in Multimodal Zero-shot Anomaly Detection: From Semantic Fusion to Conditional Modulation

    Aug 14, 2026Peng Wu, Xin Ge, Yujia Sun +1Zero-Shot Anomaly DetectionMultimodal Anomaly Detection

    School of Computer Science, Northwestern Polytechnical University, China · School of Artificial Intelligence, Xidian University, China · School of Computing and Information Systems, Singapore Management University, Singapore

  8. EGM-Det: Entropy-Guided Multimodal Adaptive Fusion for UAV RGB-IR Object Detection

    Aug 12, 2026Cunzheng Fan, Dawei Yan, Guanlin Wang +4Infrared Object DetectionMulti-Scale Feature Fusion

    School of Cybersecurity, Northwestern Polytechnical University, Xi’an 710072, China · School of Automation and Software Engineering, Shanxi University, Taiyuan 030006, China

  9. Conformal Fusion Under Missing Modalities

    Aug 7, 2026Alireza MoayedikiaMissing-Modality LearningGroup-Conditional Conformal Prediction

    Department of Business Technology and Entrepreneurship Swinburne University of Technology Hawthorn, VIC 3122, Australia

  10. MIDAS: Mutual Information Disentanglement with Uncertainty-Aware Fusion for Incomplete Multimodal Sentiment Analysis

    Aug 6, 2026Yuhua Wen, Yingying Zhou, Qifei Li +4Missing-Modality LearningMultimodal Disentangled Representation Learning

    School of Artificial Intelligence, Beijing University of Posts and Telecommunications, Beijing 100876, China · Zhongguancun Academy, Beijing 100094, China · Beijing National Research Center for Information Science and Technology, Tsinghua University, Beijing 100084, China +1

  11. UG-UMRE: Uncertainty-Guided Modality Augmentation and Distributional Calibration for Unified Multimodal Relation Extraction

    Aug 5, 2026Bo Kong, Liruiz Jia, Yi Liang +5Relation ExtractionCross-Modal Alignment

    Xinjiang University · Urumqi, China · Chongqing Jiaotong University +1

  12. Uncertainty-Aware Crossmodal Fusion for Classification of Animal Behavior

    Aug 3, 2026Ehsan Yaghoubi, Florian HaselbeckUncertainty-Aware Multimodal FusionBioacoustics

    Weihenstephan-Triesdorf University of Applied Sciences, Department of Sustainable Agricultural and Energy Systems, Freising, Am Staudengarten 1, 85354, Germany

  13. Adaptive Confidence-weighted Expansion for Trustworthy Multi-Omics Multimodal Fusion

    Jul 22, 2026Mohammad Raahemi, Ali Sekhavati, Alireza Maleki +1Probability CalibrationUncertainty Quantification

    Faculty of Engineering, University of Ottawa, Ottawa ON K1N 1A2, Canada · RIV Lab, Department of Computer Engineering, Bu-Ali Sina University, Hamedan, Iran · School of Computing and Communications, Lancaster University, Lancaster, UK

  14. EmoEUS: Uncertainty Supervision for Multimodal Emotion Recognition in Conversation

    Jul 19, 2026Zilong Huang, Kong Aik Lee, Junjie Li +2Emotion Recognition in ConversationsMultimodal Emotion Recognition

    Dept. of Electrical and Electronic Engineering, The Hong Kong Polytechnic University · Speech, Language, and Cognition Laboratory, The University of Hong Kong

  15. MRUF: Multi-granularity Routing with Uncertainty-Aware Fusion for Robust Multimodal Sentiment Analysis

    Jul 12, 2026Haoran Ma, Yinfeng Yu, Liejun WangMultimodal Sentiment AnalysisMultimodal Robustness

    School of Computer Science and Technology, Xinjiang University, Urumqi 830017, China. · Joint International Research Laboratory of Silk Road Multilingual Cognitive Computing. · Xinjiang Multimodal Intelligent Processing and Information Security Engineering Technology Research Center. +2

  16. When Does Quality-Aware Multimodal Fusion Matter? A Leakage-Safe Diagnostic for Decision-Level Dependence

    Jun 25, 2026Jaden Moon, Arvind Pillai, Andrew CampbellUncertainty-Aware Multimodal FusionMultimodal Fusion

    Dartmouth College, United States

  17. EnTrust: Modeling Inter-Modal Conflict for Trustworthy Multimodal Medical Image Analysis

    Jun 19, 2026Dwarikanath Mahapatra, Abhijit Das, Behzad Bozorgtabar +5Cross-Modality Medical Image SegmentationDiffusion Models for Medical Image Segmentation

    Khalifa University, Abu Dhabi, UAE. · MBZUAI, Abu Dhabi, UAE. · Aarhus University, Denmark. +5

  18. Evidential Fusion Network for Multimodal Survival Prediction under Missing Modalities

    Jun 18, 2026Yucheng Xing, Hailan Mo, Zi Wang +2Missing-Modality LearningUncertainty Quantification

    National University of Singapore, Singapore · Imperial College London, London, UK

  19. Multimodal Confidence Modeling in Audio-Visual Quality Assessment

    May 2, 2026Mayesha Maliha R. Mithila, Mylene C. Q. FariasVideo Quality AssessmentUncertainty-Aware Multimodal Fusion

    Texas State University Department of Computer Science

  20. Dynamic Emotion and Personality Profiling for Multimodal Deception Detection

    Apr 18, 2026Li Zheng, Yanyi Luo, Hao Fei +5Multimodal Emotion RecognitionUncertainty-Aware Multimodal Fusion

    Key Laboratory of Aerospace Information Security and Trusted Computing, Ministry of Education, School of Cyber Science and Engineering, Wuhan University, Wuhan, China · National University of Singapore, Singapore, Singapore

  21. Cross-Attentive Bayesian Low-Rank Adaptation for Multimodal Uncertainty Estimation

    Apr 17, 2026Habibeh Naderi, Behrouz Haji Soleimani, Stan MatwinLow-Rank AdaptationUncertainty-Aware Multimodal Fusion

    Dalhousie University, Halifax NS, Canada

  22. UNRIO: Uncertainty-Aware Velocity Learning for Radar-Inertial Odometry

    Apr 15, 2026Jui-Te Huang, Tianshu Huang, Anthony Rowe +1Radar-Inertial OdometryRobot State Estimation

    Carnegie Mellon University, Pittsburgh, PA 15213, USA

  23. Probabilistic Feature Imputation and Uncertainty-Aware Multimodal Federated Aggregation

    Apr 14, 2026Nafis Fuad Shahid, Maroof Ahmed, Md Akib Haider +3Multimodal Federated LearningMissing-Modality Learning

    Department of Computer Science and Engineering, Islamic University and Technology (IUT), Gazipur, Bangladesh

  24. VLMaterial: Vision-Language Model-Based Camera-Radar Fusion for Physics-Grounded Material Identification

    Apr 13, 2026Jiangyou Zhu, He ChenCamera-Radar FusionMaterials Property Prediction

    Department of Inforation Engineering, The Chinese University of Hong Kong, Hong Kong SAR, China

  25. Are Independently Estimated View Uncertainties Comparable? Unified Routing for Trusted Multi-View Classification

    Apr 10, 2026Yilin Zhang, Cai Xu, Haishun Chen +2Evidential Deep LearningUncertainty Quantification

    Xidian University, Xi’an, China

  26. URMF: Uncertainty-aware Robust Multimodal Fusion for Multimodal Sarcasm Detection

    Apr 8, 2026Zhenyu Wang, Weichen Cheng, Weijia Li +3Uncertainty-Aware Multimodal FusionMultimodal Robustness

    School of Artificial Intelligence, China University of Mining and Technology-Beijing, Beijing 100083, China

  27. QA-MoE: Towards a Continuous Reliability Spectrum with Quality-Aware Mixture of Experts for Robust Multimodal Sentiment Analysis

    Apr 7, 2026Yitong Zhu, Yuxuan Jiang, Guanxuan Jiang +4Multimodal Sentiment AnalysisMultimodal Robustness

    The Hong Kong University of Science and Technology (Guangzhou) · Tsinghua University · Aarhus University

  28. Uncertainty-Aware Foundation Models for Clinical Data

    Apr 5, 2026Qian Zhou, Yuanyun Zhang, Shi LiEpistemic UncertaintyMultimodal Foundation Models

    University of the Chinese Academy of Sciences · Columbia University

  29. A physics-informed, behavior-aware digital twin for robust multimodal forecasting of core body temperature in precision livestock farming

    Apr 5, 2026Riasad Alvi, Mohaimenul Azam Khan Raiaan, Sadia Sultana Chowa +6Precision Livestock FarmingTime Series Forecasting

    Applied Artificial INtelligence and Intelligent Systems (AAIINS) Laboratory, Dhaka 1217, Bangladesh · Department of Computer Science and Engineering, United International University, Dhaka 1212, Bangladesh · Department of Data Science and Artificial Intelligence, Monash University, Melbourne 3800, Australia +5

  30. BVFLMSP : Bayesian Vertical Federated Learning for Multimodal Survival with Privacy

    Apr 2, 2026Abhilash Kar, Basisth Saha, Tanmay Sen +1Multimodal Survival AnalysisFederated Learning

    SQC and OR Unit, Indian Statistical Institute, Kolkata, India · Department of Statistics, Presidency University, Kolkata, India