Uncertainty-Aware Multimodal Fusion

Ask about this topic

Overview

Uncertainty-aware multimodal fusion studies how to combine signals such as images, audio, text, and sensor measurements when their quality or relevance varies. Common approaches estimate uncertainty with Bayesian or evidential models, predict variance or confidence scores, and use them to gate features, route experts, or combine evidence with methods such as Dempster-Shafer theory. Researchers evaluate these methods on tasks including medical imaging, sentiment analysis, robotics, and autonomous sensing, often testing missing modalities, noise, corruption, and sensor mismatch. A key open challenge is ensuring that estimated reliability is calibrated and actually predicts which inputs are useful, especially beyond controlled benchmarks.

Latest papers

CardsList

82 papers18 papers in the last four weeks, up 64% on the four weeks before. 0.1% of all new papers.

  1. Reliability-Aware Probabilistic Fusion for Multimodal Learning with Incomplete Evidence

    Oct 21, 2026Patitapaban Palo, Abhisek Tiwari, Mattia Carletti +3Reliability-Aware Multimodal FusionUncertainty-Aware Multimodal Fusion

    Institute of Biomedical Engineering, Department of Engineering Science, University of Oxford, UK

  2. Compositional neural networks for multimodal X-ray and neutron scattering inversion

    Oct 21, 2026Valentin Munteanu, Alexander Hinderhofer, Dmitry Lapkin +2Inverse ProblemsUncertainty-Aware Multimodal Fusion

    Institute of Applied Physics, University of Tübingen, Auf der Morgenstelle 10, 72076 Tübingen, Germany · Cluster of Excellence ”Machine Learning” - new perspectives for science, University of Tübingen, Germany

  3. Text-Anchored Modality Experts with Uncertainty-Guided Routing for Multimodal Sentiment Analysis

    Oct 20, 2026Xiaohua Huang, Bingrui Lei, Xiuyan Shao +4Multimodal Sentiment AnalysisModality Reliance in Multimodal Models

    the School of Communication and Information Engineering, Nanjing Institute of Technology, Nanjing 210096, China. · the School of Economics and Management, Southeast University, Nanjing 211196, China. · the School of Computer Science and Engineering, Macau University of Science and Technology, Macau SAR, 999078, China. +2

  4. Evidential Multimodal Transformers with Dempster-Shafer Fusion for Emotion Recognition

    Oct 9, 2026Muhammad Usama, Yeoreum JungMultimodal Emotion RecognitionUncertainty-Aware Multimodal Fusion

    Braindeck Inc., Seoul, Republic of Korea · School of Electrical Engineering, Korea Advanced Institute of Science and Technology (KAIST), Daejeon 34141, Republic of Korea

  5. OA-MAP: Evidence-Grounded Multi-Agent Multimodal Framework for Interpretable Knee Osteoarthritis Progression

    Oct 8, 2026Sixu Chen, Mingrui Yang, Qiang Guan +1Clinical Outcome PredictionMultimodal Agents

    Cleveland Clinic, Cleveland, OH, USA

  6. CoPoE: Multimodal Fusion via Decomposable Disease-Coordinate Product-of-Experts for Missing-Modality Alzheimer's Diagnosis

    Oct 8, 2026Chihun An, Ikbeom JangAlzheimer's Disease ClassificationMultimodal Fusion

    Department of Artificial Intelligence Hanyang University Seoul, Republic of Korea · Division of Computer Engineering Hankuk University of Foreign Studies Yongin, Republic of Korea

  7. Controlled Acquisition and Abstention in Three-Channel Score Conflicts

    Oct 7, 2026Mengzhe GengSelective PredictionUncertainty-Aware Multimodal Fusion

    National Research Council Canada

  8. Knowing When to Trust a Prior: Reliability-Gated Cue Fusion for Video Gaze Prediction

    Oct 6, 2026Lichen Zhu, Yueqian Lin, Yiheng Wang +2Gaze EstimationSelective Prediction

    Duke University, Durham, NC, USA

  9. Navigation with RF Cues: Embodied Perception Action under Multipath Uncertainty

    Oct 6, 2026Wenlihan Lu, Tianshun Li, Liuqing Yang +1Robotic InspectionRobot Navigation

    Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China

  10. Dynamic Alignment and Calibration for Multimodal Learning

    Oct 6, 2026Jinghao Xu, Zhenhua Guo, Xiaofeng Zhu +1Multimodal RobustnessCross-Modal Alignment

    School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu,China · Tianyijiaotong Technology Ltd.,China · School of Computer Science and Technology, Hainan University, Haikou, China

  11. Integrating Survival-Based Aging Models with Data-Driven RUL Prognostics

    Oct 5, 2026Abhishek Srinivasan, Juan Carlos Andresen, Sepideh Pashami +1RUL EstimationPredictive Maintenance

    Traton AB, Södertälje, Sweden · Halmstad University, Halmstad, Sweden · RISE Research Institutes of Sweden AB, Stockholm, Sweden +1

  12. ReDiffNet: Differential RGB-Infrared Learning for Low-Light UAV Oriented Vehicle Detection

    Oct 4, 2026Qifan Zhang, Ziran Zhou, Ruijie Li +4Remote Sensing Small Object DetectionInfrared Object Detection

    Dalian Maritime University, Dalian, China · The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China · Hubei University of Economics, Wuhan, China

  13. TIMBRE: Teaching Time Series Forecasters to Read, Remember, and Reconcile

    Oct 3, 2026Xinyu Guan, Zhirong Zhang, Hongyuan Liu +4Time Series ForecastingRetrieval-Augmented Time Series Forecasting

    Alibaba Group · Chang’an University · University of Electronic Science and Technology of China +4

  14. Neurodiversity-Aware Multimodal Affective Computing for Neurodevelopmental Assessment: From Norm-Referenced Classification to Context-Sensitive Decision Support

    Oct 3, 2026Mateusz Pomianek, Anna Łężniak-SerugaAffective ComputingUncertainty-Aware Multimodal Fusion

    Department of Computer and Control Engineering, Rzeszow University of Technology, Rzeszów, Poland · School of Human Sciences, VIZJA University, Warsaw, Poland

  15. NM-LIO: Multiple LiDAR-Inertial Odometry Addressing LiDAR Measurement Noise Discrepancy

    Oct 3, 2026Gunhee Shin, Seungjae Lee, Minho Oh +4Adaptive Lidar-Inertial OdometryLiDAR Odometry

    School of Electrical Engineering, KAIST (Korea Advanced Institute of Science and Technology), Daejeon 34141, Republic of Korea · Hanwha Aerospace, 6, Pangyo-ro 319 beon-gil, Bundang-gu, Seongnam-si, Gyeonggi-do, Republic of Korea

  16. When Satellite and Street Views Disagree: Cross-View Complementarity in Disaster Damage Assessment

    Oct 3, 2026Yifan Yang, Lei ZouDisaster Damage AssessmentBuilding Damage Assessment

    Department of Geography, Texas A&M University, College Station, Texas, USA

  17. Reliability-Gated Fusion of Consumer Head and Foot IMUs for Lower-Body 3D Pose

    Sep 28, 2026Zhilin Guo, Boqiao Zhang, Oszkár Urbán +8Markerless Motion Capture3D Human Pose Estimation

    University of Cambridge · Chalmers University of Technology

  18. TRACE: Expert-Aligned ECG Representation Learning with Rigorous Benchmarking and Real-World Validation in Acute Cardiac Care

    Sep 28, 2026Lovely Yeswanth Panchumarthi, Andrew Lu, Saurabh Kataria +11Representation LearningCross-Modal Representation Learning

    Department of Computer Science, Emory University, GA, USA · School of Nursing, Emory University, GA, USA · Institute for Medical Engineering and Science, Massachusetts Institute of Technology, MA, USA +3

  19. Reliability-aware Cross-sample Enhancement for Robust Multimodal Sentiment Analysis

    Sep 24, 2026Menghua Jiang, Haokai Gao, Xiangui Kang +2Multimodal Sentiment AnalysisMultimodal Robustness

    School of Computer Science and Engineering, Sun Yat-sen University · School of Computer Science, South China Normal University · School of Electronics and Information Technology, Sun Yat-sen University

  20. When Point Clouds Outperform Pixels: Rethinking Zero-Shot Multimodal Anomaly Detection

    Sep 22, 2026Chenglin Ye, Lupeng Liu, Dongbo Yu +2Multimodal Anomaly DetectionPoint Cloud Anomaly Detection

    School of Artificial Intelligence, University of Chinese Academy of Sciences

  21. MIGU: Multimodal Instruction Grounding under Uncertainty for Manipulation Planning

    Sep 21, 2026Mingke Lu, Anxing Xiao, David HsuMultimodal GroundingRobot Task Planning

    School of Computing and Smart Systems Institute, National University of Singapore, Singapore · Department of Electrical and Computer Engineering, University of California, Los Angeles, Los Angeles, CA, USA

  22. Knowing When to Trust Images: Reliability-Aware Multi-modal Entity Alignment

    Sep 20, 2026Chenxiao Li, Yunhe Feng, Dongfang Liu +3Cross-Modal AlignmentUncertainty-Aware Multimodal Fusion

    University of North Texas · Purdue University · Meta Inc

  23. GaitVista: Reliability-Aware AI Measurement toward Accessible Longitudinal Gait Assessment

    Sep 18, 2026Nethmi Jayasinghe, Mihir Parashar, Amit Ranjan TrivediGait AnalysisMultimodal Sensor Fusion

    Department of Electrical and Computer Engineering, University of Illinois Chicago, Chicago, IL, USA. · Illinois Mathematics and Science Academy, Aurora, IL, USA.

  24. Refine Then Fusion: Training-Free 3D Point Cloud Adaptation with Priority Refinement and Multi-Modal Knowledge Fusion

    Sep 18, 2026Hang Cheng, Yan Chen, Mingyu Fan +1Few-Shot LearningPoint Cloud Learning

    Tsinghua Shenzhen International Graduate School, Tsinghua University

  25. ReH-FUSE: Reliability-Aware Hierarchical Fusion of Experts for Multimodal Emotion Recognition in Conversation

    Sep 12, 2026Guan-Hua Wen, Hou-Chiang Tseng, Kuan-Yu ChenEmotion Recognition in ConversationsMultimodal Emotion Recognition

    National Taiwan University of Science and Technology, Taipei, Taiwan

  26. Hyper-LLaVA: Hyperbolic Uncertainty-aware Modality-Balanced Routing for Multimodal Continual Instruction Tuning

    Sep 12, 2026Kunlun Xu, Yanqin Zhang, Wenwen Qiang +1Multimodal Instruction TuningMultimodal Large Language Models

    Wangxuan Institute of Computer Technology, Peking University, Beijing, China · Institute of Software Chinese Academy of Sciences, Beijing, China · University of Chinese Academy of Sciences, Beijing, China

  27. RiVaT-Fuse: Reliability-Calibrated Variational Tensor Fusion with Matrix-Valued Trust for Multimodal Prediction under Modality Uncertainty

    Sep 9, 2026Yingfan Xu, Tieming Liu, Ye LiangMultimodal RobustnessUncertainty-Aware Multimodal Fusion

    School of Industrial Engineering and Management Oklahoma State University · Department of Statistics, Oklahoma State University

  28. A Confidence-Aware Multimodal Fusion Framework for Industrial Human-Robot Collaboration

    Sep 9, 2026Xinyu Liu, Qiqi Dong, Boya Jia +2Intent ClassificationHuman-Robot Collaboration

  29. MURAL: Multimodal Uncertainty-aware Recommendation via Adaptive edge Learning

    Sep 4, 2026Ahmad Mousavi, Majid Alikhani, Yeon-Chang Lee +2Multimodal RecommendationMultimodal Graph Learning

    Department of Mathematics and Statistics American University · Independent Researcher Toronto, Ontario, Canada · Independent Researcher +4

  30. Uncertainty-Aware Multimodal Anti-UAV Detection via Evidential Fusion and Conflict-Discounted Belief Aggregation

    Aug 29, 2026Sharanda Suttorp, Seyed Sahand Mohammadi Ziabari, Ali Mohammed Mansour AlsahagUncertainty-Aware Multimodal FusionAerial Object Detection

    University of Amsterdam Amsterdam, The Netherlands · Department of Computer Science and Technology, SUNY Empire State University Saratoga Springs, NY, USA