Multimodal Sensor Fusion

Ask about this topic

Overview

The field studies how to combine complementary measurements—such as camera and LiDAR, radar and vision, or wearable motion and physiological signals—when no single sensor provides a complete or reliable picture. Common approaches include probabilistic filters and factor graphs for state estimation, feature- or decision-level fusion, and attention-based networks that learn cross-modal relationships; alignment and uncertainty modeling help account for mismatched timing, geometry, or sensor quality. Methods are evaluated in settings such as autonomous driving on nuScenes and KITTI, robotic manipulation, wearable activity recognition, and satellite image analysis. A central challenge is making fusion robust to missing, degraded, or misaligned sensors: adding modalities can help, but can also introduce noise, negative transfer, extra computation, and poor performance when conditions differ from training.

Latest papers

CardsList

418 papers37 papers in the last four weeks, down 16% on the four weeks before. 0.2% of all new papers.

  1. FusionCast: Enhancing Precipitation Nowcasting with Asymmetric Cross-Modal Fusion and Future Radar Priors

    Mar 2, 2026Henan Wang, Shengwu Xiong, Yifang Zhang +4Precipitation NowcastingPrecipitation Forecasting

    School of Computer Science and Artificial Intelligence, Wuhan University of Technology · School of Earth and Space Science and Technology, Wuhan University

  2. Interpretable Multimodal Gesture Recognition for Drone and Mobile Robot Teleoperation via Log-Likelihood Ratio Fusion

    Feb 27, 2026Seungyeol Baek, Jaspreet Singh, Lala Shakti Swarup Ray +3Gesture RecognitionMultimodal Fusion

    Department of Artificial Intelligence, Korea University, Seoul, Republic of Korea · Department of Computer Science, RPTU Kaisersalutern-Landau, Kaiserslautern, Germany · Embedded Intelligence, German Research Center for Artificial Intelligence (DFKI), Kaiserslautern, Germany

  3. FujiView: Multimodal Late-Fusion for Predicting Scenic Visibility

    Feb 25, 2026Bryceton Bible, Shah Md Nehal Hasnaeen, Hairong QiMultimodal Sensor FusionMultimodal Fusion

    University of Tennessee, Knoxville

  4. RT-RMOT: A Dataset and Framework for RGB-Thermal Referring Multi-Object Tracking

    Feb 25, 2026Yanqiu Yu, Zhifan Jin, Sijia Chen +4Referring MOTVisual Object Tracking

    Huazhong University of Science and Technology, Wuhan, Hubei, China · South-Central Minzu University, Wuhan, Hubei, China

  5. LiREC-Net: A Target-Free and Learning-Based Network for LiDAR, RGB, and Event Calibration

    Feb 25, 2026Aditya Ranjan Dash, Ramy Battrawy, René Schuster +1Lidar-Camera Extrinsic CalibrationCamera-LiDAR Fusion

    RPTU – University Kaiserslautern-Landau · DFKI – German Research Center for Artificial Intelligence

  6. ADM-DP: Adaptive Dynamic Modality Diffusion Policy through Vision-Tactile-Graph Fusion for Multi-Agent Manipulation

    Feb 25, 2026Enyi Wang, Wen Fan, Dandan ZhangMulti-Robot CoordinationDiffusion Policy

    Department of Bioengineering, Imperial-X Initiative, Imperial College London, London, United Kingdom.

  7. EKF-Based Depth Camera and Deep Learning Fusion for UAV-Person Distance Estimation and Following in SAR Operations

    Feb 24, 2026Luka Šiktar, Branimir Ćaran, Bojan Šekoranja +1Search-and-Rescue RoboticsAerial Robotics

    Faculty of Mechanical Engineering and Naval Architecture, University of Zagreb, Zagreb, Croatia

  8. SODA-CitrON: Static Object Data Association by Clustering Multi-Modal Sensor Detections Online

    Feb 24, 2026Jan Nausner, Kilian Wohlleben, Michael HubnerData AssociationMultimodal Sensor Fusion

    Center for Digital Safety & Security, Austrian Institute of Technology GmbH, Vienna, Austria

  9. Towards Dexterous Embodied Manipulation via Deep Multi-Sensory Fusion and Sparse Expert Scaling

    Feb 23, 2026Yirui Sun, Guangyu Zhuge, Keliang Liu +5Robotic ManipulationMultimodal Sensor Fusion

    College of Intelligent Robotics and Advanced Manufacturing, Fudan University · School of Information Science and Engineering, Harbin Institute of Technology

  10. Comparative Assessment of Multimodal Earth Observation Data for Soil Moisture Estimation

    Feb 20, 2026Ioannis Kontogiorgakis, Athanasios Askitopoulos, Iason Tsardanidis +4Agricultural Remote SensingMultimodal Sensor Fusion

    BEYOND EO Centre, IAASARS, National Observatory of Athens, Athens, Greece · Artificial Intelligence, Wageningen University & Research, The Netherlands

  11. MUOT_3M: A 3 Million Frame Multimodal Underwater Benchmark and the MUTrack Tracking Method

    Feb 20, 2026Ahsan Baidar Bakht, Mohamad Alansari, Muhayy Ud Din +5Visual Object TrackingMultimodal Visual Object Tracking

    Khalifa University, Abu Dhabi, UAE · Czech Technical University, Prague, Czech Republic · Information Technology University, Lahore, Pakistan

  12. 3D Scene Rendering with Multimodal Gaussian Splatting

    Feb 19, 2026Chi-Shiang Gau, Konstantinos D. Polyzos, Athanasios Bacharis +23D Reconstruction3D Gaussian Splatting

    Department of Electrical and Computer Engineering, University of California San Diego, USA · NVIDIA

  13. Boreas Road Trip: A Multi-Sensor Autonomous Driving Dataset on Challenging Roads

    Feb 18, 2026Daniil Lisus, Katya M. Papais, Cedric Le Gentil +4Autonomous Driving DatasetsAutonomous Driving Benchmarks

    University of Toronto Institute of Aerospace Studies, Toronto, Canada · Trimble, Canada

  14. Axle Sensor Fusion for Online Continual Wheel Fault Detection in Wayside Railway Monitoring

    Feb 18, 2026Afonso Lourenço, Francisca Osório, Diogo Risca +1Fault DiagnosisOnline Anomaly Detection

    GECAD, ISEP, Polytechnic of Porto, Rua Dr. António Bernardino de Almeida, Porto, 4249-015, Portugal

  15. Combining scEEG and PPG for reliable sleep staging

    Feb 4, 2026Jiawei Wang, Liang Xu, Shuntian Zheng +7Sleep Stage ClassificationWearable Health Monitoring

    School of Engineering, The University of Warwick, Coventry, UK · Department of Computer Science, The University of Warwick, Coventry, UK · College of Biomedical Engineering, Fudan University, Shanghai, China +2

  16. Multimodal Benchmark for Safety Assessment in Industrial Inspection Scenarios

    Jan 29, 2026Zeyi Liu, Shuang Liu, Jihai Min +7Robotic InspectionIndustrial Inspection

    Department of Automation, Tsinghua University, Beijing 100084, China · Institute for Embodied Intelligence and Robotics, Tsinghua University, Beijing 100084, China · TetraBOT Intelligence Co., Ltd., Nanjing 210000, China +2

  17. M2I2HA: Multi-modal Object Detection Based on Intra- and Inter-Modal Hypergraph Attention

    Jan 21, 2026Xiaofan Yang, Yubin Liu, Wei Pan +5Cross-Modal AlignmentHypergraph Neural Networks

  18. AsyncBEV: Cross-modal Flow Alignment in Asynchronous 3D Object Detection

    Jan 19, 2026Shiming Wang, Holger Caesar, Liangliang Nan +1Autonomous Driving Perception3D Object Detection

    Intelligent Vehicles Group, Delft University of Technology · 3D Geoinformation Group, Delft University of Technology

  19. CREPES-X: Hierarchical Bearing-Distance-Inertial Direct Cooperative Relative Pose Estimation System

    Dec 31, 2025Zhehan Li, Jiadong Lu, Zheng Wang +6Relative Pose EstimationMulti-Robot Systems

    State Key Laboratory of Industrial Control Technology, Institute of Cyber-Systems and Control, Zhejiang University, Hangzhou, China. · Huzhou Institute of Zhejiang University, Huzhou, China. · University of Michigan, Ann Arbor, MI, USA. +1

  20. HAGI++: Head-Assisted Gaze Imputation and Generation

    Nov 4, 2025Chuhan Jiao, Zhiming Hu, Andreas BullingGaze EstimationTime Series Imputation

    University of Stuttgart, Germany · The Hong Kong University of Science and Technology (Guangzhou), China

  21. Vision-Based Tactile Sensing for the Perception of the Object's Compliance and Hardness

    Oct 14, 2025Muxing Huang, Zibin Chen, Weiliang Xu +5Vision-Based Tactile SensingTactile Sensing

    Guangdong Basic Research Center of Excellence for Structure and Fundamental Interactions of Matter, Guangdong Provincial Key Laboratory of Nanophotonic Functional Materials and Devices, School of Optoelectronic Science and Engineering, South China Normal University, Guangzhou 510006, China

  22. QuantV2X: A Fully Quantized Multi-Agent System for Cooperative Perception

    Sep 3, 2025Seth Z. Zhao, Huizhi Zhang, Zhaowei Li +11V2X CommunicationCooperative Perception

    UCLA · UW-Madison · NCSU +3

  23. MMS-VPR: A Fine-Grained Multimodal Street-Level Visual Place Recognition Dataset and Evaluation Benchmark for Dense Pedestrian Environments

    May 18, 2025Yiwei Ou, Xiaobin Ren, Ronggui Sun +3Visual Place RecognitionMultimodal Model Evaluation

    The University of Auckland · Harbin Institute of Technology, Shenzhen

  24. M4-SAR: A Multi-Resolution, Multi-Polarization, Multi-Scene, Multi-Source Dataset and Benchmark for optical-SAR Object Detection

    May 16, 2025Chao Wang, Wei Lu, Xiang Li +2Remote SensingObject Detection

    Nanjing University of Science and Technology, Nanjing 210094, China · Anhui University, Hefei 230601, China · Nankai University, Tianjin 300350, China

  25. DogLegs: Robust Proprioceptive State Estimation for Legged Robots Using Multiple Leg-Mounted IMUs

    Mar 6, 2025Yibin Wu, Jian Kuang, Shahram Khorshidi +4Legged RoboticsRobot State Estimation

    Center for Robotics, University of Bonn, Germany · GNSS Research Center, Wuhan University, China · Humanoid Robots Lab, University of Bonn, Germany +1

  26. Sonicmesh: Enhancing 3D Human Mesh Reconstruction in Vision-Impaired Environments With Acoustic Signals

    Dec 15, 2024Xiaoxuan Liang, Hong Zhou, Zhaolong Wei +33D ReconstructionHuman Mesh Recovery

    University of Massachusetts at Amherst, Location, Country · Trine University · University of Illinois at Chicago +1

  27. Multi-Sensor Fusion for UAV Classification Based on Feature Maps of Image and Radar Data

    Oct 21, 2024Nikos Sakellariou, Antonios Lalas, Konstantinos Votis +1Camera-Radar FusionImage Classification

    Centre for Research and Technology Hellas, Information Technologies Institute

  28. EPMF: Efficient Perception-aware Multi-sensor Fusion for 3D Semantic Segmentation

    Jun 21, 2021Mingkui Tan, Zhuangwei Zhuang, Sitao Chen +4Camera-LiDAR FusionAutonomous Driving Perception

    School of Software Engineering, South China University of Technology · Pazhou Laboratory, Guangzhou, China · School of Electronic and Information Engineering, South China University of Technology +2