Multimodal Sensor Fusion

Ask about this topic

Overview

The field studies how to combine complementary measurements—such as camera and LiDAR, radar and vision, or wearable motion and physiological signals—when no single sensor provides a complete or reliable picture. Common approaches include probabilistic filters and factor graphs for state estimation, feature- or decision-level fusion, and attention-based networks that learn cross-modal relationships; alignment and uncertainty modeling help account for mismatched timing, geometry, or sensor quality. Methods are evaluated in settings such as autonomous driving on nuScenes and KITTI, robotic manipulation, wearable activity recognition, and satellite image analysis. A central challenge is making fusion robust to missing, degraded, or misaligned sensors: adding modalities can help, but can also introduce noise, negative transfer, extra computation, and poor performance when conditions differ from training.

Latest papers

CardsList

418 papers37 papers in the last four weeks, down 16% on the four weeks before. 0.2% of all new papers.

  1. DSERT-RoLL: Robust Multi-Modal Perception for Diverse Driving Conditions with Stereo Event-RGB-Thermal Cameras, 4D Radar, and Dual-LiDAR

    Apr 4, 2026Hoonhee Cho, Jae-Young Kang, Yuhwan Jeong +4Autonomous Driving DatasetsAutonomous Driving Benchmarks

    Visual Intelligence Lab, KAIST

  2. MOMO: Mars Orbital Model Foundation Model for Mars Orbital Applications

    Apr 3, 2026Mirali Purohit, Bimal Gajera, Irish Mehta +8Remote Sensing Image UnderstandingGeospatial Foundation Models

    Arizona State University · Jet Propulsion Laboratory, California Institute of Technology

  3. Adaptive Learned State Estimation based on KalmanNet

    Apr 2, 2026Arian Mehrfard, Bharanidhar Duraisamy, Stefan Haag +2Robot State EstimationKalman Filtering

    Research and Development, Mercedes-Benz AG, Germany · Institute for Autonomous Driving, University of the Bundeswehr Munich, Germany

  4. LEO: Graph Attention Network based Hybrid Multi Sensor Extended Object Fusion and Tracking for Autonomous Driving Applications

    Apr 2, 2026Mayank Mayank, Bharanidhar Duraisamy, Florian GeissMulti-Object TrackingAutonomous Driving

    Research and Development, Mercedes-Benz AG, Germany

  5. Semantically Annotated Multimodal Dataset for RF Interpretation and Prediction

    Apr 1, 2026Steve Blandino, Jelena Senic, Raied Caromi +4Radio Propagation ModelingMultimodal Sensor Fusion

    National Institute of Standards and Technology (NIST) · Prometheus Computing LLC

  6. Learning When to See and When to Feel: Adaptive Vision-Torque Fusion for Contact-Aware Manipulation

    Apr 1, 2026Jiuzhou Lei, Chang Liu, Yu She +2Robotic ManipulationContact-Rich Robotic Manipulation

    Jiuzhou Lei, Chang Liu, and Minghui Zheng are with the J. Mike Walker ’66 Department of Mechanical Engineering, Texas A&M University, College Station, TX 77843, USA. · Yu She is with the Edwardson School of Industrial Engineering, Purdue University, West Lafayette, Indiana, USA. · Xiao Liang is with the Zachry Department of Civil and Environmental Engineering, Texas A&M University, College Station, TX 77843, USA.

  7. PollutionNet: A Vision Transformer Framework for Climatological Assessment of NO2_2 and SO2_2 Using Satellite-Ground Data Fusion

    Mar 31, 2026Prasanjit Dey, Soumyabrata Dev, Bianca Schoen-PhelanAir Quality PredictionMultimodal Sensor Fusion

    School of Computer Science, Technological University Dublin, Ireland. · ADAPT Research Ireland Centre, Ireland. · School of Computer Science, University College Dublin, Ireland.

  8. CReF: Cross-modal and Recurrent Fusion for Depth-conditioned Humanoid Locomotion

    Mar 31, 2026Yuan Hao, Ruiqi Yu, Shixin Luo +3Humanoid Robot LocomotionRobot Locomotion

    The authors are with Institute of Cyber-Systems and Control, Zhejiang University, Hangzhou 310027, China. · Guoteng Zhang is with the School of Control Science and Engineering, Shandong University, Jinan 250061, China.

  9. Active Stereo-Camera Outperforms Multi-Sensor Setup in ACT Imitation Learning for Humanoid Manipulation

    Mar 30, 2026Robin Kühn, Moritz Schappler, Thomas Seel +1Robot Imitation LearningRobot Learning

    The authors are with the Institute of Mechatronic Systems, Leibniz University Hannover, 30167 Hannover, Germany.

  10. Energy-Aware Imitation Learning for Steering Prediction Using Events and Frames

    Mar 30, 2026Hu Cao, Jiong Liu, Xingzhuo Yan +5Autonomous DrivingImitation Learning

    School of Automation, Southeast University, Sipailou 2, Nanjing, 210096, China. · Chair of Robotics, Artificial Intelligence and Real-time Systems, Technical University of Munich, Boltzmannstraße 15, Munich, 85748, Germany. · School of Computer Science and Technology, Tongji University, Cao’an Road, Shanghai, 200070, China.

  11. WearBCI Dataset: Understanding and Benchmarking Real-World Wearable Brain-Computer Interfaces Signals

    Mar 29, 2026Haoxian Liu, Hengle Jiang, Lanxuan Hong +1EEG Artifact RemovalBrain-Computer Interfaces

    Hong Kong University of Science and Technology, Hong Kong SAR, China

  12. An End-to-end Flight Control Network for High-speed UAV Obstacle Avoidance based on Event-Depth Fusion

    Mar 28, 2026Dikai Shang, Jingyue Zhao, Shi Xu +2UAV NavigationDynamic Obstacle Avoidance

    Shanghai jiaotong University, Shanghai 200240, China · Defense Innovation Institute, AMS, Beijing 10071, China

  13. Material Identification using Multi-Modal Intrinsic Radiation and Radiography

    Mar 27, 2026Khoa Nguyen, Brendt Wohlberg, Oleg Korobkin +1Multimodal ClassificationMultimodal Sensor Fusion

    Los Alamos National Laboratory, Los Alamos, NM 87545, USA

  14. Synergistic Event-SVE Imaging for Quantitative Propellant Combustion Diagnostics

    Mar 26, 2026Jing Tao, Taihang Lei, Banglei Guan +5Computational ImagingHDR Imaging

    College of Aerospace Science and Engineering, National University of Defense Technology, Changsha 410073, China · Hunan Provincial Key Laboratory of Image Measurement and Vision Navigation, National University of Defense Technology, Changsha 410073, China · Hypersonic Technology Laboratory, National University of Defense Technology, Changsha 410073, China

  15. A Neuro-Symbolic System for Interpretable Multimodal Physiological Signals Integration in Human Fatigue Detection

    Mar 25, 2026Mohammadreza Jamalifard, Yaxiong Lei, Parasto Azizinezhad +2Multimodal Model InterpretabilityNeuro-Symbolic Reasoning

    School of Computer Science and Electronic Engineering, University of Essex, United Kingdom

  16. Comparative analysis of dual-form networks for live land monitoring using multi-modal satellite image time series

    Mar 25, 2026Iris Dumeur, Jérémy Anger, Gabriele FaccioloEfficient Multimodal InferenceLinear Attention

    CB · Kayrros SAS · Université Paris-Saclay, ENS Paris-Saclay, CNRS, Centre Borelli, 91190, Gif-sur-Yvette, France +1

  17. Multimodal Industrial Anomaly Detection via Geometric Prior

    Mar 24, 2026Min Li, Jinghui He, Gang Li +3Industrial Anomaly DetectionAnomaly Localization

    Faculty of Data Science, City University of Macau, Macau, China. · Key Laboratory of Computing Power Network and Information Security, Ministry of Education, Shandong Computer Science Center (National Supercomputer Center in Jinan), Qilu University of Technology (Shandong Academy of Sciences), Jinan, China. · Shandong Provincial Key Laboratory of Computing Power Internet and Service Computing, Shandong Fundamental Research Center for Computer Science, Jinan, China.

  18. LRC-WeatherNet: LiDAR, RADAR, and Camera Fusion Network for Real-time Weather-type Classification in Autonomous Driving

    Mar 23, 2026Nour Alhuda Albashir, Lars Pernickel, Danial Hamoud +2Autonomous DrivingMultimodal Sensor Fusion

    Halmstad University, School of Information Technology, Center for Applied Intelligent Systems Research, Halmstad, Sweden

  19. A Multi-Modal Sensor Fusion Instrument for Measuring Regional Human Mobility: The Distributed Human Data Engine (DHDE)

    Mar 23, 2026Amil Khanzada, Takuji TakemotoMultimodal Sensor FusionVirtual Sensing

    Specially Appointed Assistant Professor, Headquarters for Regional Revitalization, University of Fukui, Japan. · Professor, Headquarters for Regional Revitalization, University of Fukui, Japan.

  20. Detecting Neurovascular Instability from Multimodal Physiological Signals Using Wearable-Compatible Edge AI: A Responsible Computational Framework

    Mar 20, 2026Truong Quynh Hoa, Hoang Dinh Cuong, Truong Xuan KhanhWearable Health MonitoringMultimodal Sensor Fusion

    H&K Research Studio, Clevix LLC, Hanoi, Vietnam · the Department of Cardiology, Hanoi Heart Hospital, Hanoi, Vietnam

  21. A Multi-Modal CNN-LSTM Framework with Multi-Head Attention and Focal Loss for Real-Time Elderly Fall Detection

    Mar 20, 2026Lijie Zhou, Luran WangFall DetectionWearable Sensing

    School of Computer Science, University of Nottingham Ningbo China, Ningbo, China · School of Mathematical Sciences, University of Nottingham Ningbo China, Ningbo, China

  22. PFM-VEPAR: Prompting Foundation Models for RGB-Event Camera based Pedestrian Attribute Recognition

    Mar 20, 2026Minghe Xu, Rouying Wu, ChiaWei Chu +2Event-Based VisionMultimodal Sensor Fusion

    City University of Macau, Macau SAR, China · Zhuhai College of Science and Technology, Zhuhai, China · Macau University of Science and Technology, Macau SAR, China +1

  23. An Annotation-to-Detection Framework for Autonomous and Robust Vine Trunk Localization in the Field by Mobile Agricultural Robots

    Mar 20, 2026Dimitrios Chatziparaschis, Elia Scudiero, Brent Sams +1Multimodal Object DetectionObject Detection

    Dept. of Electrical and Computer Engineering, Univ. of California, Riverside, 900 University Avenue, Riverside, CA 92521, USA · Dept. of Environmental Sciences, Univ. of California, Riverside, 900 University Avenue, Riverside, CA 92521, USA · Gallo, 600 Yosemite Blvd, Modesto, CA 95354, USA

  24. SurfaceXR: Fusing Smartwatch IMUs and Egocentric Hand Pose for Seamless Surface Interactions

    Mar 19, 2026Vasco Xu, Brian Chen, Eric J. Gonzalez +4Gesture RecognitionMultimodal Sensor Fusion

    Vasco Xu and Henry Hoffmann are with the University of Chicago. · Brian Chen is with Northwestern University. · The remaining co-authors are with Google. +1

  25. SEAR: Simple and Efficient Adaptation of Visual Geometric Transformers for Unpaired RGB+Thermal 3D Reconstruction

    Mar 19, 2026Vsevolod Skorokhodov, Chenghao Xu, Shuo Sun +23D ReconstructionMulti-View 3D Reconstruction

    Schindler EPFL Lab, Lausanne, Switzerland · EPFL, Lausanne, Switzerland · Örebro University, Örebro, Sweden

  26. Feeling the Space: Egomotion-Aware Video Representation for Efficient and Accurate 3D Scene Understanding

    Mar 18, 2026Shuyao Shi, Kang G. Shin3D Scene Understanding3D Spatial Reasoning

    Department of Computer Science, University of Michigan, Ann Arbor, MI, USA

  27. MM-OVSeg:Multimodal Optical-SAR Fusion for Open-Vocabulary Segmentation in Remote Sensing

    Mar 18, 2026Yimin Wei, Aoran Xiao, Hongruixuan Chen +2Remote Sensing Image SegmentationOpen-Vocabulary Semantic Segmentation

    The University of Tokyo · RIKEN AIP

  28. Edge-Efficient Two-Stream Multimodal Architecture for Non-Intrusive Bathroom Fall Detection

    Mar 17, 2026Haitian Wang, Yiren Wang, Xinyu Wang +2Fall DetectionMultimodal Sensor Fusion

    The University of Western Australia, 35 Stirling Highway, Crawley, WA 6009, Australia

  29. x2x^2-Fusion: Cross-Modality and Cross-Dimension Flow Estimation in Event Edge Space

    Mar 17, 2026Ruishan Guo, Ciyu Ruan, Haoyang Wang +3Scene Flow EstimationOptical Flow

    Shenzhen International Graduate School, Tsinghua University · Harbin Institute of Technology · The University of Hong Kong

  30. MAND: Modality-Aware Novelty Detection for Open-World Egocentric Activity Recognition

    Mar 17, 2026Hyejeong Im, Wonseon Lim, Dae-Won KimContinual LearningEgocentric Video Understanding

    Department of Computer Science and Engineering, Chung-Ang University, Seoul, Republic of Korea.