Multimodal Sensor Fusion

Ask about this topic

Overview

The field studies how to combine complementary measurements—such as camera and LiDAR, radar and vision, or wearable motion and physiological signals—when no single sensor provides a complete or reliable picture. Common approaches include probabilistic filters and factor graphs for state estimation, feature- or decision-level fusion, and attention-based networks that learn cross-modal relationships; alignment and uncertainty modeling help account for mismatched timing, geometry, or sensor quality. Methods are evaluated in settings such as autonomous driving on nuScenes and KITTI, robotic manipulation, wearable activity recognition, and satellite image analysis. A central challenge is making fusion robust to missing, degraded, or misaligned sensors: adding modalities can help, but can also introduce noise, negative transfer, extra computation, and poor performance when conditions differ from training.

Latest papers

CardsList

418 papers37 papers in the last four weeks, down 16% on the four weeks before. 0.2% of all new papers.

  1. BadmintonGRF: A Multimodal Dataset and Benchmark for Markerless Ground Reaction Force Estimation in Badminton

    May 3, 2026Kuoye Niu, Jianwei Li, Shengze Cai +6Benchmark ConstructionMultimodal Sensor Fusion

    Beijing Sport University · Beijing, China · Zhejiang University +4

  2. EduGage: A Multimodal Dataset and Benchmark for Sensor-Based Momentary Assessment of Engagement in Self-Guided Video Learning

    May 2, 2026Zikang Leng, Edan Eyal, Yingtian Shi +3Wearable SensingUser Modeling

    College of Computing, Georgia Institute of Technology, Atlanta, Georgia, USA · School of Computing Technologies, RMIT University, Melbourne, Australia

  3. Representative Spectral Correlation Network for Multi-source Remote Sensing Image Classification

    Apr 30, 2026Chuanzheng Gong, Feng Gao, Junyan Lin +2Remote Sensing Image UnderstandingFeature Selection

    State Key Laboratory of Physical Oceanography, Ocean University of China, Qingdao 266100, China · Department of Computing, The Hong Kong Polytechnic University, Hong Kong, China · Department of Electrical and Computer Engineering, Mississippi State University, Starkville, MS 39762 USA

  4. Continuous Flood Nowcasting in South Asia: A Multi-Sensor Ensemble Remote Sensing Framework for Flood Extent

    Apr 29, 2026Usman Nazir, Disha Gomathinayagam, Muhammad Kamran +1Flood Inundation MappingRemote Sensing

    Planetary Health Informatics (PHI) Lab, University of Oxford · Blavatnik School of Government, University of Oxford · PMIU Secretariat, Irrigation Department, Pakistan

  5. Star-Fusion: A Multi-modal Transformer Architecture for Discrete Celestial Orientation via Spherical Topology

    Apr 29, 2026May Hammad, Menatallh HammadSpacecraft Attitude DeterminationMultimodal Sensor Fusion

    Department of Aerospace Informatics, Julius-Maximilians-Universität Würzburg

  6. Camera-RFID Fusion for Robust Asset Tracking in Forested Environments

    Apr 29, 2026John Hateley, Sriram Narasimhan, Omid AbariMultimodal Sensor Fusion

    Department of Mechanical & Aerospace Engineering at the University of California, Los Angeles · Department of Electrical and Computer Engineering at the University of California, Los Angeles

  7. RADIO-ViPE: Online Tightly Coupled Multi-Modal Fusion for Open-Vocabulary Semantic SLAM in Dynamic Environments

    Apr 28, 2026Zaid Nasser, Mikhail Iumanov, Tianhao Li +3Simultaneous Localization and MappingVisual SLAM

    Biomechatronics and Energy-Efficient Robotics (BE2R) Lab, ITMO University, Saint Petersburg, Russia

  8. Improving Sensing Coverage and Compliance of 3D-Printed Artificial Skins Through Multi-Modal Sensing and Soft Materials

    Apr 28, 2026Carson Kohlbrenner, Caleb Escobedo, Sayak Ray +5Robotic Tactile SensingSoft Robotics

    Human Interaction and Robotics [HIRO] Group, University of Colorado Boulder, 1111 Engineering Drive, Boulder, CO USA.

  9. Bridging the Indoor-Outdoor Gap: Cross-Technology Ranging for Seamless Robot Navigation

    Apr 28, 2026Paul SchwarzbachRobot NavigationWireless Localization

    Institute of Traffic Telematics, TUD Dresden University of Technology, 01069 Dresden, Germany

  10. An analysis of sensor selection for fruit picking with suction-based grippers

    Apr 27, 2026Eva Krueger, Marcus Rosette, Joseph R. DavidsonRobotic GraspingTactile Sensing

    Collaborative Robotics and Intelligent Systems (CoRIS) Institute, Oregon State University, Corvallis OR 97331, USA

  11. Modular Sensory Stream for Integrating Physical Feedback in Vision-Language-Action Models

    Apr 25, 2026Jimin Lee, Huiwon Jang, Myungkyu Koo +2Vision-Language-Action ModelsVLMs for Robotics

    KAIST · RLWRLD

  12. Ufil: A Unified Framework for Infrastructure-based Localization

    Apr 23, 2026Simon Schäfer, Lucas Hegerath, Marius Molz +2Intelligent Transportation SystemsRobot Localization

    The authors are with the Chair of Embedded Software, RWTH Aachen University · B. Alrifaee is with the Department of Aerospace Engineering, University of the Bundeswehr Munich, Germany

  13. Channel-Free Human Activity Recognition via Inductive-Bias-Aware Fusion Design for Heterogeneous IoT Sensor Environments

    Apr 23, 2026Tatsuhito HasegawaHuman Activity RecognitionInternet of Things

    Graduate School of Engineering, University of Fukui, Bunkyo, Fukui, 9108507, Fukui, Japan

  14. FingerEye: Learning Dexterous Manipulation with Continuous Vision-Tactile Sensing

    Apr 22, 2026Zhixuan Xu, Yichen Li, Xuanye Wu +2Dexterous ManipulationRobotic Tactile Sensing

    National University of Singapore · RoboScience · Huazhong University of Science and Technology +1

  15. Multimodal embodiment-aware navigation transformer

    Apr 21, 2026Louis Dezons, Quentin Picard, Rémi Marsal +2Off-Road Robot NavigationRobot Navigation

    AMIAD Pˆole Recherche, Palaiseau, France · U2IS, ENSTA, Institut Polytechnique de Paris, Palaiseau, France

  16. DGHMesh: A Large-scale Dual-radar mmWave Dataset and Generalization-focused Benchmark for Human Mesh Reconstruction

    Apr 19, 2026Rongxiao Guo, Qingchao Chen3D ReconstructionHuman Mesh Recovery

    Institute of Medical Technology, Peking University Health Science Center · National Institute of Health Data Science at Peking University · State Key Laboratory of General Artificial Intelligence, Peking University

  17. Tri-Modal Fusion Transformers for UAV-based Object Detection

    Apr 17, 2026Craig Iaboni, Pramod AbichandaniInfrared Object DetectionMultimodal Object Detection

    New Jersey Institute of Technology

  18. MMGait: Towards Multi-Modal Gait Recognition

    Apr 17, 2026Chenye Wang, Qingyuan Cai, Saihui Hou +2Cross-Modal Representation LearningGait Analysis

    School of Artificial Intelligence, Beijing Normal University

  19. FreqTrack: Frequency Learning based Vision Transformer for RGB-Event Object Tracking

    Apr 16, 2026Jinlin You, Muyu Li, Xudong ZhaoVision TransformerFrequency-Domain Feature Learning

    Dalian University of Technology

  20. Radar-Informed 3D Multi-Object Tracking under Adverse Conditions

    Apr 15, 2026Bingxue Xu, Emil Hedemalm, Ajinkya Khoche +13D MOTMultimodal Robustness

    KTH Royal Institute of Technology, Stockholm 10044, Sweden.

  21. Reliability-Guided RGB-D Sensor Fusion for Glare-Resilient Navigation Costmaps

    Apr 14, 2026Shang-En Tsai, Wei-Cheng SunDepth EstimationRobot Navigation

    Department of Computer Science and Information Engineering, Chang Jung Christian University

  22. Stress Detection Using Wearable Physiological and Sociometric Sensors

    Apr 14, 2026Oscar Martinez Mozos, Virginia Sandulescu, Sally Andrews +4Multimodal Sensor FusionWearable Sensing

    Dept. of Electronics, Computer Technology and Projects, Polytechnic University of Cartagena, Plaza del Hospital, n1, 30202, Cartagena, Spain · Faculty of Automatic Control and Computer Science, Politehnica University of Bucharest, 313 Splaiul Independentei, Bucharest, 060042, Romania · Division of Psychology, Nottingham Trent University, Burton Street, Nottingham, NG1 4BU, UK +2

  23. CDPR: Cross-modal Diffusion with Polarization for Reliable Monocular Depth Estimation

    Apr 13, 2026Rongjia Yu, Tong Jia, Hao Wang +4Depth EstimationMonocular Depth Estimation

  24. Spatio-Temporal Difference Guided Motion Deblurring with the Complementary Vision Sensor

    Apr 12, 2026Yapeng Meng, Lin Yang, Yuguo Chen +6Image DeblurringMultimodal Sensor Fusion

    Department of Precision Instrument, Tsinghua University, Beijing, China · Communication University of China, Beijing, China · Primevision Technology, Shanghai, China +1

  25. Sense Less, Infer More: Agentic Multimodal Transformers for Edge Medical Intelligence

    Apr 12, 2026Chengwei Zhou, Zhaoyan Jia, Haotian Yu +6Efficient Multimodal InferenceWearable Health Monitoring

    Case Western Reserve University, USA · University of Southern California, USA

  26. RAGE-XY: RADAR-Aided Longitudinal and Lateral Forces Estimation For Autonomous Race Cars

    Apr 9, 2026Davide Malvezzi, Nicola Musiu, Eugenio Mascaro +2Vehicle DynamicsAutonomous Racing

  27. Multi-Modal Landslide Detection from Sentinel-1 SAR and Sentinel-2 Optical Imagery Using Multi-Encoder Vision Transformers and Ensemble Learning

    Apr 7, 2026Ioannis NasiosRemote Sensing Image UnderstandingRemote Sensing

    Nodalpoint Systems, Pireos 205, Athens, 118 53, Greece

  28. Weather-Conditioned Branch Routing for Robust LiDAR-Radar 3D Object Detection

    Apr 7, 2026Hongsheng Li, Lingfeng Zhang, Zexian Yang +4Radar-LiDAR FusionObject Detection

    Shenzhen International Graduate School, Tsinghua University, Shenzhen, Guangdong, China · College of Computer and Data Science, Fuzhou University, Fuzhou, Fujian, China · Institute of Information Engineering, CAS, Beijing, China +1

  29. Synchronous Observer Design for Landmark-Inertial SLAM with Magnetometer and Intermittent GNSS Measurements

    Apr 6, 2026Arkadeep Saha, Pieter van Goor, Ravi BanavarSimultaneous Localization and MappingMultimodal Sensor Fusion

    Centre for Systems and Control, Indian Institute of Technology Bombay, Powai, Mumbai-400076, India. · School of Aerospace, Mechanical and Mechatronic Engineering (AMME), Faculty of Engineering, University of Sydney, NSW, 2006, Australia.

  30. PanLUNA: An Efficient and Robust Query-Unified Multimodal Model for Edge Biosignal Intelligence

    Apr 5, 2026Marija Zelic, Anna Tegon, Yawei Li +2Multimodal Foundation ModelsMultimodal Sensor Fusion

    Integrated Systems Laboratory, ETH Zürich, Zürich, Switzerland · DEI, University of Bologna, Bologna, Italy