Multimodal Sensor Fusion

Ask about this topic

Overview

The field studies how to combine complementary measurements—such as camera and LiDAR, radar and vision, or wearable motion and physiological signals—when no single sensor provides a complete or reliable picture. Common approaches include probabilistic filters and factor graphs for state estimation, feature- or decision-level fusion, and attention-based networks that learn cross-modal relationships; alignment and uncertainty modeling help account for mismatched timing, geometry, or sensor quality. Methods are evaluated in settings such as autonomous driving on nuScenes and KITTI, robotic manipulation, wearable activity recognition, and satellite image analysis. A central challenge is making fusion robust to missing, degraded, or misaligned sensors: adding modalities can help, but can also introduce noise, negative transfer, extra computation, and poor performance when conditions differ from training.

Latest papers

CardsList

418 papers37 papers in the last four weeks, down 16% on the four weeks before. 0.2% of all new papers.

  1. PulmoWorld: An Asynchronous Multimodal Medical World Model for Longitudinal Pneumonia Dynamics

    Oct 28, 2026Cheng Tang, Xinyu Zhang, Jiawei Chen +14Disease Progression ModelingWorld Model Learning

    Shanghai Artificial Intelligence Laboratory · University of the Chinese Academy of Sciences · Fudan University +7

  2. RESPLE2: Inertial-Informed Recursive Non-Uniform Spline Estimation for Multi-Sensor Odometry with Global Refinement

    Oct 27, 2026Ziyu Cao, William Talbot, Song Li +4Robot State EstimationMultimodal Sensor Fusion

    Department of Electrical Engineering, Linköping University, 58183 Linköping, Sweden · Robotic Systems Lab (RSL), ETH Zürich, 8092 Zurich, Switzerland · Department of Mechanical and Industrial Engineering, University of Toronto, Toronto, ON M5S 3G8, Canada +1

  3. SVIn+: Multi-Modal Framework for Robust Underwater State Estimation using 3D Sonar, Visual-Inertial, Water-Pressure, and DVL

    Oct 27, 2026Chinmay Burgul, Xiaoming Zhao, Yewei Huang +4Robot State EstimationUnderwater Robotics

    Mechanical Engineering Department, University of Delaware, Newark, DE, USA. · Dartmouth College, Hanover, NH, USA · Aquaculture Robotics and Automation Group, SINTEF Ocean

  4. Fed-MAPTrack: Federated Modality-aware Prompt Tuning for RGB-X Tracking

    Oct 26, 2026Xiangyu Liu, Yijie Shen, Yue Yin +3Multimodal Visual Object TrackingMultimodal Federated Learning

    Beijing Normal-Hong Kong Baptist University, 519087, Zhuhai, Guangdong, China · Zhejiang University, 310058, Hangzhou, Zhejiang, China · Peking University, 100871, Beijing, China +1

  5. MMGNet: Multi-Modal 6-DoF Grasp Synthesis for Cluttered Scenes

    Oct 26, 2026Rui Cao, Jia-Hui Pan, Yichuan Li +4Robotic GraspingMultimodal Sensor Fusion

    Department of Mechanical and Automation Engineering, The Chinese University of Hong Kong, Hong Kong, China · Department of Computer Science and Engineering, The Chinese University of Hong Kong, Hong Kong, China · Sino-German College of Intelligent Manufacturing, Shenzhen Technology University, Shenzhen, China

  6. AgriDefend: A $205 Offline Edge Device that Detects Crop Stress Before Visible Symptoms via Multimodal AI Fusion

    Oct 26, 2026George Xu, Nikhil Vincent, Elizabeth S. ForbesPrecision AgricultureAgricultural Remote Sensing

    Independent Researcher Bothell, WA, USA · Univ. of Michigan, School for Environment and Sustainability Ann Arbor, MI, USA

  7. An Error Budget for Shuffled Multimodal Windows in the 3rd WEAR Challenge: Stacking, Count Constraints and Seam Matching

    Oct 23, 2026Soheil KooklanHuman Activity RecognitionMultimodal Classification

    Independent Researcher

  8. GelSlim 5.0: Unifying Vision and Touch with a Single Fingertip Camera

    Oct 22, 2026Xiangdong Wang, I-Hsuan Huang, Xiaohao Xu +3Tactile Sensing for Robotic ManipulationMultimodal Sensor Fusion

    University of Michigan, Ann Arbor, MI, USA

  9. Multimodal Progress-Residual Fusion for Anticipating Navigation Retreat in Robotic Flexible Endoscopy

    Oct 22, 2026Chi Kit Ng, Tianchun Wu, Jiatan Sun +3Robotic EndoscopySurgical Robotics

  10. AeroIntent: A Synchronized Multimodal Benchmark for UAV Mission Intent Recognition

    Oct 19, 2026Yutong Chen, Jingxin Mao, Zhiwei Wei +3Multimodal Sensor Fusion

    Y. Chen, Z. Wei, and B. Li are with Tongji University. · J. Mao and R. Zhang are with The Hong Kong University of Science and Technology (Guangzhou). · J. Ni is with Nanchang Highly Electrical Appliances Co., Ltd.

  11. Multimodal Interaction Estimation for Contact-Aware Adaptive Impedance Control

    Oct 16, 2026Mateusz Jaszczuk, Paris Perdikaris, Nadia FigueroaImpedance ControlContact Force Estimation

    Department of Mechanical Engineering, University of Pennsylvania

  12. Recording-Aware Evaluation of Multimodal OT Intrusion Detection

    Oct 16, 2026Mustafa Umut Ozbek, Khalil El-Khatib, Li YangNetwork Intrusion DetectionMultimodal Sensor Fusion

    Ontario Tech University, Oshawa, Ontario, Canada

  13. STC-MGNet: Spatial-Temporal-Channel Attention-Enhanced Multimodal Graph Network for Pedestrian Intention Prediction

    Oct 16, 2026Yuanzhe Li, Meng WangPedestrian Intention PredictionSpatiotemporal GNNs

    Chair of Automotive Engineering, Technische Universität Berlin, Berlin, Germany

  14. VesperMUSE: Multi-Modal Fusion Vision Transformer for Image Semantic Segmentation Under Diverse Light Conditions

    Oct 15, 2026Raul BalmezSemantic SegmentationMultimodal Sensor Fusion

    The University of Manchester, Department of Computer Science

  15. Robust Real-Time Monocular Human Pose Estimation with Insole-Based Sensor Fusion

    Oct 14, 2026Liang Yu, Tao Zhu, Xiangqing Lu +2Human Pose EstimationMultimodal Sensor Fusion

    University of South China, Hengyang, Hunan, China · Harbin Institute of Technology, Harbin, China · Dalian University of Technology, Dalian, China

  16. Learning Multimodal Sensor Fusion with JEPA Representations for Breathing Activity and Pose Recognition

    Oct 11, 2026Yunxiang Luo, Constantino Álvarez Casado, Xiaoting Wu +4Multimodal LearningMultimodal Sensor Fusion

    Center for Machine Vision and Signal Analysis, University of Oulu, Finland · Candour Ltd, Oulu, Finland

  17. SpikeOcc: Towards Resilient Multimodal 3D Occupancy Perception via Spiking Experts

    Oct 11, 2026Yiding Sun, Zijie Xu, Hongqiang Lin +73D Occupancy PredictionMultimodal Robustness

    Peking University · Xi’an Jiaotong University · Zhejiang University +1

  18. Multimodal Sensor-Gated Mid-Air Launch and Vision-Based Recovery of a Child UAV on a Hovering Mothership Drone

    Oct 11, 2026Aashish Sahu, R. Prasanth KumarAerial RoboticsUAV Landing on Moving Platforms

    Department of Mechanical and Aerospace Engineering, Indian Institute of Technology Hyderabad, Kandi, Sangareddy 502284, India · Department of Artificial Intelligence, Indian Institute of Technology Hyderabad, Kandi, Sangareddy 502284, India

  19. Harnessing SPAD Noise: Test-Time Gated Attention for Indoor Depth Estimation

    Oct 11, 2026Yiquan GaoDepth EstimationMetric Depth Estimation

    Heriot-Watt University

  20. Sensor-Aware Joint Data Fusion and State Estimation under Sparse Multi-rate Distribution Grid Sensing

    Oct 10, 2026Pooja Algikar, Stefano Fenu, Kumarsinh Jhala +1Multimodal Sensor FusionNeural ODEs

    Argonne National Laboratory, Lemont, Illinois, USA

  21. ToF-Wheel Odometry: Navigating Degeneration of Localization With Limited Sensing Capabilities

    Oct 10, 2026Xu Yang, Wenhao Li, Zhengyu Wei +3Robot State EstimationRobot Localization

    Deptrum Ltd.

  22. UniMergeTrack: Hierarchical Model Merging for General RGB-X Visual Object Tracking

    Oct 9, 2026Shiao Wang, Xiao Wang, Yuhe Ding +5Visual Object TrackingModel Merging

    School of Computer Science and Technology, Anhui University, Hefei 230601, China · Peng Cheng Laboratory, Shenzhen, China · School of Computer Science, Peking University, China +1

  23. What Egocentric Video Embeddings Encode and How to Combine Them with Inertial Sensors for Human Activity Recognition

    Oct 9, 2026Kansuke HabanoGraph-Based Label PropagationHuman Activity Recognition

    Independent Researcher, Japan

  24. GLIO2: A GPU-Parallelized Tightly-Coupled LiDAR-Inertial-GNSS System for Robust and Real-Time Global Localization and Mapping

    Oct 8, 2026Qi Zhang, Xikun Liu, Qijun Qin +5Simultaneous Localization and MappingLiDAR Localization

    Department of Aeronautical and Aviation Engineering, The Hong Kong Polytechnic University, Hong Kong SAR, China

  25. SenseWeather: Multimodal Sensing for Robust Robotic Perception under Extreme Conditions

    Oct 8, 2026Fucai Ke, Sukai Huang, Junyi Hao +4Robotic PerceptionMultimodal Robustness

    Faculty of Information Technology, Monash University, Clayton, VIC 3800, Australia

  26. Two-Stage Multi-modal Learning for Dynamic Audio-Visual Navigation: From Physical Alignment to Scene Aggregation

    Oct 8, 2026Jia Li, Yinfeng YuAudio-Visual NavigationAudio-Visual Alignment

    School of Computer Science and Technology, Xinjiang University, Urumqi 830017, China · Joint Research Laboratory for Embodied Intelligence, Xinjiang University · Joint International Research Laboratory of Silk Road Multilingual Cognitive Computing, Xinjiang University

  27. A method for multimodal analysis of TAIGA experiment data using essential features

    Oct 6, 2026Alexander Kryukov, Julia Dubenskaya, Elena Fedotova +6Multimodal Sensor FusionHigh-Energy Physics

    Lomonosov Moscow State University, Skobeltsyn Institute of Nuclear Physics, Moscow, 119991 Russia · Research Institute of Applied Physics, Irkutsk State University, Irkutsk, 664003 Russia

  28. Source-Learned Reliance for Selective Test-Time Adaptation of Multimodal Time Series

    Oct 5, 2026Payal Mohapatra, Yueyuan Sui, Haodong Yang +3Missing-Modality LearningTest-Time Adaptation

    Arm Inc., USA · Northwestern University, USA

  29. SURGE: Sonar-fUsed Reconstruction and localization via image-gated Graph Estimation

    Oct 5, 2026Mohammed Ibrahim M, Vallabh Deogaonkar, Trung Dong +3Underwater 3D ReconstructionRobot Localization

    Department of Electrical and Computer Engineering University of South Florida Tampa, FL, USA · Department of Ocean Engineering Indian Institute of Technology Madras Chennai, India · Department of Naval Architecture and Ocean Engineering Seoul National University Seoul, South Korea

  30. EVFormer: An Egocentric Vision-EMG Bidirectional Attention Model for Bimanual Hand Pose Estimation

    Oct 3, 2026JiaCheng Ge, SiYu Zhang, ShengJie Li +13D Hand Pose EstimationMultimodal Sensor Fusion