Multimodal Robustness

Follow this topic Ask about this topic

Momentum

22 papers in the last four weeks, up 120% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 183

All topics

Reinforcement Learning-Guided Retrieval with Soft Fusion for Robust Multimodal Imitation Learning under Missing Modalities
Jun 13, 2026Hassan Ismkhan, Hamid BouchahciaMissing-Modality Learning Multimodal Robustness
Bournemouth University Bournemouth, UK
Forced Deferral: Manipulating Routing Decisions in Multimodal LLM Cascades
Jun 13, 2026Zhongye Liu, Yaopei Zeng, Yurui Chang +1Adversarial Attacks on VLMs Multimodal Robustness
Pennsylvania State University
Fusion is not one-size-fits-all: Cross-Modal Representation Alignment for Time-to-Event Modeling
Jun 13, 2026Zhemin Zhang, Weijie Chen, David Le +7Event Time Prediction Multimodal Robustness
Arizona State University · Mayo Clinic
Augmentation techniques for video surveillance in the visible and thermal spectral range
Jun 11, 2026Vanessa Buhrmester, Ann-Kristin Grosselfinger, David Munch +1Multimodal Robustness Data Augmentation
Fraunhofer IOSB, Gutleuthausstraße 1, 76275 Ettlingen, Germany
Does AI Reviewer See the Full Picture? Attacking and Defending Multimodal Peer Review
Jun 10, 2026Xinyu Zhao, Rana Muhammad Shahroz Khan, Zhen Xu +2Adversarial Attacks on VLMs Multimodal Robustness
The University of North Carolina at Chapel Hill · Arizona State University
Missing-Token Prompted Reliability-Aware Fusion for Robust Polyglot Speaker Identification
Jun 10, 2026Peng Jia, Li Dai, Jia Li +3Missing-Modality Learning Multimodal Robustness
Hefei University of Technology Hefei, China · Intelligent Interconnected Systems Laboratory of Anhui Province Hefei University of Technology Hefei, China
Understanding Cross-Sensor Feature Variations for Generalizable 3D Perception
Jun 10, 2026Xin Qiu, Wenjie Liu, Fuyuan Ai +3Camera-Radar Fusion Multimodal Robustness
Zhejiang University
Heterogeneous 2D/1D Signal Representation Fusion for Underwater Acoustic Modulation Recognition Under Distribution Shift
Jun 8, 2026Ronglai Qian, Liang An, Xiaoyan Wang +3Multimodal Robustness Multimodal Fusion
Key Laboratory of Underwater Acoustic Signal Processing of the Ministry of Education, Southeast University, Nanjing 210096, China
CAMF-Det: Closure-Aware Multimodal Fusion for LiDAR-Camera 3D Object Detection on UAV Platforms
Jun 8, 2026Yanze Jiang, Yanfeng Gu, Xian LiLidar-Based 3D Object Detection Multimodal Robustness
School of Electronics and Information Engineering, Harbin Institute of Technology, Harbin 150001, China
When Correct Decisions Hide Internal Stress: Decision-State Probing in Multimodal Language Models
Jun 7, 2026Haoran Zhao, Soyeon Caren Han, Eduard HovyMultimodal Robustness Multimodal Large Language Models
1The University of Melbourne
LongMoE: Longitudinal Multimodal Learning via Trajectory-Aware Mixture-of-Experts
Jun 6, 2026Maxx Richard Rahman, Prakhar Kumar, Wolfgang MaassMissing-Modality Learning Multimodal Robustness
German Research Centre for Artificial Intelligence (DFKI) Saarbrücken, Germany
A robust PPG foundation model using multimodal physiological supervision
Jun 5, 2026Eloy Geenjaar, Vince Calhoun, Scott Daly +4Multimodal Robustness Multimodal Pretraining
Department of Electrical and Computer Engineering, Georgia Institute of Technology, Atlanta, USA · Tri-Institutional Center for Translational Research in Neuroimaging and Data Science (TReNDS), Georgia State University, Georgia Institute of Technology, Emory University, Atlanta, USA · Dolby Laboratories, San Francisco, USA
From Vision to Text: A Compact Multimodal Approach for Robust, Cross-Domain Presentation Attack Detection on ID Cards
Jun 5, 2026Qingwen Zeng, Juan E. Tapia, Sneha Das +1Multimodal Robustness Domain Generalization
Technical University of Denmark (DTU), Denmark · da/sec-Biometrics and Security Research Group, Hochschule Darmstadt, Germany
M2S-AVSR: Modality-aware Multi-view Self-supervised Representation for Robust Audio-Visual Speech Recognition
Jun 4, 2026Fei Su, Cancan Li, Ming Li +1Multimodal Robustness Audio-Visual Representation Learning
School of Artificial Intelligence and the School of Computer Science, Wuhan University, China · School of Artificial Intelligence, The Chinese University of Hong Kong, Shenzhen, China · School of Artificial Intelligence, Wuhan University, China
PereStruct: Multimodal Semantic Assembly for Robust Historical Document Parsing
Jun 3, 2026Maksim Shandybo, Ivan Bespalov, Daniil Yefimov +2Historical Document Analysis Multimodal Robustness
1IGIC RAS, Moscow, Russian Federation · 2Yandex Cloud, Moscow, Russian Federation · 3National University of Science and Technology MISIS, Moscow, Russian Federation +1
Exploring Adversarial Robustness and Safety Alignment in Multilingual Multi-Modal Large Language Models
Jun 2, 2026Hashmat Shadab Malik, Muzammal Naseer, Salman KhanMultilingual Language Model Evaluation Adversarial Attacks on VLMs
Mohamed Bin Zayed University of AI, UAE · Khalifa University, UAE · Australian National University, Australia
FAF-CD: Frequency-Aware Fusion for Change Detection under Imperfect Multimodal Remote Sensing
Jun 2, 2026Yufan Wang, Sokratis Makrogiannis, Chandra KambhamettuMultimodal Robustness Frequency-Domain Feature Learning
University of South Florida · 2Delaware State University
AVI-Bench: Toward Human-like Audio-Visual Intelligence of Omni-MLLMs
Jun 1, 2026Yaoting Wang, Ziyi Zhang, Wenming Tu +10Audio-Visual Understanding Multimodal Robustness
Institute of Big Data, College of Computer Science and Artificial Intelligence, Fudan University, Shanghai, China · Huazhong University of Science and Technology · Shanghai Jiaotong University +5
Multi-modal Video Representation Alignment for Robust Self-supervised Driver Distraction Detection
Jun 1, 2026David J. Lerch, Livien Majer, Zeyun Zhong +3Multimodal Robustness Cross-Modal Alignment
Fraunhofer IOSB, Karlsruhe, Germany · Karlsruhe Institute of Technology (KIT), Karlsruhe, Germany
Before Fusion, Ask What to Keep: Contextual Calibration of Multimodal Signals
Jun 1, 2026Jiyuan Liu, Liangwei Nathan Zheng, Wei Emma Zhang +2Multimodal Robustness Multimodal Fusion
Adelaide University Adelaide, Australia · Shandong University Jinan, China
Jailbreaking Multimodal Large Language Models using Multi-Clip Video
Jun 1, 2026Choongwon Kang, Seungjong Sun, Hyunmin Jun +1Multimodal Robustness Video-Language Models
Department of Applied Artificial Intelligence, Sungkyunkwan University · Department of Human-Artificial Intelligence Interaction, Sungkyunkwan University
Reusing Fusion-Time Spectral Reliability for Adaptive Fusion and Expert Routing in RGB-Infrared Object Detection
May 31, 2026Yefeng WuMultimodal Robustness Adaptive Model Routing
MMDG-Bench: A Benchmark for Multimodal Domain Generalization
May 30, 2026Qianshan Zhan, Qian Wang, Da Li +2Multimodal Robustness Domain Generalization
University of Manchester · Jiyue AI · Samsung AI Centre Cambridge +1
Beyond the Mouth: Upper-Face Affective Cues in Audiovisual Sentence Recognition under Acoustic Uncertainty
May 30, 2026Zhou Yang, Yueyi YangMultimodal Robustness Automatic Speech Recognition
Faculty of Education and Psychology, University of Oulu, Finland · Center for Machine Vision and Signal Analysis, University of Oulu, Finland
4D Radar Meets LiDAR and Camera: Cooperative Perception under Adverse Weather
May 29, 2026Melih Yazgan, Iramm Hamdard, Qiyuan Wu +1Camera-Radar Fusion Multimodal Robustness
1FZI Research Center for Information Technology · 2Karlsruhe Institute of Technology
Geometry-based Schrödinger Bridges for Trustworthy Multimodal Fusion
May 29, 2026Jiayu Xiong, Jing Wang, Qi Zhang +2Multimodal Robustness Multimodal Fusion
Department of Computer Science and Techonology, Huaqiao University · Xiamen Key Laboratory of Computer Vision and Pattern Recognition, Huaqiao University · Tongji University +1
Can BEV Perception Gracefully Degrade under Sensor Failures?
May 29, 2026Haifa Zhang, Yijing Wang, Haoyu Wang +2Missing-Modality Learning Multimodal Robustness
Tianjin Key Laboratory of Intelligent Unmanned Swarm Technology and System, School of Electrical and Information Engineering, Tianjin University, Tianjin, China · Key Laboratory of System Control and Information Processing, Ministry of Education of China, Shanghai, China
Text-guided Feature Disentanglement for Cross-modal Gait Recognition
May 29, 2026Zhiyang Lu, Ming ChengDisentangled Representation Learning Multimodal Robustness
Fujian Key Laboratory of Urban Intelligent Sensing and Computing, Xiamen University. · Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University, 361005, P.R. China.
Context-driven Missing-Modality Learning for Robust Medical Diagnosis with Image-Tabular Data
May 25, 2026Tianling Liu, Lequan Yu, Tong Han +1Missing-Modality Learning Multimodal Robustness
College of Intelligence and Computing, Tianjin University, Tianjin 300350, China. · Department of Statistics and Actuarial Science, School of Computing and Data Science, The University of Hong Kong, Hong Kong. · Department of Radiology, Tianjin Huanhu Hospital, Tianjin 300350, China. +2
Do Image-Text Metrics Respect Semantic Invariances?
May 23, 2026Amit Agarwal, Hitesh Laxmichand Patel, Meizhu Liu +9VLM Evaluation Multimodal Robustness
Oracle AI

Jun 13, 2026, cs.ROJ/K move · Enter open · S save

Reinforcement Learning-Guided Retrieval with Soft Fusion for Robust Multimodal Imitation Learning under Missing Modalities

Hassan Ismkhan, Hamid Bouchahcia

Bournemouth University Bournemouth, UK

Missing-Modality Learning Multimodal Robustness Imitation Learning Interactive Imitation Learning Robot Imitation Learning

Open paper AskPDF arXiv

Robotic systems perceive the world through multiple input modalities -- including visual camera streams and natural language instructions -- and must select appropriate actions based on these signals. However, assuming the permanent availability of all input devices is unrealistic, as sensors may fail, become occluded, or drop out entirely during deployment. Robust handling of such missing-modality scenarios is therefore essential for real-world robot operation. This paper introduces RL4IL, a reinforcement learning guided method for imitation learning that selects the most suitable action for a given observation by identifying the most relevant expert demonstrations from a training library. A reinforcement learning policy, trained via Proximal Policy Optimisation over Breadth-First Search candidate sets, ranks candidate demonstrations and a soft cross-attention fusion head aggregates their action signals to produce the final prediction. When a modality is missing at inference time, a dedicated per-modality RL retrieval policy identifies donor demonstrations from the training library, and a soft imputation head reconstructs the missing embedding via cross-attention over the top-ranked donors -- without requiring any retraining of the system. Experiments on three LIBERO benchmark suites demonstrate that RL4IL substantially outperforms state-of-the-art imitation learning methods under sensor dropout conditions, while requiring no policy network training. The code can be found at https://github.com/h-ismkhan/Reinforcement-Learning-via-kNN-for-Robotic-Learning-with-Missing-Camera

Momentum

Related topics

Latest papers 183