Multimodal Robustness

Follow this topic Ask about this topic

Momentum

22 papers in the last four weeks, up 120% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 183

All topics

InCarEmo: A Multimodal Dataset for In-Cabin Emotion Recognition and Driver State Monitoring
Jul 16, 2026Hao Yang, Yanyan Zhao, Kewei Zhao +11Multimodal Robustness Multimodal Emotion Recognition
Harbin Institute of Technology · SERES
Automatic Hard Example Synthesis with Multi-Level Agentic Data Curation
Jul 15, 2026Genglin Liu, Muye Zhang, Krishnamurthy Viswanathan +5Multimodal Robustness Adversarial Scenario Generation
1UCLA · *Work done as a student researcher at Google. · 2Google
Learning Speaker Identity Beyond Language and Modality Constraints: Insights from the POLY-SIM 2026 Challenge
Jul 15, 2026Marta Moscati, Muhammad Saad Saeed, Marina Zanoni +9Missing-Modality Learning Multimodal Robustness
Institute of Computational Perception, Johannes Kepler University Albatross AI Linz, Austria · University of Michigan-Flint Flint, United States of America · BDO DIGITAL Gmbh Düsseldorf, Germany +5
Trust Before Fusion: QIMG-7 and Source-Aware Resolution for Polluted Multimodal RAG
Jul 12, 2026Saadeldine Eletter, Owais Aijaz, Preslav NakovMultimodal RAG Retrieval-Augmented Generation
Mohamed bin Zayed University of Artificial Intelligence
MRUF: Multi-granularity Routing with Uncertainty-Aware Fusion for Robust Multimodal Sentiment Analysis
Jul 12, 2026Haoran Ma, Yinfeng Yu, Liejun WangMultimodal Robustness Uncertainty-Aware Multimodal Fusion
School of Computer Science and Technology, Xinjiang University, Urumqi 830017, China. · Joint International Research Laboratory of Silk Road Multilingual Cognitive Computing. · Xinjiang Multimodal Intelligent Processing and Information Security Engineering Technology Research Center. +2
Multimodal Routing for Interpretable, Robust, and Auditable Clinical Prediction
Jul 10, 2026Nikkie Hooman, Zhongjie Wu, Eric C. Larson +1Multimodal Robustness Clinical Prediction
Southern Methodist University Dallas, USA
ShapKO: Shapley-Adaptive Modality Knockout for Robust Multimodal Learning
Jul 10, 2026Nusrat Binta Nizam, Fengbei Liu, Sunwoo Kwak +3Missing-Modality Learning Multimodal Robustness
Cornell University, New York, NY, USA · Cornell Tech, New York, NY, USA · Weill Cornell Medicine, New York, NY, USA
VSRo-200: A Romanian Visual Speech Recognition Dataset for Studying Supervision and Multimodal Robustness
Jul 9, 2026Iulia-Maria Udrea, Alexandra Diaconu, Bogdan AlexeMultimodal Robustness Visual Speaker Recognition
Department of Computer Science, University of Bucharest
Two-Stage Multi-Modal Fusion with Adaptive Alignment for Action Quality Assessment
Jul 8, 2026Kanglei Zhou, Ruizhi Cai, Xinning Wang +4Multimodal Robustness Cross-Modal Alignment
Dept. of Psychological and Cognitive Sciences, Tsinghua University, Beijing, China · State Key Lab of Virtual Reality Technology and Systems, Beihang University, Beijing, China · Dept. of Rheumatology and Immunology, Children’s Hospital, Capital Institute of Pediatrics, Beijing, China
General Incomplete Multimodal Learning via Dynamic Quality Perception
Jul 8, 2026Xiangyu Meng, Shicai WeiMissing-Modality Learning Multimodal Robustness
University of Electronic Science and Technology of China
HoloCount: A Holistic Visual Counting Benchmark for MLLMs
Jul 7, 2026Jinhong Deng, Limeng Qiao, Guanglu WanMultimodal Robustness Object Counting
Meituan
Benchmarking the Robustness of Autonomous Driving to Environmental Illusions: A Lane Perception Perspective
Jul 7, 2026Tianyuan Zhang, Xianglong Liu, Aishan Liu +6Autonomous Driving Benchmarks VLM Robustness
SKLCCSE, the School of Computer Science and Engineering, Beihang University, Beijing 100191, China · School of Cyber Science and Technology, Sun Yat-sen University, Shenzhen 518107, China · Henan University of Science and Technology, Luoyang 471023, China +2
InfraNet: Quality-Aware RGB Guidance for Efficient Infrared Object Detection
Jul 4, 2026Zichao Feng, Haodong Zhu, Jingying Yang +8Efficient Multimodal Inference Multimodal Robustness
Beihang University, Beijing, China · Zhongguancun Academy, Beijing, China · Communication University of China, Beijing, China +1
CoRE-VLA: Towards Scalable and Robust Vision-Language-Action Modeling via Conditional Routing of Experts
Jul 4, 2026Haozhe Zhang, Sixian Li, Yifei Zhang +5Multimodal Robustness Language-Conditioned Robot Manipulation
Zhejiang University · Shanghai Innovation Institute · Fudan University +2
$C^3$ ASD: Multi-Level Consistency-Driven Representation Learning
Jul 3, 2026Jin Hong, Jisoo Park, Junseok KwonMultimodal Robustness Audio-Visual Representation Learning
Chung-Ang University, Seoul, Republic of Korea
Learning from Reliable Latent Prompts for Visual Recognition with Missing Modalities
Jun 29, 2026Taixi Chen, Nancy GuoMissing-Modality Learning Prompt Learning
School of Computing State University of New York at Binghamton
Set-Inclusive Uncertainty Modeling for Robust Brain Tumor Segmentation
Jun 29, 2026Seunghun Baek, Jihwan Park, Jaeyoon Sim +3Representation Learning Multimodal Robustness
Pohang University of Science and Technology, Pohang, South Korea
Residual-Guided Expert Specialization for Incomplete Multimodal Learning
Jun 29, 2026Seunghun Baek, Jihwan Park, Jaeyoon Sim +3Missing-Modality Learning Multimodal Robustness
Pohang University of Science and Technology (POSTECH), South Korea
Consistency as Inductive Bias: Learning Cross-View Invariance for Robust Multimodal Reasoning
Jun 29, 2026Xin Zou, Haolin Deng, Yibo Yan +6Multi-View Consistency Multimodal Robustness
HKUST (GZ) · HKUST · UIC +1
AMR: Adaptive Modality Routing for Multimodal Polyglot Speaker Identification
Jun 28, 2026Chuxiao Zuo, Yao Zhu, Minqiang Xu +3Multimodal Robustness Speech Processing
Honor Device Co., Ltd Nanjing, China · Honor Device Co., Ltd Shanghai, China
Cross-view Multimodal Vision-Based Assessment Framework for Traditional Chinese Medicine Rehabilitation Training
Jun 26, 2026Francis Xiatian Zhang, Hao Yao, Shengxuan Chen +4Multimodal Robustness Healthcare
Department of Computer Science, Durham University, UK · Institute for Regeneration and Repair, The University of Edinburgh, UK · Ningbo Hospital of Traditional Chinese Medicine, Ningbo, China +2
RPM-Distill: Physiology-guided Adaptive Cross-modal Distillation for Robust Remote Physiological Measurement
Jun 26, 2026Jiyao Wang, Qingyong Hu, Duoxun Tang +3Remote Photoplethysmography Cross-Modal Knowledge Distillation
McGill University, Montreal, Canada · Hong Kong University of Science and Technology, Hong Kong, China · Tsinghua University, Shenzhen, China +1
Active Adversarial Perturbation-driven Associative Memory Retrieval for RGB-Event Visual Object Tracking
Jun 24, 2026Xiao Wang, Xufeng Lou, Zikang Yan +5Event-Based Vision Multimodal Robustness
School of Electronic and Computer Engineering, Shenzhen Graduate School, Peking University, China
Same Evidence, Different Answer: Auditing Order Sensitivity in Multimodal Large Language Models
Jun 24, 2026Akshay Paruchuri, Sanmi Koyejo, Ehsan AdeliMultimodal Robustness LLM Auditing
Stanford University
Text Over Image: Auditing Multimodal Robustness in Synthetic Medical Image Detection
Jun 24, 2026Ching-Hao Chiu, Hao-Wei Chung, Gelei Xu +7Multimodal Robustness Medical VLMs
University of Notre Dame, Notre Dame, IN, USA. · IBM Research, Yorktown Heights, NY, USA. · Department of Cardiology, Boston Children’s Hospital, Boston, MA, USA. +2
ADM-Fusion: Adaptive Deep Multi-Sensor Fusion for Robust Ego-Motion Estimation in Diverse Conditions
Jun 23, 2026Hasan Moughnieh, Ibrahim Ghaddar, Hadi Elham +2Multimodal Robustness Mixture of Experts
Department of Electrical and Computer Engineering, American University of Beirut, Beirut, Lebanon. · Department of Mechanical Engineering, American University of Beirut, Beirut, Lebanon.
FUSE: Frequency-domain Unification and Spectral Energy Alignment for Multi-modal Object Re-Identification
Jun 18, 2026Xuanhao Qi, Tom H. Luan, Yukang Zhang +4Cross-Modal Representation Learning Multimodal Robustness
School of Cyber Science and Engineering, Xi’an Jiaotong University, Xi’an, China · School of Informatics, Xiamen University, Xiamen, China · National University of Singapore, Singapore
Learning Robust Pair Confidence for Multimodal Emotion-Cause Pair Extraction
Jun 17, 2026Zhuangzhuang Pan, Ning Dong, Yingna Su +1Pairwise Preference Learning Multimodal Robustness
Institute for Advanced Studies Universiti Malaya Kuala Lumpur 50603, Malaysia · School of Information Engineering Suqian University Suqian 223800, China · Digitization Department Suzhou University of Technology Suzhou 215500, China
Domain Generalizable Adaptation of 3D Vision-Language Models via Regularized Fine-Tuning
Jun 16, 2026Sneha Paul, Zachary Patterson, Nizar BouguilaMultimodal Robustness VLM Adaptation
Concordia University, Canada
Probing, Fusion, and Trustworthiness: A Systematic Evaluation of Foundation Model Representations for Multimodal Cancer Analysis
Jun 15, 2026Jingyu Hu, Giuseppe Tripodi, Reed Naidoo +2Computational Pathology Multimodal Robustness
The Alan Turing Institute, London, United Kingdom · University of Manchester, Manchester, United Kingdom · The Institute of Cancer Research, London, United Kingdom +1

Jul 16, 2026, cs.AIJ/K move · Enter open · S save

InCarEmo: A Multimodal Dataset for In-Cabin Emotion Recognition and Driver State Monitoring

Hao Yang, Yanyan Zhao, Kewei Zhao, Hongbo Zhang+10

Harbin Institute of Technology · SERES

Multimodal Robustness Multimodal Emotion Recognition Driver Monitoring

Open paper AskPDF arXiv

Understanding driver emotion and state is critical for the next generation of intelligent in-cabin systems that ensure safety and enhance human-vehicle interaction. However, existing public datasets for in-cabin affective computing are largely limited to visual modalities and rarely include conversational information, making it difficult to capture the linguistic and interactive cues underlying driver emotion. To address these gaps, we introduce InCarEmo, a multimodal dataset for in-cabin emotion recognition and driver state monitoring. InCarEmo integrates RGB and infrared video, in-cabin audio, and dialogue text collected from scripted in-cabin scenarios designed to simulate realistic driver behaviors, covering diverse lighting conditions and driving contexts. The dataset supports three primary tasks: 1) multimodal emotion recognition, 2) fatigue detection, and 3) distraction monitoring. In addition to the original Chinese data, we construct an auxiliary English benchmark to support preliminary cross-lingual evaluation. We provide a unified benchmark with extensive baseline results across unimodal and multimodal methods, including analyses under modality-missing and noise conditions. Experimental results demonstrate the benefits of multimodal fusion and reveal remaining challenges under real-world noise and low-light conditions. By releasing InCarEmo, we aim to establish a comprehensive foundation for robust, interpretable, and human-centric in-cabin affective understanding, promoting safer and more empathetic driver-vehicle interaction.

Momentum

Related topics

Latest papers 183