Multimodal Robustness

Follow this topic Ask about this topic

Multimodal robustness asks whether systems that combine images, text, audio, medical data, or sensors can still make reliable predictions when some inputs are missing, corrupted, or in conflict. Common approaches include modality dropout, reliability-aware gating and expert routing, uncertainty estimation, and methods that separate shared information from modality-specific or spurious features. Researchers evaluate these methods on tasks such as medical segmentation, sentiment analysis, autonomous driving, and multimodal language-model reasoning, using corruption, missing-input, domain-shift, and adversarial benchmarks. A central tension is that adding modalities can improve performance when evidence is complementary but can also introduce interference or overreliance, and gains on controlled benchmarks do not always carry over to real deployment conditions.

Latest papers

279 papers26 papers in the last four weeks, down 10% on the four weeks before. 0.2% of all new papers.

MMEarth-Bench: Global Model Adaptation via Multimodal Test-Time Training
Feb 6, 2026Lucia Gordon, Serge Belongie, Christian Igel +1Test-Time Adaptation Multimodal Robustness
Harvard University, USA · University of Copenhagen, Denmark
LaVPR: Benchmarking Language and Vision for Place Recognition
Feb 3, 2026Ofer Idan, Dan Badur, Yosi Keller +1Multimodal Robustness Visual Place Recognition
Faculty of Engineering, Bar-Ilan University cccccbkvingvvnljreflvddntlcirvicklffknccbbue Ramat Gan, Israel
MissMAC-Bench: Building Solid Benchmark for Missing Modality Issue in Robust Multimodal Affective Computing
Jan 31, 2026Ronghao Lin, Honghao Lu, Ruixing Wu +5Missing-Modality Learning Affective Computing
Sun Yat-Sen University Guangzhou, China · South China Normal University Guangzhou, China · Sun Yat-Sen University Pazhou Laboratory Guangzhou, Guangdong, China
MAD: Modality-Adaptive Decoding for Mitigating Cross-Modal Hallucinations in Multimodal Large Language Models
Jan 29, 2026Sangyun Chung, Se Yeon Kim, Youngchae Chee +1Multimodal Robustness Hallucination in Language Models
Integrated Vision Language Lab, KAIST, South Korea
SGM: Safety Glasses for Multimodal Large Language Models via Neuron-Level Detoxification
Dec 17, 2025Hongbo Wang, AprilPyone MaungMaung, Isao EchizenMLLM Safety Multimodal Robustness
1Graduate School of Information Science and Technology, The University of Tokyo, Japan · 2Information and Society Research Division, National Institute of Informatics, Japan · 3The Graduate University for Advanced Studies, SOKENDAI, Japan
MCA: Modality Composition Awareness for Robust Composed Multimodal Retrieval
Oct 17, 2025Qiyu Wu, Shuyang Cui, Satoshi Hayakawa +3Multimodal Robustness Multimodal Large Language Models
Sony Group Corporation · Sony AI
Attention Misses Visual Risk: Risk-Adaptive Steering for Multimodal Safety Alignment
Oct 15, 2025Jonghyun Park, Minhyuk Seo, Chaewon Yeo +1Multimodal Robustness LLM Safety Alignment
Seoul National University · KU Leuven
Disentangling Bias by Modeling Intra- and Inter-modal Causal Attention for Multimodal Sentiment Analysis
Aug 7, 2025Menghua Jiang, Yuxia Lin, Baoliang Chen +3Multimodal Robustness Sentiment Analysis
School of Computer Science, South China Normal University, Guangzhou, China, 510631 · School of Electronics and Information Technology, Sun Yat-sen University, Guangzhou, China, 510275
Evidence-Grounded Trustworthy Multimodal Reasoning and Evaluation Benchmark in Complex Urban Scenes
Jun 11, 2025Zhaoyang Wei, Bowen Jiang, Xumeng Han +6Multimodal Grounding Multimodal Robustness
University of Chinese Academy of Sciences(UCAS) · Tencent CDG

Feb 6, 2026J/K move · Enter open · S save

MMEarth-Bench: Global Model Adaptation via Multimodal Test-Time Training

Lucia Gordon, Serge Belongie, Christian Igel, Nico Lang

Harvard University, USA · University of Copenhagen, Denmark

Test-Time Adaptation Multimodal Robustness Geospatial Foundation Models Remote Sensing Test-Time Training Remote Sensing Benchmarks+1

Open paper AskPDF arXiv

Recent research in geospatial machine learning demonstrates that models pretrained with self-supervised learning on Earth observation data can perform well on downstream tasks with limited labeled data. However, most benchmark datasets have few data modalities and poor global representation, limiting the ability to evaluate multimodal pretrained models at global scales. In order to fill this gap, we introduce MMEarth-Bench, a collection of five new environmental tasks with 12 modalities, globally distributed data, and both random and geographic test splits. We benchmark a diverse set of pretrained models and find that while (multimodal) pretraining tends to improve model robustness in limited data settings, geographic generalization abilities remain poor. Moreover, a simple randomly initialized multimodal model is competitive given enough labeled data. Although data is abundant, models can currently only make use of the modalities on which they were pretrained. To solve this problem, we propose using all the modalities available at test time as auxiliary tasks for test-time adaptation. Our model-agnostic method for test-time training with multimodal reconstruction (TTT-MMR) can improve performance across all models and tasks on both test splits. Furthermore, geographic batching leads to a good trade-off between regularization and specialization during TTT, which is especially beneficial for long-tail distributions. Our dataset, code, and visualization tool are linked on the project page: lgordon99.github.io/mmearth-bench.