Missing-Modality Learning

Follow this topic Ask about this topic

Momentum

10 papers in the last four weeks, up 43% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 107

All topics

Pan-FM: A Pan-Organ Foundation Model with Saliency-Guided Masking for Missing Robustness
May 8, 2026Qiangqiang Wu, Grace McIlvain, Zhou Yu +1Missing-Modality Learning Medical Imaging
Laboratory of AI and Biomedical Science (LABS) · 2Columbia University
LARGO: Low-Rank Hypernetwork for Handling Missing Modalities
May 7, 2026Niels Vyncke, Pooya Ashtari, Aleksandra PižuricaMissing-Modality Learning Cross-Modality Medical Image Segmentation
Department of Telecommunications and Information Processing Ghent University Ghent, Belgium
EmoMM: Benchmarking and Steering MLLM for Multimodal Emotion Recognition under Conflict and Missingness
May 1, 2026Yueru Sun, Yimeng Zhang, Haoyu Gu +5Missing-Modality Learning Multimodal Large Language Models
School of Future Technology, South China University of Technology, Guangzhou, China
LIMSSR: LLM-Driven Sequence-to-Score Reasoning under Training-Time Incomplete Multimodal Observations
May 1, 2026Huangbiao Xu, Huanqi Wu, Xiao Ke +1Missing-Modality Learning Multimodal Reasoning
Fujian Provincial Key Laboratory of Networking Computing and Intelligent Information Processing, College of Computer and Data Science, Fuzhou University, Fuzhou 350108, China · Engineering Research Center of Big Data Intelligence, Ministry of Education, Fuzhou 350108, China · Wangxuan Institute of Computer Technology, Peking University, Beijing 100871, China
GraphPL: Leveraging GNN for Efficient and Robust Modalities Imputation in Patchwork Learning
Apr 28, 2026Xingjian Hu, Zuoyu Yan, Jianhua Zhu +3Missing-Modality Learning Multimodal Learning
Wangxuan Institute of Computer Technology, Peking University, Beijing · Weill Cornell Medicine, Cornell University, New York · Biomedical Informatics, Stony Brook University, New York
Triple-Phase Sequential Fusion Network for Hepatobiliary Phase Liver MRI Synthesis
Apr 24, 2026Qiuli Wang, Xinhuan Sun, Fengxi Chen +7Missing-Modality Learning Medical Imaging
Yu-Yue Pathology Research Center, Jinfeng Laboratory, Chongqing, 400000, China · 7T Magnetic Resonance Translational Medicine Research Center, Department of Radiology, The First Affiliated Hospital of Army Medical University, Chongqing, 400033, China · Department of Clinical Laboratory Medicine, The First Affiliated Hospital of Army Medical University, Chongqing, 400000, China +1
Federated Cross-Modal Retrieval with Missing Modalities via Semantic Routing and Adapter Personalization
Apr 24, 2026Hefeng Zhou, Xuan Liu, Sicheng Chen +7Cross-Modal Learning Multimodal Federated Learning
Shanghai Jiao Tong University · Hohai University · Shenzhen Univ of Advanced Technology
Handling and Interpreting Missing Modalities in Patient Clinical Trajectories via Autoregressive Sequence Modeling
Apr 20, 2026Andrew Wang, Ellie Pavlick, Ritambhara SinghMissing-Modality Learning Multimodal Contrastive Learning
Brown University
Conditional Evidence Reconstruction and Decomposition for Interpretable Multimodal Diagnosis
Apr 18, 2026Shaowen Wan, Yanjun Lv, Lu Zhang +5Missing-Modality Learning Medical Diagnosis
Department of Biomedical Engineering, New Jersey Institute of Technology, Newark, NJ 07102, USA · Department of Computer Science and Engineering, The University of Texas at Arlington, Arlington, TX 76019, USA · Department of Computer Science, Indiana University Indianapolis, Indianapolis, IN 46202, USA +1
Robust Multispectral Semantic Segmentation under Missing or Full Modalities via Structured Latent Projection
Apr 17, 2026Irem Ulku, Erdem Akagündüz, Ömer Özgür TanrıöverMissing-Modality Learning Remote Sensing Image Segmentation
Department of Computer Engineering, Ankara University, Ankara, Türkiye · Department of Modeling and Simulation, Graduate School of Informatics, METU, Ankara, Türkiye
Backbone-Conditional Behavior of Modality Gating in Multi-Modal Prostate MRI Segmentation: A 5-Fold Cross-Validation and Gate Mechanism Analysis
Apr 12, 2026Yongbo Shu, Wenzhao Xie, Shanhu Yao +4Missing-Modality Learning Cross-Modality Medical Image Segmentation
The Second Xiangya Hospital of Central South University, Changsha, 410011, Hunan, China · School of Life Sciences, Central South University, Changsha, 410013, Hunan, China · Hunan Provincial Key Laboratory of Medical Information Research (Central South University), Changsha, 410011, Hunan, China +2
A Clinical Point Cloud Paradigm for In-Hospital Mortality Prediction from Multi-Level Incomplete Multimodal EHRs
Apr 6, 2026Bohao Li, Tao Zou, Junchen Ye +2Electronic Health Records Clinical Risk Prediction
Beihang University, Beijing, China · The Hong Kong Polytechnic University, Hong Kong, China
COMPASS: Fusion-Matched Supervision for Missing-Modality Human Sensing
Apr 2, 2026Hao Wang, Yanyu Qian, Pengcheng Weng +4Missing-Modality Learning Human Activity Recognition
Universität Bern, Switzerland · Xi’an Jiaotong University, China · Nanyang Technological University, Singapore
MissMAC-Bench: Building Solid Benchmark for Missing Modality Issue in Robust Multimodal Affective Computing
Jan 31, 2026Ronghao Lin, Honghao Lu, Ruixing Wu +5Missing-Modality Learning Affective Computing
Sun Yat-Sen University Guangzhou, China · South China Normal University Guangzhou, China · Sun Yat-Sen University Pazhou Laboratory Guangzhou, Guangdong, China
ADMC: Attention-based Diffusion Model for Missing Modalities Feature Completion
Jul 8, 2025Yuhan Li, Wei Zhang, Juan Chen +3Cross-Modal Learning Missing-Modality Learning
National University of Defense Technology · The Fifth Electronic Research Institute of MIIT · University of Chinese Academy of Sciences +2
Hyper-modal Imputation Diffusion Embedding with Dual-Distillation for Federated Multimodal Knowledge Graph Completion
Jun 27, 2025Ying Zhang, Yu Zhao, Xuhui Sui +5Multimodal Federated Learning Missing-Modality Learning
College of Computer Science, VCIP, DISSec, Nankai University, Tianjin, China · School of Software, Tiangong University, Tianjin, China · Shenzhen Campus of Sun Yat-sen University, Shenzhen, China +1
Efficient Audiovisual Speech Processing via MUTUD: Multimodal Training and Unimodal Deployment
Jan 30, 2025Joanna Hong, Sanjeel Parekh, Honglie Chen +4Missing-Modality Learning Audio-Visual Understanding
Meta

May 8, 2026, cs.CVJ/K move · Enter open · S save

Pan-FM: A Pan-Organ Foundation Model with Saliency-Guided Masking for Missing Robustness

Qiangqiang Wu, Grace McIlvain, Zhou Yu, Junhao Wen

Laboratory of AI and Biomedical Science (LABS) · 2Columbia University

Missing-Modality Learning Medical Imaging Medical Imaging Foundation Models Multimodal Pretraining Multimodal Foundation Models Medical Image Representation Learning+1

Open paper AskPDF arXiv

Foundation models (FMs) have shown great promise in medical imaging, but most FMs are trained on unimodal data within isolated domains, such as brain MRI alone. Human aging and disease arise through coordinated biological processes across organs, therefore motivating multimodal FMs that learn whole-body representations. A key challenge, however, is that real-world multimodal biomedical data are often missing not at random, which can reduce power, limit generalizability, and introduce bias. We propose Pan-FM, a pan-organ foundation model pre-trained on imaging from seven organs (Brain, Heart, Adipose, Liver, Kidney, Spleen, and Pancreas) under realistic missing-organ scenarios. Pan-FM uses a unified backbone that handles organ missingness during both training and inference, and is pre-trained with masking-based self-distillation. We find that naive multimodal pre-training leads to dominant-organ shortcut learning bias, with the model over-relying on dominant organs such as adipose and heart. To address this, we introduce Saliency-Guided Masking (SGM), which uses the model attention distribution to adaptively mask dominant organs during pre-training, thus encouraging more balanced cross-organ, whole-body learning. Notably, SGM introduces negligible computational overhead and can be seamlessly integrated into existing self-supervised learning frameworks to improve multi-organ representation learning. On the UK Biobank, Pan-FM achieves stronger prediction across 13 disease categories and 14 single disease entities than single-organ and multi-organ baselines, with improved robustness under missing-organ settings. Pan-FM serves as a scalable solution to realistic modality-missingness in multimodal learning in system neuroscience and as a step toward more generalizable whole-body FMs.

Momentum

Related topics

Latest papers 107