Missing-Modality Learning

Follow this topic Ask about this topic

Missing-modality learning studies how to make systems use information from multiple sources—such as images and clinical records, audio and video, or different sensors—even when some inputs are unavailable. Common approaches simulate missing inputs with modality dropout, adapt fusion according to availability or reliability, or reconstruct missing features using generative models, prototypes, retrieval, and cross-modal distillation; some methods avoid reconstruction and fuse only observed inputs. Research spans medical imaging and electronic health records, sentiment analysis, remote sensing, and robotics, with evaluations increasingly testing varied missing patterns and corruption levels. A central tension is that reconstruction can add useful information but can also introduce errors, so systems must learn when to repair, ignore, or downweight missing or unreliable evidence while maintaining calibration and generalization.

Latest papers

173 papers22 papers in the last four weeks, up 38% on the four weeks before. 0.1% of all new papers.

GraphPL: Leveraging GNN for Efficient and Robust Modalities Imputation in Patchwork Learning
Apr 28, 2026Xingjian Hu, Zuoyu Yan, Jianhua Zhu +3Missing-Modality Learning Multimodal Learning
Wangxuan Institute of Computer Technology, Peking University, Beijing · Weill Cornell Medicine, Cornell University, New York · Biomedical Informatics, Stony Brook University, New York
Triple-Phase Sequential Fusion Network for Hepatobiliary Phase Liver MRI Synthesis
Apr 24, 2026Qiuli Wang, Xinhuan Sun, Fengxi Chen +7Missing-Modality Learning Medical Imaging
Yu-Yue Pathology Research Center, Jinfeng Laboratory, Chongqing, 400000, China · 7T Magnetic Resonance Translational Medicine Research Center, Department of Radiology, The First Affiliated Hospital of Army Medical University, Chongqing, 400033, China · Department of Clinical Laboratory Medicine, The First Affiliated Hospital of Army Medical University, Chongqing, 400000, China +1
Federated Cross-Modal Retrieval with Missing Modalities via Semantic Routing and Adapter Personalization
Apr 24, 2026Hefeng Zhou, Xuan Liu, Sicheng Chen +7Cross-Modal Learning Multimodal Federated Learning
Shanghai Jiao Tong University · Hohai University · Shenzhen Univ of Advanced Technology
Handling and Interpreting Missing Modalities in Patient Clinical Trajectories via Autoregressive Sequence Modeling
Apr 20, 2026Andrew Wang, Ellie Pavlick, Ritambhara SinghMissing-Modality Learning Multimodal Contrastive Learning
Brown University
Conditional Evidence Reconstruction and Decomposition for Interpretable Multimodal Diagnosis
Apr 18, 2026Shaowen Wan, Yanjun Lv, Lu Zhang +5Missing-Modality Learning Medical Diagnosis
Department of Biomedical Engineering, New Jersey Institute of Technology, Newark, NJ 07102, USA · Department of Computer Science and Engineering, The University of Texas at Arlington, Arlington, TX 76019, USA · Department of Computer Science, Indiana University Indianapolis, Indianapolis, IN 46202, USA +1
Robust Multispectral Semantic Segmentation under Missing or Full Modalities via Structured Latent Projection
Apr 17, 2026Irem Ulku, Erdem Akagündüz, Ömer Özgür TanrıöverMissing-Modality Learning Remote Sensing Image Segmentation
Department of Computer Engineering, Ankara University, Ankara, Türkiye · Department of Modeling and Simulation, Graduate School of Informatics, METU, Ankara, Türkiye
Probabilistic Feature Imputation and Uncertainty-Aware Multimodal Federated Aggregation
Apr 14, 2026Nafis Fuad Shahid, Maroof Ahmed, Md Akib Haider +3Multimodal Federated Learning Missing-Modality Learning
Department of Computer Science and Engineering, Islamic University and Technology (IUT), Gazipur, Bangladesh
Backbone-Conditional Behavior of Modality Gating in Multi-Modal Prostate MRI Segmentation: A 5-Fold Cross-Validation and Gate Mechanism Analysis
Apr 12, 2026Yongbo Shu, Wenzhao Xie, Shanhu Yao +4Missing-Modality Learning Cross-Modality Medical Image Segmentation
The Second Xiangya Hospital of Central South University, Changsha, 410011, Hunan, China · School of Life Sciences, Central South University, Changsha, 410013, Hunan, China · Hunan Provincial Key Laboratory of Medical Information Research (Central South University), Changsha, 410011, Hunan, China +2
Retrieving to Recover: Towards Incomplete Audio-Visual Question Answering via Semantic-consistent Purification
Apr 12, 2026Jiayu Zhang, Shuo Ye, Qilang Ye +3Audio-Visual QA Missing-Modality Learning
School of Computing and Information Technology, Great Bay University · Tsinghua Shenzhen International Graduate School, Tsinghua University · College of Computer Science, Nankai University +2
Improving Pediatric Emergency Department Triage with Modality Dropout in Late Fusion Multimodal EHR Models
Apr 10, 2026Tyler Yang, Romal MitrClinical Triage Healthcare
Department of Computer Science, Stanford University · Department of Bioengineering, Stanford University
Decoupling and Regulating Modality Heads for Fake News Detection under Missing Modality
Apr 8, 2026Kai Qian, Dongjie Xu, Weijie Shi +7Fake News Detection Multimodal Robustness
Soochow University · Hong Kong University of Science and Technology · Computer Science and Engineering, The Chinese University of Hong Kong +2
Hidden in the Multiplicative Interaction: Uncovering Fragility in Multimodal Contrastive Learning
Apr 7, 2026Tillmann Rheude, Stefan Hegselmann, Roland Eils +1Multimodal Contrastive Learning Cross-Modal Retrieval
Berlin Institute of Health, Charité - Universitätsmedizin Berlin · Department of Mathematics and Computer Science, Freie Universität Berlin · Intelligent Medicine Institute, Fudan University
QA-MoE: Towards a Continuous Reliability Spectrum with Quality-Aware Mixture of Experts for Robust Multimodal Sentiment Analysis
Apr 7, 2026Yitong Zhu, Yuxuan Jiang, Guanxuan Jiang +4Multimodal Sentiment Analysis Multimodal Robustness
The Hong Kong University of Science and Technology (Guangzhou) · Tsinghua University · Aarhus University
Purify-then-Align: Towards Robust Human Sensing under Modality Missing with Knowledge Distillation from Noisy Multimodal Teacher
Apr 7, 2026Pengcheng Weng, Yanyu Qian, Yangxin Xu +1Missing-Modality Learning Multimodal Robustness
School of Software Engineering, Xi’an Jiaotong University, China · Institute of Computer Science, Universität Bern, Switzerland · College of Computing and Data Science, Nanyang Technological University, Singapore
Evaluation Before Generation: A Paradigm for Robust Multimodal Sentiment Analysis with Missing Modalities
Apr 7, 2026Rongfei Chen, Tingting Zhang, Xiaoyu Shen +1Multimodal Sentiment Analysis Missing-Modality Learning
School of Computer Science and Technology, Eastern Institute of Technology, Ningbo, China · School of Mechatronic Engineering and Automation, Shanghai University, China
A Clinical Point Cloud Paradigm for In-Hospital Mortality Prediction from Multi-Level Incomplete Multimodal EHRs
Apr 6, 2026Bohao Li, Tao Zou, Junchen Ye +2Electronic Health Records Clinical Risk Prediction
Beihang University, Beijing, China · The Hong Kong Polytechnic University, Hong Kong, China
PRIME: Prototype-Driven Multimodal Pretraining for Cancer Prognosis with Missing Modalities
Apr 5, 2026Kai Yu, Shuang Zhou, Yiran Song +9Multimodal Pretraining Missing-Modality Learning
Division of Computational Health Sciences, Department of Surgery, University of Minnesota, Minneapolis, MN, USA. · University of North Carolina at Chapel Hill, Chapel Hill, NC, USA. · North Carolina State University, Raleigh, NC, USA. +2
Bridging the Missing-Modality Gap: Improving Text-Only Calibration of Vision Language Models
Apr 3, 2026Mingyeong Kim, Jungwon Choi, Chaeyun Jang +1Modality Gap in VLMs VLM Inference
Kim Jaechul Graduate School of AI, KAIST
COMPASS: Fusion-Matched Supervision for Missing-Modality Human Sensing
Apr 2, 2026Hao Wang, Yanyu Qian, Pengcheng Weng +4Missing-Modality Learning Human Activity Recognition
Universität Bern, Switzerland · Xi’an Jiaotong University, China · Nanyang Technological University, Singapore
Multicentric thrombus segmentation using an attention-based recurrent network with gradual modality dropout
Apr 1, 2026Sofia Vargas-Ibarra, Vincent Vigneron, Hichem Maaref +1Medical Image Segmentation 3D Medical Image Segmentation
Universite Evry Paris-Saclay, IBISC, 34 Rue du Pelvoux, Evry-Courcouronnes, France · SAMOVAR, Télécom SudParis, Institut Polytechnique de Paris, 34 Rue du Pelvoux, Palaiseau, France
Brain MR Image Synthesis with 3D Multi-Contrast Self-Attention GAN
Mar 31, 2026Zaid A. Abod, Furqan AzizMedical Image Synthesis 3D Medical Imaging
School of Computing and Mathematical Sciences, University of Leicester, Leicester, United Kingdom.
BLOSSOM: Block-wise Federated Learning Over Shared and Sparse Observed Modalities
Mar 29, 2026Pranav M R, Jayant Chandwani, Ahmed M. Abdelmoniem +1Multimodal Federated Learning Federated Learning
DaSH Lab, BITS Pilani, KK Birla Goa Campus, India · Queen Mary University of London, United Kingdom
MUST: Modality-Specific Representation-Aware Transformer for Diffusion-Enhanced Survival Prediction with Missing Modality
Mar 27, 2026Kyungwon Kim, Dosik HwangMultimodal Survival Analysis Missing-Modality Learning
Yonsei University · Korea Institute of Science and Technology
ARMOR: A Robust Self-Supervised Framework for Root Cause Analysis in Microservices under Missing Modality
Mar 26, 2026Wenzhuo Qian, Hailiang Zhao, Ziqi Wang +4Root Cause Analysis Multimodal Anomaly Detection
Zhejiang University, Hangzhou, China
POLY-SIM: Polyglot Speaker Identification with Missing Modality Grand Challenge 2026 Evaluation Plan
Mar 25, 2026Marta Moscati, Muhammad Saad Saeed, Marina Zanoni +8Missing-Modality Learning Speech Processing
Institute of Computational Perception, Johannes Kepler University Linz, Austria · University of Michigan-Flint, USA · Sapienza University of Rome, Italy +5
Multimodal Training to Unimodal Deployment: Leveraging Unstructured Data During Training to Optimize Structured Data Only Deployment
Mar 23, 2026Zigui Wang, Minghui Sun, Jiang Shu +3Multimodal Learning Multimodal Classification
Department of Biostatistics and Bioinformatics, Duke University School of Medicine, Durham, NC, USA · Duke Center for Autism and Brain Development, Duke University School of Medicine, Durham, NC, USA · Department of Psychiatry and Behavioral Sciences, Durham, NC, USA +1
BALM: A Model-Agnostic Framework for Balanced Multimodal Learning under Imbalanced Missing Rates
Mar 20, 2026Phuong-Anh Nguyen, Tien Anh Pham, Duc-Trong Le +1Modality Imbalance Multimodal Learning
VNU University of Engineering and Technology, Hanoi, Vietnam
EgoAdapt: Enhancing Robustness in Egocentric Interactive Speaker Detection Under Missing Modalities
Mar 18, 2026Xinyuan Qian, Xinjia Zhu, Alessio Brutti +1Multimodal Robustness Egocentric Video Understanding
School of Computer and Communication Enngineering, University of Science and Technology Beijing, China · MIIT Key Laboratory of Pattern Analysis and Machine Intelligence, College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics, China · Fondazione Bruno Kessler, Italy +1
Adaptive Clinical-Aware Latent Diffusion for Multimodal Brain Image Generation and Missing Modality Imputation
Mar 10, 2026Rong Zhou, Houliang Zhou, Yao Su +4Medical Image Synthesis Missing-Modality Learning
Department of Computer Science and Engineering, Lehigh University, PA, USA · Worcester Polytechnic Institute, MA, USA · Department of Psychiatry and Behavioral Sciences, Stanford University School of Medicine, CA, USA +3
MissBench: Benchmarking Multimodal Affective Analysis under Imbalanced Missing Modalities
Mar 10, 2026Tien Anh Pham, Phuong-Anh Nguyen, Duc-Trong Le +1Multimodal Model Evaluation Missing-Modality Learning
VNU University of Engineering and Technology, Hanoi, Vietnam

Apr 28, 2026J/K move · Enter open · S save

GraphPL: Leveraging GNN for Efficient and Robust Modalities Imputation in Patchwork Learning

Xingjian Hu, Zuoyu Yan, Jianhua Zhu, Liangcai Gao+2

Wangxuan Institute of Computer Technology, Peking University, Beijing · Weill Cornell Medicine, Cornell University, New York · Biomedical Informatics, Stony Brook University, New York

Missing-Modality Learning Multimodal Learning

Open paper AskPDF arXiv

Current research on distributed multi-modal learning typically assumes that clients can access complete information across all modalities, which may not hold in practice. In this paper, we explore patchwork learning, in which the modalities available to different clients vary, and the objective is to impute the missing modalities for each client in an unsupervised manner. Existing methods are shown not to fully utilize the modality information as they tend to rely on only a subset of the observed modalities. To address this issue, we propose GraphPL, which combines graph neural networks with patchwork learning to flexibly integrate all observed modalities and remains robust with noisy inputs. Experimental results show that GraphPL achieves SOTA performance on benchmark datasets. Our results on real-world distributed electronic health record dataset show GraphPL learns strong downstream features and enables tasks like disease prediction via superior modality imputation.