Multimodal Federated Learning

Ask about this topic

Overview

Multimodal federated learning studies how institutions or devices can jointly learn from combinations such as images and text, or audio and sensor streams, without pooling their raw data. Common approaches combine local multimodal fusion with federated aggregation, and use representation or prototype alignment, personalized adapters, distillation, and imputation to handle clients with different data distributions or missing modalities. Work spans medical imaging, sentiment analysis, graph learning, activity recognition, and vision-language models, with methods also targeting communication costs and privacy. A central challenge is balancing useful knowledge sharing with client-specific needs when modalities, data quality, and model architectures differ across participants.

Latest papers

CardsList

51 papers3 papers in the last four weeks, down 40% on the four weeks before. 0.0% of all new papers.

  1. Fed-MAPTrack: Federated Modality-aware Prompt Tuning for RGB-X Tracking

    Oct 26, 2026Xiangyu Liu, Yijie Shen, Yue Yin +3Multimodal Visual Object TrackingMultimodal Federated Learning

    Beijing Normal-Hong Kong Baptist University, 519087, Zhuhai, Guangdong, China · Zhejiang University, 310058, Hangzhou, Zhejiang, China · Peking University, 100871, Beijing, China +1

  2. FederatedBytes: Tokenizer-Free, Byte-Level Small Language Models for Efficient Multimodal Federated Learning

    Oct 16, 2026Ali Sarabadani, Massimo Piccardi, Mehran Abolhasan +3Byte-Level Language ModelMultimodal Federated Learning

    Cyber Digital Centre, University of Technology Sydney

  3. FedSSMCoOp: SSM Encoders for light-weight Federated Prompt Learning for Few-shot Classification

    Oct 7, 2026Ankita Das, Ambarish Parthasarathy, Sumohana S. Channappayya +1Few-Shot Image ClassificationMultimodal Federated Learning

    Department of Computer Science and Engineering Indian Institute of Technology Hyderabad · Department of Artificial Intelligence Indian Institute of Technology Hyderabad · Department of Electrical Engineering Indian Institute of Technology Hyderabad

  4. Prototype-guided Bilateral Alignment Multimodal Federated Learning

    Sep 30, 2026Tianchi Liao Tianchi_Liao, Lele Fu, Sheng Huang +3Multimodal Federated LearningFederated Learning Aggregation

    Department of Computer Science, Hong Kong Baptist University, Hong Kong SAR, China · School of Computer Science and Engineering, Sun Yat-Sen University, Guangzhou, China

  5. FedMHAR: Federated Multimodal Human Activity Recognition using Multi-Agent Reinforcement Learning

    Sep 27, 2026Debasmita Dey, Tanmay Sen, Himel MallickMultimodal Federated LearningHuman Activity Recognition

    SQC & OR Unit Indian Statistical Institute Kolkata Kolkata, India · Department of Population Health Sciences Cornell University 425 E 61st Street New York

  6. Coupling Perception and Reasoning in Federated Multimodal Graph Foundation Models

    Sep 24, 2026Zekai Chen, Xun Wu, Hailin Zhang +7Multimodal Federated LearningGraph Foundation Models

    Beijing Institute of Technology · Shandong University · Institute of Information Engineering +1

  7. QoS-Aware Federated Learning for Multimodal In-Cabin Interaction in Smart Vehicles

    Sep 17, 2026Baran Can Gül, Mert Nakıp, Nasser Jazdi +1Multimodal Federated LearningAsynchronous Federated Learning

    Institute of Industrial Automation and Software Engineering, University of Stuttgart, Pfaffenwaldring 47, 70550 Stuttgart, Germany · Institute of Theoretical and Applied Informatics, Polish Academy of Sciences (IITIS PAN), 44-100 Gliwice, Poland · Ideatrum Ltd. Sti., 35560, İzmir, Türkiye

  8. OmniMed-FL: A Robust Multimodal Federated Learning Framework for Clinical Diagnosis

    Sep 9, 2026Ayush Debnath, Ruelia Saha, Sudip MisraMultimodal Federated LearningMedical Diagnosis

    Indian Institute of Technology Kharagpur, India · KTH Royal Institute of Technology, Sweden · SRM University-AP, India +1

  9. Modality-Decoupled Federated Learning for Privacy-Preserving Embodied Intelligence in 6G

    Sep 9, 2026Zhuodong Liu, Xiangyu Li, Chunhong Yuan +5Multimodal Federated LearningWireless Communications

    Artificial Intelligence Thrust, The Hong Kong University of Science and Technology (Guangzhou), Guangzhou 511453, China · Department of Electronic Engineering, Shanghai Jiao Tong University, Shanghai 200240, China · School of Electronic Science and Technology, Eastern Institute of Technology, Ningbo, Zhejiang 315200, China +4

  10. MOSAIC: Modality-agnostic Spectral Alignment for Federated Image-level Weakly Supervised Tumor Segmentation under Client-specific Missing Modalities

    Aug 20, 2026Tarun Kumar Garg, Vaanathi SundaresanWeakly Supervised Image SegmentationMissing-Modality Medical Image Segmentation

    Department of IISc Mathematics Initiative, Indian Institute of Science, Bengaluru 560012, Karnataka, India · Department of Computational and Data Sciences, Indian Institute of Science, Bengaluru 560012, Karnataka, India

  11. FedCoRe: Target-Adaptive Completion for Missing Modalities in Healthcare Federated Learning

    Aug 18, 2026Holger R. Roth, Ziyue Xu, Peter CnuddeMissing-Modality LearningMultimodal Federated Learning

    NVIDIA, Santa Clara, CA, USA

  12. UniFed-VLM: Federated Instruction Tuning for Vision-Language Models with Multiple Heterogeneity

    Aug 16, 2026Pengyu Wang, Baochen Xiong, Xiaoshan Yang +4Multimodal Federated LearningHeterogeneous Federated Learning

    State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, China · School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China · Pengcheng Laboratory, Shenzhen, China +2

  13. MedMix: Specialization-Consistent Federated Sparse MoEs under Modality Heterogeneity

    Aug 14, 2026Adiba Orzikulova, Dong Min Kim, Jaehong Yoon +1Multimodal Federated LearningFederated Learning

    KAIST · NTU Singapore

  14. Multimodal Federated Learning under Dual-Axis Modality Missingness

    Aug 10, 2026Adiba Orzikulova, Jaehyun Kwak, Jaemin Shin +5Multimodal Federated LearningMissing-Modality Learning

    1KAIST · 2CUHK · 3HKUST

  15. Towards Effective Federated Multimodal Graph Learning via Navigating Multifaceted Heterogeneity

    Aug 1, 2026Yinlin Zhu, Di Wu, Yi Zhang +5Multimodal Federated LearningFederated Graph Learning

    Sun Yat-sen University, Guangzhou, China1 · Shandong University, Weihai, China2 · Beijing Institute of Technology, Beijing, China3

  16. MMFGU: Multimodal Federated Graph Unlearning

    Jul 30, 2026Haodong Lu, Zekai Chen, Weiwei Ji +5Multimodal Federated LearningFederated Unlearning

    Beijing Institute of Technology Beijing, China

  17. FedOGL: Combating Catastrophic Forgetting in Federated Open-World Multimodal Graph Learning

    Jul 30, 2026Zekai Chen, Haodong Lu, Shihao Li +5Multimodal Federated LearningFederated Continual Learning

    Beijing Institute of Technology, Beijing, China

  18. FedTaste: Topology-Aware Structural Transfer for Multimodal Federated Learning with Missing Modalities

    Jul 25, 2026Haochen Liang, Jie Zhang, Hideya OchiaiMultimodal Federated LearningMissing-Modality Learning

    The University of Tokyo Tokyo, Japan · Great Bay University Dongguan, China

  19. Trustworthy Privacy-Preserving Multimodal Federated Learning for Personalised Breast Cancer Prediction

    Jul 21, 2026Ruth Amey, Muhammad Arifur Rahman, Taha Osman +4Breast CancerMultimodal Federated Learning

  20. Toward Federated Multimodal Graph Foundation Models: A Topology-Aware Multimodal Alignment Framework

    Jul 17, 2026Xunkai Li, Guohao Fu, Yuming Ai +4Multimodal Federated LearningCross-Modal Alignment

    Beijing Institute of Technology, Beijing, 100811, China

  21. FM2^2: Unified Federated Foundation Models for Heterogeneous Multimodal Medical Imaging

    Jul 15, 2026Shengchao Chen, Ting ShuMultimodal Federated LearningCross-Modal Alignment

    School of Artificial Intelligence, Shenzhen University Shenzhen, China · Australian AI Institute, University of Technology Sydney Sydney, Australia

  22. Continual Learning with Elastic Regularization and Synthetic Replay for Federated MLLM Fine-Tuning

    Jul 13, 2026Jing Liu, Chenxuanyin Zou, Jiayang Ren +5Multimodal Federated LearningGenerative Replay

    Department of Electrical and Computer Engineering, The University of British Columbia, Vancouver, BC V6T 1Z4, Canada · College of Future Information Technology, Fudan University, Shanghai 200433, China · Department of Chemical and Biological Engineering, The University of British Columbia, Vancouver, BC V6T 1Z4, Canada +3

  23. PFAdapter: Hierarchical LoRA Decomposition for Personalized Federated MLLMs

    Jul 13, 2026Jing Liu, Kun Yang, Yan Wang +5Multimodal Federated LearningFederated Learning Aggregation

    College of Future Information Technology, Fudan University, Shanghai 200433, China · Division of Natural and Applied Sciences, Duke Kunshan University, Suzhou 215316, China · Department of Electrical and Computer Engineering, The University of British Columbia, BC V6T 1Z4, Canada +8

  24. Entropy-Guided Tensor Compression for Multimodal Federated Learning on Edge Devices

    Jul 7, 2026Quoc Bao Phan, Tuy Tan NguyenTensor NetworksMultimodal Federated Learning

    Department of Electrical and Computer Engineering, Center for Advanced Power Systems, FAMU-FSU College of Engineering, Florida State University, Tallahassee, FL 32310, USA

  25. ProMoE-FL: Prototype-conditioned Mixture of Experts for Multimodal Federated Learning with Missing Modalities

    Jul 7, 2026Aavash Chhetri, Bibek Niroula, Eduard Vazquez +4Multimodal Federated LearningMissing-Modality Learning

    NepAl Applied Mathematics and Informatics Institute for research, Nepal · Fogsphere (Redev.AI Ltd), UK · University of Lausanne, Switzerland +4

  26. QFedAgent: Quantum-Enhanced Personalized Federated Learning for Multi-Agent Activity Recognition

    Jul 2, 2026Quoc Bao Phan, Tuy Tan NguyenNon-IID Federated LearningHuman Activity Recognition

    Department of Electrical and Computer Engineering FAMU-FSU College of Engineering, Florida State University Tallahassee, FL 32310, USA

  27. FedLAB: Traceable Semantic Codebooks for Federated Multimodal Graph Foundation Learning

    Jun 30, 2026Zekai Chen, Kairui Yang, Xuaner Chen +4Multimodal Federated LearningRepresentation Learning

    Beijing Institute of Technology, Beijing, China

  28. Towards Modality-imbalanced Federated Graph Learning: A Data Synthesis-based Approach

    Jun 18, 2026Zhengyu Wu, Hongchao Qin, Xunkai Li +3Multimodal Federated LearningMissing-Modality Learning

  29. PRISM: Topology-Aware Cross-Modal Imputation for Modality-Deficient Federated Graph Learning

    Jun 8, 2026Zekai Chen, Miao Zhang, Jiayang Xing +4Multimodal Federated LearningMissing-Modality Learning

    Beijing Institute of Technology, Beijing, 100081, China

  30. Federated Foundation Models over Vehicular Networks

    Jun 5, 2026Kasra Borazjani, Fardis Nadimi, Payam Abdisarabshali +5Multimodal Federated LearningIntelligent Transportation Systems