Multimodal Federated Learning

Follow this topic Ask about this topic

Momentum

5 papers in the last four weeks, against 1 the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28

Latest papers 34

All topics

Federated Cross-Modal Retrieval with Missing Modalities via Semantic Routing and Adapter Personalization
Apr 24, 2026Hefeng Zhou, Xuan Liu, Sicheng Chen +7Cross-Modal Learning Multimodal Federated Learning
Shanghai Jiao Tong University · Hohai University · Shenzhen Univ of Advanced Technology
FDRMFL: Multimodal Federated Feature Extraction Model Based on Information Maximization and Contrastive Learning
Nov 30, 2025Haozhe WuMultimodal Federated Learning Non-IID Federated Learning
School of Computing and Data Science, The University of Hong Kong, Hong Kong SAR, China
Hyper-modal Imputation Diffusion Embedding with Dual-Distillation for Federated Multimodal Knowledge Graph Completion
Jun 27, 2025Ying Zhang, Yu Zhao, Xuhui Sui +5Multimodal Federated Learning Missing-Modality Learning
College of Computer Science, VCIP, DISSec, Nankai University, Tianjin, China · School of Software, Tiangong University, Tianjin, China · Shenzhen Campus of Sun Yat-sen University, Shenzhen, China +1
MLLM-LLaVA-FL: Multimodal Large Language Model Assisted Federated Learning
Sep 9, 2024Jianyi Zhang, Hao Frank Yang, Ang Li +5Multimodal Federated Learning Non-IID Federated Learning
Duke University · Johns Hopkins University · University of Maryland College Park +1

Apr 24, 2026, cs.CVJ/K move · Enter open · S save

Federated Cross-Modal Retrieval with Missing Modalities via Semantic Routing and Adapter Personalization

Hefeng Zhou, Xuan Liu, Sicheng Chen, Wutong Zhang+6

Shanghai Jiao Tong University · Hohai University · Shenzhen Univ of Advanced Technology

Cross-Modal Learning Multimodal Federated Learning Missing-Modality Learning Cross-Modal Alignment Cross-Modal Retrieval Federated Learning+1

Open paper AskPDF arXiv

Federated cross-modal retrieval faces severe challenges from heterogeneous client data, particularly non-IID semantic distributions and missing modalities. Under such heterogeneity, a single global model is often insufficient to capture both shared cross-modal knowledge and client-specific characteristics. We propose RCSR, a personalization-friendly federated framework that integrates prototype anchoring, retrieval-centric semantic routing, and optional client-specific adapters. Built on a frozen CLIP backbone, RCSR leverages lightweight shared adapters for global knowledge transfer while supporting efficient local personalization. Prototype anchoring helps unimodal clients align with global cross-modal semantics, and a server-side semantic router adaptively assigns aggregation weights based on retrieval consistency to mitigate alignment drift during heterogeneous updates. Extensive experiments on MS-COCO, Flickr30K, and other benchmarks show that RCSR consistently improves global retrieval accuracy and training stability, while further enhancing client-level retrieval performance, especially for clients with incomplete modalities. Code is available at https://github.com/RezinChow/RCSR-Retrieval-Centric-Semantic-Routing.

Momentum

Related topics

Latest papers 34

Federated Cross-Modal Retrieval with Missing Modalities via Semantic Routing and Adapter Personalization

FDRMFL: Multimodal Federated Feature Extraction Model Based on Information Maximization and Contrastive Learning

Hyper-modal Imputation Diffusion Embedding with Dual-Distillation for Federated Multimodal Knowledge Graph Completion

MLLM-LLaVA-FL: Multimodal Large Language Model Assisted Federated Learning