Heterogeneous Federated Learning

Follow this topic Ask about this topic

Heterogeneous Federated Learning asks how clients can learn together without sharing raw data when their data distributions, tasks, modalities, models, or hardware differ. Common approaches include personalized or clustered models, drift-correcting optimization and adaptive aggregation, and knowledge sharing through prototypes, distillation, or aligned subspaces. Systems methods also adapt client selection, model size, communication, and training schedules to resource limits, with evaluations spanning non-IID image and language benchmarks, multimodal and medical data, and edge or wireless deployments. The central tension is how to gain from shared learning without negative transfer or poor outcomes for individual clients, while keeping communication, computation, privacy, and fairness costs manageable.

Latest papers

297 papers35 papers in the last four weeks, up 30% on the four weeks before. 0.2% of all new papers.

Communication-Efficient Personalized Adaptation via Federated-Local Model Merging
Feb 20, 2026Yinan Zou, Md Kamran Chowdhury Shisher, Christopher G. Brinton +1Federated Learning Federated Fine-Tuning
Elmore Family School of Electrical and Computer Engineering, Purdue University.
ZorBA: Zeroth-order Federated Fine-tuning of LLMs with Heterogeneous Block Activation
Feb 19, 2026Chuiyang Meng, Ming Tang, Vincent W. S. WongFederated Fine-Tuning Memory-Efficient Fine-Tuning
Department of Electrical and Computer Engineering, The University of British Columbia, Vancouver, Canada · Department of Computer Science and Engineering, Southern University of Science and Technology, China
Federated Graph AGI for Cross-Border Insider Threat Intelligence in Government Financial Schemes
Feb 18, 2026Srikumar Nayak, James WalmesleyCyber Threat Intelligence Federated Graph Learning
Incedo Inc., New Jersey, USA · Indian Institute of Technology, Chennai, Tamil Nadu, India · University of Kent, Kent, UK
On the Geometric Coherence of Global Aggregation in Federated Graph Neural Networks
Feb 17, 2026Chethana Prasad Kabgere, Shylaja SSFederated Graph Learning Federated Learning Aggregation
Department of Computer Science and Engineering, PES University, Bengaluru, Karnataka, 560085, India
Evaluating Federated Learning for Cross-Country Mood Inference from Smartphone Sensing Data
Feb 17, 2026Sharmad Kalpande, Saurabh Shirke, Haroon R. LoneFederated Learning Emotion Recognition
Indian Institute of Science Education and Research Bhopal, India
pFedNavi: Structure-Aware Personalized Federated Vision-Language Navigation for Embodied AI
Feb 16, 2026Qingqian Yang, Hao Wang, Sai Qian Zhang +6Federated Learning Aggregation Non-IID Federated Learning
Stevens Institute of Technology · Hoboken, NJ, USA · New York University +9
Federated Learning of Nonlinear Temporal Dynamics with Graph Attention-based Cross-Client Interpretability
Feb 13, 2026Ayse Tursucular, Ayush Mohanty, Nazal Mohamed +1Federated Learning Nonlinear System Identification
Georgia Institute of Technology Atlanta, GA, USA
An Adaptive Differentially Private Federated Learning Framework
Feb 6, 2026Jin Wang, Hui Ma, Yajun Zhang +4Non-IID Federated Learning Differentially Private Federated Learning
Xinjiang Key Laboratory of Intelligent Computing and Smart Applications, School of Software, Xinjiang University, Urumqi 830017, China · College of Geography and Remote Sensing Sciences, Xinjiang University, Urumqi, China · Jilin University, Changchun 130012, China
ELSA: Efficient LLM-Centric Split Aggregation for Privacy-Aware Hierarchical Federated Learning over the Network Edge
Jan 20, 2026Xiaohong Yang, Tong Xie, Minghui Liwang +6Federated Fine-Tuning Federated Learning
the School of Informatics, Xiamen University, Fujian, China · the National University of Singapore · the Department of Control Science and Engineering, the National Key Laboratory of Autonomous Intelligent Unmanned Systems, and also with Frontiers Science Center for Intelligent Autonomous Systems, Ministry of Education, Tongji University, Shanghai, China +2
A Privacy-Preserving Federated Learning Method for Heterogeneous Clients
Nov 8, 2025Yusaku Negoya, Feifei Cui, Zilong Zhang +3Privacy-Preserving Federated Learning Differentially Private Federated Learning
Department of Computer and Network Engineering, The University of Electro-Communications, Tokyo, Japan · School of Computer Science and Technology, Hainan University, Haikou 570228, China. · Department of Electrical and Computer Engineering, University of Houston, Houston, TX, 77204, USA +1
FedHUG: Federated Heterogeneous Unsupervised Generalization for Remote Physiological Measurements
Oct 14, 2025Xiao Yang, Dengbo He, Duoxun Tang +2Federated Domain Generalization Federated Learning
Systems Hub, the Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China
An Efficient Subspace Algorithm for Federated Learning on Heterogeneous Data
Sep 5, 2025Jiaojiao Zhang, Yizhao Fan, Yuqi Xu +1Communication-Efficient Distributed Training Non-IID Federated Learning
Intelligent Computing Research Center, Great Bay University, Dongguan, China · Center for Machine Learning Research, Peking University, Beijing, China
FediLoRA: Practical Federated Fine-Tuning of Foundation Models Under Missing-Modality Constraints
Sep 1, 2025Lishan Yang, Wei Emma Zhang, Nam Kha Nguygen +4Federated Fine-Tuning Federated Learning Aggregation
Adelaide University · Central China Normal University · Harbin Institute of Technology
FAIRVAR: Fair Federated Learning via Variance Regularization
Aug 16, 2025Zahra Kharaghani, Ali Dadras, Tommy LöfstedtAlgorithmic Fairness Federated Learning
Department of Computing Science, Umeå University, Umeå, Sweden · Department of Mathematics and Mathematical Statistics, Umeå University, Umeå, Sweden
FedUHD: Unsupervised Federated Learning using In-Memory Hyperdimensional Computing
Aug 16, 2025You Hak Lee, Keming Fan, Xiaofan Yu +4Compute-in-Memory Federated Learning Aggregation
University of California San Diego, USA
Hyper-modal Imputation Diffusion Embedding with Dual-Distillation for Federated Multimodal Knowledge Graph Completion
Jun 27, 2025Ying Zhang, Yu Zhao, Xuhui Sui +5Multimodal Federated Learning Missing-Modality Learning
College of Computer Science, VCIP, DISSec, Nankai University, Tianjin, China · School of Software, Tiangong University, Tianjin, China · Shenzhen Campus of Sun Yat-sen University, Shenzhen, China +1
Inclusive Federated Learning Through Compliance-Weighted Noise Allocation in Healthcare AI
May 28, 2025Santhosh Parampottupadam, Melih Coşğun, Sarthak Pati +7Federated Learning Aggregation Differentially Private Stochastic Gradient Descent
German Cancer Research Center (DKFZ), Division of Medical Image Computing, Heidelberg, Germany · Department of Computer Engineering, Bilkent University, Universiteler 06800 Çankaya/Ankara, Türkiye · Medical Research Group, MLCommons, San Francisco, CA, USA +2
Federated Independent Component Analysis via Spectral Alignment and Robust Aggregation
May 26, 2025Xin Bing, Dian Jin, Yuqian ZhangFederated Learning Aggregation Independent Component Analysis
University of Toronto · National University of Singapore · Rutgers University
FedHQ: Hybrid Runtime Quantization for Federated Learning
May 17, 2025Zihao Zheng, Ziyao Wang, Jiawei Yang +7Federated Learning Adaptive Quantization
Peking University · University of Maryland
Heterogeneity-Aware Client Sampling for Optimal and Efficient Federated Learning
May 16, 2025Shudi Weng, Chao Ren, Ming Xiao +1Federated Learning Heterogeneous Federated Learning
Division of Information Science and Engineering, School of Electrical Engineering and Computer Science (EECS), KTH Royal Institute of Technology, 11428 Stockholm, Sweden
Unified Robust and Convergent Over-The-Air Federated Learning under Channel Noise and Heterogeneity
Mar 25, 2025Zubair Shaban, Nazreen Shah, Ranjitha PrasadFederated Learning Over-the-Air Federated Learning
ECE dept., IIIT Delhi, New Delhi.
Correcting Before Adapting: Gradient Correction for Federated Adaptive Optimization
Feb 4, 2025Evan Chen, Shiqiang Wang, Dong-Jun Han +2Federated Learning Heterogeneous Federated Learning
Elmore Family School of Electrical and Computer Engineering, Purdue University, West Lafayette, IN · IBM T. J. Watson Research Center, Yorktown Heights, NY, USA · Department of Computer Science and Engineering, Yonsei University, Seoul, Korea
MLLM-LLaVA-FL: Multimodal Large Language Model Assisted Federated Learning
Sep 9, 2024Jianyi Zhang, Hao Frank Yang, Ang Li +5Multimodal Federated Learning Non-IID Federated Learning
Duke University · Johns Hopkins University · University of Maryland College Park +1
Tackling Feature-Classifier Mismatch in Federated Learning via Prompt-Driven Feature Transformation
Jul 23, 2024Xinghao Wu, Xuefeng Liu, Jianwei Niu +4Prompt Learning Federated Learning
School of Computer Science and Engineering, Beihang University, Beijing, China · Zhongguancun Laboratory, Beijing, China · Sichuan University, Sichuan, China +1
Personalized Additive Modeling for Multi-level Federated Learning
May 26, 2024Shutong Chen, Guodong Long, Tianyi Zhou +3Non-IID Federated Learning Federated Learning
University of Technology Sydney, Australia · Mohamed bin Zayed University of Artificial Intelligence, UAE · The Hong Kong Polytechnic University, China
The Limits and Potentials of Local SGD for Distributed Heterogeneous Learning with Intermittent Communication
May 19, 2024Kumar Kshitij Patel, Margalit Glasgow, Ali Zindari +5Communication-Efficient Distributed Training Distributed Optimization
Toyota Technological Institute, Chicago · Stanford University · CISPA Helmholtz Center +1
Convergence Analysis of Sequential Federated Learning on Heterogeneous Data
Nov 6, 2023Yipeng Li, Xinchen LyuNon-IID Federated Learning Stochastic Optimization Convergence
National Engineering Research Center for Mobile Network Technologies Beijing University of Posts and Telecommunications Beijing, 100876, China

Feb 20, 2026J/K move · Enter open · S save

Communication-Efficient Personalized Adaptation via Federated-Local Model Merging

Yinan Zou, Md Kamran Chowdhury Shisher, Christopher G. Brinton, Vishrant Tripathi

Elmore Family School of Electrical and Computer Engineering, Purdue University.

Federated Learning Federated Fine-Tuning Heterogeneous Federated Learning Communication-Efficient Distributed Training

Open paper AskPDF arXiv

Parameter-efficient fine-tuning methods, such as LoRA, offer a practical way to adapt large vision and language models to client tasks. However, this becomes particularly challenging under task-level heterogeneity in federated deployments. In this regime, personalization requires balancing general knowledge with personalized knowledge, yet existing approaches largely rely on heuristic mixing rules and lack theoretical justification. Moreover, prior model merging approaches are also computation and communication intensive, making the process inefficient in federated settings. In this work, we propose Potara, a principled framework for federated personalization that constructs a personalized model for each client by merging two complementary models: (i) a federated model capturing general knowledge, and (ii) a local model capturing personalized knowledge. Through the construct of linear mode connectivity, we show that the expected task loss admits a variance trace upper bound, whose minimization yields closed-form optimal mixing weights that guarantee a tighter bound for the merged model than for either the federated or local model alone. Experiments on vision and language benchmarks show that Potara consistently improves personalization while reducing communication, leading to a strong performance-communication trade-off.