Federated Fine-Tuning

Ask about this topic

Overview

Federated fine-tuning studies how multiple data holders can adapt a shared pretrained model while keeping examples local, enabling collaboration in healthcare, finance, language, vision, and edge settings. Common approaches include parameter-efficient methods such as LoRA, QLoRA, and prompt tuning; client-server aggregation or split learning; and distillation when model updates are costly or architectures differ. Since clients have different data, tasks, and hardware, methods use personalized adapters, clustering or task-aware routing, update alignment, and compression, often evaluated on non-IID language, vision, and multimodal benchmarks. Keeping raw data local does not prevent gradient leakage, membership inference, poisoning, or unequal outcomes for weaker clients; differential privacy and stronger security can reduce utility and add cost, making realistic measurements of communication, memory, energy, and latency important.

Latest papers

CardsList

115 papers17 papers in the last four weeks, up 31% on the four weeks before. 0.1% of all new papers.

  1. Optimizing Multilingual LLMs via Federated Learning: A Study of Client Language Composition

    Mar 25, 2026Aleix Sant, Jordi Luque, Carlos EscolanoFederated LearningFederated Fine-Tuning

    Scientific Research, Telefónica Innovación Digital · Universitat Politècnica de Catalunya

  2. KALAVAI: Predicting When Independent Specialist Fusion Works -- A Quantitative Model for Post-Hoc Cooperative LLM Training

    Mar 24, 2026Ramchand KumaresanLanguage Model MergingMixture-of-Experts Expert Specialization

    Murai Labs

  3. FedPDPO: Federated Personalized Direct Preference Optimization for Large Language Model Alignment

    Mar 20, 2026Kewen Zhu, Liping Yi, Zhiming Zhao +3Personalized Language Model AlignmentDirect Preference Optimization

    College of Intelligence and Computing, Tianjin University, Tianjin, China · College of Computing and Data Science, Nanyang Technological University, Singapore

  4. Scalable Cross-Facility Federated Learning for Scientific Foundation Models on Multiple Supercomputers

    Mar 20, 2026Yijiang Li, Zilinghan Li, Kyle Chard +4Federated LearningHeterogeneous Federated Learning

    Mathematics and Computer Science Division, Argonne National Laboratory · Data Science and Learning Division, Argonne National Laboratory · Department of Computer Science, University of Chicago

  5. Federated Few-Shot Learning on Neuromorphic Hardware: An Empirical Study Across Physical Edge Nodes

    Mar 13, 2026Steven Motta, Gioele NanniFederated LearningFederated Learning Aggregation

  6. FedBPrompt: Federated Domain Generalization Person Re-Identification via Body Distribution Aware Visual Prompts

    Mar 13, 2026Xin Xu, Weilong Li, Wei Liu +5Federated Domain GeneralizationPerson Re-Identification

    School of Computer Science and Technology, Wuhan University of Science and Technology, China · Nanyang Technological University, Singapore · NERC for Multimedia Software, School of Computer Science, Wuhan University, China +3

  7. Med-DualLoRA: Local Adaptation of Foundation Models for 3D Cardiac MRI

    Mar 11, 2026Joan Perramon-Llussà, Amelia Jiménez-Sánchez, Grzegorz Skorupko +4Federated Fine-TuningMedical Imaging Foundation Models

    Artificial Intelligence in Medicine Lab (BCN-AIM), Departament de Matemàtiques i Informàtica, Universitat de Barcelona, Spain · Institució Catalana de Recerca i Estudis Avançats (ICREA), Barcelona, Spain

  8. Stabilized Fine-Tuning with LoRA in Federated Learning: Mitigating the Side Effect of Client Size and Rank via the Scaling Factor

    Mar 9, 2026Jiayu Huang, Xiaohu Wu, Tiantian He +1Federated Fine-TuningLow-Rank Adaptation

    Beijing University of Posts and Telecommunications, Beijing, China · Agency for Science, Technology and Research, Singapore, Singapore

  9. FedMomentum: Preserving LoRA Training Momentum in Federated Fine-Tuning

    Mar 9, 2026Peishen Yan, Yang Hua, Hao Wang +4Federated Fine-TuningFederated Learning Aggregation

    Shanghai Jiao Tong University · Queen’s University Belfast · Stevens Institute of Technology +2

  10. FedEU: Evidential Uncertainty-Driven Federated Fine-Tuning of Vision Foundation Models for Remote Sensing Image Segmentation

    Mar 8, 2026Xiaokang Zhang, Xuran Xiong, Jianzhong Huang +1Federated Fine-TuningRemote Sensing Image Segmentation

    School of Artificial Intelligence, Wuhan University, Wuhan 430072, China · School of Information Science and Engineering, Wuhan University of Science and Technology, Wuhan 430081, China · School of Computer Science, Wuhan University, Wuhan 430072, China

  11. Resource-Adaptive Federated Text Generation with Differential Privacy

    Mar 7, 2026Jiayi Wang, John Gounley, Heidi HansonDifferentially Private Federated LearningSynthetic Data Generation for Language Models

    Oak Ridge National Laboratory Oak Ridge, TN, USA

  12. HiLoRA: Hierarchical Low-Rank Adaptation for Personalized Federated Learning

    Mar 3, 2026Zihao Peng, Nan Zou, Jiandian Zeng +4Federated LearningFederated Fine-Tuning

    Beijing Normal University · Zhengzhou University

  13. FedTreeLoRA: Reconciling Statistical and Functional Heterogeneity in Federated LoRA Fine-Tuning

    Feb 27, 2026Jieming Bian, Lei Wang, Letian Zhang +1Federated Fine-TuningFederated Learning

    University of Florida, Gainesville, FL 32611 · Middle Tennessee State University Murfreesboro, TN 37132

  14. FedRot-LoRA: Mitigating Rotational Misalignment in Federated LoRA

    Feb 27, 2026Haoran Zhang, Dongjun Kim, Seohyeon Cha +1Federated Fine-TuningFederated Learning

    Chandra Department of Electrical and Computer Engineering, The University of Texas at Austin, TX, USA.

  15. Wireless Federated Multi-Task LLM Fine-Tuning via Sparse-and-Orthogonal LoRA

    Feb 24, 2026Nuocheng Yang, Sihua Wang, Ouwen Huan +3Federated Fine-TuningFederated Learning

    Beijing Laboratory of Advanced Information Network, and the Beijing Key Laboratory of Network System Architecture and Convergence, Beijing University of Posts and Telecommunications, Beijing 100876, China · Department of Electrical and Computer Engineering and Institute for Data Science and Computing, University of Miami, Coral Gables, FL, 33146 USA · Information Systems Technology and Design Pillar, Singapore University of Technology and Design, 487372, Singapore

  16. DP-FedAdamW: An Efficient Optimizer for Differentially Private Federated Large Models

    Feb 23, 2026Jin Liu, Yinbin Miao, Ning Xi +1Differentially Private Federated LearningFederated Learning

    School of Cyber Engineering, Xidian University, Xi’an, China · College of Intelligence and Computing, Tianjin University, Tianjin, China

  17. Rethinking LoRA for Privacy-Preserving Federated Learning in Large Models

    Feb 23, 2026Jin Liu, Yinbin Miao, Ning Xi +1Federated Fine-TuningLow-Rank Adaptation

    School of Cyber Engineering, Xidian University · College of Intelligence and Computing, Tianjin University

  18. Personalized Longitudinal Medical Report Generation via Temporally-Aware Federated Adaptation

    Feb 23, 2026He Zhu, Ren Togo, Takahiro Ogawa +8Medical Report GenerationFederated Learning

    Hokkaido University

  19. Communication-Efficient Personalized Adaptation via Federated-Local Model Merging

    Feb 20, 2026Yinan Zou, Md Kamran Chowdhury Shisher, Christopher G. Brinton +1Federated LearningFederated Fine-Tuning

    Elmore Family School of Electrical and Computer Engineering, Purdue University.

  20. ZorBA: Zeroth-order Federated Fine-tuning of LLMs with Heterogeneous Block Activation

    Feb 19, 2026Chuiyang Meng, Ming Tang, Vincent W. S. WongFederated Fine-TuningMemory-Efficient Fine-Tuning

    Department of Electrical and Computer Engineering, The University of British Columbia, Vancouver, Canada · Department of Computer Science and Engineering, Southern University of Science and Technology, China

  21. FLoRG: Federated Fine-tuning with Low-rank Gram Matrices and Procrustes Alignment

    Feb 19, 2026Chuiyang Meng, Ming Tang, Vincent W. S. WongLLM Fine-TuningFederated Fine-Tuning

    The University of British Columbia · Southern University of Science and Technology

  22. ELSA: Efficient LLM-Centric Split Aggregation for Privacy-Aware Hierarchical Federated Learning over the Network Edge

    Jan 20, 2026Xiaohong Yang, Tong Xie, Minghui Liwang +6Federated Fine-TuningFederated Learning

    the School of Informatics, Xiamen University, Fujian, China · the National University of Singapore · the Department of Control Science and Engineering, the National Key Laboratory of Autonomous Intelligent Unmanned Systems, and also with Frontiers Science Center for Intelligent Autonomous Systems, Ministry of Education, Tongji University, Shanghai, China +2

  23. Leveraging Soft Prompts for Privacy Attacks in Federated Prompt Tuning

    Jan 10, 2026Quan Minh Nguyen, Min-Seon Kim, Hoang M. Ngo +3Membership Inference AttacksFederated Fine-Tuning

    CISE, University of Florida, FL, USA · North Carolina State University, NC, USA · Washington State University, WA, USA +1

  24. How (Mis)calibrated is your Federated CLIP and what to do about it?

    Dec 3, 2025Mainak Singha, Masih Aminbeidokhti, Paolo Casari +3Vision-Language ModelsVLM Adaptation

    University of Trento, Italy · ´Ecole de technologie sup´erieure, QC, Canada · AMIAD, Pˆole Recherche, Palaiseau, France +2

  25. FlexP-SFT: A Flexible Aggregation-Free Framework for On-Device Personalized Split Federated Fine-Tuning of LLMs

    Aug 14, 2025Jiaxiang Geng, Tianjun Yuan, Pengchao Han +3Communication-Efficient Distributed TrainingLLM Fine-Tuning

    Duke Kunshan University, Suzhou, China · Guangdong University of Technology, Guangzhou, China · Jade Bird Fire (JBF), China +1