Federated Fine-Tuning

Ask about this topic

Overview

Federated fine-tuning studies how multiple data holders can adapt a shared pretrained model while keeping examples local, enabling collaboration in healthcare, finance, language, vision, and edge settings. Common approaches include parameter-efficient methods such as LoRA, QLoRA, and prompt tuning; client-server aggregation or split learning; and distillation when model updates are costly or architectures differ. Since clients have different data, tasks, and hardware, methods use personalized adapters, clustering or task-aware routing, update alignment, and compression, often evaluated on non-IID language, vision, and multimodal benchmarks. Keeping raw data local does not prevent gradient leakage, membership inference, poisoning, or unequal outcomes for weaker clients; differential privacy and stronger security can reduce utility and add cost, making realistic measurements of communication, memory, energy, and latency important.

Latest papers

CardsList

115 papers17 papers in the last four weeks, up 31% on the four weeks before. 0.1% of all new papers.

  1. Shift-Dependent Asymmetry: Orthogonal Inverse Low-Rank Adaptation for Federated Medical Segmentation

    Jun 7, 2026Xingyue Zhao, Wenke Huang, Linghao Zhuang +7Vision Foundation Model AdaptationNon-IID Federated Learning

    Department of Neurosurgery, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College · The Key Laboratory of Cognition and Decision Intelligence for Complex Systems, Institute of Automation, Chinese Academy of Sciences · College of Computing and Data Science, Nanyang Technological University, Singapore +4

  2. Amortizing Federated Adaptation: Hypernetwork Driven LoRA for Personalized Foundation Models

    Jun 4, 2026Sunny Gupta, Shambhavi Shanker, Amit SethiFederated Learning AggregationNon-IID Federated Learning

    Indian Institute of Technology, Bombay

  3. GuidaPA: Privacy-Preserving Chatbot for Public Administration via Federated Learning

    May 31, 2026Daniel M. Jimenez-Gutierrez, Albenzio Cirillo, Raffaele Nicolussi +2Privacy-Preserving Language ModelsConversational Agents

    Sapienza University of Rome, Via Ariosto 25, 00185 Rome, Italy · Fondazione Ugo Bordoni, Viale del Policlinico 147, 00161 Rome, Italy

  4. FedSmoothLoRA: Toward Smoother and Faster Convergence in Federated Low-Rank Adaptation

    May 28, 2026Zehao Wang, Guanglei Yang, Yihan Zeng +4Low-Rank AdaptationFederated Fine-Tuning

    1Harbin Institute of Technology · 2Huawei Noah’s Ark Lab · University College Dublin

  5. Decoupled Training with Local Reinforcement Fine-Tuning in Federated Learning

    May 27, 2026Yuting Ma, Lechao Cheng, Xiaohua XuVLM AdaptationVision-Language Alignment

    School of Computer Science and Technology, University of Science and Technology of China, Hefei, Anhui, China · School of Computer Science and Information Engineering, Hefei University of Technology, Hefei, Anhui, China.

  6. Federated LoRA Fine-Tuning for LLMs via Collaborative Alignment

    May 20, 2026Shuaida He, Liwen Chen, Long FengLLM Fine-TuningLow-Rank Adaptation

    School of Computing & Data Science, The University of Hong Kong

  7. FedSDR: Federated Self-Distillation with Rectification

    May 18, 2026Ziheng Ren, Zhanming Shen, Hao Wang +2LLM Fine-TuningHallucination in Language Models

    Beijing University of Aeronautics and Astronautics · Zhejiang University · Stevens Institute of Technology +1

  8. TSFLora: Token-Compressed Split Fine-Tuning for Wireless Edge Networks

    May 17, 2026Xianke Qiang, Zheng Chang, Li Wang +1Communication-Efficient Distributed TrainingEdge Computing

    School of Computer Science and Engineering, UESTC, 611731 Chengdu, China · School of Computer Science, Beijing University of Posts and Telecommunications, Beijing, 100876, China. · Center for Intelligent Networking and Communications, UESTC, 611731 Chengdu, China.

  9. UB-SMoE: Universally Balanced Sparse Mixture-of-Experts for Resource-adaptive Federated Fine-tuning of Foundation Models

    May 15, 2026Van-Tuan Tran, Hong-Hanh Nguyen-Le, Marco Ruffini +1Expert Load BalancingSparse Mixture-of-Experts

    School of Computer Science and Statistics, Trinity College Dublin, Ireland · School of Computer Science, University College Dublin, Ireland

  10. Towards the Next Frontier of LLMs, Training on Private Data: A Cross-Domain Benchmark for Federated Fine-Tuning

    May 13, 2026Daniel M. Jimenez-Gutierrez, Enrique Zuazua, Georgios Kellaris +3LLM Fine-TuningNon-IID Federated Learning

  11. Beyond Parameter Aggregation: Semantic Consensus for Federated Fine-Tuning of LLMs

    May 12, 2026Amr Abourayya, Jens Kleesiek, Michael KampLLM Fine-TuningFederated Knowledge Distillation

    Lamarr Institute for ML and AI, Technical University Dortmund · Institute for AI in medicine (IKIM),University Hospital Essen

  12. DP-LAC: Lightweight Adaptive Clipping for Differentially Private Federated Fine-tuning of Language Models

    May 11, 2026Haaris Mehmood, Jie Xu, Karthikeyan Saravanan +2Privacy-Preserving Language ModelsDifferentially Private Stochastic Gradient Descent

    Samsung R&D Institute UK (SRUK) · Samsung AI Centre Cambridge

  13. Concordia: Self-Improving Synthetic Tables for Federated LLMs

    May 11, 2026Jimin Huang, Duanyu Feng, Nuo Chen +8LLM Fine-TuningSynthetic Data Generation

    University of Manchester · The Fin AI · National University of Singapore +5

  14. When More Parameters Hurt: Foundation Model Priors Amplify Worst-Client Disparity Under Extreme Federated Heterogeneity

    May 9, 2026Kiran Naseer, Umar ShoaibNon-IID Federated LearningFederated Fine-Tuning

    University of Gujrat, Pakistan

  15. EdgeFlowerTune: Evaluating Federated LLM Fine-Tuning Under Realistic Edge System Constraints

    May 9, 2026Jiaxiang Geng, Yiyi Lu, Lunyu Zhao +3LLM EvaluationLLM Fine-Tuning

    Duke Kunshan University · Flower Labs · University of Cambridge

  16. Self-Play Enhancement via Advantage-Weighted Refinement in Online Federated LLM Fine-Tuning

    May 8, 2026Seohyun Lee, Wenzhi Fang, Dong-Jun Han +2LLM Fine-TuningLanguage Model Self-Improvement

    Purdue University · Yonsei University · University at Buffalo-SUNY

  17. Graph Representation Learning Augmented Model Manipulation on Federated Fine-Tuning of LLMs

    May 8, 2026Hanlin Cai, Kai Li, Houtianfu Wang +4LLM Fine-TuningLLM Security

    Centre for neXt Communications (CXC), Department of Engineering, University of Cambridge, CB3 0FA Cambridge, U.K. · Interdisciplinary Centre for Security, Reliability and Trust (SnT), University of Luxembourg, L-1855, Luxembourg · Telecommunication Networks group (TKN) at the School of Electrical Engineering and Computer Science, TU Berlin, Germany +1

  18. FedAttr: Towards Privacy-preserving Client-Level Attribution in Federated LLM Fine-tuning

    May 7, 2026Su Zhang, Junfeng Guo, Heng HuangPrivacy-Preserving Language ModelsLLM Fine-Tuning

    Department of Computer Science, University of Maryland, College Park

  19. Beyond Factor Aggregation: Gauge-Aware Low-Rank Server Representations for Federated LoRA

    May 7, 2026Jinqian Chen, Chang Liu, Jihua ZhuFederated Learning AggregationLow-Rank Adaptation

    School of Software Engineering Xi’an Jiaotong University Xi’an, China 710000

  20. Adaptive Selection of LoRA Components in Privacy-Preserving Federated Learning

    May 7, 2026Myoungjun Kim, Sangwoo Park, Yoseob Han +1Differentially Private Federated LearningFederated Learning

    Myongji University · King’s College London · Soongsil University

  21. SplitFT: An Adaptive Federated Split Learning System For LLMs Fine-Tuning

    Apr 29, 2026Yimeng Shan, Zhaorui Zhang, Sheng Di +3Communication-Efficient Distributed TrainingLLM Fine-Tuning

    Hong Kong Polytechnic University Hong Kong · Argonne National Labratory USA · University of California, Merced USA +1

  22. FED-FSTQ: Fisher-Guided Token Quantization for Communication-Efficient Federated Fine-Tuning of LLMs on Edge Devices

    Apr 28, 2026Changyu Li, Shuanghong Huang, Jiashen Liu +5Federated Fine-TuningFederated Learning

    School of Computing and Information Technology, Great Bay University, Dongguan, China · Beijing Institute of Technology, Beijing, China · University of Warwick, Coventry, U.K. +3

  23. FedProxy: Federated Fine-Tuning of LLMs via Proxy SLMs and Heterogeneity-Aware Fusion

    Apr 21, 2026Tao Fan, Guoqiang Ma, Yuanfeng Song +3Privacy-Preserving Language ModelsLLM Fine-Tuning

    Hong Kong University of Science and Technology, Hong Kong, China · WeBank Co., Ltd, Shenzhen, China · Hong Kong Polytechnic University, Hong Kong, China

  24. PubSwap: Public-Data Off-Policy Coordination for Federated RLVR

    Apr 14, 2026Anupam Nayak, Baris Askin, Muhammed Ustaomeroglu +2Federated RLReinforcement Learning with Verifiable Rewards

    Carnegie Mellon University

  25. Enhancing Cross-Problem Vehicle Routing via Federated Learning

    Apr 12, 2026Xiangchi Meng, Jianan Zhou, Jie Gao +4Vehicle RoutingNeural Combinatorial Optimization

    Dalian University of Technology · Nanyang Technological University · Delft University of Technology +1

  26. PrivFedTalk: Privacy-Aware Federated Diffusion with Identity-Stable Adapters for Personalized Talking-Head Generation

    Apr 9, 2026Soumya Mazumdar, Vineet Kumar Rakesh, Tapas SamantaTalking Head GenerationFederated Learning

    Department of Computer Science and Business Systems Gargi Memorial Institute of Technology Affiliated to Maulana Abul Kalam Azad University of Technology Balarampur, Mouza Beralia, Baruipur, Kolkata 700144, West Bengal, India · Engineering Sciences, Homi Bhabha National Institute Anushaktinagar, Mumbai, Maharashtra 400094, India · Computer and Informatics Group / Radioactive Ion Beam Facilities Group Variable Energy Cyclotron Centre 1/AF, Bidhannagar, Kolkata 700064, West Bengal, India

  27. Can LoRA Fusion Support Cross-Domain Tasks in Cloud-Edge Collaboration?

    Apr 9, 2026Yatong Wang, Fali Wang, Naibin Gu +6Model MergingDomain Adaptation for LLMs

    Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China · School of Cyber Security, University of Chinese Academy of Sciences, Beijing, China · The Pennsylvania State University, University Park, USA +1

  28. FedSQ: Optimized Weight Averaging via Fixed Gating

    Apr 3, 2026Cristian Pérez-Corral, Jose I. Mestre, Alberto Fernández-Hernández +3Federated Learning AggregationFederated Fine-Tuning

    Universitat Politècnica de València Valencia, Spain · Universitat Jaume I Castelló de la Plana, Spain · Openchip & Software Technologies Barcelona, Spain

  29. CoLLM: Continuous Adaptation for SLO-Aware LLM Serving on Shared GPU Clusters

    Mar 31, 2026Shaoyuan Huang, Yunfeng Zhao, Na Yan +5LLM Post-TrainingLLM Inference Scheduling

    Tianjin University, Tianjin, China · King’s College London, London, United Kingdom · Paiou Cloud Computing (Shanghai) Company, Ltd., Shanghai, China

  30. Task-Centric Personalized Federated Fine-Tuning of Language Models

    Mar 30, 2026Gabriel U. Talasso, Meghdad Kurmanji, Allan M. de Souza +2Federated LearningFederated Fine-Tuning

    Universidade Estadual de Campinas · University of Cambridge · Flower Labs