Federated Fine-Tuning
Overview
Federated fine-tuning studies how multiple data holders can adapt a shared pretrained model while keeping examples local, enabling collaboration in healthcare, finance, language, vision, and edge settings. Common approaches include parameter-efficient methods such as LoRA, QLoRA, and prompt tuning; client-server aggregation or split learning; and distillation when model updates are costly or architectures differ. Since clients have different data, tasks, and hardware, methods use personalized adapters, clustering or task-aware routing, update alignment, and compression, often evaluated on non-IID language, vision, and multimodal benchmarks. Keeping raw data local does not prevent gradient leakage, membership inference, poisoning, or unequal outcomes for weaker clients; differential privacy and stronger security can reduce utility and add cost, making realistic measurements of communication, memory, energy, and latency important.
Latest papers
128 papers12 papers in the last four weeks, down 33% on the four weeks before. 0.1% of all new papers.
Sustainable Federated Fine-Tuning with LoRA: Environmental Impact and Performance Footprint
Stochastic Projections and Rank-Aware Learning Rates for Federated LoRA
RBLA+: Efficient Rank-Specific LoRA Adapter Extraction for Federated Fine-Tuning with Heterogeneous Clients
FedLoVA: Value-Only Aggregation for Federated LoRA Fine-Tuning of Large Language Models
Breaking the Privacy Promise: Gradient Inversion Attacks on Federated Low-Rank Adapted Vision Transformers
Align Before Aggregation: Basis-Consistent Federated LoRA under Heterogeneous Ranks
Vision Paper: Why Distributed LLM Post-Training and Decentralized Alignment Are Now Essential
Federated Transfer Learning for Medical Image Classification under Mixed Skew
DICE: Decentralized Incentive-driven Federated Learning for Efficient & Scalable Customization of Pretrained Models in Web3
Communication-Efficient Federated Domain Generalization in Edge Networks via Dynamic Prototype Regulation
PFedQFT: Personalized Federated Quantum-Inspired Fine-Tuning for Medical Large Language Models
Federated Art Chase: Privacy Leakage and Data Poisoning in Federated Fine-Tuning
Optimal Hydra: Model Heterogeneous Federated Learning for LLMs using Data-Informed Hydra-Nets
FedGraph: Defending Federated Large Language Model Fine-Tuning Against Backdoor Attacks via Graph-Based Aggregation
BLEND: Balancing Personalization vs. Generalization in Federated Vision-Language Models
Personalized Federated Direct Preference Optimization via LoRA Adapters
HE-OFT: Privacy-Preserving One-Shot Federated Fine-Tuning under Homomorphic Encryption
Collaborative Personalized Preference Alignment for LLMs under Data Deficiency
Beyond Domain-Level Adaptation: Margin-Oriented Semantic-Appearance Interaction Correction for Personalized Federated Vision-Language Models
FedFit: Federated Fine-Tuning of LLMs via Vector-Bank Parameterization and Quantization
Federated Learning for LLMs over Mobile Networks: Issues and Solutions in the RAN Transport
From Task Mixtures to Specialized Experts
Unapologetically Distributed: A Call for Decentralized Document Analysis
FedLAFP: Low-Rank Aggregation Meets Full-Rank Personalization in Federated Fine-Tuning
CF-LoRA: Decoupled Factor Aggregation and Adaptation-Aware Client Clustering for Federated LoRA Fine-Tuning
RoboFL: Federated Expert Assembly for World Action Models
CFLoRA: Federated Fine-tuning of LLMs with Complementary Factors for Error-free Aggregation
CFLoRA, a federated LoRA scheme that partitions latent LoRA channels into two complementary sets in every communication round. By ensuring that columns and rows are complementary across factors, we eliminate bilinear terms in matrix multiplications, making federated aggregation exact. Crucially, our framework also supports clients with heterogeneous rank budgets. Convergence analysis validates CFLoRA achieves convergence rate of the original LoRA objective in homogeneous-rank cases. Extensive experiments with RoBERTa on the GLUE benchmark and with LLaMA-3.2-3B-Instruct on commonsense reasoning tasks demonstrate that CFLoRA achieves superior performance and training efficiency compared to state-of-the-art federated LoRA baselines.