Federated Learning

This paper studies robust estimation for Independent Component Analysis (ICA) in the federated learning setting, where data are locally distributed across clients and may exhibit substantial heterogeneity. The goal is to recover a common global mixing matrix by aggregating local estimators computed by individual clients. The main difficulty is that local ICA estimators are identifiable only up to sign flips and column permutations and may have highly heterogeneous estimation quality. We propose a novel three-step aggregation method that first aligns the permutations across all local estimators via a particular spectral clustering approach, then aligns the signs within each estimated cluster via another tailored spectral approach, and finally applies the geometric median for robust aggregation. The proposed estimator is shown to remain accurate even when a substantial fraction of local estimators are of low quality or inconsistent, as long as each cluster contains a majority of accurate estimators. This contrasts with its analogue based on simple averaging, whose performance is determined by the worst local estimator. In the homogeneous setting, the robust estimator is also shown to be minimax optimal, up to a logarithmic factor, both when the number of clients remains fixed and when it diverges. The theoretical findings are corroborated by simulation studies and a real-data analysis.

This book offers a hands-on introduction to building and understanding federated learning (FL) systems. FL enables multiple devices -- such as smartphones, sensors, or local computers -- to collaboratively train machine learning (ML) models, while keeping their data private and local. It is a powerful solution when data cannot or should not be centralized due to privacy, regulatory, or technical reasons. The book is designed for students, engineers, and researchers who want to learn how to design scalable, privacy preserving FL systems. Our main focus is on personalization: enabling each device to train its own model while still benefiting from collaboration with relevant devices. This is achieved by leveraging similarities between (the learning tasks associated with) devices that are encoded by the weighted edges (or links) of a federated learning network (FL network). The key idea is to represent real-world FL systems as networks of devices, where nodes correspond to device and edges represent communication links and data similarities between them. The training of personalized models for these devices can be naturally framed as a distributed optimization problem. This optimization problem is referred to as generalized total variation minimization (GTVMin) and ensures that devices with similar learning tasks learn similar model parameters. Our approach is both mathematically principled and practically motivated. While we introduce some advanced ideas from optimization theory and graph-based learning, we aim to keep the book accessible. Readers are guided through the core ideas step by step, with intuitive explanations.

In federated learning (FL), a center trains a shared model from updates that clients compute on their private data. Collecting and training on data is costly, so a strategic client may contribute little data and free-ride on the model trained by the others. We study mechanisms that incentivize data contribution and learn the model at the same time. Our first mechanism, UPBReD, updates the clients ' contributions and the model jointly, and converges at a geometric rate to a Nash equilibrium of the contribution game together with the welfare-maximizing model for those contributions. This equilibrium can involve under-contribution, so our second mechanism, 2P-UPBReD, adds budget-balanced transfers that make full contribution a dominant strategy, after which training converges to the optimal model; we also give conditions under which every client prefers to participate. Both mechanisms trust what clients report about the data they use and the gradients they compute. We show that no payment rule can reward unverifiable contributions unless something the center observes depends on the data actually used. Building on this, T-2P-UPBReD checks each client's reported contribution against the other clients ' realized valuations of the trained model, and prices gradient reports with a budget-balanced charge. It elicits truthful contribution reports, makes gradient reports approximately truthful, still reaches full contribution and the optimal model, and never runs a deficit. Experiments on CIFAR-10, FEMNIST, and Twitter show that 2P-UPBReD matches the welfare of FedAvg trained on all data, while strategic clients without incentives lower the welfare of the learned model.

Federated Learning (FL) must balance training efficiency and model accuracy across heterogeneous clients. Post-training quantization (PTQ) avoids the extra computation of quantization-aware training (QAT), but can incur greater accuracy loss. We propose FedHQ, a hybrid quantization framework that assigns PTQ or QAT according to clients' hardware and data characteristics. Hardware-based speed estimates and local data-distribution indicators guide coarse-grained global initialization, followed by fine-grained training-based adjustment of selected client strategies. Experiments on three image-classification datasets show that FedHQ achieves up to 2.47x training speedup over FedAQT and up to 11.15 percentage points higher accuracy than FedFQ.

Federated learning (FL) commonly involves clients with diverse communication and computational capabilities. Such heterogeneity can significantly distort the optimization dynamics and lead to objective inconsistency, where the global model converges to an incorrect stationary point potentially far from the pursued optimum. Despite its critical impact, the joint effect of communication and computation heterogeneity has remained largely unexplored, due to the intrinsic complexity of their interaction. In this paper, we reveal the fundamentally distinct mechanisms through which heterogeneous communication and computation drive inconsistency in FL. To the best of our knowledge, this is the first unified theoretical analysis of general heterogeneous FL, offering a principled understanding of how these two forms of heterogeneity jointly distort the optimization trajectory under arbitrary choices of local solvers. Motivated by these insights, we propose Federated Heterogeneity-Aware Client Sampling, FedACS, a universal method to eliminate all types of objective inconsistency. We theoretically prove that FedACS converges to the correct optimum at a rate of $O(1/\sqrt{R})$, even in dynamic heterogeneous environments. Extensive experiments across multiple datasets show that FedACS outperforms state-of-the-art and category-specific baselines by 4.3%-36%, while reducing communication costs by 22%-89% and computation loads by 14%-105%, respectively.

Personalization addresses one of the defining challenges in federated learning: statistical heterogeneity among clients. We show that the dissimilarity between the ideal personalized models of two clients can be upper bounded by the dissimilarity of their distributions. Motivated by this, we propose Karula, a data-driven regularization method that constrains pairwise model dissimilarities between clients according to their distributional differences. Distributional similarity is measured using a surrogate for the 1-Wasserstein distance adapted for the federated setting, allowing the method to adapt to highly complex interrelations between clients that approaches based on hard clustering may fail to recover. We propose an approximate projected stochastic gradient algorithm to solve the resulting constrained optimization problem, and prove that it converges at a rate of $O(1/K)$ to a neighborhood of a stationary point.

Federated Learning (FL) has the potential for simultaneous global learning amongst a large number of parallel agents, enabling emerging AI systems to be trained across demographically diverse data. Central to this being efficient is the ability for FL to perform sparse gradient updates and remote direct memory access at the central server. Most of the research in FL security focuses on protecting data privacy at the edge client or in the communication channels between the client and server. Client-facing attacks on the server are less well investigated as the assumption is that a large collective of clients offer resilience. Here, for the first time, we show that by attacking clients that lead to repetitive clustered memory updates in the server, we can remote initiate a rowhammer attack to corrupt the server memory with induced bit-flips. A reinforcement learning (RL) attacker without backdoor access to the server can learn how to maximize server repetitive clustered memory updates by manipulating the clients' sensor observation. We demonstrate the feasibility of our attack using FL systems with sparse updates, tasked for automatic speech recognition (ASR) as well as CIFAR10 classification, our adversarial attacking agent can achieve around 70% repeated update rate (RUR) in the targeted server model, effectively inducing bit flips on server DRAM. The security implications are that such adversarial observations can cause disruptions to learning or may inadvertently cause elevated privilege, which could pave the way for further research on practical mitigation strategies in FL and hardware design.

The use of DNS over HTTPS (DoH) tunneling by an attacker to hide malicious activity within encrypted DNS traffic poses a serious threat to network security, as it allows malicious actors to bypass traditional monitoring and intrusion detection systems while evading detection by conventional traffic analysis techniques. ML techniques can be used to detect DoH tunnels; however, their effectiveness relies on large datasets containing both benign and malicious traffic. Sharing such datasets across entities is challenging due to privacy concerns. In this work, we propose CO-DEFEND framework that enables multiple entities to collaboratively train a classification machine learning model for DoH threat detection while preserving data privacy, enhancing scalability and resilience against single points of failure. The proposed DFL framework provides a realistic implementation for DoH threat detection, enabling multiple entities to train their local models online with incoming DoH flows in real-time batches as they are processed - an approach that fits naturally within modern Internet architectures. This framework adapts four classical machine learning algorithms, Support Vector Machines, Logistic Regression, Decision Trees, and Random Forest, for federated scenarios and efficient training. In addition, a key methodological feature of CO-DEFEND is the use of DT and RF as model selection rather than aggregation mechanisms, allowing each participant to retain interpretable and locally optimal decision structures while benefiting from collective updates. We compare our proposed method by using the dataset CIRA-CIC-DoHBrw-2020 with existing machine learning approaches, including more computationally complex alternatives such as neural networks, to demonstrate its effectiveness in detecting malicious DoH tunnels while improving scalability and computational efficiency.

Federated learning (FL) has emerged as a key enabler of distributed model training across ubiquitous wireless edge devices while preserving data privacy in next-generation wireless systems. One communication-efficient approach to realize this is leveraging Over-the-Air Federated Learning (OTA-FL), which exploits the superposition property of wireless multiple access channels (MAC) to perform low-latency model aggregation. However, practical OTA-FL deployments are significantly affected by the joint effects of channel noise at the server, fading, and both data and system heterogeneity across participating edge devices, which can degrade the model performance and thereby convergence. In this work, we propose the novel Robust and Convergent Federated Learning (RoCo-FL), a framework that jointly handles these challenges in federated wireless networks. RoCo-FL formulates local updates through inexact optimization, capturing the effects of wireless noise, partial participation, and heterogeneous system capabilities via structured inexactness conditions. These conditions naturally manifest as a proximal constraint at the edge devices that stabilizes local updates and mitigates the impact of channel impairments, data heterogeneity, and stragglers. We establish convergence guarantees for non-convex objectives under the proposed zeroth- and first-order inexactness models, providing theoretical insight into the interplay between channel impairments, data, and system heterogeneity. Experimentally, we validate RoCo-FL on real-world datasets, including CIFAR10, CIFAR100, and FEMNIST, demonstrating its robustness in noisy and heterogeneous environments. Compared to state-of-the-art baselines, RoCo-FL achieves significantly more stable convergence and higher accuracy, particularly in the presence of stragglers.

Federated learning seeks to foster collaboration among distributed clients while preserving the privacy of their local data. Traditional federated learning methods typically assume a fixed setting, where participating clients, client data, and learning objectives remain unchanged. However, in real-world scenarios, a federation may evolve over time, with changes in both its client composition and target label space. In this evolving federated setting, conventional round-wise model aggregation becomes inflexible, as each federation update requires repeated communication, repeated local computation, and synchronized participation from all accumulated clients. To address this limitation, we propose CA-MMDS, a continual multiple-model distillation framework for federated continual segmentation with asynchronous clients and evolving label spaces. Instead of repeatedly aggregating model parameters from all clients, CA-MMDS maintains a server-side archive of client models and updates the global model through proxy-based distillation from multiple archived local models. When new clients join or existing clients evolve, only the newly added or updated local models need to be uploaded, while unchanged clients can remain offline and continue to contribute through their archived models. This design substantially reduces communication and computation costs while enabling flexible asynchronous cooperation among evolving clients. Using multi-class 3D abdominal CT segmentation as an application task, we demonstrate that CA-MMDS efficiently incorporates evolving client knowledge while achieving competitive segmentation performance.

Federated learning systems are increasingly threatened by data poisoning attacks, where malicious clients compromise global models by contributing tampered updates. Existing defenses often rely on impractical assumptions, such as access to a central test dataset, or fail to generalize across diverse attack types, particularly those involving multiple malicious clients working collaboratively. To address this, we propose Federated Noise-Induced Activation Analysis (FedNIA), a novel defense framework to identify and exclude adversarial clients without relying on any central test dataset. FedNIA injects random noise inputs to analyze the layerwise activation patterns in client models leveraging an autoencoder that detects abnormal behaviors indicative of data poisoning. FedNIA can defend against diverse attack types, including sample poisoning, label flipping, and backdoors, even in scenarios with multiple attacking nodes. Experimental results on non-iid federated datasets demonstrate its effectiveness and robustness, underscoring its potential as a foundational approach for enhancing the security of federated learning systems.

Client-drift compensation methods mitigate the impact of data heterogeneity in federated learning (FL). While drift correction is well studied for stochastic gradient descent (SGD)-based FL, its interaction with adaptive client optimizers remains less understood. Adaptive optimizers such as Adam build each update from running averages of past gradients, so correcting only the final update direction leaves those averages, and the drift they contain, unchanged. We propose Federated Adaptive Gradient Correction ({\tt FA-GC}), which corrects each gradient before it enters these running averages, and compare it with Federated Adaptive Naive Tracking ({\tt FA-NT}), which adds the correction to the final direction. We show that, in an idealized setting without gradient noise, the direction FA-GC follows vanishes only at stationary points of the global objective, whereas the direction FA-NT follows can vanish elsewhere when the optimizer is nonlinear. We also prove convergence of FA-GC for Adam-family methods, gradient normalization, and Muon, which orthogonalizes matrix updates. Unlike our bound for FA-NT, these bounds contain no term for client heterogeneity. On seven image and language tasks, FA-GC with Adam needs the fewest rounds on average to reach the target accuracy, and on the three image tasks it also uses less total communication than uncorrected local Adam and FA-NT. On the QNLI question-entailment task with SOAP, which applies Adam in a rotated basis, and Muon under strong heterogeneity, FA-GC gives the highest accuracy, while FA-NT falls below the uncorrected optimizers.

The increasing demand for data privacy, alongside the benefits of aggregating data from networked devices, has catalyzed the emergence of federated learning (FL). In FL, clients jointly train a global model by sharing gradients computed over private data. While this paradigm eliminates the need to exchange raw data, inference attacks can still be launched to extract sensitive information from gradients. To this end, partial gradient encryption has emerged as a promising design for balancing privacy and efficiency in practical FL systems, as encrypting only the classification-head gradients is believed to prevent known inference attacks while avoiding the high computational cost of encrypting the entire model. However, this design provides a false sense of privacy. By proposing GDBR, we show that sharing even a single unencrypted layer of gradients can lead to serious privacy leakage. GDBR is the first attack capable of high-fidelity label recovery with partial access to the gradients. It exploits a vulnerability in a commonly used neural building block, constructs a gradient bridge from the unencrypted layer to the final output layer, and approximates the logits information for accurate inference of private labels. These inferred labels not only reveal sensitive information about a client's private dataset but also serve as a prerequisite for many downstream attacks, such as data reconstruction and membership inference. GDBR brings these threats squarely into scope for FL systems employing partial encryption. In addition to theoretical analysis, extensive experiments demonstrate the severity of the problem across a wide variety of datasets and model architectures, including convolutional and transformer-based networks. Overall, our findings challenge the widespread assumption that encrypting only the output layer suffices for privacy protection.

To alleviate the training burden in federated learning while enhancing convergence speed, Split Federated Learning (SFL) has emerged as a promising approach by combining the advantages of federated and split learning. However, despite its advantages, existing SFL studies have largely overlooked the strategic interactions among self-interested participants during the SFL process. In this framework, the SFL model owner can choose the cut layer to balance the training load between the server and clients, ensuring the necessary level of privacy for the clients. Additionally, the SFL model owner sets incentives to encourage client participation in the SFL process. The optimization strategies employed by the SFL model owner influence clients' decisions regarding the amount of data they contribute, taking into account the shared incentives over clients and anticipated energy consumption from both computation and networking during SFL. To address this framework, we model the problem using a hierarchical decision-making approach, formulated as a single-leader multi-follower Stackelberg game. We demonstrate the existence and uniqueness of the Nash equilibrium among clients and analyze the Stackelberg equilibrium by examining the leader's game. Furthermore, we discuss privacy concerns related to differential privacy and the criteria for selecting the minimum required cut layer. Our findings show that the Stackelberg equilibrium solution maximizes the utility for both the clients and the SFL model owner while achieving a well-balanced trade-off between model accuracy and the associated computing and networking overhead during the SFL process.

Federated learning (FL) has gained wide popularity as a collaborative learning paradigm allowing collaborative Artificial Intelligence (AI) in sensitive healthcare applications. Nevertheless, the practical implementation of FL presents technical and organizational challenges, as it generally requires complex communication infrastructures. In this context, consensus-based learning (CBL) may represent a promising alternative for collaborative learning, allowing the combination of local knowledge into a federated decision system, while potentially reducing deployment overhead. Nevertheless, a comprehensive assessment of the viability of consensus-based learning as a cost-effective and sustainable alternative to federated learning has not yet been conducted. In this work we propose an extensive benchmark of the accuracy, cost-effectiveness, and sustainability of a panel of FL and CBL methods in a wide range of collaborative medical data analysis scenarios. The benchmark includes 7 different medical datasets, encompassing 3 machine learning tasks, 8 different data modalities, and multi-centric settings involving 3 to 23 clients. Our results reveal that CBL is a cost-effective and more sustainable alternative to FL. When compared across the panel of medical datasets in the considered benchmark, CBL methods provide equivalent accuracy to the one achieved by FL. Nonetheless, CBL significantly reduces training time and communication cost (resp. 15 fold and 131 fold decrease) (p-value < 0.05). This study opens a novel perspective on the deployment of collaborative AI in real-world applications, whereas the adoption of cost-effective methods is instrumental to achieve sustainability and democratization of AI by alleviating the need for extensive computational resources.

With the rapid development of the Internet of Things (IoT), federated learning (FL) has gained increasing attention for its privacy-preserving use of distributed data. However, conventional two-tier FL architectures are poorly suited to the hierarchical and heterogeneous nature of real-world IoT systems. Hierarchical Federated Learning (HFL) introduces multi-layer aggregation to better match IoT environments, but still suffers from communication inefficiencies and performance limitations caused by large data transfers, non-IID data distributions, and uneven device participation. These challenges hinder the realization of low-latency and high-accuracy training in practical IoT deployments. To address these limitations, we propose H-FedSN for practical IoT environments. H-FedSN leverages a binary mask mechanism with shared and personalized layers to reduce communication overhead by creating a sparse network without altering original weights. To tackle data heterogeneity and imbalanced device distribution, H-FedSN incorporates personalized layers for local data adaptation and employs Bayesian aggregation with cumulative Beta distribution updates at edge and cloud levels, effectively balancing contributions from diverse client groups. Experiments on three real-world IoT datasets and MNIST under non-IID conditions show that H-FedSN reduces communication costs by up to 477 times compared to baseline methods while maintaining high accuracy, making it well-suited for hierarchical FL in IoT deployments.

Federated learning (FL) enables collaborative analysis of biomedical data without exchanging sensitive patient-level information, but its performance in multi-center studies may be compromised by batch effects which can obscure biological signals. Here, we systematically assess the impact of uncorrected batch effects on FL outcomes using four multi-center omics datasets, including transcriptomic, proteomic, and metabolomic data, and two representative algorithms: federated k-means clustering and federated random forest classification. Our results demonstrate that uncorrected batch effects undermine unsupervised FL and can substantially degrade supervised FL performance, indicating that privacy-aware batch-effect correction is essential for reliable FL. To enable privacy-preserving BEC in distributed bulk omics data, we introduce fedRBE ( https://featurecloud.ai/app/fedrbe ), a federated implementation of limma's removeBatchEffect() method enhanced by secure multi-party computation, suitable for datasets with missing values and non-identical feature sets across clients, including proteomics and metabolomics data.

In federated learning (FL), a central server typically allocates training efforts to clients. However, from a market-oriented perspective, clients may independently choose their training efforts based on rational self-interest. To study this setting, we propose a potential game framework in which each client's payoff is determined by its individual effort and the rewards provided by the server. The rewards are influenced by the collective efforts of all clients and can be modulated by a reward factor. We first establish the existence of Nash equilibria (NEs) and then investigate their uniqueness in a stationary setting. We show that the NEs depend nonlinearly on the reward factor and exhibit a nonsmooth transition at a critical value, where the stationary potential loses strict curvature, leading to nonunique NEs and a jump between low-effort and high-effort branches. Furthermore, we prove the convergence of the best-response algorithm for computing NEs in our FL game. Finally, we apply the clients' rational efforts derived from the NEs to FL training with various datasets and models, thereby validating the effectiveness of the identified critical reward factor. The source code is available at https://github.com/DCN-FAU-AvH/FL-Potential-Game

Federated learning (FL) is a learning paradigm that enables collaborative training of models using decentralized data. Recently, the utilization of pre-trained weight initialization in FL has been demonstrated to effectively improve model performance. However, the evolving complexity of current pre-trained models, characterized by a substantial increase in parameters, markedly intensifies the challenges associated with communication rounds required for their adaptation to FL. To address these communication cost issues and increase the performance of pre-trained model adaptation in FL, we propose an innovative model interpolation-based local training technique called ``Local Superior Soups.'' Our method enhances local training across different clients, encouraging the exploration of a connected low-loss basin within a few communication rounds through regularized model interpolation. This approach acts as a catalyst for the seamless adaptation of pre-trained models in in FL. We demonstrated its effectiveness and efficiency across diverse widely-used FL datasets. Our code is available at https://github.com/ubc-tea/Local-Superior-Soups.

Personalized collaborative learning in federated settings faces a critical trade-off between customization and participant trust. Existing approaches typically rely on centralized coordinators or trusted peer groups, limiting their applicability in open, trust-averse environments. While recent decentralized methods explore anonymous knowledge sharing, they often lack global scalability and robust mechanisms against malicious peers. To bridge this gap, we propose TPFed, a Trust-free Personalized Decentralized Federated Learning framework. TPFed replaces central aggregators with a blockchain-based bulletin board, enabling participants to dynamically select global communication partners based on Locality-Sensitive Hashing (LSH) and peer ranking. Crucially, we introduce an ``all-in-one'' knowledge distillation protocol that simultaneously handles knowledge transfer, model quality evaluation, and similarity verification via a public reference dataset. This design ensures secure, globally personalized collaboration without exposing local models or data. Extensive experiments demonstrate that TPFed significantly outperforms traditional federated baselines in both learning accuracy and system robustness against adversarial attacks.

Federated learning (FL) is a machine learning paradigm where clients with different behaviors and preferences can learn collaboratively without compromising data privacy. Typical FL methods often rely on fixed weighting for parameter aggregation, thereby neglecting the mutual influence among clients. In practice, clients with similar preferences or backgrounds may provide more useful knowledge to each other, which can be leveraged to improve local performance. However, how to quantify such cross-client influence and how to exploit it for personalized aggregation remain underexplored. To address this gap, we propose an influence-oriented Federated learning framework which quantitatively measures Client-level and Class-level Influence to realize adaptive parameter aggregation for each client (FedC^2I for short). Our core idea is to explicitly model the inter-client influence within an FL system via the well-crafted influence vector and influence matrix. Specifically, FedC^2I incorporate influence vectors to quantify client-level influence, enables clients to selectively acquire knowledge from others, and guides the aggregation of feature representation layers. Meanwhile, the influence matrix captures class-level influence in a more fine-grained manner to achieve personalized classifier aggregation. We evaluate the performance of FedC^2I against existing federated learning methods under non-IID settings, and the results demonstrate the superiority of our method in terms of effectiveness, robustness, and interpretability.

Privacy attacks, particularly data reconstruction attacks, pose significant challenges to federated learning despite its promise in privacy-preserving collaborative training. This paper presents a comprehensive theoretical analysis of privacy leakage in federated learning through dual perspectives: linear algebra and optimization theory. By analyzing the client-update Jacobian, we identify conditions under which distinct data batches produce the same model update and derive a sufficient batch-size criterion for non-unique reconstruction. Through optimization theory, we establish a novel upper bound for privacy leakage that incorporates batch size, distortion extent, and other critical parameters. These theoretical insights elucidate the fundamental relationships between system parameters and privacy vulnerability, laying the groundwork for developing more robust privacy-preserving federated learning frameworks.

Federated Learning (FL) faces challenges due to data heterogeneity, which limits the global model's performance across diverse client distributions. Personalized Federated Learning (PFL) addresses this by enabling each client to possess an individual model adapted to its local distribution. Many existing methods assume that certain global model parameters are difficult to train effectively in a collaborative manner under heterogeneous data. Consequently, they localize or fine-tune these parameters to obtain personalized models. In this paper, we reveal that both the feature extractor and classifier of the global model are inherently strong, and the primary cause of its suboptimal performance is the mismatch between local features and the global classifier. Although existing methods alleviate this mismatch to some extent and improve performance, we find that they either (1) fail to fully resolve the mismatch while degrading the feature extractor, or (2) address the mismatch only post-training, allowing it to persist during training. This increases inter-client gradient divergence, hinders model aggregation, and ultimately leaves the feature extractor suboptimal for client data. To address this issue, we propose FedPFT, a novel framework that resolves the mismatch during training using personalized prompts. These prompts, along with local features, are processed by a shared self-attention-based transformation module, ensuring alignment with the global classifier. Additionally, this prompt-driven approach offers strong flexibility, enabling task-specific prompts to incorporate additional training objectives (e.g., contrastive learning) to further enhance the feature extractor. Extensive experiments show that FedPFT outperforms state-of-the-art methods by up to 5.07%, with further gains of up to 7.08% when collaborative contrastive learning is incorporated.

Federated Learning (FL) aims at unburdening the training of deep models by distributing computation across multiple devices (clients) while safeguarding data privacy. On top of that, Federated Continual Learning (FCL) also accounts for data distribution evolving over time, mirroring the dynamic nature of real-world environments. While previous studies have identified Catastrophic Forgetting and Client Drift as major factors of performance degradation in FCL, we shed light on the importance of Incremental Bias and Federated Bias, which cause models to prioritize classes that are recently introduced or locally predominant, respectively. Our proposal constrains both biases to the last layer by efficiently fine-tuning a pre-trained backbone using learnable prompts, resulting in clients that produce less biased representations and more biased classifiers. Therefore, instead of solely relying on parameter aggregation, we leverage generative prototypes to effectively balance the predictions of the global model. Our proposed methodology significantly improves the current state of the art across six datasets, each including three different scenarios.

Contemporary AI faces the challenge of balancing generality with user-specific personalization. In federated learning (FL), this challenge is amplified by highly heterogeneous client data with complex non-IID patterns beyond standard IID assumptions. Many existing FL methods are designed for relatively restricted heterogeneity settings (e.g., a fixed number of clusters or a fixed form of personalization), limiting their robustness under complex structures. In this work, we study FL from a multi-level non-IID perspective, where client similarity is captured by multiple granularities of shared knowledge: global, subgroup, and client-specific components. This view captures coarse-to-fine relationships while requiring less prior knowledge of task boundaries. Building on this insight, we propose Federated Multi-level Additive Modeling (FeMAM), which learns multiple levels of shareable models and constructs personalized predictors via additive composition across levels. To move beyond a fixed structure, FeMAM allows models to grow and be pruned dynamically during training, adapting to diverse federated scenarios. Despite employing multiple models, FeMAM remains cost-friendly by unlocking only a small subset (one level) of models for training at a time. Extensive experiments show that FeMAM effectively approximates diverse complex non-IID structures and consistently outperforms representative clustered and personalized FL baselines.

Mitigating cybersecurity risk in electric vehicle (EV) charging demand forecasting plays a crucial role in the safe operation of collective EV chargings, the stability of the power grid, and the cost-effective infrastructure expansion. However, existing methods either suffer from the data privacy issue and the susceptibility to cyberattacks or fail to consider the spatial correlation among different stations. To address these challenges, a federated graph learning approach involving multiple charging stations is proposed to collaboratively train a more generalized deep learning model for demand forecasting while capturing spatial correlations among various stations and enhancing robustness against potential attacks. Firstly, for better model performance, a Graph Neural Network (GNN) model is leveraged to characterize the geographic correlation among different charging stations in a federated manner. Secondly, to ensure robustness and deal with the data heterogeneity in a federated setting, a message passing that utilizes a global attention mechanism to aggregate personalized models for each client is proposed. Thirdly, by concerning cyberattacks, a special credit-based function is designed to mitigate potential threats from malicious clients or unwanted attacks. Extensive experiments on a public EV charging dataset are conducted using various deep learning techniques and federated learning methods to demonstrate the prediction accuracy and robustness of the proposed approach.

Instruction tuning aligns large language models (LLMs) with human intentions but requires diverse, high-quality data that are difficult to collect in privacy-sensitive domains. Federated instruction tuning (FedIT) enables collaborative training across data owners, yet existing methods typically assume sufficient local data. In realistic few-shot settings, limited samples can cause overfitting, degrade performance, and increase vulnerability to training data extraction attacks. We propose PPFedIT, a federated algorithm that improves both model performance and privacy protection in federated few-shot learning. It comprises three client-side steps: (1) synthetic data generation, which uses LLMs to diversify and enrich local data; (2) parameter isolation training, which updates the shared global LLM on synthetic data and local LLMs on private local data to mitigate synthetic-data noise; and (3) local aggregation then sharing, which mixes global and local model parameters before uploading them for server aggregation to mitigate data extraction attacks. Experiments on three open-source datasets show that PPFedIT improves model performance by an average of 8.4% and reduces the risk of data extraction attacks by approximately 20% in challenging federated few-shot settings.

Federated learning has emerged as a decentralized approach for training high-performance models without accessing user data. Despite its effectiveness, it is vulnerable to poisoning attacks, where malicious users manipulate the global model by uploading poisoned updates. In this paper, we propose FedReview, a review-based mechanism to identify and dispose the potential poisoned updates in federated learning. Under FedReview, the server randomly assigns a subset of clients as reviewers to evaluate model updates on their training datasets in each round. The reviewers rank the updates based on evaluation results and estimate the number of low-quality updates as potential poisoned ones. Based on the review reports, the server applies a majority voting mechanism to aggregate rankings, which tolerates wrong rankings from malicious reviewers and guides the removal of suspicious updates during model aggregation. In contrast to prior works such as FLTrust, FedReview does not require a server-side validation dataset or prior knowledge of clients, allowing flexible client participation. Extensive experiments demonstrate that FedReview enables the server to learn a well-performing global model in adversarial environments.

There are two categories of methods in Federated Learning (FL) for joint training across multiple clients: (i) parallel FL (PFL), where clients train models in a parallel manner; and (ii) sequential FL (SFL), where clients train models in a sequential manner. In contrast to that of PFL, the convergence theory of SFL on heterogeneous data is still lacking. In this paper, we establish the convergence guarantees of SFL for strongly/general/non-convex objectives on heterogeneous data. The convergence guarantees of SFL are better than that of PFL on heterogeneous data with both full and partial client participation. Experimental results validate the counterintuitive analysis result that SFL outperforms PFL on extremely heterogeneous data in cross-device settings.

Federated Learning (FL) systems are susceptible to adversarial attacks, such as model poisoning attacks and backdoor attacks. Existing defense mechanisms face critical limitations in deployments, such as relying on impractical assumptions (e.g., adversaries acknowledging the presence of attacks before attacking) or undermining accuracy in model training, even in benign scenarios. To address these challenges, we propose CustodianFL, a two-staged anomaly detection method specifically designed for FL deployments. In the first stage, it flags suspicious client activities. In the second stage that is activated only when needed, it further examines these candidates using Three-Sigma Rule to identify and exclude truly malicious local models from FL training. To ensure integrity and transparency within the FL system, CustodianFL integrates zero-knowledge proofs, enabling clients to cryptographically verify the server's detection process without relying on the server's goodwill. CustodianFL operates without unrealistic assumptions and avoids interfering with FL training in attack-free scenarios. It bridges the gap between theoretical advances in FL security and the practical demands of real FL systems. Experimental results demonstrate that CustodianFL consistently delivers performance comparable to benign cases, highlighting its effectiveness in identifying and eliminating malicious models with high accuracy.