Non-IID Federated Learning

Follow this topic Ask about this topic

Momentum

14 papers in the last four weeks, up 75% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 130

Oct 5, 2026cs.LG

Fed-BRDECS: Privacy-Preserving and Heterogeneity-Aware Federated Deep Embedded Clustering

Federated deep clustering seeks to learn clustering-friendly representations from decentralized unlabeled data while preserving client privacy. However, Deep Embedded Clustering (DEC)-style objectives depend on global soft-assignment statistics that require clients to reveal their sensitive information. We propose Fed-BRDECS, a privacy-preserving and heterogeneity-aware federated deep embedded clustering framework. Fed-BRDECS replaces the globally normalized clustering objective with a locally computable sample-stability loss, avoiding the transmission of local soft-assignment distributions. To tackle non-IID client distributions, we introduce prediction-balanced sampling, which oversamples locally rare predicted clusters without requiring ground-truth labels, and centroid-level restarting, which periodically refreshes biased or inactive centroids. Experiments on image and text clustering benchmarks show that Fed-BRDECS consistently outperforms representative federated clustering and deep clustering baselines under both IID and non-IID partitions. We further demonstrate its applicability to federated time-series anomaly detection, where it improves reconstruction-based detectors without adding inference-time cost.

Haemin Park, Diego Klabjan, Martin W. Braun +2

Northwestern University · Intel Corporation

Clustering Deep Clustering Non-IID Federated Learning Privacy-Preserving ML Federated Learning

Oct 1, 2026cs.LG

FedSAP: Federated Learning with Structured Adaptive Partitioning for Multi-Domain Heterogeneous Edge Devices

Federated learning (FL) on heterogeneous edge devices must jointly accommodate unequal resource budgets and domain-shifted local data. Existing resource-adaptive methods decide how much of a model each client trains but not where retained capacity should reside or how it should be shared, whereas federated domain-generalization methods usually assume a shared full architecture. Uniform compression can therefore discard high-utility channels, and a single aggregation path can mix transferable features with domain-sensitive updates. We propose FedSAP, a domain-aware heterogeneous FL framework that casts structured pruning as budget-constrained tri-state channel allocation. FedSAP converts each keep ratio into non-uniform layer budgets, assigns stable channels to a Global pool, useful domain-sensitive channels to pseudo-domain-specific Private pools, and low-utility channels to a Dropped state. This partition lets broadly useful features benefit from cross-client pooling while isolating domain-sensitive updates from incompatible clients. Domain-Guided Assignment infers pseudo-domains from shallow-gradient similarity, while Type-Matched Aggregation restricts each channel to its intended sharing scope. Across three random seeds, FedSAP reaches 76.00% and 72.67% mean global accuracy on Digits and Office-Caltech, exceeding the strongest baseline by 1.70 and 4.92 percentage points while supporting client pruning ratios of up to 80% across heterogeneous clients.

Wentao Yue, Tianyou Lai, Hongji Li +2

Non-IID Federated Learning Structured Pruning Federated Learning Heterogeneous Federated Learning

Sep 30, 2026cs.CV

FedMAD: Modulation-Aware Directional Aggregation for Federated Learning in Remote Sensing Image Classification

Federated learning (FL) has recently attracted increasing attention in remote sensing (RS) since it enables collaborative model training across decentralized RS image archives without requiring direct access to local data. However, FL performance significantly degrades when the data distributions between clients are heterogeneous, which often occurs due to geographical differences, seasonal changes, and varying image acquisition and atmospheric conditions. To address this challenge, in this letter, we propose a novel personalized FL framework (denoted as FedMAD) for RS image classification problems. The proposed framework separates globally shared representation parameters from client-specific adaptation parameters to preserve client-specific features while maintaining globally transferable representations. This is achieved by integrating lightweight modulation modules and local batch normalization layers into the backbone network. Although globally shared parameters are collaboratively optimized between clients, client-specific parameters remain local to preserve domain-specific feature characteristics. In addition, FedMAD introduces a modulation-aware directional aggregation strategy that dynamically adjusts the importance of aggregation for each client according to the alignment of local modulation updates. This allows the global optimization process to suppress conflicting client updates originating from heterogeneous data distributions while enhancing the contribution of clients with consistent adaptation behaviors. The experimental results obtained on the BigEarthNet-S2 and EuroSAT datasets demonstrate the effectiveness of FedMAD compared to state-of-the-art FL algorithms under heterogeneous RS data distributions. The code of the proposed framework will be publicly available at https://git.tu-berlin.de/rsim/fedmad.

Barış Büyüktaş, Begüm Demir

Faculty of Electrical Engineering and Computer Science, Technische Universität Berlin, 10587 Berlin, Germany · BIFOLD - Berlin Institute for the Foundations of Learning and Data, 10587 Berlin, Germany

Remote Sensing Image Understanding Federated Learning Aggregation Non-IID Federated Learning Federated Learning Heterogeneous Federated Learning

Sep 30, 2026cs.LG

Client and Training Data Selection for Computationally Efficient Synchronized Federated Learning

Federated learning (FL) is a promising paradigm of machine learning, which preserves user privacy by enabling learning without sharing raw data with a cloud server. Straggling clients have been a problem for FL as they introduce delays in aggregating the local models and hence, the convergence of the global model. Therefore, it is important to have a mechanism that ensures fast convergence of the global model as well as good FL participation rate. Another issue for the convergence of a model in FL is the non-independent and identically distributed (non-iid) data across the clients. Prior approaches based on probabilistic client selection do not work well under non-iid data especially when the number of clients is small. We show scenarios where such approaches fail and propose a joint client-training data selection algorithm for fast convergence of FL models. Our experiments on CIFAR-100 dataset show that convergence of the FL model can be significantly improved over prior works that can consider non-iid data and heterogeneous computation and higher model accuracy.

Muzaffer Citir, Hiroki Nishikawa, Sangyoung Park

Smart Mobility Systems, Technical University of Berlin, Germany · Graduate School of Information Science and Technology, The University of Osaka, Japan

Client Selection in Federated Learning Non-IID Federated Learning Training Data Selection Federated Learning Heterogeneous Federated Learning

Sep 28, 2026cs.MA

Agentic Federated Learning: Rule-Based Client and Server Agents for Adaptive Training

Federated Learning (FL) enables collaborative model training across distributed clients without sharing raw data, making it suitable for privacy-sensitive applications such as healthcare, finance, and edge intelligence. However, conventional FL approaches rely on static client participation and fixed aggregation strategies, which limits their effectiveness under non-IID data distributions, heterogeneous client behavior, and noisy or unreliable updates. To overcome these issuess, this paper proposes an Agentic Federated Learning (AFL) framework that integrates lightweight rule-based autonomous agents at both client and server levels. The proposed framework introduces a Client-Side Agent (CSA) that dynamically adapts local training parameters, controls participa- tion, and evaluates update reliability, while a Server-Side Orchestrator Agent (SSOA) performs quality-aware client selection and adaptive aggregation. Unlike traditional FL methods, AFL enables context-aware decision-making during the training process, improving adaptability and robustness in dynamic distributed environments. Extensive experiments conducted on the CIFAR-10 dataset under IID, non-IID, and noisy-client settings demonstrate that AFL consistently outperforms standard base- lines including FedAvg and FedProx. Experimental results show improvements in classification accuracy, convergence speed, robustness against corrupted updates, and communication efficiency. Ablation studies further confirm the complementary contributions of CSA and SSOA, while statistical analysis validates the significance of the observed gains. The proposed AFL framework demonstrates that incorporating autonomous agentic reasoning into federated learning provides an effective and practical solution for intelligent, adaptive, and robust distributed learning systems.

Deepthy K. Bhaskar, VP Binu, B Minimol

Department of Computer Engineering, Model Engineering College, APJ Abdul Kalam Technological University, Thiruvananthapuram 695016, Kerala, India. · Department of Biomedical Engineering, Model Engineering College, APJ Abdul Kalam Technological University, Thiruvananthapuram 695016, Kerala, India.

Federated Learning Aggregation Client Selection in Federated Learning Non-IID Federated Learning Federated Learning

Sep 22, 2026stat.ML

On the Gradient Heterogeneity Dynamics of Adversarially Robust Federated Regression

Federated learning (FL) is intrinsically heterogeneous: honest clients may have different data-generating models. On top of that, adversarial clients can make heterogeneity even more pronounced by sharing arbitrary updates. Existing analyses typically control the interaction between statistical heterogeneity and adversarial behavior through gradient-dissimilarity conditions. However, the underlying bound is imposed a priori and may yield conservative guarantees even for least-squares regression. We instead derive the gradient heterogeneity from the statistical model of linear and nonlinear regression with fresh data samples at every round. Our bounds separate heterogeneity among the honest clients' ground-truth model parameters, finite-sample label noise, and initialization. We then demonstrate that, for any

(f,κ)

-robust aggregator with coefficient

κ= O(f/n)

, where

f

is the number of adversarial clients and

n

the total number of clients (with

f/n < 1/2

), convergence holds after an explicit sample burn-in.

Leonardo F. Toso, James Anderson, Nirupam Gupta +1

Columbia University, USA · University of Copenhagen, Denmark · Sorbonne Université and Université Paris Cité, CNRS, LPSM, France

Byzantine-Robust Federated Learning Non-IID Federated Learning Statistical Learning Theory Federated Learning

Sep 17, 2026cs.LG

Distributionally Robust Federated Learning with Multi-Source Data

Federated learning trains a shared model from private client data. In practice, data-generating distributions may differ, and the true mixture across clients is often unknown, making the underlying group distribution difficult to specify. Existing approaches address cross-client mixture uncertainty by optimizing against the worst-case mixture, yet assume accurate client-wise distribution estimates. However, these estimates can be unreliable when based on finite samples. To handle both cross-client mixture uncertainty and within-client distributional ambiguity, we construct a global ambiguity set as the union of admissible mixtures of local ambiguity sets. The construction allows client-specific ambiguity radii and admits a client-wise separable reformulation. Leveraging this structure, we establish a high-probability out-of-sample performance guarantee. We further develop a federated algorithm for a penalty-based reformulation and prove its convergence under milder regularity conditions. Simulations validate the algorithm's effectiveness.

Yingzhu Liu, Zhongkui Li, Pengcheng You +1

Department of Control Science and Systems Engineering, Peking University, Beijing, China. · Engineering and Technology Institute Groningen and the Jan C. Willems Center for Systems and Control, University of Groningen, The Netherlands.

Distributionally Robust Optimization Non-IID Federated Learning Federated Learning

Sep 17, 2026cs.LG

FedFIbOS: Fisher Importance based Optimal Submodelling for Heterogeneous Federated Learning

Heterogeneous federated learning requires clients with diverse computational capacities to collaboratively train a global model, where each client trains a capacity-constrained submodel. Existing methods select submodel parameters using heuristic importance measures---most prominently parameter magnitude---without theoretical justification for why these measures support convergence. We identify a fundamental gap: existing parameter selection criteria lack theoretical grounding in the convergence framework, partial client participation introduces additional estimation effects in the Fisher scores. We propose \textbf{FedFIbOS}: Fisher Importance-based Optimal Submodelling for heterogeneous federated learning, using Fisher Information in a principled criterion derived from minimizing submodel masking error. %We formally establish when magnitude selection is equivalent to Fisher selection fail under non-IID heterogeneous federated learning. We theoretically formulate submodel selection through a Fisher-weighted quadratic masking surrogate and show that the raw Fisher top-

k

rule implemented by FedFIbOS solves this surrogate under a Fisher-dominant ranking condition. The resulting method retains the convergence structure of the underlying masked federated optimization bound. Fisher scores are efficiently estimated from empirical diagonal Fisher information using squared gradients, enabling stable and adaptive parameter selection without additional optimization overhead. Experiments on CIFAR-10, CIFAR-100, and AGNews under pathological and Dirichlet non-IID settings show FedFIbOS achieves

{\approx}10\%

higher accuracy than the state of the art, with improvements becoming more pronounced under stronger heterogeneity.

Yasmeen Afzal, Jeremiah D. Deng, Haibo Zhang

School of Computing, University of Otago, New Zealand · School of Computer Science and Engineering, University of New South Wales, Australia

Non-IID Federated Learning Federated Learning Heterogeneous Federated Learning

Sep 16, 2026stat.ML

Federated Soft Clustering via Generalized Total Variation Minimization

We study federated soft clustering over federated learning (FL) networks of devices that each hold a private local dataset and fit a personalized Gaussian mixture model (GMM). Generalized total variation minimization (GTVMin) couples the local maximum likelihood problems through a graph regularizer that penalizes a discrepancy between the models of connected nodes. The choice of discrepancy measure is a key design decision: we compare a squared Euclidean distance between model parameters, which requires component matching, with two measures that compare the local model distributions directly and hence need no matching: a Monte-Carlo approximated Kullback-Leibler (KL) divergence and a closed-form maximum mean discrepancy (MMD). All three resulting GTVMin instances are optimized by synchronous projected gradient updates; for the smooth MMD instance we provide a convergence guarantee to stationary points. We characterize their computational cost and evaluate their robustness to data heterogeneity.

Shamsiiat Abdurakhmanova, Alexander Jung

Department of Computer Science, Aalto University, Espoo, Finland

Gaussian Mixture Models Non-IID Federated Learning Federated Learning

Sep 15, 2026cs.LG

Personalized Federated Learning through Global Knowledge Distillation and Local Head Adaptation

Statistical heterogeneity limits federated learning when a single global classifier cannot represent client-specific label distributions. In this work, we propose Personalized Federated Knowledge Distillation with Head Adaptation (pFedKDH), which aggregates only the shared backbone, keeps persistent client-specific heads, and uses a recalibrated global head as a teacher during local training. Across MNIST, Fashion-MNIST, CIFAR10, and CIFAR100 under class-wise Dirichlet partitions, pFedKDH obtains the best accuracy in most settings, with accuracy gaps up to 37.67% over the weakest baseline and consistently low standard deviation across repetitions. Component-wise diagnostics and convergence results support the role of persistent heads and distillation-guided local optimization under label-skewed data.

Polycarpo Souza Neto, José Mairton Barros da Silva Júnior, Charles Casimiro Cavalcante

Universidade Federal do Cear´a, Fortaleza, CE, Brazil · Uppsala University, Uppsala, Sweden

Federated Knowledge Distillation Non-IID Federated Learning Knowledge Distillation Federated Learning

Sep 15, 2026cs.CV

Decentralized Gossip Learning and Federated Averaging for Histopathology Image Classification

Breast histopathology analysis increasingly relies on distributed learning because direct data pooling across institutions is often restricted by privacy, governance, and communication constraints. This study compares server-based Federated Averaging (FedAvg), fully decentralized gossip learning, and Hybrid Gossip-FedAvg for invasive ductal carcinoma (IDC) patch classification. Experiments used 277,524 color image patches with patient-disjoint training, validation, and test partitions and a workload-balanced, Dirichlet-guided allocation across six nodes. Ring, random degree-3, and fully connected gossip topologies were evaluated together with sensitivity analyses for statistical heterogeneity, mixing coefficient, learning rate, model drift, prediction disagreement, calibration, clinically motivated operating points, communication payload, and patient-level IDC burden, together with auxiliary backbone robustness analyses. In the principal alpha=0.3 experiment, Hybrid Gossip-FedAvg achieved a test area under the receiver operating characteristic curve (ROC-AUC) of 0.8811, closely followed by FedAvg at 0.8801 and fully connected gossip at 0.8751. Across three independent patient-level repetitions, FedAvg and Hybrid Gossip-FedAvg obtained the same mean ROC-AUC of 0.9082, with standard deviations of 0.0037 and 0.0043, respectively. Hybrid achieved the highest mean area under the precision-recall curve of 0.8240, whereas FedAvg produced the lowest mean Brier score of 0.1335. Denser gossip graphs improved discrimination but increased theoretical model payload, while ring gossip remained sensitive to learning rate and mixing strength. Overall, FedAvg provided the most consistently reliable server-based baseline, topology-aware gossip offered a viable decentralized alternative, and Hybrid Gossip-FedAvg provided a balanced compromise between peer-to-peer diffusion and periodic global coordination.

Yusuf Ozturk, Bengisu Atli, Enes Goktekin +2

Antalya Bilim University, Electrical and Electronics Engineering Dep. Antalya, Türkiye · Antalya Bilim University, Computer Engineering Dep. Antalya, Türkiye · Ankara University, Graduate School of Natural and Applied Sciences, Ankara, Türkiye +1

Federated Learning Aggregation Non-IID Federated Learning Decentralized Learning Histopathology Image Classification Federated Learning

Sep 15, 2026cs.LG

Adaptive Bayesian Partner Selection for Federated Clinical Centers

Federated learning (FL) in healthcare faces pronounced heterogeneity and temporal concept drift across clinical centers, where evolving patient populations and care practices shift data distributions. Existing approaches rely on persistent global communication, incurring substantial bandwidth overhead while risking negative transfer from poorly aligned peers. We propose Adaptive Bayesian Partner Selection (ABPS), a peer-to-peer framework that governs who collaborates, when, and at what cost. Each center maintains a Beta-Bernoulli posterior over prospective peers' Shapley marginal utility, ranks candidates with an Upper Confidence Bound (UCB) criterion, and forms collaborations through a lightweight propose-reject mechanism, with the option to abstain from communication when no mutually beneficial partner exists. The framework admits a stochastic decision interpretation, yielding finite-sample concentration guarantees and O(kappa log T) regret in partner selection, along with conditions under which intentional isolation is optimal under negative transfer. Lightweight extensions (head personalization, bfloat16 quantized communication, and a tunable active-set size) further improve efficiency, and a goal-aware metadata filter enables institution-specific collaboration strategies. On binary in-hospital mortality prediction over the first 24 hours of an ICU stay, with 230 non-IID clinical centers drawn from MIMIC-IV, the full ABPS-X variant matches the strongest federated baseline (FedDyn, AUROC 0.758) at 0.09x the communication cost of FedAvg, with reduced variability. A diversity-driven configuration activates intentional isolation for a substantial fraction of centers. These results show that adaptive, utility-aware collaboration reduces communication without sacrificing accuracy when centers are numerous and small, offering a scalable paradigm for healthcare FL.

Navid Seidi, Satyaki Roy, Sajal K. Das

Department of Computer Science Missouri University of Science and Technology Rolla, MO, USA · Department of Mathematical Sciences University of Alabama in Huntsville Huntsville, AL, USA

Healthcare Communication-Efficient Distributed Training Client Selection in Federated Learning Non-IID Federated Learning Federated Learning

Sep 14, 2026cs.LG

FedLTLib: A Comprehensive Benchmark for Federated Long-Tail Learning

Driven by the escalating demand for privacy-preserving computing, Federated Learning (FL) has witnessed remarkable progress, becoming a cornerstone technology for bridging distributed data silos in mobile edge networks. However, in real-world mobile computing environments, data is generated by heterogeneous mobile devices with varying user behaviors, leading to a significant Long-Tail Distribution. Unlike idealized balanced datasets, data in the wild manifests an acute imbalance where a minority of head classes dominate the sample space while a vast number of tail classes, often representing rare but critical edge-case events, are extremely scarce. This data heterogeneity, which we formally characterize as "Double Heterogeneity", referring to the superposition of global class imbalance and local statistical skew, precipitates severe performance deterioration on tail classes, thereby spurring the vital research direction of Federated Long-Tail Learning (FL-LT). To standardize evaluation and accelerate research in this field, we introduce FedLTLib, a comprehensive benchmark tailored for FL-LT. Addressing the critical issues of inconsistent experimental configurations and unfair comparisons in prior work, FedLTLib establishes a standardized evaluation framework. The platform not only incorporates diverse benchmark datasets reflecting mobile data characteristics but also implements 13 state-of-the-art FL algorithms (4 traditional FL algorithms and 9 FL-LT algorithms). By leveraging FedLTLib, researchers can perform fair and reproducible evaluations of algorithm robustness and generalization capabilities under a unified experimental protocol, ultimately advancing the deployment of robust intelligence in mobile computing ecosystems.

Changkun Lin, Junxiao Wang

Guangzhou University, China

Class-Imbalanced Learning Non-IID Federated Learning Federated Learning

Sep 9, 2026cs.LG

OmniMed-FL: A Robust Multimodal Federated Learning Framework for Clinical Diagnosis

Simultaneous assessment of medical imaging and patient records is often required in clinical diagnosis. However, standard machine learning algorithms cannot analyze these data types together. Meanwhile, compliance with HIPAA and GDPR can constrain centralized aggregation of sensitive patient data. This leaves a crucial void of secure fusion of visual and textual context across distant networks. Thus, we present OmniMed-FL, a controlled systems study of multimodal federated learning for five-class clinical condition classification (Normal, Pneumonia, COVID-19, Pleural Effusion, Cardiomegaly). Our proxy corpus pairs 3,000 public chest radiographs with 3,000 class-conditioned synthetic notes, matched by class, not by patient. The framework benchmarks eight fusion strategies, three initializations, four missing-text imputation rules, and matched federated baselines under non-IID Dirichlet partitioning across 3 to 20 hospital clients. As all notes are synthetic and pairing is not patient-level, these are descriptive proxy comparisons, not estimates of diagnostic performance or deployment readiness. Within those limits with clients (

K=5

) and severe skew (

α=0.1

), local-only training achieves a macro-F1 score of 0.297, FedAvg achieves

0.662\pm0.074

, FedProx

0.737\pm0.085

, a matched FedMME-style one-shot ensemble

0.647\pm0.080

, and our SCAFFOLD-AdamW adaptation

0.070\pm0.015

, the 0.075 FedProx-FedAvg gap falling inside the wider of the two two-seed standard deviations. Over a

4\times3

grid, label skew costs up to 0.27 F1 whereas a near-sevenfold client increase costs at most 0.10, while bidirectional volume grows linearly to 183.5 GiB at

K=20

. Multimodal fusion leads on both corpora, scoring 0.956 against 0.934 for text and 0.664 for images on the synthetic corpus and 0.906 against 0.880 and 0.737 on the radiograph corpus, for

2.3\times

the model state of text alone.

Ayush Debnath, Ruelia Saha, Sudip Misra

Indian Institute of Technology Kharagpur, India · KTH Royal Institute of Technology, Sweden · SRM University-AP, India +1

Multimodal Federated Learning Medical Diagnosis Non-IID Federated Learning Multimodal Classification Multimodal Fusion

Sep 7, 2026cs.LG

FedRAW: Preserving Rare-Label Influence in Asynchronous Federated Learning

Asynchronous federated learning improves scalability by updating the global model from a server-side buffer of client updates as they arrive, rather than waiting for all selected clients to finish. While efficient, this arrival-driven aggregation can silently distort representation learning under heterogeneous participation. We identify silent rarity failure, a hidden failure mode in which clients holding rare labels contribute too weakly to the global model even though its overall accuracy appears largely unaffected. This failure arises from two coupled effects: rare-label clients may submit updates less frequently when they are slower or less available, creating participation bias; and once their updates enter the buffer, standard asynchronous aggregation assigns them no compensating influence, creating aggregation bias. We propose FedRAW, a fully server-side aggregation method that preserves rare-label influence without changing local training, client objectives, or communication protocols. FedRAW combines client-level update deduplication, which prevents frequently arriving clients from repeatedly dominating the update buffer, with rare-label-aware weighting, which increases the influence of clients carrying low-coverage labels. We formalize silent rarity failure through participation and aggregation bias, and show that FedRAW increases rare-label client influence over uniform aggregation while preserving convergence. Across EMNIST Balanced, CIFAR-10, HAM10000, and ISIC-2019, FedRAW improves rarelabel accuracy while preserving comparable global accuracy and adding negligible server-side computation.

Prashant Bajpai, Divya Saxena, Philippe Lalanda +1

School of Artificial Intelligence and Data Science, Indian Institute of Technology Jodhpur, India · Department of Computer Science and Engineering, Indian Institute of Technology Jodhpur, India · Laboratoire d’Informatique de Grenoble (LIG), University of Grenoble Alpes, Grenoble, France

Federated Learning Aggregation Asynchronous Federated Learning Non-IID Federated Learning Federated Learning

Sep 2, 2026cs.LG

Similarity-Aware Personalized Federated Learning in Heterogeneous Environments

Federated Learning (FL) allows decentralized clients to train models collaboratively while preserving data privacy. However, distribution mismatch across clients often leads to poor global generalization and degraded local client-level performance. In such scenarios, some of the clients with their local models trained solely on local data may perform better than the globally learnt model, thus nullifying the benefits of collaborative federated learning. To address this, we propose SAPE-FL (Similarity-Aware Personalized Federated Learning), a novel personalization framework that anchors each client's model to both the global model and a similarity-weighted peer averaged model. By incorporating dynamic, client-specific regularization based on both model similarity and output similarity, SAPE-FL adaptively balances global knowledge transfer and peer collaboration while filtering out dissimilar clients. This dual anchoring mitigates negative transfer and enhances robustness in heterogeneous settings. We theoretically analyze our algorithm establishing its convergence guarantees and empirically show that SAPE-FL outperforms state-of-the-art methods under high statistical heterogeneity and low client data regimes.

Arun Kumar A, Sunil Gupta, Dang Ngyuen +2

Applied Artificial Intelligence Initiative (A2I2) Deakin University, Australia

Federated Learning Non-IID Federated Learning Negative Transfer

Sep 1, 2026cs.LG

D-FROST: Decentralized Federated pRompt-tuning via Optimal tranSporT for Non-IID and Imbalanced Data

Prompt tuning provides a parameter-efficient way to adapt foundation models (FMs) by freezing the pretrained backbone and updating only a small set of learnable prompts. This property makes prompt tuning especially suitable for decentralized federated learning (DFL), where exchanging full-model updates can be prohibitively expensive. However, prompt tuning in DFL introduces new challenges. Prompt sets learned from heterogeneous local data may not be index-wise aligned, making standard decentralized averaging unsuitable. In addition, the algorithm should be theoretically guaranteed to achieve consensus and make progress toward the shared objective. In this work, we provide the first study of prompt tuning in DFL. We formulate decentralized prompt tuning as a Wasserstein-based optimization problem over prompt measures, which captures the set-valued structure of prompts. We then propose D-FROST, an optimal-transport-based (OT-based) decentralized prompt-tuning algorithm that merges neighborhood prompts into compact representative prompt sets through transportation-based matching. We further analyze D-FROST by bounding the Wasserstein consensus error across clients, and establishing convergence of the network-level prompt barycenter to a neighborhood of stationarity. Experiments under heterogeneous client data demonstrate the effectiveness of D-FROST for decentralized prompt tuning.

Quan Minh Nguyen, Hoang M. Ngo, Trong Nghia Hoang +1

University of Florida, FL, USA · Washington State University, WA, USA

Non-IID Federated Learning Decentralized Learning

Sep 1, 2026cs.LG

Breaking the Structural Identity: Personalized Federated LoRA Fine-tuning under Rank Heterogeneity

Large Language Models (LLMs) have achieved remarkable success across diverse domains, but their adaptation to privacy-sensitive, distributed datasets remains a challenge. While Federated Learning (FL) combined with Low-Rank Adaptation (LoRA) provides a resource-efficient paradigm for collaborative fine-tuning, practical deployments are hindered by the dual challenges of resource heterogeneity and data heterogeneity. Existing rank-heterogeneous methods primarily focus on bridging dimension mismatches for aggregation but typically provide a unified global model for all clients sharing the same rank, failing to capture client-specific features in non-IID scenarios. In this paper, we propose FedRoRA (Federated Rank-wise Personalized LoRA), a novel framework that enables fine-grained personalization within rank-heterogeneous federations. FedRoRA decouples adaptation into shared global directions and personalized rank-wise magnitudes governed by learnable diagonal scales. On the server side, it extracts a global subspace via singular value decomposition (SVD) and redistributes client-specific initializations through a personalized projection and top-

k

selection mechanism. Extensive experiments on NLU and NLG benchmarks demonstrate that FedRoRA consistently outperforms state-of-the-art methods.

Lei Wang, Jieming Bian, Letian Zhang +1

University of Florida · Gainesville, FL 32611 · Middle Tennessee State University +1

Non-IID Federated Learning Low-Rank Adaptation Federated Fine-Tuning Federated Learning Heterogeneous Federated Learning

Aug 31, 2026cs.LG

CRAD: Class-wise Reliability-Aware Distillation for Decentralized Heterogeneous Federated Learning

Conventional federated learning (FL) relies on parameter averaging, which forces clients to be doubly homogeneous: it demands an identical architecture and degrades under non-IID data. Real-world deployments usually break both assumptions. We sidestep both by building a decentralized knowledge distillation framework in which each client evaluates its peers' model snapshots on its own local data and distills from the resulting soft predictions. Because knowledge is transferred through the shared class posterior, clients are free to run different architectures; and because every teacher is evaluated on the student's own device, raw data never leaves the client, with no central server or public dataset required. Within this setting, we identify and address an under-examined problem: how to combine the peer teacher predictions. Existing methods, like uniform averaging, ignore how knowledge reliability varies across teachers and classes. We propose Class-wise Reliability-Aware Distillation (CRAD), which, per class, first discards teachers that disagree with the peer consensus and then takes a weighted average of the rest, weighting each teacher by its per-class reliability (precision, or inverse variance). Since the variance of an accuracy from

n

samples scales as

1/n

, support enters automatically: among the teachers that survive filtering, a teacher is trusted for a class to the degree that it is both accurate and well-evidenced for it. On three image-classification benchmarks (CIFAR-10, CIFAR-100, and PathMNIST colon pathology), across heterogeneous architectures under severe non-IID skew, CRAD consistently outperforms competing methods in global accuracy.

Baraa Bilbeisi, Mengchen Fan, Baocheng Geng +1

Dept. of Electrical & Computer Engineering · Dept. of Computer Science

Federated Learning Aggregation Federated Knowledge Distillation Non-IID Federated Learning Cross-Architecture Knowledge Distillation Knowledge Distillation Federated Learning

Aug 13, 2026cs.LG

Federated Compositional Muon Optimizer for Matrix-Wise Models

Muon, a more recently developed optimizer, is useful for matrix-wise models in AI areas. Although many works have studied Muon and its variants, these methods are still not particularly well-suited for hierarchical structured problems. To fill this gap, we propose an effective federated compositional Muon (FedCoMuon) optimizer to solve distributed matrix-wise compositional optimization problems. Specifically, our FedCoMuon optimizer builds on compositional gradient tracking and orthogonalized momentum. Moreover, we propose a variance reduced variant of FedCoMuon (FedCoMuon-VR) based on a momentum-based variance reduced technique. In theory, we analyze the convergence properties of our algorithms under the non-i.i.d. and non-convex settings. In particular, we prove that our FedCoMuon-VR obtains a lower sample complexity of

O(ε^{-3})

for finding an

ε

-stationary solution than the existing FedMuon algorithms. Extensive numerical experiments on robust federated learning and task-distributed risk-sensitive meta learning show that our proposed methods are competitive with existing compositional baselines and achieve the best reported accuracy in several settings.

Wang Yan, Feihu Huang

College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics, Nanjing, China · MIIT Key Laboratory of Pattern Analysis and Machine Intelligence, Nanjing, China

Muon Optimizer Non-IID Federated Learning Federated Learning

Aug 10, 2026cs.LG

FedOrbit: Adaptive Personalized Federated Learning for Non-IID LEO Satellite Constellations

Federated learning (FL) in Low Earth Orbit (LEO) satellite constellations is affected by non-IID data and irregular ground-station visibility, both driven by orbital geometry. Global aggregation performs poorly when orbit-level class distributions are disjoint, while strong personalisation can be excessive when these distributions overlap. We present FedOrbit, which combines continuous orbit-level training over inter-satellite links, class-aware hierarchical aggregation, quality-weighted feature aggregation with return-rate dampening, and adaptive feature decomposition based on inter-orbit class similarity. Across three remote-sensing benchmarks and two non-IID partitions, FedOrbit achieves the highest accuracy in five of six settings and is within

0.9

percentage points of the best result in the sixth. The gains over the strongest baseline reach

16.1

percentage points under Dirichlet partitioning and

8.6

under pathological partitioning, with the smallest per-orbit accuracy spread in five of six settings.

Satwat Bashir, Tasos Dagiuklas, Muddesar Iqbal

School of Engineering London South Bank University, London, UK

Hierarchical Federated Learning Federated Learning Aggregation Non-IID Federated Learning LEO Satellite Networks Federated Learning

Aug 10, 2026cs.LG

FedTVD: Balancing Data Quality and Quantity for Robust Federated Learning

Federated Learning (FL) enables collaborative model training across distributed client devices while preserving data privacy. However, FL faces significant challenges due to data heterogeneity, particularly in terms of label distribution skewness and variations in dataset sizes, which can lead to biased model updates and hinder convergence. To address this, we propose FedTVD, a novel FL algorithm that weights client contributions during aggregation by considering both data quality and quantity. Unlike traditional FL approaches such as FedAvg, which rely solely on dataset size for client weighting, FedTVD integrates Total Variation Distance (TVD) to measure the divergence between each client's local label distribution and a uniform global distribution. Clients with highly skewed distributions receive lower weights, preventing unbalanced datasets with imbalances from disproportionately influencing the global model. At the same time, dataset size is incorporated to ensure scalability and fairness. This dual-weighting mechanism effectively mitigates the impact of data imbalance, leading to more stable and generalized global models. Experimental results show that FedTVD consistently outperforms state-of-the-art methods across all datasets (FMNIST, CIFAR-10, and CIFAR-100) and all levels of data heterogeneity. Notably, it achieves up to 10.6% improvement over FedAvg on CIFAR-10 under highly skewed data, while maintaining top performance even under moderate and IID settings.

Radwan Selo, Majid Kundroo, Taehong Kim

School of Information and Communication Engineering, Chungbuk National University, Cheongju, 28644, Republic of Korea

Class-Imbalanced Learning Non-IID Federated Learning Federated Learning

Aug 10, 2026cs.LG

FedA2L: Adaptive layer-wise learning rate adjustment in decentralized federated learning

Decentralized intelligence systems with heterogeneous devices and limited coordination increasingly rely on decentralized federated learning (DFL). However, DFL suffers from convergence inefficiency under data heterogeneity due to the use of a uniform learning rate (LR) that ignores layer-specific optimization needs. Foundational layers are responsible for maintaining network consensus, while specialized layers adapt to local data characteristics, leading to conflicting gradients and degraded performance under non-IID conditions. To address this fundamental tension, this work introduces FedA2L, a method that dynamically adjusts layer-wise LRs based on model divergence signals. By leveraging local update intensity and network consensus constraints, FedA2L seamlessly integrates into existing DFL protocols without additional communication or coordination. Extensive evaluations across DFL algorithms, various model architectures, and datasets demonstrate that FedA2L achieves up to 4.94 times faster convergence than vanilla DFL and reduces communication rounds by up to 59% compared to scheduler-based baselines. Furthermore, FedA2L exhibits resilience to severe data heterogeneity, larger network sizes, and sparse topologies, reducing communication overhead and establishing it as a versatile optimization tool for resource-constrained or large-scale distributed learning in edge and IoT deployments. The code is released at https://github.com/nclabteam/FedA2L.

Van Truong Vo, Khoa Nguyen, Taehong Kim

School of Information and Communication Engineering, Chungbuk National University, Cheongju, South Korea

Non-IID Federated Learning Federated Learning Layer-Wise Learning Rate Adaptation

Aug 7, 2026cs.LG

Capacity Confounds and Coverage Guarantees in Adaptive Sub-model Federated Learning

Sub-model federated learning lets resource-constrained clients train width-reduced versions of a global model, but existing methods allocate capacity by device resources alone. A natural next step, allocating capacity by each client's data heterogeneity as estimated from the updates the server already observes, is suggested by recent methods that size sub-models from training-derived signals. We ask whether that step is possible, using HAS-FL, an adaptive capacity-allocation framework, as a test case. First, validated against ground-truth label-distribution divergence on reproducible partitions, update-divergence estimates of client heterogeneity are dominated by capacity rather than data: on both image benchmarks and every seed, the estimates correlate strongly and negatively with device capacity, and once capacity is controlled for their association with data heterogeneity is near zero or negative. Any method estimating client statistics from sub-model updates is exposed to this previously undocumented confound. Second, adaptive allocation has a hidden failure mode: when every client is capped below full width, the uncovered parameters stay at random initialization and progressively corrupt the global model. A simple coverage guarantee removes the failure and explains why uniform allocation collapses. Third, a matched-budget control settles what adaptivity contributes: random allocation to the same average budget matches the adaptive policy to within seed-to-seed variation on both image benchmarks, and on the naturally partitioned text benchmark the adaptive policy is the weakest of the three strategies while consuming the most capacity. Sub-model training admits constrained clients at quadratically reduced cost but gives up substantial accuracy relative to full-model training, and what protects that accuracy is parameter coverage rather than allocation intelligence.

Alireza Moayedikia

Department of Business Technology and Entrepreneurship Swinburne University of Technology Melbourne, Australia

Non-IID Federated Learning Federated Learning Heterogeneous Federated Learning

Aug 7, 2026cs.AI

FedLBW: A Loss-Based Weighting Strategy for Federated Learning on Non-IID Data in Wireless Networks

Federated Learning (FL) enables collaborative machine learning (ML) across distributed clients while preserving privacy. However, efficient model convergence in FL remains challenging, especially in wireless networks where non-independent and identically distributed (non-IID) data and frequent client dropouts are common. Traditional FL algorithms, such as FedAvg, rely solely on dataset size to weight client updates. This introduces biases towards clients with larger datasets and makes the process sensitive to non-IID data, outliers, and client dropouts. To address these challenges, we propose Federated Learning with Loss-Based Weighting (FedLBW), a novel aggregation method that assigns each client's update a weight proportional to the inverse of its validation loss, computed using a small proxy dataset on the server, rather than its dataset size. This ensures that lower-loss models exert greater influence during aggregation, prioritizing the most reliable updates and boosting overall performance. Through extensive experiments across multiple datasets, including FashionMNIST (CNN), CIFAR-10 (ResNet-18), and CIFAR-100 (ResNet-34), we demonstrate that FedLBW achieves higher accuracy and faster convergence compared to baseline algorithms such as FedAvg, FedAvgM, FedProx, FedNova, FedLAW and FedDkw, with notable improvements of up to 7.6 % higher accuracy on CIFAR-10 in extreme non-IID cases. Moreover, FedLBW showcases exceptional resilience to increasing dropout probabilities, consistently maintaining significantly higher accuracy even in challenging conditions. These results establish FedLBW as an effective and resilient solution for FL in wireless network environments, offering marked improvements in model accuracy, convergence speed, and robustness to non-IID data and client dropouts.

Majid Kundroo, Tinku Singh, Taehong Kim

aSchool of Information and Communication Engineering, Chungbuk National University, Cheongju, 28644, Republic of Korea · bSchool of Computer Science Engineering and Technology, Bennett University, Greater Noida, 201310, India

Federated Learning Aggregation Non-IID Federated Learning Adaptive Loss Weighting Federated Learning

Aug 7, 2026cs.CV

FedVAR: Prototype-Aligned Federated Framework for Video Anomaly Recognition

In the era of Industrial Internet of Things (IIoT) and Cyber-Physical Systems (CPS), Federated Learning (FL) offers a promising decentralized intelligence paradigm for Video Anomaly Recognition (VAR). This task is vital for maintaining high-fidelity Digital Twins and ensuring safety in mission-critical environments. However, the inherent data heterogeneity across distributed edge clients leads to a fundamental challenge known as semantic misalignment, where clients learn divergent feature representations of "normal" and "abnormal" events. The problem becomes particularly pronounced in VAR, where the presence of diverse and fine-grained anomaly categories leads each client to develop distinct semantic interpretations of abnormality. Existing federated methods primarily focus on binary anomaly detection and fail to address this misalignment, preventing effective fine-grained recognition. In this paper, we introduce FedVAR, a weakly-supervised FL framework explicitly designed for VAR. Leveraging the rich representations of Vision-Language Models (VLMs), FedVAR employs a prototype-based alignment mechanism that creates a shared semantic anchor for all clients to re-center and align their visual and textual feature spaces. This process enforces a consistent representation of "normality" across the decentralized network, directly mitigating semantic misalignment and enabling robust prompt-learning of anomaly direction vectors with minimal communication overhead. We conduct extensive experiments on challenging benchmarks under various non-IID data partitioning schemes, unseen domains, and novel anomaly classes. The results demonstrate that FedVAR consistently outperforms state-of-the-art federated baselines, establishing a robust framework for distributed intelligence in video-based CPS.

Ghani Haider, Majid Kundroo, Boyun Eom +3

School of Information and Communication Engineering, Chungbuk National University, Cheongju, 28644, Republic of Korea · Electronics and Telecommunications Research Institute (ETRI), Daejeon, 34129, Republic of Korea · Institute of Artificial Intelligence (IAI), University of Central Florida, Orlando, 32816, USA

Prototype-Based Learning Non-IID Federated Learning Video Anomaly Detection Weakly Supervised Video Anomaly Detection

Aug 5, 2026cs.CV

On the Effectiveness of Adaptation Strategies for VLM-Based Federated Learning in Remote Sensing

Federated learning (FL) enables collaborative training of deep learning models across decentralized image archives without requiring data centralization. This paradigm is particularly relevant in remote sensing (RS), where legal regulations, privacy concerns, and bandwidth constraints restrict data sharing. However, the presence of training data heterogeneity across clients (known as non-IID data) can impede convergence and limit the generalization capability of the aggregated global model. To mitigate the adverse effects of training data heterogeneity, vision-language models (VLMs) can be leveraged in FL due to their transferable representations, which have demonstrated robustness under distribution shifts. However, their large parameter size may substantially increase communication overhead and local computational complexity in federated settings. Therefore, it is crucial to select an appropriate VLM adaptation strategy that balances the generalization ability with the communication and computational constraints. To address this issue, in this paper, we present the first comparative study of VLM adaptation strategies for FL in the context of RS image classification. We investigate full fine-tuning, encoder-specific fine-tuning, prompt learning, and low-rank adaptation (LoRA) tuning, and analyze them with respect to three criteria: 1) generalization capability under non-IID data, 2) communication overhead, and 3) local computational complexity. Experiments on BigEarthNet-S2, EuroSAT, RESISC45, and ImageNet reveal distinct trade-offs between task specialization, cross-domain generalization, and efficiency. Based on our findings, we derive a guideline for the selection of an appropriate VLM adaptation strategy in FL for RS image classification under different operational constraints. The code of this work is publicly available at https://git.tu-berlin.de/rsim/FL-RS-VLM.

Simon Lösche, Barış Büyüktaş, Mathis Adler +3

BIFOLD - Berlin Institute for the Foundations of Learning and Data, 10587 Berlin, Germany · Technische Universität Berlin, 10623 Berlin, Germany · Orion Lab, School of Rural, Surveying and Geoinformatics Engineering, National Technical University of Athens, 15772 Athens, Greece +2

Remote Sensing Image Understanding Prompt Learning VLM Adaptation Non-IID Federated Learning Low-Rank Adaptation Federated Fine-Tuning

Aug 5, 2026cs.LG

Attention, Anomalies! Handling Attention Layers in Unsupervised Federated Outlier Detection

Attention layers are the backbone of today's most powerful and impactful models. Models with multi-million and billion parameters rely on contextual knowledge provided by attention layers. However, their use goes well beyond just being the core component of large language models. One particularly interesting application is in Memory Augmented Autoencoders (MemAE), specifically for unsupervised representation learning in outlier detection tasks. It was shown that attention helps these models be more effective in centralized learning scenarios. Our work aims to address the lack of specialized aggregation techniques in Federated Learning (FL) when it comes to MemAE models. In this paper we analyze the intricacies of the architecture behind Memory Augmented Autoencoders, and propose novel, guided approaches to effectively aggregate these models in federated scenarios. We demonstrate our approach on non-IID datasets and show that these novel aggregation schemes are more robust when dealing with numerous edge nodes in environments with unbalanced datasets, specifically for unsupervised anomaly detection scenarios. This approach improves the performance of even very shallow autoencoders, allowing them to be used in resource constrained environments.

Mihailo Ilić, Miloš Savić, Vladimir Kurbalija +3

Department of Mathematics and Informatics, Faculty of Sciences, University of Novi Sad, Serbia · Department of Informatics, Modeling, Electronics, and Systems, University of Calabria, Italy

Memory-Augmented Neural Networks Federated Learning Aggregation Non-IID Federated Learning Unsupervised Anomaly Detection Attention Mechanisms Federated Learning+1

Aug 4, 2026cs.CR

FBID: Adaptive Personalized Federated Learning for Robust Out-of-Distribution Attack Detection in IoT Networks

Personalized Federated Learning (PFL) has emerged as a promising solution for intrusion detection in heterogeneous IoT environments, as it can improve local adaptation under highly Non-Independent and Identically Distributed (non-IID) data distributions. However, existing PFL methods often rely on client-side self-adjustment, which may lead to over-personalization and substantial degradation in out-of-distribution (OOD) attack detection. In this paper, we propose Federated Bandit Intrusion Detection (FBID), a novel adaptive PFL framework to address this limitation through server-side personalization control. In particular, FBID employs a contextual multi-armed bandit at the server to dynamically regulate each client's local training intensity according to its observed behavior and update quality. Moreover, FBID introduces a trust-based blending mechanism to derive client-specific interpolation coefficients between the global and local models, thereby preserving global attack-detection knowledge while still allowing beneficial local specialization. Through extensive experiments on the CICIoT2023 dataset under heterogeneous client distributions and OOD stress-test settings, we show that FBID improves individual client OOD Detection Rate (DR) by up to 7.66% and F1-Score (F1) by up to 5.08% (relative) over the strongest stable baseline, while also improving robustness to previously unseen attack classes.

An Khanh Bui, Cong Thanh Nguyen, Hoang-Anh Pham +2

UTS-HCMUT JTIRC, Ho Chi Minh City University of Technology (HCMUT), Ho Chi Minh City 700000, Vietnam. · Vietnam National University Ho Chi Minh City (VNU-HCM), Ho Chi Minh City 700000, Vietnam. · School of Electrical and Data Engineering, University of Technology Sydney, Sydney, NSW 2007, Australia.

Network Intrusion Detection Internet of Things Non-IID Federated Learning OOD Detection Federated Learning

Aug 4, 2026cs.LG

Robust and Personalized Federated Learning for Aircraft-Engine Prognostics under Benign and Adversarial Client Heterogeneity

Federated learning (FL) enables aircraft fleet operators to jointly train remaining-useful-life (RUL) models from engine sensor telemetry without sharing raw data. This study examines two complementary challenges: benign heterogeneity, where honest operators observe different operating conditions and fault modes, and adversarial heterogeneity, where compromised operators submit poisoned updates. We conduct a controlled, safety-oriented evaluation using a multi-task one-dimensional convolutional neural network and a structurally non-IID partition of the Commercial Modular Aero-Propulsion System Simulation (C-MAPSS) benchmark. We compare four remedies for benign heterogeneity and evaluate five attacks against four aggregation methods, including a physically motivated sensor-value backdoor designed to mask engine degradation. Shared-representation personalization closes approximately 70% of the local-to-centralized root-mean-square-error gap, compared with 21% for proximal regularization and 10% for server-side reweighting. The backdoor achieves a 94.9% attack success rate against standard averaging while leaving clean accuracy statistically unchanged, demonstrating that accuracy alone cannot certify model safety and that attack success must be evaluated explicitly. Krum reduces attack success by an order of magnitude and is the only evaluated aggregator that withstands coordinated attackers, whereas personalization alone provides no protection. Combining personalization with robust aggregation restores robustness (2.8% attack success) with only a small accuracy cost, revealing a trade-off between robust update selection and collaborative representation learning. Results remain consistent across client counts and on a harder six-condition dataset. Code and data partitions are released for reproducibility.

Chinmoy Mitra, Md. Mehedi Hasan Nipu, Mohammad Sakib Mahmood +2

Department of Computer Science and Engineering, Rajshahi University of Engineering & Technology, Kazla, Rajshahi, 6204, Bangladesh · Department of Electrical and Computer Engineering, North South University, Dhaka 1229, Bangladesh. · Department of Computer Science, Missouri State University, Springfield, MO, 65809, USA. +2

Federated Learning Aggregation RUL Estimation Byzantine-Robust Federated Learning Backdoor Attacks in Federated Learning Non-IID Federated Learning Federated Learning