Catastrophic Forgetting Resilient One-Shot Incremental Federated Learning

Obaidullah Zaland; Zulfiqar Ahmad Khan; Monowar Bhuyan

cs.LGFeb 19, 2026

Catastrophic Forgetting Resilient One-Shot Incremental Federated Learning

Authors: Obaidullah Zaland, Zulfiqar Ahmad Khan, Monowar Bhuyan

Organizations: Department of Computing Science, Umeå University, Umeå, SE-90187, Sweden

Abstract

Modern big-data systems generate massive, heterogeneous, and geographically dispersed streams that are large-scale and privacy-sensitive, making centralization challenging. While federated learning (FL) provides a privacy-enhancing training mechanism, it assumes a static data flow and learns a collaborative model over multiple rounds, making learning with incremental data challenging in limited-communication scenarios. This paper presents One-Shot Incremental Federated Learning (OSI-FL), the first FL framework that addresses the dual challenges of communication overhead and catastrophic forgetting. OSI-FL communicates category-specific embeddings, devised by a frozen vision-language model (VLM) from each client in a single communication round, which a pre-trained diffusion model at the server uses to synthesize new data similar to the client's data distribution. The synthesized samples are used on the server for training. However, two challenges still persist: i) tasks arriving incrementally need to retrain the global model, and ii) as future tasks arrive, retraining the model introduces catastrophic forgetting. To this end, we augment training with Selective Sample Retention (SSR), which identifies and retains the top-p most informative samples per category and task pair based on sample loss. SSR bounds forgetting by ensuring that representative retained samples are incorporated into training in further iterations. The experimental results indicate that OSI-FL outperforms baselines, including traditional and one-shot FL approaches, in both class-incremental and domain-incremental scenarios across three benchmark datasets.

Figures & tables

Panel of Figure 1 — Fig. 1 : Motivation behind OSI-FL. One-shot FL approaches degrade in performance similar to traditional FL approaches as tasks arrive incrementally, specially when the tasks differ significantly.

Fig. 2 : T-SNE data visualization for multiple tasks, where each task contains a subset of the total classes, in an incremental FL setup (NICO Common dataset). As depicted, the distribution of data varies significantly among tasks, and as these tasks arrive incremetnally, learning the most recent task affects the knowledge about previous tasks (catastrophic forgetting).

Symbol	Explanation
$\mathcal{C}$	Set of participating clients
$\mathcal{T}$	Set of all tasks
$n$	Number of participating clients
$m$	Number of tasks
$\mathbf{D}_{t}$	Data for task $t$
$\mathbf{D}$	Global Dataset

TABLE I : Symbols used

Criteria	OSCAR	OSI-FL
Description Generation	OSCAR primarily utilizes the BLIP OPT pretrained model for caption generation, which is large, weighing around 5 GB .	In contrast, OSI-FL employs the GPT ViT pretrained model for captioning, which is compact, totaling around 0.9 GB .
Data Division	OSCAR mainly use domain-based data division, incorporating feature-skewed non-IID division to simulate a distributed setup.	OSI-FL adds a label-skewed, non-IID setup in which data is divided by label across clients. Although we also provide experimental data for a feature-skewed setup.
Learning Setup	OSCAR assume that all encodings from all clients are available at the start of the experiments, which is impractical in real-world settings.	OSI-FL incorporates a more realistic setup in which clients join the FL setup incrementally.

TABLE II : Comparison between OSCAR and OSI-FL approaches

Fig. 3 : Overview of OSI-FL framework, as new tasks arrive.

Hyperparameter	Value
Backbone	ResNet-18
Optimizer	Adam
Learning rate	0.001
Batch size	256
Global epochs	20
Weight decay	1e-4

TABLE III : Hyperparameters

Model	FL	OS	IL
FedAvg	✔	✗	✗
FedProx	✔	✗	✗
FedDyn	✔	✗	✗
FedET	✔	✗	✔
FedIL+	✔	✗	✔
FedEWC	✔	✗	✔

TABLE IV : Baselines

Model	Client Test Set Accuracy after each task $\uparrow$
	$\mathcal{T}_{1}$	$\mathcal{T}_{2}$	$\mathcal{T}_{3}$	$\mathcal{T}_{4}$	$\mathcal{T}_{5}$	$\mathcal{T}_{6}$
OpenImage
FedAvg	93.69	41.54	33.97	28.94	24.75	25.22
FedProx	93.69	41.54	33.98	29.64	24.73	25.22
FedEWC	93.69	41.64	34.22	29.02	25.02	25.19
FedET	95.22	44.52	39.94	34.14	33.61	32.27

TABLE V : Accuracy (in %) on the test set for the baselines and our approach on class incremental settings after each task. Ceiling performance in italic, best performance excluding ceiling in bold.

Model	Client Test Set Accuracy after each task $\uparrow$
	$\mathcal{T}_{1}$	$\mathcal{T}_{2}$	$\mathcal{T}_{3}$	$\mathcal{T}_{4}$	$\mathcal{T}_{5}$	$\mathcal{T}_{6}$
OpenImage
FedAvg	64.38	52.64	53.08	58.44	58.68	57.14
FedProx	64.38	52.55	53.62	53.94	57.25	57.20
FedEWC	64.38	52.72	53.42	59.66	59.22	58.04
OSCAR-IL	52.74	47.40	48.72	45.66	46.67	45.78

TABLE VI : Accuracy (in %) on the test set for the baselines and our approach on domain incremental settings after each task. Ceiling performance in italic, best performance excluding ceiling in bold

Fig. 4 : Impact of

p

, the number of retained samples, on the overall performance in OSI-FL.

Fig. 5 : Class incremental accuracy with varying number of clients

Fig. 6 : Peak GPU and CPU memory usage (in GB), and training cost (in TFLOPS) for OSI-FL and baselines (NICO Common dataset and ResNet-18 backbone).

Fig. 7 : Client upload cost in OSI-FL and baselines compared to final performance on class-incremental OpenImage dataset and ResNet-18 backbone (in million parameters, log scaled).

Federated Learning

Federated learning trains shared models across data holders without requiring them to pool their raw data.

See all 951 papers

Fisher-Routed Mixture of Experts for Federated Class-Incremental Learning
Jun 27, 2026Wenhao Yuan, Chenchen Lin, Jian Chen +3Class-Incremental Learning Federated Continual Learning
Department of Electrical and Computer Engineering, The University of Hong Kong, Hong Kong SAR, China · School of Artificial Intelligence, Sun Yat-sen University, China.
ReSCENE: Server-Side Replay for Structural Mitigation of Catastrophic Forgetting in Federated Continual Learning
Sep 30, 2026Sungmin Kang, Zhengzhong Tu, Sunwoo LeeFederated Continual Learning Replay-Based Continual Learning
Texas A&M University · Inha University
Federated Class-Incremental Learning with Hierarchical Generative Prototypes
Jun 4, 2024Riccardo Salami, Pietro Buzzega, Matteo Mosconi +2Class-Incremental Learning Prototype-Based Classification
AImageLab - University of Modena and Reggio Emilia, Modena, Italy

Jun 27, 2026J/K move · Enter open · S save

Fisher-Routed Mixture of Experts for Federated Class-Incremental Learning

Wenhao Yuan, Chenchen Lin, Jian Chen, Jinfeng Xu+2

Department of Electrical and Computer Engineering, The University of Hong Kong, Hong Kong SAR, China · School of Artificial Intelligence, Sun Yat-sen University, China.

Class-Incremental Learning Federated Continual Learning Non-IID Federated Learning Incremental Learning Mixture-of-Experts Models Federated Learning

Open paper AskPDF arXiv

Federated Learning (FL) emerged as a promising distributed machine learning paradigm. However, extending FL to the class incremental learning scenarios introduces unique challenges: 1) Capacity conflict and catastrophic forgetting from the shared model overloading, 2) Heterogeneity from Non-Independent and Identically Distributed (Non-IID) data, and 3) Synchronized class misalignment. In this paper, we propose Fisher-Routed MiXture of Experts for Federated Class-Incremental Learning (FedFMX), a novel framework to address these challenges via adaptive expert specialization across clients. The crucial insight is to route each sample to an expert subset that jointly optimizes knowledge acquisition and retention. Specifically, we introduce a Fisher-Routed Expert Scoring (FRES) module to estimate expert importance via Fisher-based stability cost and gradient-based plasticity gain. Then, we design an Adaptive Expert Selection (AES) module by quantifying marginal contributions for adaptive expert subset determination. Finally, by the routing-aware regularization (RAR), we achieve load balance and efficient FL training. We theoretically prove the

\mathcal{O}(T^{-1})

convergence rate. Extensive experiments on multiple benchmarks compared with state-of-the-art methods demonstrate the superiority of FedFMX.

Abstract

Federated Learning

Explore similar work

Fisher-Routed Mixture of Experts for Federated Class-Incremental Learning

ReSCENE: Server-Side Replay for Structural Mitigation of Catastrophic Forgetting in Federated Continual Learning

Federated Class-Incremental Learning with Hierarchical Generative Prototypes