Jellyfish: Zero-Shot Federated Unlearning Scheme with Knowledge Disentanglement
Authors: Houzhe Wang, Xiaojie Zhu, Chi Chen
Organizations: Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China · King Abdullah University of Science and Technology, Thuwal, Kingdom of Saudi Arabia
With the increasing importance of data privacy and security, federated unlearning emerges as a new research field dedicated to ensuring that once specific data is deleted, federated learning models no longer retain or disclose related information. In this paper, we propose a zero-shot federated unlearning scheme, named Jellyfish. It distinguishes itself from conventional federated unlearning frameworks in four key aspects: synthetic data generation, knowledge disentanglement, loss function design, and model repair. To preserve the privacy of forgotten data, we design a zero-shot unlearning mechanism that generates error-minimization noise as proxy data for the data to be forgotten. To maintain model utility, we first propose a knowledge disentanglement mechanism that regularises the output of the final convolutional layer by restricting the number of activated channels for the data to be forgotten and encouraging activation sparsity. Next, we construct a comprehensive loss function that incorporates multiple components, including hard loss, confusion loss, distillation loss, model weight drift loss, gradient harmonization, and gradient masking, to effectively align the learning trajectories of the objectives of forgetting" and retaining". Finally, we propose a zero-shot repair mechanism that leverages proxy data to restore model accuracy within acceptable bounds without accessing users' local data. To evaluate the performance of the proposed zero-shot federated unlearning scheme, we conducted comprehensive experiments across diverse settings. The results validate the effectiveness and robustness of the scheme.
Figures & tables
Figure 1: The motivation for knowledge disentanglement. The upper diagram illustrates the phenomenon of inter-class feature mixing during the feature extraction process in conventional models. This mixing leads to the simultaneous activation of features for cats and dogs in the activation maps (as shown in the red areas), affecting the accurate representation of single-class features. The lower diagram demonstrates the effect after applying knowledge disentanglement, where the model learns to represent features with less mixing. The activation maps show clearer class specificity (blue is primarily related to cats, and yellow to dogs). This disentanglement reduces activation overlap between different classes, enhancing the model’s interpretability.
Notations
Description
Nclient
Number of clients
c
Client index
D
Complete dataset
Df
forgetting dataset
Dr
remaining dataset
Dc
Local dataset
Table 1: Summary of Notations
Figure 2: Proposed Jellyfish Scheme: ① Noise Training: The user requests data deletion locally and trains proxy data Nf to replace the deleted data. ② Knowledge Disentanglement: When the server receives the data, it first disentangles the data to reduce knowledge entanglement between categories. ③ Unlearn: The model is guided to forget by using an improved loss function and mechanisms like gradient harmonization. ④ Repair: If the model’s accuracy drops after unlearning, the proxy data of the remaining data Nr is used to restore the model’s performance.
Figure 3: Performance Metrics on Test Set. (a) JSD on Test Set, (b) L2 on Test Set, (c) p-value on Test Set.
Dataset
Dimensions
Classes
Training
Test
MNIST
784
10
60000
10000
CIFAR-10
3072
10
50000
10000
CIFAR-100
3072
100
50000
10000
Table 2: Dataset Description
Figure 4: Comparison of accuracy on remaining and forgotten data using real and proxy data for unlearning. (a) accuracy on remaining data, and (b) accuracy on forgotten data.
Figure 5: Accuracy of non-target categories before and after disentangling.
With the increasing importance of data privacy and security, federated unlearning has emerged as a novel research field dedicated to ensuring that federated learning models no longer retain or leak relevant information once specific data has been deleted. In this paper, to the best of our knowledge, we propose the first complete pipeline for federated unlearning, which includes a federated unlearning approach and an evaluation framework. Our proposed federated unlearning approach ensures high efficiency and model accuracy without the need to store historical data.It effectively leverages the knowledge distillation model alongside various optimization mechanisms. Moreover, we propose a framework named Skyeye to visualize the forgetting capacity of federated unlearning models. It utilizes the federated unlearning model as the classifier integrated into a Generative Adversarial Network (GAN). Afterward, both the classifier and discriminator guide the generator in generating samples. Throughout this process, the generator learns from the classifier's knowledge. The generator then visualizes this knowledge through sample generation. Finally, the model's forgetting capability is evaluated based on the relevance between the deleted data and the generated samples. Comprehensive experiments are conducted to illustrate the effectiveness of the proposed federated unlearning approach and the corresponding evaluation framework.
Houzhe Wang, Xiaojie Zhu, Chi Chen
School of Cyber Security, University of Chinese Academy of Sciences, Beijing, China · Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China · King Abdullah University of Science and Technology, Thuwal, Kingdom of Saudi Arabia
Federated unlearning (FU) is critical for complying with legal mandates like the right to be forgotten in decentralized systems, yet current methods face a persistent dilemma between non-target knowledge loss and high request latency. To resolve these issues, we propose FedUP, a one-shot federated unlearning framework utilizing lightweight pluggable filters that act as a "knowledge funnel" to screen out target data while preserving original model performance. By freezing original model parameters and training filters at the server side using differentially private (DP)-protected class centroid samples, FedUP bypasses the need for multi-round client-server communication and complex retraining, reducing unlearning latency from minutes to mere seconds. Additionally, the framework's pluggable architecture ensures inherent reversibility, enabling the seamless restoration of forgotten knowledge by simply removing the filters. Extensive experiments on diverse image and text tasks demonstrate that FedUP effectively reduces non-target knowledge loss and achieves superior unlearning precision and efficiency across various scenarios. Code is available at: https://github.com/suows/FedUP-code.
Feihong Nan, Zhengyi Zhong, Pan Wang +4
1National Key Laboratory of Big Data and Decision, National University of Defense Technology, China.
Federated learning (FL) increasingly needs machine unlearning to comply with privacy regulations. However, existing federated unlearning approaches may overlook the overlapping information between the unlearning and remaining data, leading to ineffective unlearning and unfairness between clients. In this work, we revisit federated unlearning through the lens of memorization. We argue that unlearning should mainly remove the unique memorized information attributable to the data to be forgotten, while preserving overlapping patterns that are also supported by the remaining data. Specifically, we propose Grouped Memorization Evaluation, an example-level metric that separates memorized knowledge from overlapping knowledge. Building on this metric, we introduce Federated Memorization Pruning (FedMemPrune), a pruning-based unlearning approach that resets redundant parameters responsible for memorization. Extensive experiments show that FedMemPrune closely matches retraining-based unlearning baselines while more effectively eliminating memorization than existing federated unlearning algorithms, yielding strong unlearning performance without sacrificing the utility of retained knowledge.
Jiaheng Wei, Yanjun Zhang, He Zhang +5
Royal Melbourne Institute of Technology · Griffith University · Swinburne University of Technology