Jellyfish: Zero-Shot Federated Unlearning Scheme with Knowledge Disentanglement

Houzhe Wang; Xiaojie Zhu; Chi Chen

cs.CRApr 5, 2026

Jellyfish: Zero-Shot Federated Unlearning Scheme with Knowledge Disentanglement

Authors: Houzhe Wang, Xiaojie Zhu, Chi Chen

Organizations: Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China · King Abdullah University of Science and Technology, Thuwal, Kingdom of Saudi Arabia

Abstract

With the increasing importance of data privacy and security, federated unlearning emerges as a new research field dedicated to ensuring that once specific data is deleted, federated learning models no longer retain or disclose related information. In this paper, we propose a zero-shot federated unlearning scheme, named Jellyfish. It distinguishes itself from conventional federated unlearning frameworks in four key aspects: synthetic data generation, knowledge disentanglement, loss function design, and model repair. To preserve the privacy of forgotten data, we design a zero-shot unlearning mechanism that generates error-minimization noise as proxy data for the data to be forgotten. To maintain model utility, we first propose a knowledge disentanglement mechanism that regularises the output of the final convolutional layer by restricting the number of activated channels for the data to be forgotten and encouraging activation sparsity. Next, we construct a comprehensive loss function that incorporates multiple components, including hard loss, confusion loss, distillation loss, model weight drift loss, gradient harmonization, and gradient masking, to effectively align the learning trajectories of the objectives of forgetting" and retaining". Finally, we propose a zero-shot repair mechanism that leverages proxy data to restore model accuracy within acceptable bounds without accessing users' local data. To evaluate the performance of the proposed zero-shot federated unlearning scheme, we conducted comprehensive experiments across diverse settings. The results validate the effectiveness and robustness of the scheme.

Figures & tables

Figure 1: The motivation for knowledge disentanglement. The upper diagram illustrates the phenomenon of inter-class feature mixing during the feature extraction process in conventional models. This mixing leads to the simultaneous activation of features for cats and dogs in the activation maps (as shown in the red areas), affecting the accurate representation of single-class features. The lower diagram demonstrates the effect after applying knowledge disentanglement, where the model learns to represent features with less mixing. The activation maps show clearer class specificity (blue is primarily related to cats, and yellow to dogs). This disentanglement reduces activation overlap between different classes, enhancing the model’s interpretability.

Notations	Description
$N_{\text{client}}$	Number of clients
$c$	Client index
$D$	Complete dataset
$D_{f}$	forgetting dataset
$D_{r}$	remaining dataset
$D_{c}$	Local dataset

Table 1: Summary of Notations

Figure 2: Proposed Jellyfish Scheme: ① Noise Training: The user requests data deletion locally and trains proxy data $N_{f}$ to replace the deleted data. ② Knowledge Disentanglement: When the server receives the data, it first disentangles the data to reduce knowledge entanglement between categories. ③ Unlearn: The model is guided to forget by using an improved loss function and mechanisms like gradient harmonization. ④ Repair: If the model’s accuracy drops after unlearning, the proxy data of the remaining data $N_{r}$ is used to restore the model’s performance.

Panel of Figure 3 — Figure 3: Performance Metrics on Test Set. (a) JSD on Test Set, (b) L2 on Test Set, (c) p-value on Test Set.

Dataset	Dimensions	Classes	Training	Test
MNIST	784	10	60000	10000
CIFAR-10	3072	10	50000	10000
CIFAR-100	3072	100	50000	10000

Table 2: Dataset Description

Panel of Figure 4 — Figure 4: Comparison of accuracy on remaining and forgotten data using real and proxy data for unlearning. (a) accuracy on remaining data, and (b) accuracy on forgotten data.

Figure 5: Accuracy of non-target categories before and after disentangling.

Dataset	Data Partition	Before	Unlearned	Retrained
MNIST	$D_{r}$	81.13%	79.40%	78.83%
MNIST	$D_{f}$	80.57%	50.68%	50.10%
CIFAR-10	$D_{r}$	96.60%	83.45%	86.07%
CIFAR-10	$D_{f}$	97.96%	53.28%	50.07%
CIFAR-100	$D_{r}$	89.28%	89.61%	89.22%
CIFAR-100	$D_{f}$	89.34%	52.55%	51.87%

Table 4: Membership Inference Attack Success Rate.

Metric	Method	Dataset
Metric	Method	MNIST	CIFAR-10	CIFAR-100
Average Batch Time (s/epoch)	$B_{1}$	2.64	15.36	23.75
	$B_{7}$	6.91	4.51	5.14
	Ours	3.25	4.49	5.62
Total Time (s)	$B_{1}$	903.27	2379.3	3081.23
	$B_{7}$	15.44	115.39	154.05

Table 5: Computational Efficiency Comparison of Different Unlearning Methods.

Variant	Acc on $D_{r}$	Acc on $D_{f}$	L2	JSD
Without Disentanglement	84.02%	0.00%	0.28344	0.63013
Without Gradient Harmonization	75.20%	0.00%	0.59916	0.63013
Without Gradient Masking	90.21%	81.10%	0.94388	0.69315
Without Hard Loss	97.87%	32.60%	0.93813	0.69315
Without Confusion Loss	85.60%	0.00%	0.22123	0.63013
Without Distillation Loss	87.13%	0.00%	0.17709	0.63013

Table 6: Ablation Study of the Importance of Different Components.

Stage	Accuracy	Accuracy Gap vs. Retrained Model
Before Unlearning	93.90%	+1.64%
Unlearned	85.41%	-6.85%
Repaired	91.88%	-0.38%
Retrained	92.26%	0.00%

Table 7: Evaluation of Repair Mechanism.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 6 — Figure 6: Performance Metrics on Train Set. (a) JSD on Train Set, (b) L2 on Train Set, (c) p-value on Train Set.

Federated Unlearning

Federated unlearning removes a client’s or record’s influence from a collaboratively trained model while preserving useful knowledge.

See all 27 papers

Forgetting to Witness: Efficient Federated Unlearning and Its Visible Evaluation
Apr 6, 2026Houzhe Wang, Xiaojie Zhu, Chi ChenFederated Unlearning Machine Unlearning Evaluation
School of Cyber Security, University of Chinese Academy of Sciences, Beijing, China · Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China · King Abdullah University of Science and Technology, Thuwal, Kingdom of Saudi Arabia
FedUP: One-Shot Federated Unlearning via Centroid-Guided Plug-in Filters
Jun 23, 2026Feihong Nan, Zhengyi Zhong, Pan Wang +4Federated Unlearning Machine Unlearning
1National Key Laboratory of Big Data and Decision, National University of Defense Technology, China.
Rethinking Federated Unlearning via the Lens of Memorization
May 23, 2026Jiaheng Wei, Yanjun Zhang, He Zhang +5Federated Unlearning Neural Network Memorization
Royal Melbourne Institute of Technology · Griffith University · Swinburne University of Technology

Apr 6, 2026J/K move · Enter open · S save

Forgetting to Witness: Efficient Federated Unlearning and Its Visible Evaluation

Houzhe Wang, Xiaojie Zhu, Chi Chen

School of Cyber Security, University of Chinese Academy of Sciences, Beijing, China · Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China · King Abdullah University of Science and Technology, Thuwal, Kingdom of Saudi Arabia

Federated Unlearning Machine Unlearning Evaluation

Open paper AskPDF arXiv

With the increasing importance of data privacy and security, federated unlearning has emerged as a novel research field dedicated to ensuring that federated learning models no longer retain or leak relevant information once specific data has been deleted. In this paper, to the best of our knowledge, we propose the first complete pipeline for federated unlearning, which includes a federated unlearning approach and an evaluation framework. Our proposed federated unlearning approach ensures high efficiency and model accuracy without the need to store historical data.It effectively leverages the knowledge distillation model alongside various optimization mechanisms. Moreover, we propose a framework named Skyeye to visualize the forgetting capacity of federated unlearning models. It utilizes the federated unlearning model as the classifier integrated into a Generative Adversarial Network (GAN). Afterward, both the classifier and discriminator guide the generator in generating samples. Throughout this process, the generator learns from the classifier's knowledge. The generator then visualizes this knowledge through sample generation. Finally, the model's forgetting capability is evaluated based on the relevance between the deleted data and the generated samples. Comprehensive experiments are conducted to illustrate the effectiveness of the proposed federated unlearning approach and the corresponding evaluation framework.

Abstract

Federated Unlearning

Explore similar work

Forgetting to Witness: Efficient Federated Unlearning and Its Visible Evaluation

FedUP: One-Shot Federated Unlearning via Centroid-Guided Plug-in Filters

Rethinking Federated Unlearning via the Lens of Memorization