Enhancing Gradient Inversion Attacks in Federated Learning via Hierarchical Feature Optimization

Hao Fang; Wenbo Yu; Bin Chen; Xuan Wang; Shu-Tao Xia; Qing Liao; Ke Xu

cs.CVApr 1, 2026

Enhancing Gradient Inversion Attacks in Federated Learning via Hierarchical Feature Optimization

Authors: Hao Fang, Wenbo Yu, Bin Chen, Xuan Wang, Shu-Tao Xia, Qing Liao, Ke Xu

Organizations: Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen 518055, Guangdong, China · School of Computer Science and Technology, Harbin Institute of Technology, Shenzhen 518055, Guangdong, China · Department of Computer Science and Technology, Tsinghua University, Beijing 100084, China

Abstract

Federated Learning (FL) has emerged as a compelling paradigm for privacy-preserving distributed machine learning, allowing multiple clients to collaboratively train a global model by transmitting locally computed gradients to a central server without exposing their private data. Nonetheless, recent studies find that the gradients exchanged in the FL system are also vulnerable to privacy leakage, e.g., an attacker can invert shared gradients to reconstruct sensitive data by leveraging pre-trained generative adversarial networks (GAN) as prior knowledge. However, existing attacks simply perform gradient inversion in the latent space of the GAN model, which limits their expression ability and generalizability. To tackle these challenges, we propose Gradient Inversion over Feature Domains (GIFD), which disassembles the GAN model and searches the hierarchical features of the intermediate layers. Instead of optimizing only over the initial latent code, we progressively change the optimized layer, from the initial latent space to intermediate layers closer to the output images. In addition, we design a regularizer to avoid unreal image generation by adding a small ${l_1}$ ball constraint to the searching range. We also extend GIFD to the out-of-distribution (OOD) setting, which weakens the assumption that the training sets of GANs and FL tasks obey the same data distribution. Furthermore, we consider the challenging OOD scenario of label inconsistency and propose a label mapping technique as an effective solution. Extensive experiments demonstrate that our method can achieve pixel-level reconstruction and outperform competitive baselines across a variety of FL scenarios.

Figures & tables

Panel of Figure 1 — Fig. 1 : The reconstructed results of our proposed GIFD on ImageNet [ deng2009imagenet ] and FFHQ [ karras2019style ] . The first column contains the randomly initialized images generated by GAN models. The next two columns show the reconstruction samples of the latent space search and our proposed GIFD.

Fig. 2 : Overview of our proposed GIFD attack. The intermediate layer optimizer minimizes the matching loss computed from the dummy gradients and the shared gradients from the victim client to update the latent vector and the intermediate features successively. The image fidelity regularization helps improve the quality of generated images. The reconstructed image from the layer with the corresponding least gradient matching loss is selected as the final output.

Fig. 3 : An illustration of our proposed label mapping mechanism. We first conduct only a small number of iterations based on the initially inferred label for a coarse reconstruction $\mathbf{\hat{x}}$ . It is then fed into $f_{m}(\cdot)$ for a refined label, which serves as the conditional input for GAN models in subsequent fine-grained attack. This technique corrects the label inconsistency and offers a more aligned guidance in the semantic space, further enhancing the attack effectiveness.

Panel of Figure 4 — Fig. 4 : Comparison of PSNR mean on BigGAN and StyleGAN2 under different values of hyper-parameter $K$ ( i.e. , the last intermediate layer to optimize). Notably, the figures exclude the results where the corresponding values are below the starting point of the y-axis.

Metric	ImageNet					FFHQ
Metric	IG [ geiping2020inverting ]	GI [ yin2021see ]	GGL [ li2022auditing ]	GIAS [ jeon2021gradient ]	GIFD	IG [ geiping2020inverting ]	GI [ yin2021see ]	GGL [ li2022auditing ]	GIAS [ jeon2021gradient ]	GIFD
PSNR $\uparrow$	17.0756	16.5109	13.3885	17.4923	20.0534	15.3523	14.9485	15.1335	20.1799	21.3368
LPIPS $\downarrow$	0.3078	0.3297	0.3678	0.2536	0.1559	0.4172	0.4503	0.2009	0.1266	0.1023
SSIM $\uparrow$	0.2908	0.2673	0.1251	0.3381	0.4713	0.2272	0.2044	0.2453	0.5379	0.5768
MSE $\downarrow$	0.0223	0.0258	0.0553	0.0236	0.0141	0.0311	0.0343	0.0339	0.0121	0.0098

TABLE I : Comparison of GIFD with state-of-the-art methods on every 1000th image of the ImageNet and FFHQ validation set. We calculate the average value of metrics on reconstructed images.

Panel of Figure 6 — Fig. 5 : Qualitative results of different methods on ImageNet and FFHQ.

Datset	Method	Art Painting				Photo				Cartoon
Datset	Method	PSNR $\uparrow$	LPIPS $\downarrow$	SSIM $\uparrow$	MSE $\downarrow$	PSNR $\uparrow$	LPIPS $\downarrow$	SSIM $\uparrow$	MSE $\downarrow$	PSNR $\uparrow$	LPIPS $\downarrow$	SSIM $\uparrow$	MSE $\downarrow$
ImageNet*	IG [ geiping2020inverting ]	18.3476	0.2286	0.3870	0.0172	15.6647	0.3575	0.2409	0.0325	15.8766	0.3183	0.3970	0.0288
	GI [ yin2021see ]	17.4681	0.2625	0.3445	0.0203	15.2700	0.3888	0.2201	0.0346	15.3905	0.3112	0.3926	0.0327
	GGL [ li2022auditing ]	12.8011	0.3639	0.1356	0.0571	12.9246	0.3159	0.1507	0.0667	11.0315	0.3294	0.2832	0.0895
	GIAS [ jeon2021gradient ]	17.2804	0.2774	0.3346	0.0227	20.4539	0.1724	0.4913	0.0111	19.0247	0.1862	0.5740	0.0149
	GIFD	19.3311	0.1700	0.4503	0.0151	21.9281	0.1137	0.5765	0.0082	22.8055	0.1030	0.6970	0.0067

TABLE II : Comparison of GIFD with state-of-the-art baselines on OOD data of different styles.

Panel of Figure 8 — Fig. 6 : Visual comparison of different methods on ImageNet* and FFHQ*.

Method	PSNR $\uparrow$	LPIPS $\downarrow$	SSIM $\uparrow$	MSE $\downarrow$
IG [ geiping2020inverting ]	15.3523	0.4172	0.2272	0.0311
GI [ yin2021see ]	14.9485	0.4503	0.2044	0.0343
GGL [ li2022auditing ]	11.1235	0.4498	0.0995	0.0843
GIAS [ jeon2021gradient ]	17.4565	0.2534	0.3825	0.0201
GIFD	19.9793	0.1597	0.5095	0.0126
GIFD*	20.3664	0.1463	0.5288	0.0114

TABLE III : Comparison of GIFD with SOTA baselines on more challenging OOD scenarios of label inconsistency. We attack the FL model trained on FFHQ using the GAN model learned from ImageNet. Note that GIFD* denotes GIFD plus label mapping technique.

TABLE IV : PSNR mean of different attack methods under different defense strategies.

Fig. 7 : PSNR results of different attacks when changing the values of

q

(the probability of assigning a training example with a certain label to a particular client) on ImageNet. These settings strictly align with the previous works [ wei2025extracting , cao2021fltrust ] considering data heterogeneity.

Panel of Figure 12 — Fig. 8 : Comparison of GIFD with state-of-the-art baselines on ImageNet when increasing the FL training rounds.

TABLE V : PSNR mean of different attack methods under different batch sizes.

Method	CIFAR-10	CIFAR-100	MNIST	KMNIST	SVHN
IG [ geiping2020inverting ]	25.9120	21.0027	26.4194	26.3165	24.8600
GI [ yin2021see ]	26.1532	22.9966	25.7285	27.1744	24.4489
GGL [ li2022auditing ]	12.5193	11.9671	10.0464	8.6165	17.8009
GIAS [ jeon2021gradient ]	27.5338	26.4250	39.5232	28.5140	28.3148
GIFD	28.9481	28.6634	40.4472	34.5892	30.6602

TABLE VI : PSNR mean of different attack methods across different private datasets.

Panel of Figure 15 — Fig. 9 : PSNR results of different methods for different global models on the ImageNet dataset.

TABLE VII : Ablation study of GIFD and its three variants on every 1000th image of ImageNet and FFHQ validation set.

Method	IG [ geiping2020inverting ]	GIAS [ jeon2021gradient ]	GIFD
Time(s)↓	726.8008	1360.7808	907.727
PSNR↑	14.1896	16.6995	18.4018
Loss↓	0.008029	0.007803	0.006865

TABLE VIII : Converge time and PSNR results of GIFD and baselines that use the same distance metric at batch size 4.

Gradient Inversion Attacks

Gradient inversion attacks infer private training examples or labels from gradients shared during distributed model training.

See all 20 papers

Practical Feasibility of Gradient Inversion Attacks in Federated Learning
Aug 27, 2025Viktor Valadi, Lucas Beerens, Mattias Åkesson +4Gradient Inversion Attacks Adversarial Attacks
Scaleout Systems Sweden · Recorded Future Sweden · AI Sweden Sweden +1
No More Guessing: a Verifiable Gradient Inversion Attack in Federated Learning
Apr 16, 2026Francesco Diana, Chuan Xu, André Nusser +1Gradient Inversion Attacks Adversarial Attacks
Université Côte d’Azur, France · Inria, France · CNRS, France +1
Cascading Gradient Inversion via LT-Code Inspired Peeling in Federated Learning
Sep 9, 2026Saeed Shariati, Mohsen Alambardar MeybodiGradient Inversion Attacks Federated Learning
Department of Applied Mathematics and Computer Science, Faculty of Mathematics and Statistics, University of Isfahan, Isfahan 81746-73441, Iran

Aug 27, 2025J/K move · Enter open · S save

Practical Feasibility of Gradient Inversion Attacks in Federated Learning

Viktor Valadi, Lucas Beerens, Mattias Åkesson, Johan Östman+3

Scaleout Systems Sweden · Recorded Future Sweden · AI Sweden Sweden+1

Gradient Inversion Attacks Adversarial Attacks Federated Learning

Open paper AskPDF arXiv

Gradient inversion attacks are often presented as a serious privacy threat in federated learning, with recent work reporting increasingly strong reconstructions under favorable experimental settings. However, it remains unclear whether such attacks are feasible in modern, performance-optimized systems deployed in practice. In this work, we evaluate the practical feasibility of gradient inversion for image-based federated learning. We conduct a systematic study across multiple datasets and tasks, including image classification and object detection, using canonical vision architectures at contemporary resolutions. Our results show that while gradient inversion remains possible for certain legacy or transitional designs under highly restrictive assumptions, modern, performance-optimized models consistently resist meaningful reconstruction visually. We further demonstrate that many reported successes rely on upper-bound settings, such as inference mode operation or architectural simplifications which do not reflect realistic training pipelines. Taken together, our findings indicate that, under an honest-but-curious server assumption, high-fidelity image reconstruction via gradient inversion does not constitute a critical privacy risk in production-optimized federated learning systems, and that practical risk assessments must carefully distinguish diagnostic attack settings from real-world deployments.

Abstract

Gradient Inversion Attacks

Explore similar work

Practical Feasibility of Gradient Inversion Attacks in Federated Learning

No More Guessing: a Verifiable Gradient Inversion Attack in Federated Learning

Cascading Gradient Inversion via LT-Code Inspired Peeling in Federated Learning