Enhancing Gradient Inversion Attacks in Federated Learning via Hierarchical Feature Optimization
Authors: Hao Fang, Wenbo Yu, Bin Chen, Xuan Wang, Shu-Tao Xia, Qing Liao, Ke Xu
Organizations: Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen 518055, Guangdong, China · School of Computer Science and Technology, Harbin Institute of Technology, Shenzhen 518055, Guangdong, China · Department of Computer Science and Technology, Tsinghua University, Beijing 100084, China
Federated Learning (FL) has emerged as a compelling paradigm for privacy-preserving distributed machine learning, allowing multiple clients to collaboratively train a global model by transmitting locally computed gradients to a central server without exposing their private data. Nonetheless, recent studies find that the gradients exchanged in the FL system are also vulnerable to privacy leakage, e.g., an attacker can invert shared gradients to reconstruct sensitive data by leveraging pre-trained generative adversarial networks (GAN) as prior knowledge. However, existing attacks simply perform gradient inversion in the latent space of the GAN model, which limits their expression ability and generalizability. To tackle these challenges, we propose Gradient Inversion over Feature Domains (GIFD), which disassembles the GAN model and searches the hierarchical features of the intermediate layers. Instead of optimizing only over the initial latent code, we progressively change the optimized layer, from the initial latent space to intermediate layers closer to the output images. In addition, we design a regularizer to avoid unreal image generation by adding a small l1 ball constraint to the searching range. We also extend GIFD to the out-of-distribution (OOD) setting, which weakens the assumption that the training sets of GANs and FL tasks obey the same data distribution. Furthermore, we consider the challenging OOD scenario of label inconsistency and propose a label mapping technique as an effective solution. Extensive experiments demonstrate that our method can achieve pixel-level reconstruction and outperform competitive baselines across a variety of FL scenarios.
Figures & tables
Fig. 1 : The reconstructed results of our proposed GIFD on ImageNet [ deng2009imagenet ] and FFHQ [ karras2019style ] . The first column contains the randomly initialized images generated by GAN models. The next two columns show the reconstruction samples of the latent space search and our proposed GIFD.
Fig. 2 : Overview of our proposed GIFD attack. The intermediate layer optimizer minimizes the matching loss computed from the dummy gradients and the shared gradients from the victim client to update the latent vector and the intermediate features successively. The image fidelity regularization helps improve the quality of generated images. The reconstructed image from the layer with the corresponding least gradient matching loss is selected as the final output.
Fig. 3 : An illustration of our proposed label mapping mechanism. We first conduct only a small number of iterations based on the initially inferred label for a coarse reconstruction x^ . It is then fed into fm(⋅) for a refined label, which serves as the conditional input for GAN models in subsequent fine-grained attack. This technique corrects the label inconsistency and offers a more aligned guidance in the semantic space, further enhancing the attack effectiveness.
Fig. 4 : Comparison of PSNR mean on BigGAN and StyleGAN2 under different values of hyper-parameter K ( i.e. , the last intermediate layer to optimize). Notably, the figures exclude the results where the corresponding values are below the starting point of the y-axis.
Metric
ImageNet
FFHQ
IG [ geiping2020inverting ]
GI [ yin2021see ]
GGL [ li2022auditing ]
GIAS [ jeon2021gradient ]
GIFD
IG [ geiping2020inverting ]
GI [ yin2021see ]
GGL [ li2022auditing ]
GIAS [ jeon2021gradient ]
GIFD
PSNR ↑
17.0756
16.5109
13.3885
17.4923
20.0534
15.3523
14.9485
15.1335
20.1799
21.3368
LPIPS ↓
0.3078
0.3297
0.3678
0.2536
0.1559
0.4172
0.4503
0.2009
0.1266
0.1023
SSIM ↑
0.2908
0.2673
0.1251
0.3381
0.4713
0.2272
0.2044
0.2453
0.5379
0.5768
MSE ↓
0.0223
0.0258
0.0553
0.0236
0.0141
0.0311
0.0343
0.0339
0.0121
0.0098
TABLE I : Comparison of GIFD with state-of-the-art methods on every 1000th image of the ImageNet and FFHQ validation set. We calculate the average value of metrics on reconstructed images.
Fig. 5 : Qualitative results of different methods on ImageNet and FFHQ.
Datset
Method
Art Painting
Photo
Cartoon
PSNR ↑
LPIPS ↓
SSIM ↑
MSE ↓
PSNR ↑
LPIPS ↓
SSIM ↑
MSE ↓
PSNR ↑
LPIPS ↓
SSIM ↑
MSE ↓
ImageNet*
IG [ geiping2020inverting ]
18.3476
0.2286
0.3870
0.0172
15.6647
0.3575
0.2409
0.0325
15.8766
0.3183
0.3970
0.0288
GI [ yin2021see ]
17.4681
0.2625
0.3445
0.0203
15.2700
0.3888
0.2201
0.0346
15.3905
0.3112
0.3926
0.0327
GGL [ li2022auditing ]
12.8011
0.3639
0.1356
0.0571
12.9246
0.3159
0.1507
0.0667
11.0315
0.3294
0.2832
0.0895
GIAS [ jeon2021gradient ]
17.2804
0.2774
0.3346
0.0227
20.4539
0.1724
0.4913
0.0111
19.0247
0.1862
0.5740
0.0149
GIFD
19.3311
0.1700
0.4503
0.0151
21.9281
0.1137
0.5765
0.0082
22.8055
0.1030
0.6970
0.0067
TABLE II : Comparison of GIFD with state-of-the-art baselines on OOD data of different styles.
Fig. 6 : Visual comparison of different methods on ImageNet* and FFHQ*.
Method
PSNR ↑
LPIPS ↓
SSIM ↑
MSE ↓
IG [ geiping2020inverting ]
15.3523
0.4172
0.2272
0.0311
GI [ yin2021see ]
14.9485
0.4503
0.2044
0.0343
GGL [ li2022auditing ]
11.1235
0.4498
0.0995
0.0843
GIAS [ jeon2021gradient ]
17.4565
0.2534
0.3825
0.0201
GIFD
19.9793
0.1597
0.5095
0.0126
GIFD*
20.3664
0.1463
0.5288
0.0114
TABLE III : Comparison of GIFD with SOTA baselines on more challenging OOD scenarios of label inconsistency. We attack the FL model trained on FFHQ using the GAN model learned from ImageNet. Note that GIFD* denotes GIFD plus label mapping technique.
TABLE IV : PSNR mean of different attack methods under different defense strategies.
Fig. 7 : PSNR results of different attacks when changing the values of q (the probability of assigning a training example with a certain label to a particular client) on ImageNet. These settings strictly align with the previous works [ wei2025extracting , cao2021fltrust ] considering data heterogeneity.
Fig. 8 : Comparison of GIFD with state-of-the-art baselines on ImageNet when increasing the FL training rounds.
TABLE V : PSNR mean of different attack methods under different batch sizes.
Method
CIFAR-10
CIFAR-100
MNIST
KMNIST
SVHN
IG [ geiping2020inverting ]
25.9120
21.0027
26.4194
26.3165
24.8600
GI [ yin2021see ]
26.1532
22.9966
25.7285
27.1744
24.4489
GGL [ li2022auditing ]
12.5193
11.9671
10.0464
8.6165
17.8009
GIAS [ jeon2021gradient ]
27.5338
26.4250
39.5232
28.5140
28.3148
GIFD
28.9481
28.6634
40.4472
34.5892
30.6602
TABLE VI : PSNR mean of different attack methods across different private datasets.
Fig. 9 : PSNR results of different methods for different global models on the ImageNet dataset.
TABLE VII : Ablation study of GIFD and its three variants on every 1000th image of ImageNet and FFHQ validation set.
Method
IG [ geiping2020inverting ]
GIAS [ jeon2021gradient ]
GIFD
Time(s)↓
726.8008
1360.7808
907.727
PSNR↑
14.1896
16.6995
18.4018
Loss↓
0.008029
0.007803
0.006865
TABLE VIII : Converge time and PSNR results of GIFD and baselines that use the same distance metric at batch size 4.
Gradient inversion attacks are often presented as a serious privacy threat in federated learning, with recent work reporting increasingly strong reconstructions under favorable experimental settings. However, it remains unclear whether such attacks are feasible in modern, performance-optimized systems deployed in practice. In this work, we evaluate the practical feasibility of gradient inversion for image-based federated learning. We conduct a systematic study across multiple datasets and tasks, including image classification and object detection, using canonical vision architectures at contemporary resolutions. Our results show that while gradient inversion remains possible for certain legacy or transitional designs under highly restrictive assumptions, modern, performance-optimized models consistently resist meaningful reconstruction visually. We further demonstrate that many reported successes rely on upper-bound settings, such as inference mode operation or architectural simplifications which do not reflect realistic training pipelines. Taken together, our findings indicate that, under an honest-but-curious server assumption, high-fidelity image reconstruction via gradient inversion does not constitute a critical privacy risk in production-optimized federated learning systems, and that practical risk assessments must carefully distinguish diagnostic attack settings from real-world deployments.
Viktor Valadi, Lucas Beerens, Mattias Åkesson +4
Scaleout Systems Sweden · Recorded Future Sweden · AI Sweden Sweden +1
Gradient inversion attacks threaten client privacy in federated learning by reconstructing training samples from clients' shared gradients. Gradients aggregate contributions from multiple records and existing attacks may fail to disentangle them, yielding incorrect reconstructions with no intrinsic way to certify success. In vision and language, attackers may fall back on human inspection to judge reconstruction plausibility, but this is far less feasible for numerical tabular records, fueling the impression that tabular data is less vulnerable. We challenge this perception by proposing a verifiable gradient inversion attack (VGIA) that provides an explicit certificate of correctness for reconstructed samples. Our method adopts a geometric view of ReLU leakage: the activation boundary of a fully connected layer defines a hyperplane in input space. VGIA introduces an algebraic, subspace-based verification test that detects when a hyperplane-delimited region contains exactly one record. Once isolation is certified, VGIA recovers the corresponding feature vector analytically and reconstructs the target via a lightweight optimization step. Experiments on tabular benchmarks with large batch sizes demonstrate exact record and target recovery in regimes where existing state-of-the-art attacks either fail or cannot assess reconstruction fidelity. Compared to prior geometric approaches, VGIA allocates hyperplane queries more effectively, yielding faster reconstructions with fewer attack rounds.
Francesco Diana, Chuan Xu, André Nusser +1
Université Côte d’Azur, France · Inria, France · CNRS, France +1
Federated learning shares model updates rather than raw data, yet these updates can be inverted to reconstruct the clients' training data. Analytic reconstruction attacks, which invert a gradient in closed form, degrade as the batch grows: prior single-round attacks recover only about half of a batch of size 100 even when the attacker fully controls the network parameters, and known upper bounds limit what any such method can recover. We establish a connection between gradient inversion and the theory of erasure-correcting codes, and use it to construct attacks that exceed these bounds. Our attacks recover batches exactly, together with every sample's label, from a single FedSGD round, and certify each recovery without ground-truth data. On eight image and tabular benchmarks they outperform prior single-round attacks by a wide margin. Even a passive attacker who only observes an honestly trained network recovers 94--100% of ImageNet batches at sizes up to 128, more than prior single-round attacks achieve even with active manipulation of the model, and in the active setting more than 90% is recovered at batch sizes of several hundred. These results show that the privacy leakage of federated learning has been underestimated.
Saeed Shariati, Mohsen Alambardar Meybodi
Department of Applied Mathematics and Computer Science, Faculty of Mathematics and Statistics, University of Isfahan, Isfahan 81746-73441, Iran