Privacy-preserving face recognition (PPFR) and face anonymization have different goals, but both must retain some identity-related information for their intended use. We show that an adaptive attacker can learn this information. We propose FaceLinkGen, a simple distillation-based attack that trains a face recognition model to map protected inputs back to standard face embeddings. FaceLinkGen applies to keyless PPFR systems and perception-preserving face de-identification (De-ID) systems. For PPFR, the recovered embeddings can be used to regenerate faces that match the original person. Across MinusFace, PartialFace, and DecoyFace, the regenerated faces achieve acceptance rates of 81.0--99.4% on Face++ and 74.9--99.6% on Amazon. For De-ID, FaceLinkGen links protected faces to unprotected images of the same person, reaching Recall@1 values of 48.4--89.6% in the one-side-protected setting across the evaluated methods. FaceLinkGen exposes identity leakage across all evaluated methods, including DecoyFace and WDP, whose protection resists the tested U-Net attacks on face recovery and protected-to-unprotected linkage, respectively. The attack also remains effective when trained with limited paired data. Code is available at https://github.com/weathon/FaceLinkGenRelease.
Figures & tables
Figure 1: Method overview of FaceLinkGen. (a) For PPFR, training extracts the ground-truth identity embedding from the original face and encourages the student to produce the same embedding from the protected template by matching the teacher’s output using a cosine-similarity loss. At inference time, the student extracts an embedding from an unseen template, and Arc2Face is used to synthesize a face image from that embedding. (b) For De-ID systems, we use two images of the same identity along with their protected counterparts. The goal is to maximize cosine similarity across the 2×2 matching grid between the two original images and their protected counterparts. To preserve the student’s ability to recognize unprotected faces, we also feed the student an original image and enforce that its output for that image matches the teacher’s.
Figure 2: U-Net attack results for PartialFace (top row) and MinusFace (bottom row). IN denotes instance normalization. From left to right, the columns show the original image, SGD training without IN, SGD training with IN at an earlier training step, SGD training with IN at a later training step, and AdamW+IN training.
Figure 3: Left: Reconstructions produced by each attack. Rows correspond to PartialFace, MinusFace, and DecoyFace; columns show the original image, the U-Net attack, and the distillation attack. Right: More uncurated examples of reconstruction comparisons between U-Net and distillation for DecoyFace.
Attack Method
Metric
MinusFace
PartialFace
DecoyFace
AdamW+IN U-Net
Face++
0.998
1.000
0.014
Amazon
0.998
1.000
0.000
AdaFace Sim
0.803
0.940
0.074
SGD U-Net
Face++
0.000
0.008
–
Amazon
0.000
0.064
–
AdaFace Sim
0.009
0.268
–
Table 1: PPFR method results under U-Net and distillation attacks. Both AdamW+IN U-Net and distillation successfully invert MinusFace and PartialFace, achieving high acceptance rates on Face++ and Amazon and exceeding the AdaFace similarity threshold (0.19, selected to maximize F1 on LFW). Distillation is the only evaluated attack that succeeds against DecoyFace.
U-Net
Distillation (Ours)
Before Attack
Recall@1
Recall@5
MRR
Recall@1
Recall@5
MRR
Recall@1
Recall@5
MRR
TIP-IM
One side
0.969
0.974
0.973
0.896
0.958
0.923
0.031
0.052
0.045
Both sides
0.974
0.979
0.975
0.885
0.922
0.905
0.255
0.391
0.317
Average
0.971
0.977
0.974
0.891
0.940
0.914
0.143
0.222
0.181
WDP
One side
0.031
0.042
0.036
0.484
0.703
0.578
0.005
0.021
0.011
Both sides
0.490
0.667
0.577
0.620
0.745
0.673
0.682
0.839
0.762
Table 2: Attack results on the De-ID systems: recall on the 192-ID test pool (gallery size 107,676). “Average” is the arithmetic mean of the scores from the one-side and both-sides settings, reflecting a realistic scenario where protected and unprotected images coexist in the gallery.
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 4: t-SNE plot of each method’s embeddings compared to the original embeddings. Gray dots represent embeddings of unprotected distractor images.
Metric
Reproduced Result
Original Paper Result
Cosine similarity ( ↓ )
0.081
0.027
Identity Leakage Ratio ( ↓ )
2.61 %
2.93%
Identity Redirection Ratio ( ↑ )
97.39 %
97.07%
Face Validity Ratio ( ↑ )
99.99 %
99.88%
Appendix
Table 3: Reproduction results compared with those reported in the original DecoyFace paper. Our reproduction performs better on three of the four metrics, with a lower identity leakage ratio, a higher identity redirection ratio, and a higher face validity ratio. It has slightly higher cosine similarity with the original face (0.081), but the value remains within the near-zero range of similarities between different people, and thus the Identity Leakage Ratio remains low.
Figure 5: Distributions of cosine similarity for raw same-person image pairs and student-teacher embedding pairs from MinusFace and PartialFace.
Attack Method
Metric
MinusFace
PartialFace
DecoyFace
AdamW+IN U-Net
Amazon
0.962
0.986
0.000
AdaFace Sim
0.519
0.600
0.041
SGD U-Net
Amazon
0.000
0.030
–
AdaFace Sim
0.007
0.163
–
Distillation (Ours)
Amazon
0.784
0.890
0.419
AdaFace Sim
0.425
0.470
0.307
Appendix
Table 4: PPFR method results under U-Net and distillation attacks, evaluated against a different image from the same identity directory. MinusFace and PartialFace distillation use the best checkpoints after continued training. The teacher baseline uses embeddings of the original unprotected inputs. The teacher baseline is shown in separate rows because it is not associated with any protection method.
Gender Acc
Race Acc
Age Group Acc
Age MAE
MinusFace
92.52%
64.97%
54.43%
5.28
PartialFace
93.49%
68.64%
56.55%
4.92
DecoyFace
89.39%
62.20%
53.43%
5.66
Raw Images
94.29%
71.28%
60.30%
4.40
Appendix
Table 5: Soft biometric recovery performance from the protected template without regeneration. Acc denotes accuracy, and MAE denotes mean absolute error in years. Age MAE is calculated using the midpoint of each age group, with ages greater than 70 marked as 75.
Figure 6: Few-sample attack success rate as a function of N . The De-ID ASR is the average Recall@1 across the one-side-protected and both-sides-protected settings, and the PPFR attack success rate is defined as the rate of successful verification by Face++. N refers to the number of identities.
Attack
PPFR method
Amazon
Face++
AdaFace Sim
AdamW+IN U-Net
MinusFace
1.000
1.000
0.722
AdamW+IN U-Net
PartialFace
1.000
1.000
0.908
AdamW+IN U-Net
DecoyFace
0.005
0.015
0.060
Distillation
MinusFace
0.985
0.980
0.515
Distillation
PartialFace
1.000
1.000
0.638
Distillation
DecoyFace
0.415
0.660
0.333
Appendix
Table 6: PPFR attacks on 200 aligned synthetic faces from the TPDNE dataset. Each reconstructed image is compared with the original cropped input from which the protected representation was generated. Amazon and Face++ report acceptance rates; AdaFace reports mean cosine similarity.
Publishing private face recognition~(FR) training datasets is privacy-sensitive because faces expose identity information. Private FR training dataset publication mitigates this risk by releasing protected proxies as substitutes for private training faces. However, training FR models with such data introduces an identity paradox: \emph{the identity cues that make released faces useful for recognition supervision are also the cues that make them linkable to real individuals.} A protected face should be decoupled from the original identity, yet still behave as a reliable identity sample for training. Removing these cues too aggressively may destroy the class structure needed for recognition learning, whereas preserving them too faithfully may increase source-identity linkability. We argue that this paradox stems from conflating source-aligned identity semantics with recognition-useful proxy identity geometry. The former should be suppressed to reduce linkage to private individuals, while the latter should be preserved for FR learning. Based on this insight, we propose \textbf{Private Face Distillation}, an identity-decoupling and geometry-preserving framework. It uses Orthogonal Geometry Preservation to construct decoupled proxy identities from private identity representations while maintaining hyperspherical geometry, and Relational Topology Alignment to preserve identity relations for recognition learning. Experiments across multiple domain-shifted FR scenarios show that Private Face Distillation achieves stronger utility than the evaluated publication baselines. On IJB-C surveillance, it improves TAR@FAR=1e-3 by 3.94% over the baseline while reducing source-identity linkability. These results suggest that private FR training dataset publication should decouple source-identity correspondence while preserving proxy identity geometry.
Split face recognition reduces client-side computation but exposes intermediate features to feature inversion attacks and unauthorized analysis by honest-but-curious (HBC) servers. Existing privacy-preserving face recognition methods mainly aim to resist unauthorized reconstruction, typically producing features whose inversion yields visibly degraded results, which may reveal the existence of protection and motivate adaptive attacks. To address this issue, we propose DecoyFace, an imperceptible decoy-oriented framework that steers unauthorized reconstruction toward a plausible but incorrect identity while preserving recognition utility. The key idea is to decompose the intermediate representation into a reconstruction-sensitive subspace and its complementary subspace. The client injects decoy identity cues into the reconstruction-sensitive subspace, while limited recognition-relevant evidence from the true sample is retained in the complementary subspace. On the server side, an authorized canonicalization module suppresses decoy-dominant components and recovers a recognition-friendly representation. This design addresses both attacker-side inversion from intercepted features and HBC server-side reconstruction from canonicalized representations. Experiments show that DecoyFace preserves competitive recognition accuracy while substantially reducing identity leakage to 2.93% under U-Net attacks and 0.74% under Flow-Matching attacks while yielding visually plausible and imperceptible reconstructions, with over 99.78% face validity on LFW dataset.
Zhihan Ren, Lijun He, Xinyao Wang +2
Shaanxi Key Laboratory of Deep Space Exploration Intelligent Information Technology, School of Information and Communications Engineering, Xi’an Jiaotong University, Xi’an 710049, China
In this paper, we investigate the impact of adversarial attacks on identity encoders within a realistic de-identification framework. Our experiments show that the transferability of attacks transfers from an external surrogate model to the system model (e.g., CosFace to ArcFace) allows the adversary to cause identity information to leak in a sufficiently sensitive face recognition system. We present experimental evidence and propose strategies to mitigate this vulnerability. Specifically, we show how fine-tuning on adversarial examples helps to mitigate this effect for distortion-based attacks (i.e., snow, fog, etc.), while a simple low-pass filter can attenuate the effect of adversarial noise without affecting the de-identified images. Our mitigation results in a de-identification system that preserves its functionality while being significantly more robust to adversarial noise.
Felix Rosberg, Cristofer Englund, Eren Erdal Aksoy +1