The proliferation of facial recognition (FR) systems has raised privacy concerns in the digital realm, as malicious uses of FR models pose a significant threat. Traditional countermeasures, such as makeup style transfer, have suffered from low transferability in black-box settings and limited applicability across various demographic groups, including males and individuals with darker skin tones. To address these challenges, we introduce a novel facial privacy protection method, dubbed MAP, a pioneering approach that employs human emotion modifications to disguise original identities as target identities in facial images. Our method uniquely fine-tunes a score network to learn dual objectives, target identity and human expression, which are jointly optimized through gradient projection to ensure convergence at a shared local optimum. Additionally, we enhance the perceptual quality of protected images by applying local smoothness regularization and optimizing the score matching loss within our network. Empirical experiments demonstrate that our innovative approach surpasses previous baselines, including noise-based, makeup-based, and freeform attribute methods, in both qualitative fidelity and quantitative metrics. Furthermore, MAP proves its effectiveness against an online FR API and shows advanced adaptability in uncommon photographic scenarios.
Figures & tables
Figure 1: Comparison of makeup-based baselines with our MAP . Baseline methods (top) use makeup styles obscure original identities, optimizing objectives independently, which reduces efficiency and limits applicability across demographics ( e.g. , males). In contrast, MAP (bottom) leverages human emotion modifications and a unified optimization strategy to disguise original identities as target identities, achieving universal robustness.
Figure 2: Comparison between makeup transfer-based and emotion-based approaches in terms of frequency changes (with azimuthal integral). Makeup transfer primarily edits the image in the low-frequency range, while our emotion-based approach targets medium to high frequencies, resulting in a more natural appearance and being better suited for obfuscating identity adversarial noises.
Figure 3: Illustration of our dual objectives optimization strategies. Naively optimizing unrelated tasks (identity ← and emotion → ) may lead to negative transfer, canceling out each other’s gradients. Our approach can render a new update ( ↑ ) that helps guide the model towards optimal values.
Figure 4: Top: Illustration of Laplacian Smoothness Regularization for landmarks ( ★ ) 54 th and 26 th . ▲ represents the average of neighbors ( ∙ ) of ★ . Bottom: Effects of Laplacian Smoothness; without it, the eyebrow is eroded compared to the original image.
Figure 5: Visualizations of protected face images generated by different facial privacy protection methods on CelebA-HQ. The yellow numbers in each image represent confidence scores returned by Face++. Unlike makeup-based approaches, which may not be suitable for all demographics, our method successfully protects images against malicious FR systems through emotion editing (top to bottom: surprise, surprise, happy), while preserving original details like color grading and background.
Method
CelebA-HQ
LADN-Dataset
Avg.
IRSE50
IR152
FaceNet
MobileFace
IRSE50
IR152
FaceNet
MobileFace
Clean
7.29
3.80
1.08
12.68
2.71
3.61
0.60
5.11
4.61
Noise-based
PGD
36.87
20.68
1.85
43.99
40.09
19.59
3.82
41.09
25.60
MI-FGSM
45.79
25.03
2.58
45.85
48.90
25.57
6.31
45.01
30.63
TI-DIM
63.63
36.17
15.30
57.12
56.36
34.18
22.11
48.30
41.64
TIP-IM
54.40
37.23
40.74
48.72
65.89
43.57
63.50
46.48
50.06
Table 1: Protect Success Rate (PSR) with FAR@1e-2 for black-box setting on CelebA-HQ and LADN dataset.
Method
IRSE50
IR152
FaceNet
MobileFace
R1-T
R5-T
R1-T
R5-T
R1-T
R5-T
R1-T
R5-T
TIP-IM
16.2
51.4
21.2
56.0
8.1
35.8
9.6
24.0
CLIP2Protect
24.5
64.7
24.2
65.2
12.5
38.7
11.8
28.2
SD4Privacy
15.6
26.8
23.4
41.2
33.6
53.8
31.8
49.8
GIFT
21.2
57.2
34.6
49.4
33.2
65.6
41.2
67.6
Adv-CPG
24.4
56.4
33.8
51.2
36.6
67.4
43.4
70.4
Table 2: Protection success rate (PSR) of impersonation attacks under the face identification task on CelebA-HQ.
Method
FID ↓
PSNR ↑
SSIM ↑
PSR Gain ↑
Adv-Makeup
4.22
34.51
0.985
0
AMT-GAN
34.44
19.50
0.787
38.12
CLIP2Protect
26.62
19.31
0.750
50.18
DiffAM
26.10
20.52
0.886
63.13
MAP ( ours )
24.21
29.07
0.876
73.76
Table 3: Quantitative evaluations of image quality. PSR Gain is absolute gain in PSR relative to Adv-Makeup.
Figure 6: Trade-off between PSR and FID. MAP achieves a better trade-off and outperforms five freeform baselines: DiffProtect, SD4Privacy, Adv-Diffusion, GIFT, and Adv-CPG.
Figure 7: Average confidence score from the real-world face verification API, Face++, for impersonation attacks.
Settings
FID ↓
PSR ↑ with FAR
@1e-1
@1e-2
@1e-3
w/o emotion
24.1
95.6
85.0
67.0
w/o grads proj.
24.5
90.5
71.2
50.9
w/o EMA
22.5
93.6
76.5
56.8
w/o LLS
23.6
94.8
78.8
56.4
MAP ( ours )
21.5
95.4
80.3
60.5
Table 4: Ablation studies on different optimization settings.
Figure 8: Illustration of uncommon photography styles: monochrome, Rembrandt lighting, and backlighting.
Figure 9: Our human evaluation from 25 participants.
The widespread deployment of face recognition (FR) systems exposes personal images shared on social media and public platforms to identity linkage and privacy risks. Existing adversarial privacy protection methods can degrade unauthorized FR performance but are not compatible with generative face editing. Artificial intelligence-driven face editing tools are gaining popularity, which has significantly increased user demand for personalized portrait generation and social sharing. However, current editing methods often preserve identity features, making the edited images still susceptible to tracking by malicious FR systems. Thus, this paper proposes Flux-Guard, a privacy-preserving face editing framework based on adversarial attacks, which integrates face editing and privacy protection within a unified generative process. Specifically, we design a flow trajectory control method to align semantic manipulations with the generative process and introduce latent-space adversarial optimization with an adaptive perceptual-loss-driven weighting strategy, dynamically adjusting adversarial strength to maximize attack effectiveness while preserving visual quality. Extensive experiments demonstrate that Flux-Guard supports face editing while significantly improving attack success rates against cross-domain face recognition models on the CelebA-HQ and LADN datasets. Furthermore, evaluation results for commercial APIs have confirmed its effectiveness in real-world applications. The code is released at https://github.com/JLMWang/Flux-Guard.
Jie Wang, Tao Wang, Ru Zhang +1
School of Cyberspace Security, Beijing University of Posts and Telecommunications, Beijing, 100876, China · Nanjing University of Aeronautics and Astronautics, NanJing, China
Photos of faces uploaded online are vulnerable to malicious actors who can scrape facial images from online sources and intrude on personal privacy via unauthorized use of facial recognition models. This paper presents FaceCloak, a novel personalized face privacy protection system, which can generate defensive identity-specific universal face privacy masks from a single image of a user, causing facial recognition to fail. FaceCloak introduces a three-stage personalized face perturbation learning methodology: (1) It generates a small set of high-variety synthetic face images of a person based on a single image of the person. (2) It learns face cloaking by adding more protection to key facial-identity leakage regions through iterative perturbation generation over the small set of synthetic images, effectively shifting a user's identity embedding towards a distant anchor identity and away from a similar one. (3) It generates a personalized identity-protective mask in the form of pixel-wise cloaking, which is light-weight and can be efficiently applied to any facial image of a user while maintaining good perceptual quality. Extensive experiments on three popular face datasets across ten recognition models show the effectiveness of FaceCloak compared to 29 other existing representative methods. Code is available at https://github.com/zacharyyahn/FaceCloak
Zachary Yahn, Fatih Ilhan, Tiansheng Huang +5
Georgia Institute of Technology, Atlanta, Georgia, USA
While the rapid development of facial recognition algorithms has enabled numerous beneficial applications, their widespread deployment has raised significant concerns about the risks of mass surveillance and threats to individual privacy. In this paper, we introduce Adversarial Camouflage as a novel solution for protecting users' privacy. This approach is designed to be efficient and simple to reproduce for users in the physical world. The algorithm starts by defining a low-dimensional pattern space parameterized by color, shape, and angle. Optimized patterns, once found, are projected onto semantically valid facial regions for evaluation. Our method maximizes recognition error across multiple architectures, ensuring high cross-model transferability even against black-box systems. It significantly degrades the performance of all tested state-of-the-art face recognition models during simulations and demonstrates promising results in real-world human experiments, while revealing differences in model robustness and evidence of attack transferability across architectures.
Paweł Borsukiewicz, Daniele Lunghi, Melissa Tessa +2