This work aims to answer the question of whether it is possible to generate irreversible protected templates when the PolyProtect biometric template protection method is applied to face embeddings using system-specific keys (i.e., the same C and E parameters, which define the transform, are applied to all subjects' face embeddings), instead of the traditional subject-specific keys (i.e., each subject has their own C and E parameters). This is important for determining whether we can perform de-duplication of face identities in the PolyProtected domain, which is not possible in the subject-specific key scenario due to the clash with PolyProtect's unlinkability property (i.e., one could generate multiple protected templates belonging to the same identity, using different C and E parameters, such that those templates cannot be linked to each other). We present experiments (reproducible using our open-source code) to prove that there exist at least three ways of systematically selecting system-specific keys that produce irreversible PolyProtected templates: (i) from pre-selected subject-specific keys, (ii) by applying a previously proposed key selection algorithm to random vectors, and (iii) by approximating a "good" C/E pair distribution from which system-specific keys can be constructed. Our findings thus point to the conclusion that it is, indeed, possible to safely operate PolyProtect in the system-specific key scenario without degrading the template protection potential. This opens up the possibility for identity de-duplication in the PolyProtected domain.
Figures & tables
Fig. 1: We investigate the selection of system -specific keys ( C , E ) for the PolyProtect BTP method, such that the protected templates ( P1 , …, PN ), generated from the face embeddings ( E1 , …, EN ) of all identities enrolled in the face recognition system ( I1 , …, IN ) are irreversible , i.e., it is impossible to recover Ei from its corresponding Pi . (Face images from SOTERIA [ 26 ] .)
Fig. 2: Mapping 512-dimensional V to P via PolyProtect, using C=[c1,c2,...,c5] and E=[e1,e2,...,e5] , for different amounts of overlap.
Fig. 3: Verification accuracy across unprotected face templates (embeddings) generated using five different face recognition models on three face datasets.
Fig. 4: Accuracy of template matching in the PolyProtected domain, for protected templates created from EdgeFace and iResNet100 face embeddings in two scenarios: (i) System -specific keys used for PolyProtect (i.e., the same C and E parameters used for all face embeddings across all identities), and (ii) Sample -specific keys used for PolyProtect (i.e., different C and E parameters used for each face embedding, including different embeddings from the same identity). Accuracy for unprotected embeddings is shown as the baseline.
Fig. 5: Inversion scores for PolyProtected templates generated from EdgeFace and iResNet100 face embeddings when the keys ( C , E ) were: system-specific + random (gray histogram), subject-specific + selected using the key selection algorithm from [ 24 ] (cyan histogram), and system-specific + selected at random from the subject-specific keys (blue histogram). Our main focus is the blue histogram. The inversion was performed using the cosine-based numerical solver from [ 24 ] . The two vertical black lines represent different thresholds at which the inversion success rate (ISR) was computed for Table I .
Model
Dataset
ISR (%)
@ 0.1% FMR
@ 0.01% FMR
Sb.
Sy.
R-Sy.
Sb.
Sy.
R-Sy.
EdgeFace
Multi-PIE
0.5
0.8
100
0
0
99.2
SOTERIA
0
0.6
91.3
0
0
90.1
iCarB-Face
1.1
1.3
91.6
0.2
0
89.7
iResNet100
Multi-PIE
0.5
0.5
100
0
0
99.4
TABLE I: Inversion success rate (ISR) for Fig. 5 at the two FMR thresholds, for EdgeFace and iResNet100 face embeddings protected via PolyProtect when the keys ( C , E ) were: subject-specific + selected using the key selection algorithm from [ 24 ] ( Sb. ) VS system-specific + selected from the subject-specific keys ( Sy. ) VS system-specific + random ( R-Sy. ).
Fig. 6: Inversion scores for PolyProtected templates generated from EdgeFace and iResNet100 face embeddings when the keys ( C , E ) were system-specific and: selected by applying the key selection algorithm from [ 24 ] to random vectors (purple histogram) versus chosen at random from subject-specific keys selected by applying the same key selection algorithm to face embeddings (blue histogram, from Fig. 5 ). Our main focus is the purple histogram. The inversion was performed using the numerical solver from [ 24 ] . The two vertical black lines represent different thresholds at which the inversion success rate (ISR) was computed for Table II .
Model
Dataset
ISR (%)
@ 0.1% FMR
@ 0.01% FMR
RV
SK
RV
SK
EdgeFace
Multi-PIE
0.3
0.8
0
0
SOTERIA
0.2
0.6
0
0
iCarB-Face
0.3
1.3
0
0
iResNet100
Multi-PIE
0.1
0.5
0
0
TABLE II: Inversion success rate (ISR) for Fig. 6 at the two FMR thresholds, for EdgeFace and iResNet100 face embeddings protected via PolyProtect when the keys ( C , E ) were: system-specific + selected using the key selection algorithm from [ 24 ] applied to random vectors ( RV ) VS system-specific + chosen from subject-specific keys selected for individual face embeddings ( SK – corresp. to Sy. in Table I ).
Fig. 7: Left column: Distribution of ( Ci , Ei ) pairs from “good” keys selected by the key selection algorithm [ 24 ] , when it was applied to random vectors and when different thresholds (negative cosine distance) were used to define irreversibility. Right column: Spearman’s rank correlation coefficient matrix, showing weak monotonic correlations across the different ( Ci , Ei ) pairs.
Fig. 8: Inversion scores for PolyProtected templates generated from EdgeFace and iResNet100 face embeddings when the keys ( C , E ) were system-specific and based on applying the key selection algorithm from [ 24 ] to random vectors. The sequences of 5 corresponding C and E parameter pairs were: constructed by choosing samples from a “good” ( Ci , Ei ) pair distribution (orange histogram) versus chosen directly from a list of pre-determined sequences (purple histogram, from Fig. 6 ). Our main focus is the orange histogram. The inversion was performed using the numerical solver from [ 24 ] . The two vertical black lines represent different thresholds at which the inversion success rate (ISR) was computed for Table III .
Model
Dataset
ISR (%)
@ 0.1% FMR
@ 0.01% FMR
Dis.
Seq.
Dis.
Seq.
EdgeFace
Multi-PIE
2.1
0.3
0.1
0
SOTERIA
2.6
0.2
0.2
0
iCarB-Face
3.3
0.3
0.2
0
iResNet100
Multi-PIE
3.7
0.1
0.2
0
TABLE III: Inversion success rate (ISR) for Fig. 8 at the two FMR thresholds, for EdgeFace and iResNet100 face embeddings protected via PolyProtect when the keys (based on applying the key selection algorithm [ 24 ] to random vectors) were system-specific and: selected from a “good” ( Ci , Ei ) pair distribution ( Dis. ) VS chosen from a list of pre-selected ( C , E ) sequences ( Seq. – corresp. to RV in Table II ).
Fig. 9: Recognition accuracy for PolyProtected templates when the keys ( C , E ) were generated randomly versus using the key selection algorithm from [ 24 ] . In the latter scenario, the keys were either subject-specific, or system-specific and selected using the three methods from Sections III-C – III-E : from subject-specific keys, from random vectors, or from a C / E distribution. Unprotected template accuracy is shown as a baseline.
This work presents a deeper analysis of the "irreversibility" property of PolyProtect, a biometric template protection method initially proposed for securing face embeddings. PolyProtect transforms embeddings into protected templates via multivariate polynomials, whose coefficients and exponents are distinct for each subject enrolled in the face recognition system. A polynomial is applied to consecutive sets of elements from a given embedding, where the amount of overlap between the sets is a tunable parameter. We begin our irreversibility analysis by demonstrating that PolyProtected templates are easier to invert using a numerical solver based on cosine distance, as opposed to Euclidean distance (used in the earlier PolyProtect work). To make this inversion more difficult, we then propose a "key selection algorithm", which tries to choose "keys" (coefficients and exponents of the PolyProtect polynomial) that enhance the irreversibility of PolyProtected templates, compared to when the keys are purely random. Our experiments show that this algorithm is effective at generating PolyProtected templates that are significantly more difficult to invert, and that it approximately equalises the irreversibility of PolyProtected templates generated using different "overlap" parameters. This allows for better control of the irreversibility versus accuracy trade-off, known to exist across different overlaps. We also show that accuracy in the PolyProtected domain can be affected by the range in which the embedding elements lie, but that this can be improved by normalizing the embeddings prior to applying PolyProtect. This work is reproducible using our open-source code.
In face recognition systems, facial templates are widely adopted for identity authentication due to their compliance with the data minimization principle. However, facial template inversion technologies have posed a severe privacy leakage risk by enabling face reconstruction from templates. This paper proposes a Layer-Based Facial Template Inversion (LBFTI) method to reconstruct identity-preserving fine-grained face images. Our scheme decomposes face images into three layers: foreground layers (including eyebrows, eyes, nose, and mouth), midground layers (skin), and background layers (other parts). LBFTI leverages dedicated generators to produce these layers, adopting a rigorous three-stage training strategy: (1) independent refined generation of foreground and midground layers, (2) fusion of foreground and midground layers with template secondary injection to produce complete panoramic face images with background layers, and (3) joint fine-tuning of all modules to optimize inter-layer coordination and identity consistency. Experiments demonstrate that our LBFTI not only outperforms state-of-the-art methods in machine authentication performance, with a 25.3% improvement in TAR, but also achieves better similarity in human perception, as validated by both quantitative metrics and a questionnaire survey.
Zixuan Shen, Zhihua Xia, Kaikai Gan +1
College of Cyber Security, Jinan University · Guangzhou, Guangdong, China
Privacy-preserving face recognition (PPFR) and face anonymization have different goals, but both must retain some identity-related information for their intended use. We show that an adaptive attacker can learn this information. We propose FaceLinkGen, a simple distillation-based attack that trains a face recognition model to map protected inputs back to standard face embeddings. FaceLinkGen applies to keyless PPFR systems and perception-preserving face de-identification (De-ID) systems. For PPFR, the recovered embeddings can be used to regenerate faces that match the original person. Across MinusFace, PartialFace, and DecoyFace, the regenerated faces achieve acceptance rates of 81.0--99.4% on Face++ and 74.9--99.6% on Amazon. For De-ID, FaceLinkGen links protected faces to unprotected images of the same person, reaching Recall@1 values of 48.4--89.6% in the one-side-protected setting across the evaluated methods. FaceLinkGen exposes identity leakage across all evaluated methods, including DecoyFace and WDP, whose protection resists the tested U-Net attacks on face recovery and protected-to-unprotected linkage, respectively. The attack also remains effective when trained with limited paired data. Code is available at https://github.com/weathon/FaceLinkGenRelease.
Wenqi Guo, Qingyun Qian, Mohamed Shehata +1
Department of Computer Science, Mathematics, Physics and Statistics University of British Columbia, Kelowna, BC, Canada