Breaking High Confidence: Practical Face Impersonation under High-Security Thresholds

Changjin Kim; Seunghun Paik; Dongsoo Kim; Jae Hong Seo

cs.CVAug 21, 2026

Breaking High Confidence: Practical Face Impersonation under High-Security Thresholds

Authors: Changjin Kim, Seunghun Paik, Dongsoo Kim, Jae Hong Seo

Organizations: Department of Mathematics & Research Institute for Natural Sciences Hanyang University, Seoul 04763, Republic of Korea

Abstract

Face recognition systems (FRSs) are increasingly deployed in critical real-world services for authentication, such as banking applications and airport identity checks, necessitating stringent security configurations. Consequently, the security vulnerabilities of FRSs have garnered significant attention. While existing studies have extensively explored FRS security, prior analyses have primarily focused on medium-security threshold settings, which are not directly applicable to FRSs operating under high-security constraints. In this paper, we propose the first successful impersonation attack against FRSs under high-security threshold settings. Among various threat models, we focus on a practical and challenging scenario: score-based impersonation attacks under strict rate limits. To precisely evaluate the feasibility of such attacks, we provide a principled mathematical analysis characterizing the gaps in each stage of the attack pipeline. Our method significantly enhances impersonation capabilities in score-based attacks, even under elevated decision thresholds. On the LFW benchmark, with a budget of only 100 confidence score queries per identity, our attack achieves an impersonation success rate exceeding 92% against Amazon Rekognition at a confidence score threshold of 99-recommended setting for law enforcement scenarios. We further observe consistently robust performance across multiple open-source FRSs evaluated at similarly stringent decision thresholds.

Figures & tables

Method	Access	Query budget	ASR@FMR (Mean Cos. Sim.)	Notes
Shahreza et al. [ 58 ]	white-box	–	$94.90@{\color[rgb]{1,0,0}10^{-3}}$
Shahreza et al. [ 60 ]	white-box	–	$96.40@{\color[rgb]{1,0,0}10^{-3}}$	3D face reconstruction
Shahreza et al. [ 47 ]	white-box	–	$96.40@{\color[rgb]{1,0,0}10^{-3}}$	high-resolution
Shahreza et al. [ 59 ]	white-box	–	$96.48@{\color[rgb]{1,0,0}10^{-3}}$
Template-based
Mai et al. [ 42 ]	black-box	–	$95.20@{\color[rgb]{1,0,0}10^{-3}}$

Table 1: Summary of prior attacks and settings. All attack success rate (ASR) values are computed using the LFW evaluation protocol. We denote by AWS(

k

) the AWS Rekognition confidence score threshold of

k

between 0-100.

Figure 1: The errors ( E1 )-( E3 ) in the attack by [ 37 ] measured through experiments (Exp. 1–4). The dashed lines indicate the average of each distribution. The solid red line denotes the estimated threshold when $\mathrm{FMR}=10^{-6}$ . Best viewed in color.

Panel of Figure 2 — Figure 2: The error analysis with our techniques—correction matrix ( E1 ), PCA-based OFS (E2) , and SPNet (E3) —with a comparison to the baseline [ 37 ] in the black-box (BB) and white-box (WB) attacks. In the BB attack of ours, all the techniques are enabled, whereas only PCA-based OFS and SPNet are enabled in the WB attack.

FRS	Architecture	Loss	Train Dataset	Thres.
$F_{\mathsf{S}}$ [ 11 ]	ResNet-100 [ 21 ]	ArcFace	Glint360K [ 6 ]	0.5422
$F_{\mathsf{1}}$ [ 34 ]	ViT-Base [ 13 ]	AdaFace [ 33 ] +KPRPE	WebFace12M [ 74 ]	0.5195
$F_{\mathsf{2}}$ [ 9 ]	ResNet-100	CosFace [ 70 ] +TopoFR	MS1MV2 [ 11 ]	0.5216
$F_{\mathsf{3}}$ [ 39 ]	ResNet-100	SphereFace-R	MS1MV2	0.5537
$F_{\mathsf{A}}$ [ 1 ]	Not publicly disclosed

Table 2: Summary of target open-source and commercial FRSs. Thres. denotes estimated threshold when

\mathrm{FMR}=10^{-6}

for each open-source FRS.

Figure 3: Attack results of ours on various open-source FRSs in the LFW dataset. x-axis: cosine similarity. y-axis: the number of samples (left), ASR (right). Red solid lines denote the ASR for each threshold, and black dashed lines denote the threshold at $\mathrm{FMR}=10^{-6}$ . We also report the ASR at that threshold. Best viewed in color.

Used Tools/Tech.	LFW			CFP-FP			AgeDB
	80	90	99	80	90	99	80	90	99
Baseline ∗ [ 37 ]	27.0	13.9	–	20.6	9.5	–	29.6	17.7	–
Reproduced †	21.13	9.27	0.33	14.91	6.49	0.14	15.53	6.6	0.17
NbNet+C	70.89	48.98	2.83	64.14	41.43	2.46	61.17	40.27	2.10
NbNet+P	99.60	97.37	46.68	99.89	98.4	51.26	99.73	98.47	55.67
NbNet+CP	99.67	98.20	56.10	99.63	98.63	60.60	99.93	99.23	63.50

Table 3: ASRs (%) on benchmark datasets against

F_{\mathsf{A}}

. Baseline corresponds to “NbNet” without our techniques. C: correction matrix. P: PCA-based OFS.

Figure 4: True pairs of the LFW dataset (1st and 2nd rows) and the reconstructed faces from our attack (3rd row) targeting those in the 1st row. Green/red boxes denote whether the confidence score with the 1st row’s faces exceeds 99 or not, respectively.

Panel of Figure 5 — Figure 5: Ablation studies against $F_{\mathsf{A}}$ in the LFW dataset with respect to the effect of query budget and the effect of the correction matrix (C) and PCA-based OFS (P). In (b), dashed lines and solid lines denote the ASR results from the baseline NbNet in [ 37 ] and the proposed SPNet. Best viewed in color.

Model	Inv. Prec.	ASR on $F_{\mathsf{A}}$
NbNet [ 48 ]	0.8809	56.10
+ Tuning/Micro	0.9687	91.41 $\pm$ 1.75
+ DSCasConv [ 59 ]	0.9782	91.88 $\pm$ 1.60

Table 4: Ablation study on the inverse model

Appendix figures & tables13 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 6: The score-based attack pipeline we used. The gray area is not accessible to the adversary. A star and colored dots denote templates of the enrolled and queried faces in the target and surrogate embedding spaces, respectively. Best viewed in color.

Figure 8: The design of SPNet. PNorm: Pixel-wise normalization. Best viewed in color.

Datasets	LFW	CFP-FP	AgeDB	CASIA-WebFace
# of Imgs	13,233	7,000	16,488	490,623
# of IDs	5,749	500	568	10,572
# of TP/FPs	3,000/3,000	3,500/3,500	3,000/3,000	NA

Appendix

Table 5: Detailed specification of the datasets. TP/FP: True/false pairs.

FMRs	$\mathbf{10^{-2}}$	$\mathbf{10^{-3}}$	$\mathbf{10^{-4}}$	$\mathbf{10^{-5}}$	$\mathbf{10^{-6}}$
$F_{\mathsf{S}}$	0.1416	0.1981	0.2526	0.3095	0.5422
$F_{1}$	0.1390	0.1932	0.2464	0.3010	0.5195
$F_{2}$	0.1615	0.2265	0.2881	0.3463	0.5216
$F_{3}$	0.1864	0.2646	0.3324	0.3942	0.5537

Appendix

Table 6: Estimated thresholds of open-source FRSs at various FMR levels

FRSs	LFW				CFP-FP				AgeDB
FMRs	$10^{-2}$	$\tau$	$10^{-3}$	$\tau$	$10^{-2}$	$\tau$	$10^{-3}$	$\tau$	$10^{-2}$	$\tau$	$10^{-3}$	$\tau$
$F_{\mathsf{S}}$	99.80	0.1446	99.70	0.1888	98.97	0.1414	98.74	0.1985	97.53	0.1786	97.50	0.1814
$F_{1}$	99.77	0.1380	99.73	0.1670	98.89	0.1380	98.74	0.1837	97.10	0.1699	97.07	0.1771
$F_{2}$	99.80	0.1581	99.73	0.2046	98.37	0.1594	98.17	0.1835	97.50	0.1914	97.10	0.2116
$F_{3}$	99.77	0.1835	99.73	0.2500	98.37	0.1871	98.14	0.2073	97.50	0.2157	97.50	0.2161

Appendix

Table 7: Benchmark results of open-source FRSs in evaluation datasets with thresholds. We report TMR(%) at

\mathrm{FMR}=10^{-2}

and

10^{-3}

based on the benchmark.

\tau

denotes the threshold obtained from the benchmark.

FRSs	LFW			CFP-FP			AgeDB
FMRs	$10^{-4}$	$10^{-5}$	$10^{-6}$	$10^{-4}$	$10^{-5}$	$10^{-6}$	$10^{-4}$	$10^{-5}$	$10^{-6}$
$F_{\mathsf{S}}$	99.70	99.63	94.47	98.34	97.29	53.60	95.73	93.43	34.23
$F_{1}$	99.67	99.60	95.43	98.00	96.34	53.14	95.03	91.83	32.03
$F_{2}$	99.70	99.57	97.23	95.80	91.89	60.11	95.40	92.80	58.57
$F_{3}$	99.70	99.53	96.53	94.06	90.03	53.54	94.50	90.83	51.03

Appendix

Table 8: Benchmark results of open-source FRSs in evaluation datasets. We report TMR(%) at the estimated thresholds at

\mathrm{FMR}=10^{-4}

10^{-5}

, and

10^{-6}

, respectively.

Figure 9: Projection gaps between the embeddings from the LFW dataset and the PCA-based OFS measured from various FRSs. The OFS is crafted from $F_{\mathsf{S}}$ with NbNet (upper) and SPNet (lower). Solid lines indicate the average of the cosine similarities of the projection gap. Best viewed in color.

Figure 10: The left plot shows the PCA singular-value spectra of MS1MV3 [ 10 ] embeddings. The right plot shows the distributions of cosine similarities between LFW embeddings and their projections onto the OFS subspace.

Table 9: ASRs (%) on benchmark datasets against

F_{\mathsf{S}}

and

F_{1}

–

F_{3}

. Baseline corresponds to “NbNet” without our techniques. C: correction matrix. P: PCA-based OFS.

Panel of Figure 11 — Figure 11: Attack results of ours on various open-source FRSs in the CFP-FP and AgeDB datasets. Best viewed in color.

Panel of Figure 12 — Figure 12: Query sweep analysis results on open-source FRSs. Best viewed in color.

Panel of Figure 13 — Figure 13: Ablation study results on open-source FRSs. Solid/dashed lines denote that SPNet and NbNet are used as inverse models, respectively. Best viewed in color.

Figure 14: Attack results against $F_{\mathsf{A}}$ on the LFW dataset. Box colors indicate the confidence score against the target image in the first row: green for scores $>99$ , skyblue for scores $>90$ , purple for scores $>80$ , and red otherwise. The number below each reconstructed image denotes the returned confidence score.

Adversarial Attacks

Adversarial attacks exploit crafted inputs, poisoned data, or system interactions to manipulate AI predictions, behavior, privacy, or reliability.

See all 448 papers

Casting the Net! Revisiting MasterFace Impersonation Attacks
Aug 7, 2026Seunghun Paik, Sunpill Kim, Chanwoo Hwang +1Biometric Identification Adversarial Attacks
Hanyang University Seoul, Republic of Korea
Scores Know Bobs Voice: Speaker Impersonation Attack
Mar 3, 2026Chanwoo Hwang, Sunpill Kim, Yong Kiam Tan +6Black-Box Adversarial Attacks Speaker Verification
Hanyang University Seoul, Republic of Korea · I2R, A*STAR & NTU Singapore · National University of Singapore Singapore +1
Ghost Attack: Using 19th-Century Illusions to Fool 21st-Century Biometrics
Oct 16, 2026Tyler Fitzsimmons, Adam CzajkaFace Recognition Face Presentation Attack Detection
Department of Computer Science and Engineering, University of Notre Dame, Notre Dame, IN 46556, USA · Naval Surface Warfare Center, Crane Division, Crane, IN 47522, USA

Aug 7, 2026J/K move · Enter open · S save

Casting the Net! Revisiting MasterFace Impersonation Attacks

Seunghun Paik, Sunpill Kim, Chanwoo Hwang, Jae Hong Seo

Hanyang University Seoul, Republic of Korea

Biometric Identification Adversarial Attacks Black-Box Adversarial Attacks Face Recognition

Open paper AskPDF arXiv

Impersonation is a fundamental security threat in face recognition systems (FRSs). While the security of FRSs has been challenged by various attack vectors, under realistic adversarial capabilities, e.g., a limited number of decision-only authentication trials and no internal system knowledge, most attack techniques become infeasible. As a result, impersonation by zero-effort impostors, characterized by false match rate (FMR), is commonly regarded as a standalone baseline. A few years ago, impersonation attacks based on MasterFaces emerged as a notable security threat that could break the barrier of the FMR-based baseline under such realistic constraints. However, they were believed not to yield impersonation above the standard FMR in modern FRSs, as discussed by multiple follow-up studies. In this paper, we demonstrate that even legitimate access to public commercial APIs allows an adversary to amplify impersonation rates through MasterFaces, resulting in a non-trivial impersonation attack beyond FMR on downstream applications built on top of these APIs. We observe that several real-world FRS deployments are implemented using commercial APIs, and that the backend service provider is publicly disclosed or trivially inferable. As a result, the adversary can purchase these pay-as-you-go API services without requiring any additional privilege over the target FRS. From this observation, we formalize the MasterFaces attack as a maximum coverage problem over the biometric representation space, which we call a NET, and show that the adversary can construct an API-tailored NET by leveraging the geometric structure of the representation space. We demonstrate that our attack amplifies the impersonation rates of several open-source and commercial API-based FRSs by up to 9.5

\times

within at most 30 authentication trials, compared to those expected from the standard FMR.

Abstract

Adversarial Attacks

Explore similar work

Casting the Net! Revisiting MasterFace Impersonation Attacks

Scores Know Bobs Voice: Speaker Impersonation Attack

Ghost Attack: Using 19th-Century Illusions to Fool 21st-Century Biometrics