AI-Powered Facial Mask Removal Is Not Suitable For Identification
Authors: Emily A Cooper, Hany Farid
Organizations: Herbert Wertheim School of Optometry & Vision Science University of California, Berkeley · School of Information University of California, Berkeley
Recently, crowd-sourced online criminal investigations have used generative-AI to enhance low-quality visual evidence. In one high-profile case, social-media users circulated an "AI-unmasked" image of a federal agent involved in a fatal shooting, fueling a wide-spread misidentification. In response to this and similar incidents, we conducted a large-scale analysis evaluating the efficacy and risks of commercial AI-powered facial unmasking, specifically assessing whether the resulting faces can be reliably matched to true identities.
Figures & tables
Figure 1: A representative set of original unmasked faces (top) and the result of unmasking using Gemini to fill in the removal of the lower-half of the face. The value above each face is the biometric similarity (in [−1,1] ) between the original and unmasked face. Larger values correspond to a higher facial similarity.
Figure 2: Senator Elizabeth Warren’s mask (left) is removed (middle) using ChatGPT (photo sources: commons.wikimedia.org).
Figure 3: Distributions of biometric facial similarity scores: (A) pairs of images of the same identity (green), images of different identities from the same racial/gender group (red), and images of Doppelgangers (yellow); (B)-(D) pairs of a reference and unmasked image in which the lower-half was removed; and (E) pairs of a reference image and an image in which COVID masks were removed.
non match
non match
false match
Same-ID
AI-unmasked
Same-ID
AI-unmasked
1.0%
0.0000%
8.1%
0.0000%
6.3%
0.1%
0.0000%
17.3%
0.0000%
13.7%
0.01%
0.0001%
28.6%
0.0000%
23.1%
0.001%
0.0006%
40.7%
0.0001%
33.8%
0.0001%
0.0049%
52.3%
0.0012%
44.5%
Table 1: Face detection accuracy for a set of fixed false match rates (incorrectly identifying different identities as same). For each false match rate, non match rates are reported: the percentage of image pairs of the same identity incorrectly identified as different. The “Same-ID” column corresponds to the Same-ID dataset, and the “AI-unmasked” column corresponds to the combined ChatGPT and Gemini unmasking of the Senators dataset (see Figure 3 (A) and (E)).
AI-generated image detection is increasingly open-world: new image generators produce highly realistic images that make visual artifacts harder to identify. Existing detectors usually rely on a fixed set of forensic cues, so a detector that works well for one generator family may fail on another. We introduce FACT (Forensic Agent with Compiled Tool-use Trajectories), which learns an image-conditioned tool-use policy for forensic analysis. Instead of applying a fixed detector, FACT decides which forensic tools to call, interprets the returned evidence, and stops when sufficient evidence has been collected. FACT follows an Evolve--Distill--Refine pipeline: it evolves an execution-verified forensic skill, compiles the skill into action--observation tool-use trajectories, distills them into a compact agent, and refines the policy with cost-aware GRPO. Across two internal and four public benchmarks, FACT achieves the best performance among all compared methods, including on recent unseen generators, deepfakes, and manipulated images.
Jiaoyang Chen, Bin Hu, Jingyu Hu +3
Lingnan University, Hong Kong · Harbin Institute of Technology, Shenzhen · The University of Hong Kong +2
Suspect face generation remains a technical challenge in crime investigations. Traditional sketch-drawing workflows suffer from low efficiency and quality, while diffusion-based approaches still face intrinsic limitations on conditional ambiguity for text-to-image models and sampling variance for one-shot generation. We proposed IdentiFace, a novel diffusion-based framework for identifiable suspect face generation, which addressed these issues through (1) multi-modal input design to strengthen conditional control, and (2) an iterative generation pipeline enabling identifiable feature adjustment. We additionally contributed a facial identity loss and two task-specific datasets. Comprehensive experiments on synthetic datasets and in real-world scenarios indicate that IdentiFace achieves superior performance over existing methods, especially in terms of identity retrieval, and shows strong potential for practical applications.
Weichen Liu, Yixin Yang, Changsheng Chen +1
Southeast University · Nanjing, China · Shenzhen MSU-BIT University +1
AI-generated faces can be difficult to distinguish from real ones, leaving viewers to rely on source labels when judging an image. Yet prior work has made it difficult to separate the effects of what an image actually is from what viewers are told it is. We validated faces as AI-generated or human in an online study (N=169), then crossed actual source (AI, human) with label (none, Made with AI, Made by a human) in a lab study $N=30), recording event-related potentials (ERPs) and gaze. ERP responses were equivalent for AI-generated and real faces, but varied with the label: labels drew early attention (N2), while labels that conflicted with the face's actual source prompted re-evaluation of the face (P3). Affective processing and initial gaze orienting were unchanged, but labels altered visual exploration. We provide a validated stimulus set and evidence that attributed origin shapes face processing, with implications for disclosure design.
Teodora Mitrevska, Luise Donat, Andreas Butz +3
LMU Munich, Munich, Germany · HU Berlin, Berlin, Germany · Centrum Wiskunde & Informatica, Amsterdam, Netherlands +2