Rank-1 Identity Consensus Predicts Gallery Enrollment in 1:N Face Matching More Accurately than Score Thresholding
Authors: Gabriella Pangelinan, Aman Bhatta, Michael C. King, Kevin W. Bowyer
Organizations: Florida Institute of Technology, Melbourne, FL · University of Notre Dame, Notre Dame, IN
Abstract
In operational 1:N face identification, a crucial question arises for each probe: is this person enrolled in the gallery or not? The stakes are high and asymmetric. Rejecting a mate-present (MP) probe loses a valid lead; accepting a mate-absent (MA) probe makes every returned candidate a false identification, at worst a wrongful arrest. Most approaches threshold match scores, but scores shift substantially with image quality and gallery size and composition, making thresholds fixed before deployment brittle under realistic conditions. Our prior work introduced 1-consistency, the only method based on rank consensus across multiple independently trained matchers: a probe is labeled MP if all matchers return the same rank-1 identity. This work stress-tests 1-consistency across 36 (gallery, probe quality) scenarios spanning four quality levels and two structural axes: images per identity and total enrolled identities. We benchmark against two score-thresholding methods that bracket what any deployed threshold could achieve. Fixed Score-Thresholding (FST), calibrated once on baseline conditions, collapses asymmetrically as quality degrades: MP recall falls below 2% while MA recall holds near 100%. Oracle Score-Thresholding (OST), re-tuned per scenario, is the best any threshold could theoretically do, yet for degraded probes 1-consistency matches it with zero tuning. The two differ mainly in error type (OST favors MP recall, 1-consistency favors MA recall), but on one axis 1-consistency does not merely match the oracle: when it labels a probe MP, it returns the correct mate 97-100% of the time versus OST's 66-84% under severe degradation. In short, 1-consistency delivers oracle-level accuracy without the impossible requirement: it sets no threshold, so it needs no advance knowledge of the conditions a probe will arrive in, which is what makes it usable.
Face identification has achieved remarkable performance under normal conditions. Yet, its accuracy often degrades significantly when query faces are partially occluded, especially by facial masks. Existing re-ranking approaches improve robustness by exploiting patch-level similarities. Still, they often rely on costly, fine-grained matching mechanisms, which limit their efficiency in large-scale retrieval scenarios. In this paper, we propose MGFace, a mask-gated face identification pipeline that predicts the mask status of a query face and conditionally routes the similarity computation accordingly. Specifically, MGFace distinguishes between masked and unmasked queries, applies global embedding matching to unmasked queries, and activates mask-aware patch-level re-ranking only for masked queries. This design focuses on reliable upper-face regions while avoiding unnecessary fine-grained computation. Experiments on the extended LFW-Mask dataset show that MGFace achieves over 80% identification accuracy with the FaceNet backbone and over 90% with the ArcFace backbone. Compared with a previous EMD-based re-ranking method, MGFace achieves better identification performance while reducing query time by approximately 20x. These results demonstrate the effectiveness of MGFace in improving masked-face identification accuracy with low computational overhead. The source code is available at https://github.com/chequanghuy/MGFace.
One-to-many facial identification is commonly used to match a probe image from surveillance video against a gallery of driver's license and/or booking photos. The algorithm's rank-one image from the gallery, or a human examiner's selection from the algorithm's top-ranked images, may then be placed in a photo lineup shown to a witness. Witness selection of the gallery image in the photo lineup may then lead directly to the person in the gallery image being arrested. This facial identification process is involved in a number of wrongful arrests. This work specifically examines whether the probability of a witness making an incorrect identification increases with the size of the gallery searched. We compare photo lineup accuracy when the "suspect" image is drawn from galleries of 500, 5,000, and 24,000 images. We find that larger galleries increase both the likelihood of a witness making an incorrect identification and their confidence in that (incorrect) identification. These results raise questions of whether an image resulting from such a facial identification process should be used in photo lineups and of whether results of a photo lineup alone should constitute probable cause for arrest.
Identity-preserving image generation becomes increasingly unreliable when a scene must contain many specified people. Beyond retaining each identity, the model must bind every reference to a distinct person and location, while training-time identity losses must establish correspondence among several noisy predicted faces. We introduce WithEveryone, a unified framework for generating group images up to ten reference identities. WithEveryone injects each selected identity as an addressed token, predicts a structured identity--layout plan, and renders the plan as a visual condition. Its key objective, Layout-Grounded ID Loss, uses annotated face regions to supervise the intended identities directly, avoiding unstable embedding-based face matching; ID Representation Forcing additionally trains a prediction for each identity before image synthesis. On an identity-disjoint benchmark, WithEveryone achieves the highest target-context identity similarity, improving face similarity from 0.462 for GPT-Image-2 to 0.499, while reducing copy-paste artifacts from 0.169 to 0.055. It further covers 97.3% of the requested identities with a duplicate rate of only 2.8%. These results show that explicit identity--layout grounding enables identity-preserving generation to scale to larger groups without relying on direct reference-face copying.