Have I Seen You? Embedding Behavior Signals Synthetic Face Dataset Membership

Paweł Borsukiewicz; Daniele Lunghi; Wendkûuni C. Ouédraogo; Jacques Klein; Tegawendé F. Bissyandé

cs.CVJul 31, 2026

Have I Seen You? Embedding Behavior Signals Synthetic Face Dataset Membership

Authors: Paweł Borsukiewicz, Daniele Lunghi, Wendkûuni C. Ouédraogo, Jacques Klein, Tegawendé F. Bissyandé

Organizations: University of Luxembourg, Luxembourg

Abstract

Synthetic face datasets are increasingly used to reduce privacy exposure and data access constraints in biometric recognition. Yet the generators that produce these datasets are trained on real faces, so synthetic data may still reveal their real source data. We study this risk through a dataset-level membership inference attack that first identifies the synthetic dataset used to train a face recognizer and then infers the real dataset used to train the generator. Across 11 face recognition models, 11 synthetic datasets, and 7 real datasets, the attack recovers the synthetic training dataset in 100% of cases and identifies the generator's source dataset in 54.5% of cases. These results show that synthetic data can retain dataset-level traces of real training data and that privacy-preserving deployment requires stronger leakage mitigation.

Explore similar work

Benchmarking Face Recognition without Real Faces
Jul 16, 2026Paweł Borsukiewicz, Daniele Lunghi, Wendkûuni C. Ouédraogo +2Facial Recognition Synthetic Dataset
ReMIA: a Powerful and Efficient Alternative to Membership Inference Attacks against Synthetic Data Generators
May 14, 2026Davide Scassola, Andrea Coser, Sebastiano SaccaniMembership Inference Attacks Synthetic Data
Face-trace: Open-Set Attribution and Progressive Discovery of Synthetic Face Generators
Jul 8, 2026Alessia Infantino, Claudio Schiavella, Irene AmeriniSynthetic Image Detection Facial Recognition

Jul 16, 2026 · cs.CVJ/K move · Enter open · S save

Benchmarking Face Recognition without Real Faces

Paweł Borsukiewicz, Daniele Lunghi, Wendkûuni C. Ouédraogo, Jacques Klein+1

University of Luxembourg · Luxembourg

Synthetic face datasets have become effective enough to train face recognition models with accuracy rivaling that of models trained on real photographs. This progress sidesteps the ethical and legal burdens of collecting real biometric data, yet evaluation has not kept pace. Even studies that train entirely on synthetic images still rely on real-face benchmarks to measure performance, leaving the privacy problem only half solved. We ask whether synthetic datasets can replace real benchmarks for face recognition evaluation. We test 12 synthetic datasets against 7 established real benchmarks using 24 pre-trained models that span both convolutional and transformer architectures. Our evaluation covers biometric verification metrics, similarity score distributions, cross-model ranking consistency, and the underlying distributional properties of each dataset. Benchmarking fidelity varies widely across the synthetic candidates, but the two strongest, MorphFace and Vec2Face, reproduce the relative behavior of real benchmarks and reach agreement levels that fall within the natural disagreement already observed among the real benchmarks themselves. These results establish that well-constructed synthetic datasets can support reliable comparative evaluation for face recognition, moving the field closer to a fully synthetic and privacy-preserving pipeline for both training and benchmarking.

Facial Recognition Synthetic Dataset Biometrics Benchmark Datasets

Open paper Ask about it PDF

Abstract

Explore similar work

Benchmarking Face Recognition without Real Faces

ReMIA: a Powerful and Efficient Alternative to Membership Inference Attacks against Synthetic Data Generators

Face-trace: Open-Set Attribution and Progressive Discovery of Synthetic Face Generators